# Fuzzy Multicriteria Models for Decision Making in Gamification

^{1}

^{2}

## Abstract

**:**

## 1. Introduction

## 2. Fuzzy Analytic Hierarchy Process Methodology

_{max}[62], fuzzy preference programming [63], and two-stage logarithmic goal programming [64]. Although the Chang extent analysis methodology is widely used in the literature, it gives less accurate results [65]. The geometric mean method suggested by Buckley [59] is, therefore, the methodology to be used in this study as it is easier to apply and understand than other methodologies [66] and guarantees a unique solution.

- Choose the decision maker. A decision maker or a group of experts who know the problem and can provide information, judgements and the necessary validation of the results obtained from the model.
- Build the hierarchical structure. Choosing and defining the criteria and subcriteria relevant to the problem structuring them into a hierarchical tree. At the higher level of the hierarchy is the objective of the problem, while the criteria and subcriteria are placed at the following levels respectively, and finally, the alternatives are placed at the lower level.
- Select the fuzzy scale. The original scale proposed by Saaty, in which a judgement is associated with an integer from 1 to 9 or their inverses, does not include uncertainties (doubt, vagueness, hesitancy or ambiguous situations) [67] which characterises decision problems in the real world. Also, decision makers sometimes feel more confident giving interval judgments rather than crisp judgements [68]. Different fuzzy scales are proposed in the literature [67,69,70,71]. From these, the scale given by Lamata [72], which is shown in Table 2, was chosen for this study, as it corresponds better to the original scale proposed by Saaty in the crisp AHP. One example of the application of this scale can be seen in Koulinas et al. [73].
- Building the fuzzy judgement matrices among the entire hierarchy. The individual decision maker or decision group should give fuzzy judgements when comparing criteria/subcriteria or alternatives and between the scale levels of each criterion/subcriterion. The elements of the fuzzy pairwise comparison matrix $\tilde{A}$ are the fuzzy values ${\tilde{M}}_{ij}$, which express the decision maker’s judgement about the relative importance of element i over element j, using the fuzzy scale of Table 1, at the same level of the hierarchy.$$\tilde{A}=\left(\begin{array}{c}\begin{array}{c}\tilde{1}\\ {\tilde{M}}_{21}\end{array}\\ \begin{array}{c}\vdots \\ {\tilde{M}}_{n1}\end{array}\end{array}\begin{array}{c}\begin{array}{c}{\tilde{M}}_{12}\\ \tilde{1}\end{array}\\ \begin{array}{c}\vdots \\ {\tilde{M}}_{n2}\end{array}\end{array}\begin{array}{c}\begin{array}{c}\cdots \\ \cdots \end{array}\\ \begin{array}{c}\ddots \\ \cdots \end{array}\end{array}\begin{array}{c}\begin{array}{c}{\tilde{M}}_{1n}\\ {\tilde{M}}_{2n}\end{array}\\ \begin{array}{c}\vdots \\ \tilde{1}\end{array}\end{array}\right)=\left(\begin{array}{c}\begin{array}{c}\tilde{1}\\ 1/{\tilde{M}}_{12}\end{array}\\ \begin{array}{c}\vdots \\ 1/{\tilde{M}}_{1n}\end{array}\end{array}\begin{array}{c}\begin{array}{c}{\tilde{M}}_{12}\\ \tilde{1}\end{array}\\ \begin{array}{c}\vdots \\ 1/{\tilde{M}}_{2n}\end{array}\end{array}\begin{array}{c}\begin{array}{c}\cdots \\ \cdots \end{array}\\ \begin{array}{c}\ddots \\ \cdots \end{array}\end{array}\begin{array}{c}\begin{array}{c}{\tilde{M}}_{1n}\\ {\tilde{M}}_{2n}\end{array}\\ \begin{array}{c}\vdots \\ \tilde{1}\end{array}\end{array}\right)$$
- Calculating fuzzy weights. The geometric mean method [59] is used to obtain the fuzzy weights of each criterion/subcriterion via Equations (3) and (4):$${\tilde{r}}_{i}={\left({\tilde{M}}_{i1}\otimes {\tilde{M}}_{i2}\otimes \cdots \otimes {\tilde{M}}_{in}\right)}^{\frac{1}{n}}\text{}$$$${\tilde{w}}_{i}={\tilde{r}}_{i}\otimes {({\tilde{r}}_{1}\oplus {\tilde{r}}_{2}\oplus \cdots \oplus {\tilde{r}}_{n})}^{-1}\text{}\forall \text{}\mathrm{i}=1,\text{}2,\text{}\dots ,\text{}\mathrm{n}$$
- Defuzzification process. ${\tilde{w}}_{i}$ need to be defuzzified, finding the best non-fuzzy performance value (BNP) [74]. Different methods can be used for this: mean of maximum (MOM), centre of area (COA) and α-cut. The COA or centroid method is simple and practical, and there is no need to bring in the preferences of any assessors [75], and so it is the method applied in this research. Equation (5) should be applied to the ${w}_{i}$ obtained from Equation (4) [74,76]:$${w}_{i}=({l}_{i}+{m}_{i}+{u}_{i})/3,\text{}i=1,2,\dots ,n$$
- Normalisation. ${w}_{i}$ can be normalised by applying Equation (6).$${z}_{i}=\frac{{w}_{i}}{{{\displaystyle \sum}}_{i}^{n}{w}_{i}},i=1,2,\dots ,n$$
- Evaluate the consistency of the judgements. Let $\tilde{R}=[{\tilde{r}}_{ij}]$ be a fuzzy judgement matrix comprising triangular fuzzy numbers ${\tilde{r}}_{ij}=\left({\alpha}_{ij},{\beta}_{ij},{\gamma}_{ij}\right)$; a crisp matrix $R=({\beta}_{ij})$ can be produced. If $R$ is consistent, then $\tilde{R}$ is consistent [62].Saaty [47] defined the consistency ratio (CR) to quantify the consistency of the judgements given in each pairwise matrix (see Equation (7)).$$CR=\frac{CI}{RCI},$$$$CI=\frac{\left({\lambda}_{max}-n\right)}{\left(n-1\right)}$$

## 3. Fuzzy Analytic Hierarchy Process Model for Decision Making in Gamification

#### 3.1. Structuring

- Flexibility in the creation of questionnaires (FCQ). The flexibility of the applications is evaluated in order to include different types of questions (true/false, short questions, etc.), the possibility of grading each question independently, the number of answers to each question, the possibility of including questions with pictures and videos. The scale levels of the descriptor are:
- -
- L11. The flexibility in the way questions are asked is very high, as it allows all possible types of question to be included (true/false, short questions and single and multiple choice), the number of answers is unlimited, the questions can use images and video, additional explanations can be included for each question without a character limit, and the questionnaire can be downloaded as a file. (Good)
- -
- L12. The flexibility in putting the questions is high. It allows all possible types of question to be included (true/false, short questions and single and multiple choice), the number of answers is limited to five, the questions can use images and video, additional explanations can be included for each question with a character limit, and the questionnaire can be downloaded as a file.
- -
- L13. The flexibility in putting the questions is medium. It allows all possible types of question to be included (true/false, short questions and single and multiple choice), the number of answers is limited to four, the questions can use images and video, additional explanations cannot be included for each question, and the questionnaire cannot be downloaded as a file. (Neutral)
- -
- L14. The flexibility in putting the questions is low. It allows all possible types of question to be included (true/false, short questions and single and multiple choice), the number of answers is limited to four or fewer, the questions cannot use images and video, additional explanations cannot be included for each question, and the questionnaire cannot be downloaded as a file.
- -
- L15. The flexibility in putting the questions is very low. It does not allow all possible types of question to be included (true/false, short questions and single and multiple choice), the number of answers is limited to four or fewer, the questions cannot use images and video, additional explanations cannot be included for each question, and the questionnaire cannot be downloaded as a file.

- Learning rhythm (LRH). Defined as the speed with which a person can learn, a number of levels can be identified: fast, medium and slow. It is essential that the application allows the rhythm of the activities, and thus of the learning, to be controlled. Thus, this criterion uses the descriptor Capacity of the Teacher to Control the Rhythm by a time limit, an unlimited time, and repeating the test as often as necessary. The scale levels of the descriptor in decreasing order of function are:
- -
- L21. Can be controlled by the teacher, who has the option to set a time limit or let each student complete the questionnaire at their own rhythm without a time limit and as many times as they wish (with no need to wait for other people’s answers). (Good)
- -
- L22. Can be controlled by the teacher, who has the option to set a time limit or let each student complete the questionnaire at their own rhythm without a time limit but only once (with no need to wait for other people’s answers). (Neutral)
- -
- L23. A time must be set for solving each question, with a limit of 15 min or less.
- -
- L24. A time must be set for solving each question, with a limit of 120 s or less.

- Assessment of the questionnaire (AQU). The descriptor used to evaluate this criterion is the versatility in assigning a score to each question, as well as calculating the time taken to respond. The scale levels of the descriptor are:
- -
- L31. Each question can be given an independent score and, if there is a tie, the time taken by each student can be used. (Good)
- -
- L32. Each question is assessed depending on the number of right answers and the time taken to respond.
- -
- L33. Each question is assessed by the number of right answers. (Neutral)
- -
- L34.Each question is assessed by the time taken to respond.
- -
- L35. The questions are not scored at all.

- Obtaining results and reports (ORR). The ability to obtain reports and to show the results of the participants during the game is assessed. The option to hide names is considered worth bearing in mind, since in some cases students preferred not to have their results made known. The scale levels from more to less attractive are as follows:
- -
- L41. Download of the Excel file with participants’ results question by question. Visual scanning of the results on the platform. The names of participants in the score list can be hidden during the game. A final score list of participants can be shown in real time. (Good)
- -
- L42. Download of the Excel file with participants’ results question by question. Visual scanning of the results on the platform. The names of participants in the score list cannot be hidden during the game. The final score list of the participants is shown in real time.
- -
- L43. Download of the Excel file with participants’ results question by question. Visual scanning of the results on the platform is not possible. The names of participants in the score list cannot be hidden during the game. The final score list of the participants is shown in real time. (Neutral)
- -
- L44. Download of the Excel file with participants’ results as a whole, not question by question. Visual scanning of the results on the platform is not possible. The names of participants in the score list cannot be hidden during the game. The final score list of the participants is not shown in real time.
- -
- L45. An Excel file with the results of participants’ question by question cannot be downloaded, nor converted to Excel. Visual scanning of the results on the platform is not possible. The names of participants in the score list cannot be hidden during the game. The final score list of the participants is not shown in real time.

- Ability to apply just-in-time teaching (AJI). The possibility of using a weak form of Just In Time Teaching (JITT), with a need for a prior study (open questions on prior study, with questions about what is and is not understood and where to provide support and reinforcement), is assessed, and then also a strong JITT (closed questions, to directly check knowledge of content, in order to assess it). The scale levels used for the descriptor, in descending order of function, are as follows:
- -
- L51. JITT can be used easily. (Good)
- -
- L52. JITT can be applied although it requires extra time to design the questions. (Neutral)
- -
- L53. JITT can be used but the information gathered is limited, as questions calling for short answers cannot be included in the questionnaire.
- -
- L54. JITT cannot be used, as it cannot gather short answers, or grade the answers to the questionnaires by default.

- Elements of gamification (amusement) with impact/motivation on the student (EGS). The ability to assess the broadening of the application to include elements of gamification typical of games. The scale levels of the descriptors that assess this criterion are:
- -
- L61. The application allows each student to develop their own avatar or make use of those available on the platform, messages of support and encouragement (memes), the possibility of adding images, embedding Youtube videos or adding music to questions. A fun final score list is shown. (Good)
- -
- L62. The application allows students to make use of avatars available on the platform, the possibility of adding images and embedding Youtube videos; it does not offer messages of support and encouragement (memes) or the chance of adding music to questions. A fun final score list is shown. (Neutral)
- -
- L63. The application does not have avatars available on the platform, although it does offer the possibility of adding images and embedding Youtube videos. It does not offer messages of support and encouragement (memes) or the chance of adding music to questions. A fun final score list is shown.
- -
- L64. It is a sober application with no fun element.

- Quality of the question library (QQL). Quantity of available public questionnaires, capacity for sharing, duplicating or editing, and the strength of the forum for exchanging experiences and information is the descriptor used to assess this criterion. It has the following scale levels:
- -
- L71. It has a library with more than two million publicly available questionnaires that can be used. The questionnaires can be shared, duplicated and edited. It has an active Internet forum for exchanging experience and information. (Good)
- -
- L72. It has a library with up to half a million publicly available questionnaires that can be used. The questionnaires can be shared and duplicated, but not edited. There are no forums with significant information on the application. (Neutral)
- -
- L73. There is no library, or the resources of other users are not available. Questionnaires cannot be shared, duplicated or edited. There are no forums with significant information on the application.

- Ease of use in class (EUC). Versatility of application in class is analysed via different devices and the need for extra equipment. The scale levels of the descriptor in descending order of attractiveness are as follows:
- -
- L81. Can be used in class with a mobile, tablet or laptop. Does not require a projector (Good).
- -
- L82. Can be used in class with a mobile or laptop. Does not require a projector (Neutral).
- -
- L83. Can be used in class with a mobile or tablet (but not a laptop in some game modes). Does not require a projector.
- -
- L84. Can be used in class with a mobile or tablet (but not a laptop in some game modes). Requires the use of a projector (in some game modes).
- -
- L85. Can only be used in class with a mobile. Requires the use of a projector.

#### 3.2. Weighting

## 4. Fuzzy Analytic Hierarchy Process Combined with the Measuring Attractiveness by a Categorical-Based Evaluation Technique (MACBETH) Approach Model for Decision Making in Gamification

#### 4.1. Structuring

#### 4.2. Weighting

## 5. Results and Discussion

^{©}Logical Decision. Increasing the weighting of the criteria Flexibility in the creation of questionnaires, Obtaining results and reports, Ability to apply just-in-time teaching to 100% or decreasing to 0% shows no variation in the classification of the alternatives, and Socrative is the application chosen in all cases.

## 6. Conclusions

## Funding

## Conflicts of Interest

## References

- Corchuelo-Rodríguez, C.A. Gamificación en Educación Superior: Experiencia innovadora para motivar estudiantes y dinamizar contenidos en el aula. Edutec
**2018**, 63, 29–41. [Google Scholar] [CrossRef] - Rodríguez, F.; Santiago, R. Gamificación: Cómo Motivar a tu Alumnado y Mejorar el Clima en el Aula. Innovación Educativa; Editorial Océano: Barcelona, Spain, 2015. [Google Scholar]
- Contreras, R.S.; Eguia, J.L. (Eds.) Experiencias de Gamificación en Aulas; Institut de la Comunicació, Universitat Autònoma de Barcelona: Bellaterra, Barcelona, Spain, 2017. [Google Scholar]
- Hurtado, S. Gamificación: Que es y Cuáles Son sus Ventajas. 2017. Available online: http://es.eadbox.com/gamificacion-que-es-y-cuales-son-sus-ventajas/ (accessed on 23 March 2019).
- Oliva, H.A. The gamification as a methodological strategy in the university educational context. Real. Reflect.
**2016**, 44, 29–47. [Google Scholar] - Palomar, M.J. Ventajas e Inconvenientes de las TIC en la Docencia, Innovación y Experiencias Educativas. 2019. Available online: https://archivos.csif.es/archivos/andalucia/ensenanza/revistas/csicsif/revista/pdf/Numero_25/MARIA_JOSE_PALOMAR_SANCHEZ01.pdf (accessed on 2 March 2020).
- Gee, J.P. What Video Games Have to Teach Us about Learning and Literacy, 2nd ed.; St. Martin’s Griffin: New York, NY, USA, 2007. [Google Scholar]
- Torres-Toukoumidis, A.; Romero-Rodríguez, L.M.; Pérez-Rodríguez, A.M. Ludificación y sus posibilidades en el entorno de blended learning: Revisión documental. RIED. Rev. Iberoam. Educ. Distancia
**2018**, 21, 95–111. [Google Scholar] [CrossRef][Green Version] - Acuña, M. Las 5 Mejores Herramientas DE Gamificación para Universitarios. Available online: https://www.evirtualplus.com/herramientas-de-gamificacion-para-universitarios/ (accessed on 16 September 2019).
- Ruiz, R.; Tesouro, M. Beneficios e inconvenientes de las nuevas tecnologías en el aprendizaje del alumno. Propuestas formativas para alumnos, profesores y padres. Rev. Educ. Futuro Digit.
**2013**, 7, 17–27. [Google Scholar] - González-González, C.S.; del Río, N.G.; Navarro-Adelantado, V. Exploring the Benefits of Using Gamification and Videogames for Physical Exercise: A Review of State of Art. Int. J. Interact. Multimed. Artif. Intell.
**2018**, 5, 46–52. [Google Scholar] [CrossRef][Green Version] - Çakıroğlu, Ü.; Başıbüyük, B.; Güler, M.; Atabay, M.; Memiş, B.Y. Gamifying an ICT course: Influences on engagement and academic performance. Comput. Hum. Behav.
**2017**, 69, 98–107. [Google Scholar] [CrossRef] - Hamari, J.; Koivisto, J.; Sarsa, H. Does gamification work?—A literature review of empirical studies on gamification. In Proceedings of the 2014 47th Hawaii International conference on system sciences, Waikoloa, HI, USA, 6–9 January 2014; IEEE: Piscataway, NJ, USA; pp. 3025–3034. [Google Scholar]
- Wang, A.I.; Tahir, R. The effect of using Kahoot! for learning—A literature review. Comput. Educ.
**2020**, 149. [Google Scholar] [CrossRef] - Serna, E.; Mauricio, M.D.; San Miguel, T.; Mejías, J. Experiencia de gamificación en Docencia Universitaria: Aprendizaje activo y entretenido. In Procceedings of the In-Red 2016, Valencia, Spain, 7–8 July 2016; Universitat Politècnica de València: Valencia, Spain, 2016. [Google Scholar]
- Yildirim, I. The effects of gamification-based teaching practices on student achievement and students’ attitudes toward lessons. Internet High. Educ.
**2017**, 33, 86–92. [Google Scholar] [CrossRef] - Morís, G. Gamificación a través de Kahoot como innovación docente en el Grado de Logopedia. In Proceedings of the INNODOCT 2017, Valencia, Spain, 20–25 October 2017; Universitat Politècnica de València: Valencia, Spain, 2017; pp. 705–714. [Google Scholar]
- Dias, J. Teaching operations research to undergraduate management students: The role of gamification. Int. J. Manag. Educ.
**2017**, 15, 98–111. [Google Scholar] [CrossRef][Green Version] - Dell, K.A.; Chudow, M.B. A web-based review game as a measure of overall course knowledge in pharmacotherapeutics. Curr. Pharm. Teach. Learn.
**2019**, 11, 838–842. [Google Scholar] [CrossRef] - Knutas, A.; Ikonen, J.; Nikula, U.; Porras, J. Increasing collaborative communications in a programming course with gamification: A case study. In Proceedings of the 15th International Conference on Computer Systems and Technologies, Ruse, Bulgaria, 27 June 2014. [Google Scholar] [CrossRef]
- Iosup, A.; Epema, D. An experience report on using gamification in technical higher education. In Proceedings of the 45th ACM technical symposium on Computer science, education, Atlanta, GA, USA, 5–8 March 2014; ACM: New York, NY, USA; pp. 27–32. Available online: https://goo.gl/ISLuL6 (accessed on 14 March 2020).
- Laskowski, M. Implementing gamification techniques into university study path-A case study. In Proceedings of the IEE Global Engineering Education Conference (EDUCON), Tallinn, Estonia, 18–20 March 2015; IEEE: Rome, Italy, 2015; pp. 582–586. [Google Scholar]
- Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in education: A systematic mapping study. Educ. Technol. Soc.
**2015**, 18, 75–88. [Google Scholar] - Rodríguez-Palero, M.; Guadix Martín, J.; Barbadilla Martín, E.; Robles Velasco, A. Experiencia piloto de gamificación con uso de aplicaciones móviles en una asignatura de máster de Ingeniería Industrial. In Proceedings of the 13th International Conference on Industrial Engineering and Industrial Management XXIII Congreso de Ingeniería de Organización, Gijón, Spain, 11–12 July 2019. [Google Scholar]
- Gartner. Gartner Says by 2015, More than 50 Percent of Organizations That Manage Innovation Processes will Gamify Those Processes. 2011. Available online: http://www.gartner.com/it/page.jsp?id=1629214 (accessed on 14 February 2017).
- Martínez, E. La utilización de la gamificación para la formación aumenta un 18% en España, RRHH Digital. Available online: http://www.rrhhdigital.com/secciones/tecnologia-e-innovacion/120597/La-utilizacion-de-la-gamificacion-para-la-formacion-aumenta-un-18-en-Espana (accessed on 16 May 2019).
- Koivisto, J.; Hamari, J. The rise of motivational information systems: A review of gamification research. Int. J. Inf. Manag.
**2019**, 45, 191–210. [Google Scholar] [CrossRef] - Padilla-Zea, N.; Aceto, S.; Burgos, D. Social Seducement: Empowering Social Economy Entrepreneurship. The Training Approach. Int. J. Interact. Multimed. Artif. Intell.
**2019**, 5, 135–150. [Google Scholar] [CrossRef] - Gupta, P. Tools, Tips & Resources Teachers Must Know to Learn about Gamification of Education. Available online: https://edtechreview.in/trends-insights/insights/2293-gamification-of-education (accessed on 2 March 2020).
- Lynch, M. 8 Must Have Gamification Apps, Tools, and Resources. Available online: https://www.thetechedvocate.org/8-must-gamification-apps-tools-resources/ (accessed on 2 March 2020).
- Educación 3.0. 20 Herramientas de Gamificación para Clase que Engancharán a Tus Alumnos. Available online: https://www.educaciontrespuntocero.com/recursos/herramientas-gamificacion-educacion/33094.html (accessed on 2 March 2020).
- Loayza, J. The 10 Best Educational Apps That Use Gamification for Adults in 2019. Available online: https://yukaichou.com/gamification-examples/top-10-education-gamification-examples/ (accessed on 2 March 2020).
- Göksün, D.O.; Gürso, G. Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz. Comput. Educ.
**2019**, 135, 15–29. [Google Scholar] [CrossRef] - Kim, S. Decision Support Model for Introduction of Gamification Solution Using AHP. Optim. Decis. Sci.
**2014**, 714239. [Google Scholar] [CrossRef] - Putra, D.; Sobandi, M.; Andryana, S.; Gunaryati, A. Fuzzy Analytical Hierarchy Process Method to Determine the Quality of Gemstones. Adv. Fuzzy Syst.
**2018**, 9094380. [Google Scholar] [CrossRef] - Roubens, M. Fuzzy sets and decision analysis. Fuzzy Sets Syst.
**1997**, 90, 199–206. [Google Scholar] [CrossRef] - Cox, R.; Sanchez, J.; Revie, C.W. Multi-Criteria Decision Analysis Tools for Prioritising Emerging or Re-Emerging Infectious Diseases Associated with Climate Change in Canada. Plos One
**2013**, 8, e68338. [Google Scholar] [CrossRef] - Pishdar, M.; Ghasemzadeh, F.; Antuchevičienė, J. A mixed interval type-2 fuzzy best-worst MACBETH approach to choose hub airport in developing countries: Case of Iranian passenger airports. Transport
**2019**, 34, 639–651. [Google Scholar] [CrossRef][Green Version] - Gürbüz, T.; Alptekin, S.E.; Alptekin, G.I. A hybrid MCDM methodology for ERP selection problem with interacting criteria. Decis. Support Syst.
**2012**, 54, 206–214. [Google Scholar] [CrossRef] - Ertay, T.; Kahraman, C.; Kaya, İ. Evaluation of renewable energy alternatives using MACBETH and fuzzy AHP multicriteria methods: The case of Turkey. Technol. Econ. Dev. Econ.
**2013**, 19, 38–62. [Google Scholar] [CrossRef][Green Version] - Ferreria, F.A.; Santos, S.P. Comparing trade-off adjustments in credit risk analysis of mortgage loans using AHP, Delphi and MACBETH. Int. J. Strateg. Prop. Manag.
**2013**, 20, 44–63. [Google Scholar] [CrossRef] - Dhouib, D. An extension of MACBETH method for a fuzzy environment to analyze alternatives in reverse logistics for automobile tire wastes. Omega
**2014**, 42, 25–32. [Google Scholar] [CrossRef] - Herrera, F.; Martinez, L. An approach for combining linguistic and numerical information based on the 2-tuple fuzzy linguistic representation model in decision-making. Int. J. Uncertain. Fuzziness Knowl. Based Syst.
**2000**, 8, 539–562. [Google Scholar] [CrossRef] - Roger, S.; Cobos, M.; Arevalillo-Herráez, M.; García-Pineda, M. Combinación de cuestionarios simples y gamificados utilizando gestores de participación en el aula: Experiencia y percepción del alumnado. In Proceedings of the III Congreso Nacional de Innovación Educativa y Docencia en Red, Valencia, Spain, 13–14 July 2017; pp. 1–12. [Google Scholar] [CrossRef]
- Fuertes, A.; García-Pineda, M.; Castaño, M.A.; López-Iñesta, E.; Zacarés, M.; Cobos, M.; Ferris, R.; Grimaldo, F. Uso de herramientas de respuesta de audiencia en la docencia presencial universitaria. Un primer contacto. In Proceedings of the XXII JENUI, Almería, Spain, 5 July 2016; pp. 261–268. [Google Scholar]
- Morente-Molinera, J.A.; Ríos-Aguilar, S.; González-Crespo, R. Dealing with group decision-making environments that have a high amount of alternatives using card-sorting techniques. Expert Syst. Appl.
**2019**, 127, 187–198. [Google Scholar] [CrossRef] - Saaty, T.L. The Analytic Hierarchy Process; McGraw Hill: New York, NY, USA, 1980. [Google Scholar]
- Saaty, T.L.; Vargas, L.G. Decision Making IN Economic, Politic, Social and Technological Environments with the Analytic Hierarchy Process; Analytic Hierarchy Process Series; RWS Publications: Pittsburgh, PA, USA, 1994; Volume VII. [Google Scholar]
- Lin, H.F. An application of fuzzy AHP for evaluating course website quality. Comput. Educ.
**2010**, 54, 877–888. [Google Scholar] [CrossRef] - Zadeh, L.A. Fuzzy sets. Inf. Control
**1965**, 8, 338–353. [Google Scholar] [CrossRef][Green Version] - Nguyen, H.T.; Md Dawal, S.Z.; Nukman, Y.P.; Rifai, A.; Aoyama, H. An Integrated MCDM Model for Conveyor Equipment Evaluation and Selection in an FMC Based on a Fuzzy AHP and Fuzzy ARAS in the Presence of Vagueness. PLoS ONE
**2016**, 11, e0153222. [Google Scholar] [CrossRef] - Cebeci, U. Fuzzy AHP-based decision support system for selecting ERP systems in textile industry by using balanced scorecard. Expert Syst Appl
**2009**, 36, 8900–8909. [Google Scholar] [CrossRef] - Kaufmann, A.; Gupta, M.M. Fuzzy Mathematical Models in Engineering and Management Science; North Holland: Amsterdam, The Netherlands, 1988. [Google Scholar]
- Keršulienė, V.; Turskis, Z. A hybrid linguistic fuzzy multiple criteria group selection of a chief accounting officer. J. Bus. Econ. Manag.
**2014**, 15, 232–252. [Google Scholar] [CrossRef][Green Version] - Kubler, S.; Robert, J.; Derigent, W.; Voisin, A.; Le Traon, Y. A state-of-the-art survey & testbed of fuzzy AHP (FAHP) applications. Expert Syst. Appl.
**2016**, 65, 398–422. [Google Scholar] [CrossRef] - Calabrese, A.; Costa, R.; Levialdi, N.; Menichini, T. Integrating sustainability into strategic decision-making: A fuzzy AHP method for the selection of relevant sustainability issues. Technol. Forecast. Soc. Chang.
**2019**, 139, 155–168. [Google Scholar] [CrossRef] - Turskis, Z.; Zavadskas, E.K.; Antucheviciene, J.; Kosareva, N. A hybrid model based on fuzzy AHP and fuzzy WASPAS for construction site selection. Int. J. Comput. Commun. Control
**2015**, 10, 113–128. [Google Scholar] [CrossRef] - Van Laarhoven, P.J.M.; Pedrycz, W. A fuzzy extension of Saaty’s priority theory. Fuzzy Sets Syst.
**1983**, 11, 229–241. [Google Scholar] [CrossRef] - Buckley, J.J. Fuzzy hierarchical analysis. Fuzzy Sets Syst.
**1985**, 17, 233–247. [Google Scholar] [CrossRef] - Chang, D.Y. Applications of the extent analysis method on fuzzy AHP. Eur. J. Oper. Res.
**1996**, 95, 649–655. [Google Scholar] [CrossRef] - Xu, R. Fuzzy least square priority method in the analytic hierarchy process. Fuzzy Sets Syst.
**2000**, 112, 395–404. [Google Scholar] [CrossRef] - Csutora, R.; Buckley, J.J. Fuzzy hierarchical analysis: The Lambda-Max method. Fuzzy Sets Syst.
**2001**, 120, 181–195. [Google Scholar] [CrossRef] - Mikhailov, L. Deriving priorities from fuzzy pairwise comparison judgements. Fuzzy Sets Syst.
**2003**, 134, 365–385. [Google Scholar] [CrossRef] - Wang, Y.M.; Yang, J.B.; Xu, D.L. A two-stage logarithmic goal programming method for generating weights from interval comparison matrices. Fuzzy Sets Syst.
**2005**, 152, 475–498. [Google Scholar] [CrossRef] - Ahmed, F.; Kilic, K. Fuzzy Analytic Hierarchy Process: A performance analysis of various algorithms. Fuzzy Sets Syst.
**2019**, 362, 110–128. [Google Scholar] [CrossRef] - Jenatabadi, H.S.; Babashamsi, P.; Yusoff, N.I. The Combination of a Fuzzy Analytical Hierarchy Process and the Taguchi Method to Evaluate the Malaysian Users’ Willingness to Pay for Public Transportation. Symmetry
**2016**, 8, 90. [Google Scholar] [CrossRef][Green Version] - Cheng, C.H. Evaluating naval tactical missile systems by fuzzy AHP based on the grade value of membership function. Eur. J. Oper. Res.
**1996**, 96, 343–350. [Google Scholar] [CrossRef] - Isaai, M.T.; Kanani, A.; Tootoonchi, M.; Afzali, H.R. Intelligent timetable evaluation using fuzzy AHP. Expert Syst. Appl.
**2011**, 38, 3718–3723. [Google Scholar] [CrossRef] - Zhu, K.J.; Jing, Y.; Chang, D.Y. A discussion on Extent Analysis Method and applications of fuzzy AHP. Eur. J. Oper. Res.
**1999**, 116, 450–456. [Google Scholar] [CrossRef] - Wang, L.; Chu, J.; Wu, J. Selection of optimum maintenance strategies based on a fuzzy analytic hierarchy process. Int. J. Prod. Econ.
**2007**, 107, 151–163. [Google Scholar] [CrossRef] - SeongKon, L.; Gento, M.; SangKon, L.; JongWook, K. Prioritizing the weights of hydrogen energy Technologies in the sector of the hydrogen economy by using a fuzzy AHP approach. Int. J. Hydrogen Energy
**2011**, 36, 1897–1902. [Google Scholar] - Lamata, M.T. Ranking of alternatives with ordered weighted averaging operators. Int. J. Intell. Syst.
**2004**, 19, 473–482. [Google Scholar] [CrossRef] - Koulinas, G.K.; Marhavilas, P.K.; Demesouka, O.E.; Vavatsikos, A.P.; Koulouriotis, D.E. Risk analysis and assessment in the worksites using the fuzzy-analytical hierarchy process and a quantitative technique—A case study for the Greek construction sector. Saf. Sci.
**2019**, 112, 96–104. [Google Scholar] [CrossRef] - Opricovic, S.; Tzeng, G.H. Defuzzification within a multicriteria decision model, International Journal of Uncertainty. Fuzziness Knowl. Based Syst.
**2003**, 11, 635–652. [Google Scholar] [CrossRef] - Chen, V.Y.C.; Lien, H.P.; Liu, C.H.; Liou, J.J.H.; Tzeng, G.H.; Yang, L.S. Fuzzy MCDM approach for selecting the best environment-watershed plan. Appl. Soft Comput.
**2011**, 11, 265–275. [Google Scholar] [CrossRef] - Chang, T.H.; Wang, T.C. Using the fuzzy multi-criteria decision making approach for measuring the possibility of successful knowledge management. Inf. Sci.
**2009**, 179, 355–370. [Google Scholar] [CrossRef] - Saaty, T.L. Decision Making with Dependence and Feedback: The Analytic Network Process; RWS Publications: Pittsburgh, PA, USA, 2001. [Google Scholar]
- Bana e Costa, C.A.; Ensslin, L.; Correa, E.C.; Vansnick, J.C. Decision Support System in action: Integrated application in a multicriteria decision aid process. Eur. J. Oper. Res.
**1999**, 113, 315–335. [Google Scholar] [CrossRef] - Bana e Costa, C.A.; Carvalho, R. Assigning priorities for maintenance, repair and refurbishment in managing a municipal housing stock. Eur. J. Oper. Res.
**2002**, 138, 380–391. [Google Scholar] [CrossRef] - Kusuma, G.P.; Wigati, E.K.; Utomo, Y.; Suryapranata, L.K.P. Analysis of Gamification Models in Education Using MDA Framework 3rd International Conference on Computer Science and Computational Intelligence 2018. Procedia Comput. Sci.
**2018**, 135, 385–392. [Google Scholar] [CrossRef] - Bana e Costa, C.A.; Vansnick, J.C. Applications of the MACBETH approach in the framework of an additive aggregation model. J. Multi-Criteria Decis. Anal.
**1997**, 6, 107–114. [Google Scholar] [CrossRef] - Bana e Costa, C.A.; De Corte, J.M.; Vansnick, J.C. MACBETH. Int. J. Inf. Technol. Decis. Mak.
**2012**, 11, 359–387. [Google Scholar] [CrossRef] - Bana e Costa, C.A.; Vansnick, J.C. The MACBETH approach: General overview and applications. In Advances in Multicriteria Analysis; Pardalos, P.M., Siskos, Y., Zapounidis, C., Eds.; Kluwer Academic Publishers: Dordrecht, The Netherlands, 1995. [Google Scholar]
- Bana e Costa, C.A.; De Corte, J.M.; Vansnick, J.C. MACBETH User’s Guide. 2005. Available online: http://m-macbeth.com/wp-content/uploads/2017/10/M-MACBETH-Users-Guide.pdf (accessed on 24 March 2019).
- Bana e Costa, C.A.; Vansnick, J.C. The MACBETH approach: Basic ideas, software, and an application, Mathematical Modelling. Theory Appl.
**1999**, 4, 131–157. [Google Scholar] [CrossRef] - Bana e Costa, C.A.; Correa, E.; De Corte, J.M.; Vansnick, J.C. Facilitating bid evaluation in public call for tenders: A socio-technical approach. Omega Int. J. Manag.
**2002**, 30, 227–242. [Google Scholar] [CrossRef] - Bana e Costa, C.A.; Chagas, M.P. A career choice problem: An example of how to use MACBETH to build a quantitative value model based on qualitative value judgments. Eur. J. Oper. Res.
**2004**, 153, 323–331. [Google Scholar] [CrossRef]

**Figure 2.**Measuring attractiveness by a categorical-based evaluation technique (MACBETH) judgement matrix of criterion Flexibility in the creation of questionnaires.

**Figure 3.**Value functions of criteria (from left to right): Flexibility in the creation of questionnaires, Learning rhythm, Assessment of the questionnaire, Obtaining results and reports, Ability to apply just-in-time teaching, Elements of gamification (amusement) with impact/motivation on the student, Quality of the question library and Ease of use in class.

**Figure 4.**Completed MACBETH judgement matrix to give weightings for the criteria obtained via fuzzy analytic hierarchy process (AHP).

**Figure 7.**Sensitivity analysis (left to right and top to bottom): Flexibility in the creation of questionnaires, Learning rhythm, Assessment of the questionnaire, Obtaining results and reports, Ability to apply just-in-time teaching, Elements of gamification (amusement) with impact/motivation on the student, Quality of the question library and Ease of use in class.

Operations | Results |
---|---|

Addition | ${\tilde{M}}_{1}\oplus {\tilde{M}}_{2}=\left({l}_{1}+{l}_{2},{m}_{1}+{m}_{2},{u}_{1}+{u}_{2}\right)$ |

Subtraction | ${\tilde{M}}_{1}\ominus {\tilde{M}}_{2}=\left({l}_{1}-{u}_{2},{m}_{1}-{m}_{2},{u}_{1}-{l}_{2}\right)$ |

Multiplication | ${\tilde{M}}_{1}\otimes {\tilde{M}}_{2}\approx \left({l}_{1}{l}_{2},{m}_{1}{m}_{2},{u}_{1}{u}_{2}\right)$ |

Division | ${\tilde{M}}_{1}\oslash {\tilde{M}}_{2}=\left({l}_{1}/{u}_{2},{m}_{1}/{m}_{2},{u}_{1}/{l}_{2}\right)$ |

Inverse | ${{\tilde{M}}_{1}}^{-1}\approx \left(1/{u}_{1},1/{m}_{1},1/{l}_{1}\right)for\text{}l,\text{}m,\text{}u0$ |

Scalar multiplication | $k\otimes {\tilde{M}}_{1}=\left(k{l}_{1},k{m}_{1},k{u}_{1}\right),k>0,\text{}k\u220aR$ $\mathrm{k}\otimes {\tilde{M}}_{1}=\left(k{u}_{1},k{m}_{1},k{l}_{1}\right),k<0,\text{}k\u220aR$ |

Definition of Fuzzy Numbers | Saaty’s Scale | Fuzzy Numbers | Reciprocal Fuzzy Numbers |
---|---|---|---|

Equally important | 1 | $\tilde{1}=\left(1,1,1\right)$ | (1,1,1) |

Judgement values between equally and moderately | 2 | $\tilde{2}=\left(1,2,3\right)$ | (1/3,1/2,1) |

Moderately more important | 3 | $\tilde{3}=\left(2,3,4\right)$ | (1/4,1/3,1/2) |

Judgement values between moderately and strongly | 4 | $\tilde{4}=\left(3,4,5\right)$ | (1/5,1/4,1/3) |

Strongly more important | 5 | $\tilde{5}=\left(4,5,6\right)$ | (1/6,1/5,1/4) |

Judgement values between strongly and very strongly | 6 | $\tilde{6}=\left(5,6,7\right)$ | (1/7,1/6,1/5) |

Very strongly more important | 7 | $\tilde{7}=\left(6,7,8\right)$ | (1/8,1/7,1/6) |

Judgement values between very strongly and extremely | 8 | $\tilde{8}=\left(7,8,9\right)$ | (1/9,1/8,1/7) |

Extremely more important | 9 | $\tilde{9}=\left(8,9,9\right)$ | (1/9,1/9,1/8) |

FCQ | LRH | AQU | ORR | AJI | EGS | QQL | EUC | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | ${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | ${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | ${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | ${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | ${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | ${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | ${\mathit{l}}_{\mathit{i}\mathit{j}}$ | ${\mathit{m}}_{\mathit{i}\mathit{j}}$ | ${\mathit{u}}_{\mathit{i}\mathit{j}}$ | |

FCQ | 1.000 | 1.000 | 1.000 | 1.000 | 2.000 | 3.000 | 1.000 | 2.000 | 3.000 | 1.000 | 2.000 | 3.000 | 2.000 | 3.000 | 4.000 | 3.000 | 4.000 | 5.000 | 3.000 | 4.000 | 5.000 | 4.000 | 5.000 | 6.000 |

LRH | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 2.000 | 3.000 | 2.000 | 3.000 | 4.000 | 2.000 | 3.000 | 4.000 | 3.000 | 4.000 | 5.000 |

AQU | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 2.000 | 3.000 | 2.000 | 3.000 | 4.000 | 2.000 | 3.000 | 4.000 | 3.000 | 4.000 | 5.000 |

ORR | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 2.000 | 3.000 | 2.000 | 3.000 | 4.000 | 2.000 | 3.000 | 4.000 | 3.000 | 4.000 | 5.000 |

AJI | 0.250 | 0.333 | 0.500 | 0.333 | 0.500 | 1.000 | 0.333 | 0.500 | 1.000 | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 2.000 | 3.000 | 1.000 | 2.000 | 3.000 | 2.000 | 3.000 | 4.000 |

EGS | 0.200 | 0.250 | 0.333 | 0.250 | 0.333 | 0.500 | 0.250 | 0.333 | 0.500 | 0.250 | 0.333 | 0.500 | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 2.000 | 3.000 |

QQL | 0.200 | 0.250 | 0.333 | 0.250 | 0.333 | 0.500 | 0.250 | 0.333 | 0.500 | 0.250 | 0.333 | 0.500 | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 2.000 | 3.000 |

EUC | 0.200 | 0.250 | 0.333 | 0.250 | 0.333 | 0.500 | 0.250 | 0.333 | 0.500 | 0.250 | 0.333 | 0.500 | 0.333 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |

Descriptor | Utility Vector | $\mathit{C}\mathit{R}$ |
---|---|---|

FCQ | (1.000, 0.557, 0.244, 0.083, 0.000) | 0.0155 |

LRH | (1.000, 0.406, 0.122, 0.000) | 0.0438 |

AQU | (1.000, 0.476, 0.199, 0.063, 0.000) | 0.0535 |

ORR | (1.000, 0.570, 0.223, 0.070, 0.000) | 0.0416 |

AJI | (1.000, 0.386, 0.108, 0.000) | 0.0304 |

EGS | (1.000, 0.473, 0.212, 0.000) | 0.0337 |

QQL | (1.000, 0.232, 0.000) | 0.0355 |

EUC | (1.000, 0.563, 0.275, 0.099, 0.000) | 0.0153 |

© 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Carnero, M.C.
Fuzzy Multicriteria Models for Decision Making in Gamification. *Mathematics* **2020**, *8*, 682.
https://doi.org/10.3390/math8050682

**AMA Style**

Carnero MC.
Fuzzy Multicriteria Models for Decision Making in Gamification. *Mathematics*. 2020; 8(5):682.
https://doi.org/10.3390/math8050682

**Chicago/Turabian Style**

Carnero, María Carmen.
2020. "Fuzzy Multicriteria Models for Decision Making in Gamification" *Mathematics* 8, no. 5: 682.
https://doi.org/10.3390/math8050682