# An Academic Performance Indicator Using Flexible Multi-Criteria Methods

^{1}

^{2}

^{3}

^{4}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Construction of the Academic Performance Indicator

- Step 1.
- Development of the conceptual framework or theoretical model, which will allow us to identify the phenomenon to be measured, the groups involved and identify the variables.
- Step 2.
- Selection of the decision criteria. It is essential to identify the variables or simple indicators. Determining how to obtain the data, studying whether there are missing data and how to resolve it and analyzing the data structure is fundamental to constructing a quality.
- Step 3.
- Comparison of alternatives. When a composite indicator is constructed, in most cases, the units in which the variables are measured and the nature of these are very diverse [37]. To compare and aggregate variables and simple indicators, it is necessary to express them on a similar scale, i.e., normalize [38,39,40]. However, the normalization method must be carefully assessed because the results may vary depending on the method used. In this work, the method proposed by Liern et al. [9] will be used to normalize it, which will be described in more detail in Section 2.2.
- Step 4.
- Assignment of weights and aggregation. One of the problems widely discussed in the literature is the weighting of variables according to their relative importance [26,41,42,43]. Sometimes, the assignment is done based on expert judgment, but at other times, as we will develop in this paper, we will not assign the weights a priori but apply the uwTOPSIS method, in which it is not necessary to have the weights showing a priori the relative importance of the criteria [27].
- Step 5.
- Fifth and final step: Comparison of the alternatives and sensitivity analysis of the composite indicator. In this case, the total and partial performance of each student will be evaluated using the indicator proposed in this work. When an indicator is proposed, it is necessary to test its usefulness and study whether it meets the properties of the indicators [44,45], and finally, the robustness of the indicator must be determined.

#### 2.1. Academic Performance Indicator

**Definition**

**1**

**.**Given a set $\mathcal{D}=\{(x,{R}^{L},{R}^{U})\in {[0,1]}^{3}:{R}^{L}\le {R}^{U}\}$, the academic performance indicator is defined as ${P}_{A}:\mathcal{D}\to [0,1]$ such that

**Proposition**

**1.**

**Proof.**

#### 2.2. Determination of Performance Range Values

- Step 1.
- Decision matrix is determined $\left[{x}_{ij}\right]$ so that $i\in \{1,\cdots ,n\}$ and $j\in \{1,\cdots ,m\}$, where n and m are the number of alternatives and criteria, respectively.
- Step 2.
- Decision matrix is normalized $\left[{r}_{ij}\right]$, where each ${r}_{ij}\in [0,1]$ for each $i\in \{1,\cdots ,n\}$ and $j\in \{1,\cdots ,m\}$.
- Step 3.
- Positive ideal and negative ideal are determined as ${A}^{+}=({v}_{1}^{+},\cdots ,{v}_{m}^{+})$ and ${A}^{-}=({v}_{1}^{-},\cdots ,{v}_{m}^{-})$, where$${v}_{j}^{+}=\left\{\begin{array}{ccc}{\displaystyle \underset{1\le i\le n}{max}{r}_{ij}}\hfill & \mathrm{si}& j\in {J}_{max}\hfill \\ {\displaystyle \underset{1\le i\le n}{min}{r}_{ij}}\hfill & \mathrm{si}& j\in {J}_{min}\hfill \end{array}\phantom{\rule{1.em}{0ex}}1\le j\le m\right.$$$${v}_{j}^{-}=\left\{\begin{array}{ccc}{\displaystyle \underset{1\le i\le n}{min}{r}_{ij}}\hfill & \mathrm{si}& j\in {J}_{max}\hfill \\ {\displaystyle \underset{1\le i\le n}{max}{r}_{ij}}\hfill & \mathrm{si}& j\in {J}_{min}\hfill \end{array}\phantom{\rule{1.em}{0ex}}1\le j\le m\right.$$
- Step 4.
- Given $\mathsf{\Omega}=\{w=({w}_{1},\cdots ,{w}_{m})\in {[0,1]}^{m}:{\sum}_{j=1}^{m}{w}_{j}=1\}$ and d a distance defined in ${[0,1]}^{m}\times {[0,1]}^{m}$, we consider the separating functions ${D}_{i}^{+},{D}_{i}^{-}:\mathsf{\Omega}\to [0,1]$ for each of $i\in \{1,\cdots ,n\}$ as$${D}_{i}^{+}\left(w\right)=d\left(({w}_{1}{r}_{i1},\cdots ,{w}_{m}{r}_{im}),({w}_{1}{v}_{1}^{+},\cdots ,{w}_{m}{v}_{m}^{+})\right).$$$${D}_{i}^{-}\left(w\right)=d\left(({w}_{1}{r}_{i1},\cdots ,{w}_{m}{r}_{im}),({w}_{1}{v}_{1}^{-},\cdots ,{w}_{m}{v}_{m}^{-})\right).$$
- Step 5.
- The relative proximity function to the ideal solutions is defined as ${R}_{i}:\mathsf{\Omega}\to [0,1]$ for each $i\in \{1,\cdots ,n\}$$${R}_{i}\left(w\right)=\frac{{D}_{i}^{-}\left(w\right)}{{D}_{i}^{+}\left(w\right)+{D}_{i}^{-}\left(w\right)},1\le i\le n.$$
- Step 6.
- For each $i\in \{1,\cdots ,n\}$, the values ${R}_{i}^{L}$ and ${R}_{i}^{U}$ are calculated when solving mathematical programming problems over ${R}_{i}$ considering the set of weights as the problem variables$${R}_{i}^{L}=\underset{1\le i\le n}{min}\left\{{R}_{i}\left(w\right):\sum _{j=1}^{m}{w}_{j}=1,{l}_{j}\le {w}_{j}\le {u}_{j}\right\},1\le i\le n.$$$${R}_{i}^{U}=\underset{1\le i\le n}{max}\left\{{R}_{i}\left(w\right):\sum _{j=1}^{m}{w}_{j}=1,{l}_{j}\le {w}_{j}\le {u}_{j}\right\},1\le i\le n.$$
_{j}is the lower bound and u_{j}the upper bound within w_{j}, $\forall j\in \{1,\cdots ,m\}$.

_{1},k

_{2}) coefficients are the left and right exponents that determine whether each spread is convex or concave. For a better understanding of their functionality, Figure 1 depicts how η and ξ transform the data.

**Remark**

**1.**

#### 2.3. Properties of Academic Performance Indicator

_{A}will be checked.

- Existence and determination. For every tern $(x,{R}^{L},{R}^{U})\in \mathcal{D}$, the function P
_{A}is well-defined, and the value of ${P}_{A}(x,{R}^{L},{R}^{U})$ exists due to Proposition 1. - Uniqueness. By the indicator’s construction, each element of its domain returns a unique value.
- Monotonicity. Given $(x,{R}^{L},{R}^{U})\in \mathcal{D}$ for the cases where $x\le {R}^{L}$ and $x\ge {R}^{U}$, the indicator is monotonically a constant. For the case that ${R}^{L}x{R}^{U}$, the gradient of the indicator can be defined as$$\nabla {P}_{A}(x,{R}^{L},{R}^{U})=\left(\frac{1}{{R}^{U}-{R}^{L}},-\frac{x-{R}^{L}}{{({R}^{U}-{R}^{L})}^{2}},\frac{x-{R}^{U}}{{({R}^{U}-{R}^{L})}^{2}}\right).$$Therefore, the function is monotonically increasing with respect to x and monotonically decreasing with respect to R
^{L}and R^{U}.Figure 2 shows a particular case of the behavior of the excellence indicator. Three graphs are represented where one of the variables varies and the rest remain fixed. - Continuity. The indicator is continuous except for the case where ${R}^{L}={R}^{U}$. This is because, for each $(x,{R}^{L},{R}^{U})\in \mathcal{D}$, where ${R}^{L}{R}^{U}$ is satisfied$$\underset{x\to {({R}^{L})}^{+}}{lim}{P}_{A}(x,{R}^{L},{R}^{U})=0\phantom{\rule{4pt}{0ex}}\mathrm{and}\underset{x\to {({R}^{U})}^{-}}{lim}{P}_{A}(x,{R}^{L},{R}^{U})=1.$$
- Decomposability. Given ${\mathcal{D}}_{1},{\mathcal{D}}_{2}\subset \mathcal{D}$ such that ${\mathcal{D}}_{1}\cup {\mathcal{D}}_{2}=\mathcal{D}$ and ${\mathcal{D}}_{1}\cap {\mathcal{D}}_{2}=\varnothing $, it is easy to see ${P}_{A}={P}_{A}{{|}_{{\mathcal{D}}_{1}}+{P}_{A}|}_{{\mathcal{D}}_{2}}$. For the case that ${R}^{L}={R}^{U}$, the proof is trivial. Now if in each ${\mathcal{D}}_{1}$ and ${\mathcal{D}}_{2}$ is satisfied ${R}^{L}x{R}^{U}$, we have the same decomposition since the function has no discontinuities in $\mathcal{D}$ as is well demonstrated by the property 4.
- Normality. By definition, the indicator P
_{A}takes values between 0 and 1, where, in addition, the same scale is guaranteed for each of the variables in $\mathcal{D}$. - Scale invariance. Given an element $(x,{R}^{L},{R}^{U})\in \mathcal{D}$ and a scalar $\lambda \in \mathbb{R}$ such that $(\lambda x,\lambda {R}^{L},\lambda {R}^{U})\in \mathcal{D}$, so ${P}_{A}(\lambda x,\lambda {R}^{L},\lambda {R}^{U})=\frac{\lambda x-\lambda {R}^{L}}{\lambda {R}^{U}-\lambda {R}^{L}}=\frac{x-{R}^{L}}{{R}^{U}-{R}^{L}}={P}_{A}(x,{R}^{L},{R}^{U})$.
- Translation invariance. Given an element $(x,{R}^{L},{R}^{U})\in \mathcal{D}$ and a scalar $\lambda \in \mathbb{R}$ such that $(x+\lambda ,{R}^{L}+\lambda ,{R}^{U}+\lambda )\in \mathcal{D}$, so ${P}_{A}(x+\lambda ,{R}^{L}+\lambda ,{R}^{U}+\lambda )=\frac{(x+\lambda )-({R}^{L}+\lambda )}{({R}^{U}+\lambda )-({R}^{L}+\lambda )}=\frac{x-{R}^{L}}{{R}^{U}-{R}^{L}}={P}_{A}(x,{R}^{L},{R}^{U})$.

## 3. Application of the Academic Performance Indicator to UIS Students

_{A}score equal to or higher than 0.6, he/she presents an adequate academic level and meets the minimum required by the university.

#### 3.1. Dataset

- (a)
- Academic dimension. This dimension is composed of three items: a diagnostic test of UIS Math (DTM); EFAI-4 numerical ability (NUA); and 11-Math Knowledge Test (PSO).
- (b)
- Cognitive dimension. Five items are assessed: verbal reasoning (VR), numerical reasoning (NR), abstract reasoning (ABR), memory (MEM) and spatial attitude (SA).
- (c)
- Economic dimension. The variables that analyze this dimension are: Income from economic dependence, ED = wage/SMMLV (where SMMLV = current legal minimum monthly wages), the number of siblings (NS), the position between siblings (PS) and the payment of rent during the course (PRC).
- (d)
- Health dimension. The eight variables that compose this dimension are: Anxiety (ANX), Depression (DEP), Emotional Adjustment (EMA), Alcohol Dependence (ALD), Psychoactive Substance Abuse (PSA), Chronic Illnesses (CI), Disability (DI) and “Question 23” (P23), which refers to the tendency toward suicide.
- (e)
- Social dimension. This is determined by family dysfunction (FAD) through the “Family APGAR” [48].

^{1}value, and normalized with the minimum-maximum normalization. With the data obtained before and after the end of the first semester, the students will be classified depending on whether or not the academic dimension score is contained within the interval. The student can be considered to have had an expected performance if the final score is contained in the interval $[{R}^{L},{R}^{U}]$, excellent if he/she has obtained a final score higher than ${R}^{U}$ and, on the contrary, an insufficient score if he/she has had a final score lower than ${R}^{L}$. From now on, the academic performance of students are distinguished as shown in the Table 2.

**Remark**

**2.**

#### 3.2. Classification of Students at UIS Colombia According to Gender and Economic Status

_{A}taking into account whether they have received actions or not. If Table 8 is observed, it can be seen that in all semesters, except in 2019_1, the mean value of the P

_{A}indicator is higher for students who have not received complementary actions, i.e., those who had a high score in the academic dimension when they accessed the university. The differences that exist between the two groups of students in semester 2016_2 should be highlighted, where the average academic performance for those who did not participate in any action was 0.9080 and those who participated in some action was 0.6924. In 2018_2, the mean academic performance for those who did not participate in any action was 0.6229 and those who participated in any action was 0.2654. This may be due to the fact that in some semesters, the political situation in the country led to class stoppages during part of the semester and the suspension of complementary actions. In this way, students with greater needs ceased to have some support actions.

_{A}higher than 0.6, while the percentage of males is 51.88%).

_{A}above 0.6. Given that more than half of the students who participated in the complementary actions organized by the SEA have had a score higher than the threshold set as success of the strategy, it is justified to recommend that the UIS continues with the actions carried out and implement some more so that the success rate is close to that of the students who do not have support needs.

## 4. Discussion

- An interval associated with the worst and best possible results of each student is obtained, which allows knowing their aspirations.
- Results dependent on the data handled have been avoided (rank reversal [40]), using $(1,1,\cdots ,1)$ and $(0,0,\cdots ,0)$ as ideal and anti-ideal, respectively.
- A standardization has been applied that allows taking into account the criteria of the institution, which makes the future applicability of the results easier.

- In this study, stability/regularity in the data has not been taken into account. As it was said in the different sections, there are periods with clear differences with respect to the others, and this could require an in-depth study of the homogeneity between periods. However, once the results are known, it is clear that these differences should be taken into account in a subsequent study.
- The intervals assigned to the weights, although they make the results more flexible and objective, determine (in some way) the groupings. This could be avoided by allowing weights that could span the entire interval $[0,1]$. However, this would not make complete sense in a multicriteria environment. According to [52], a weight greater than 50% in any of the criteria in a decision context with more than two criteria is not sustainable with the multicriteria character.

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Appendix A

`import pandas as pd`

`import numpy as np`

`from uwTOPSIS.uwTOPSIS import *`

`# Definition of the eta normalization`

`def eta(x, A, a, b, B, k1, k2):`

`def f(x, v1, v2, k):`

`return (1-np.exp(k*(x-v2)/(v1-v2)))/(1-np.exp(k))`

`if A <= x and x < a:`

`z = f(x, a, A, k1)`

`elif a < x and x < b:`

`z = 1`

`elif b < x and x <= B:`

`z = f(-x, -b, -B, k2)`

`else:`

`z = 0`

`return z`

`# Definition of the xi normalization`

`def xi(x, A, a, b, B):`

`def f(x, v1, v2):`

`return (x-v2)/(v1-v2)`

`if A <= x and x < a:`

`z = f(x, a, A)`

`elif a < x and x < b:`

`z = 1`

`elif b < x and x <= B:`

`z = f(-x, -b, -B)`

`else:`

`z = 0`

`return z`

`# Data preparation`

`path_to_data = ’path/to/data.xlsx’`

`data = pd.read_excel(path_to_data)`

`directions = np.repeat("max", 5)`

`L = np.repeat(0.1, 5)`

`U = np.repeat(0.5, 5)`

`norm = "none"`

`p = 2`

`# Data normalization`

`data[’Academic’] = data[’Academic’].apply(lambda x: eta(x,1,6,7,7,1,0))`

`data[’Cognitive’] = data[’Cognitive’].apply(lambda x: eta(x,1,6,7,7,-1,0))`

`data[’Economic’] = data[’Economic’].apply(lambda x: xi(x,0,0.8,1,1))`

`data[’Health’] = data[’Health’].apply(lambda x: xi(x,0,0.65,0.65,0.65))`

`data[’Social’] = data[’Social’].apply(lambda x: xi(x,0.1,0.7,1,1))`

`# Application of unweighted TOPSIS method`

`x = uwTOPSIS(data, directions, L, U, norm, p, forceideal=True)`

## References

- Zhou, P.; Ang, B.W.; Zhou, D.Q. Weighting and Aggregation in Composite Indicator Construction: A Multiplicative Optimization Approach. Soc. Indic. Res.
**2010**, 96, 169–181. [Google Scholar] [CrossRef] - Zhou, P.; Ang, B.W.; Zhou, D.Q. Comparing MCDA aggregation methods in constructing composite indicators using the Shannon-Spearman measure. Soc. Indic. Res.
**2009**, 94, 83–96. [Google Scholar] [CrossRef] - Roy, B. Multicriteria Methodology for Decision Aiding; Nonconvex Optimization and Its Applications; Springer: Berlin/Heidelberg, Germany, 1996. [Google Scholar]
- Zopounidis, C.; Doumpos, M. Multiple Criteria Decision Making: Applications in Management and Engineering, 1st ed.; Springer International Publishing: Berlin/Heidelberg, Germany, 2017. [Google Scholar]
- Hwang, C.; Yoon, K. Multiple Attribute Decision Making: Methods and Applications: A State-of-the-Art Survey; Lecture Notes in Economics and Mathematical Systems; Springer: Berlin/Heidelberg, Germany, 1981. [Google Scholar]
- Rampasso, I.S.; Siqueira, R.G.; Anholon, R.; Silva, D.; Quelhas, O.L.; Filho, W.L.; Brandli, L.L. Some of the challenges in implementing Education for Sustainable Development: Perspectives from Brazilian engineering students. Int. J. Sustain. Dev. World Ecol.
**2019**, 26, 367–376. [Google Scholar] [CrossRef] - Parada, S.E.; Blasco-Blasco, O.; Liern, V. Adequacy Indicators Based on Pre-established Goals: An Implementation in a Colombian University. Soc. Indic. Res.
**2019**, 143, 1–24. [Google Scholar] [CrossRef] - Parada, S.E.; Fiallo, J.E.; Blasco-Blasco, O. Construcción de indicadores sintéticos basados en Juicio experto: Aplicación a una medida integral de la excelencia académica. Rect@
**2016**, 16, 51–67. [Google Scholar] - Liern, V.; Parada-Rico, S.E.; Blasco-Blasco, O. Construction of quality indicators based on pre-established goals: Application to a Colombian Public University. Mathematics
**2020**, 8, 1075. [Google Scholar] [CrossRef] - Parada, S.E.; Blasco-Blasco, O.; Liern, V. Construcción de indicadores basada en medidas de similitud con ideales. Una aplicación al cálculo de índices de adecuación y de excelencia. Rect@
**2017**, 18, 119–135. [Google Scholar] [CrossRef] - Blasco-Blasco, O.; Parada Rico, S.; Liern-García, M.; López-García, A. Characterization of University Students Through Indicators of Adequacy and Excellence. Analysis from Gender and Socioeconimic Status Perspective. In Proceedings of the ICERI2020, IATED, 13th Annual International Conference of Education, Research and Innovation, Seville, Spain, 9–11 November 2020; pp. 8030–8037. [Google Scholar] [CrossRef]
- Behzadian, M.; Otaghsara, S.K.; Yazdani, M.; Ignatius, J. A state-of the-art survey of TOPSIS applications. Expert Syst. Appl.
**2012**, 39, 13051–13069. [Google Scholar] [CrossRef] - Wątróbski, J.; Jankowski, J.; Ziemba, P.; Karczmarczyk, A.; Zioło, M. Generalised framework for multi-criteria method selection. Omega
**2019**, 86, 107–124. [Google Scholar] [CrossRef] - Jacquet-Lagreze, E.; Siskos, J. Assessing a set of additive utility functions for multicriteria decision-making, the UTA method. Eur. J. Oper. Res.
**1982**, 10, 151–164. [Google Scholar] [CrossRef] - Ebrahimnejad, S.; Mousavi, S.; Tavakkoli-Moghaddam, R.; Hashemi, H.; Vahdani, B. A novel two-phase group decision making approach for construction project selection in a fuzzy environment. Appl. Math. Model.
**2012**, 36, 4197–4217. [Google Scholar] [CrossRef] - Gil-Aluja, J. Elements for a Theory of Decision in Uncertainty; Springer: Boston, MA, USA, 1999. [Google Scholar] [CrossRef]
- Barron, F.H.; Barrett, B.E. Decision quality using ranked attribute weights. Manag. Sci.
**1996**, 42, 1515–1523. [Google Scholar] [CrossRef] - Hobbs, B.F. A Comparison of Weighting Methods in Power Plant Siting. Decis. Sci.
**1980**, 11, 725–737. [Google Scholar] [CrossRef] - Alemi-Ardakani, M.; Milani, A.S.; Yannacopoulos, S.; Shokouhi, G. On the effect of subjective, objective and combinative weighting in multiple criteria decision making: A case study on impact optimization of composites. Expert Syst. Appl.
**2016**, 46, 426–438. [Google Scholar] [CrossRef] - Németh, B.; Molnár, A.; Bozóki, S.; Wijaya, K.; Inotai, A.; Campbell, J.D.; Kaló, Z. Comparison of weighting methods used in multicriteria decision analysis frameworks in healthcare with focus on low- and middle-income countries. J. Comp. Eff. Res.
**2019**, 8, 195–204. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Domazetović, F.; Šiljeg, A.; Lončar, N.; Marić, I. GIS automated multicriteria analysis (GAMA) method for susceptibility modelling. MethodsX
**2019**, 6, 2553–2561. [Google Scholar] [CrossRef] - Triantaphyllou, E. Multi-Criteria Decision Making Methods. In Multi-Criteria Decision Making Methods: A Comparative Study; Springer: Boston, MA, USA, 2000; pp. 5–21. [Google Scholar] [CrossRef]
- Deng, H.; Yeh, C.H.; Willis, R.J. Inter-company comparison using modified TOPSIS with objective weights. Comput. Oper. Res.
**2000**, 27, 963–973. [Google Scholar] [CrossRef] - Fischer, G.W. Range Sensitivity of Attribute Weights in Multiattribute Value Models. Organ. Behav. Hum. Decis. Process.
**1995**, 62, 252–266. [Google Scholar] [CrossRef] - Mareschal, B. Weight stability intervals in multicriteria decision aid. Eur. J. Oper. Res.
**1988**, 33, 54–64. [Google Scholar] [CrossRef] - Ouenniche, J.; Pérez-Gladish, B.; Bouslah, K. An out-of-sample framework for TOPSIS-based classifiers with application in bankruptcy prediction. Technol. Forecast. Soc. Chang.
**2018**, 131, 111–116. [Google Scholar] [CrossRef] [Green Version] - Liern, V.; Pérez-Gladish, B. Multiple criteria ranking method based on functional proximity index: Un-Weighted TOPSIS. Ann. Oper. Res.
**2020**. [Google Scholar] [CrossRef] - Benítez, R.; Liern, V. Unweighted TOPSIS: A new multi-criteria tool for sustainability analysis. Int. J. Sustain. Dev. World Ecol.
**2021**, 28, 36–48. [Google Scholar] [CrossRef] - Vicens-Colom, J.; Holles, J.; Liern, V. Measuring Sustainability with Unweighted TOPSIS: An Application to Sustainable Tourism in Spain. Sustainability
**2021**, 13, 5283. [Google Scholar] [CrossRef] - Wu, H.Y.; Tzeng, G.H.; Chen, Y.H. A fuzzy MCDM approach for evaluating banking performance based on Balanced Scorecard. Expert Syst. Appl.
**2009**, 36, 10135–10147. [Google Scholar] [CrossRef] - Jahanshahloo, G.R.; Lotfi, F.H.; Izadikhah, M. Extension of the TOPSIS method for decision-making problems with fuzzy data. Appl. Math. Comput.
**2006**, 181, 1544–1551. [Google Scholar] [CrossRef] - Bellman, R.E.; Zadeh, L.A. Decision-Making in a Fuzzy Environment. Manag. Sci.
**1970**, 17, B141–B164. [Google Scholar] [CrossRef] - Lai, Y.J.; Hwang, C.L. Fuzzy Mathematical Programming; Springer: Berlin/Heidelberg, Germany, 1992. [Google Scholar] [CrossRef]
- Moffatt, I. On measuring sustainable development indicators. Int. J. Sustain. Dev. World Ecol.
**1994**, 1, 97–109. [Google Scholar] [CrossRef] - Fazlagić, J.; Skikiewicz, R. Measuring sustainable development—The creative economy perspective. Int. J. Sustain. Dev. World Ecol.
**2019**, 26, 635–645. [Google Scholar] [CrossRef] - León, T.; Liern, V.; Pérez-Gladish, B. A multicriteria assessment model for countries’ degree of preparedness for successful impact investing. Manag. Decis.
**2019**, 58, 2455–2471. [Google Scholar] [CrossRef] - Wang, Y.J. A fuzzy multi-criteria decision-making model by associating technique for order preference by similarity to ideal solution with relative preference relation. New Sensing and Processing Technologies for Hand-based Biometrics Authentication. Inf. Sci.
**2014**, 268, 169–184. [Google Scholar] [CrossRef] - Freudenberg, M. Composite Indicators of Country Performance: A Critical Assessment; OECD Publishing: Paris, France, 2003. [Google Scholar] [CrossRef]
- Acuña-Soto, C.; Liern, V.; Gladish, B. Multiple criteria performance evaluation of YouTube mathematical educational videos by IS-TOPSIS. Oper. Res.
**2020**, 20, 1–23. [Google Scholar] [CrossRef] - Cables, E.; Lamata, M.; Verdegay, J. RIM-reference ideal method in multicriteria decision making. Inf. Sci.
**2016**, 337–338, 1–10. [Google Scholar] [CrossRef] - Saisana, M.; Saltelli, A.; Tarantola, S. Uncertainty and Sensitivity Analysis Techniques as Tools for the Quality Assessment of Composite Indicators. J. R. Stat. Soc. Ser. (Stat. Soc.)
**2005**, 2, 307–323. [Google Scholar] [CrossRef] - Munda, G.; Nardo, M. Constructing Consistent Composite Indicators: The Issue of Weights. 2005. Available online: https://core.ac.uk/download/pdf/38619689.pdf (accessed on 20 September 2021).
- Munda, G.; Nardo, M.; Saisana, M.; Srebotnjak, T. Measuring uncertainties in composite indicators of sustainability. Int. J. Environ. Technol. Manag.
**2009**, 11, 7–26. [Google Scholar] [CrossRef] [Green Version] - OECD. Handbook on Constructing Composite Indicators: Methodology and User Guide; OECD Publishing: Paris, France, 2008. [Google Scholar]
- Murias, P.; de Miguel, J.C.; Rodríguez, D. A Composite Indicator for University Quality Assesment: The Case of Spanish Higher Education System. Soc. Indic. Res.
**2008**, 89, 129–146. [Google Scholar] [CrossRef] - Ivanova, I.; Arcelus, F.J.; Srinivasan, G. An assessment of the measurement properties of the human development index. Soc. Indic. Res.
**1999**, 46, 157–179. [Google Scholar] [CrossRef] - Zheng, B. An axiomatic characterization of the Watts poverty index. Econ. Lett.
**1993**, 42, 81–86. [Google Scholar] [CrossRef] - Smilkstein, G. The family APGAR: A proposal for a family function test and its use by physicians. J. Fam. Pract.
**1978**, 66, 1231–1239. [Google Scholar] - R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2021. [Google Scholar]
- van Rossum, G.; Drake, F. Python Language Reference, Version 3.8; Python Software Foundation, Centrum voor Wiskunde and Informatica: Amsterdam, The Netherlands, 2021. [Google Scholar]
- López-García, A. uwTOPSIS. 2021. Available online: https://github.com/Aaron-AALG/uwTOPSIS (accessed on 20 September 2021).
- Ishizaka, A.; Labib, A. Review of the main developments in the analytic hierarchy process. Expert Syst. Appl.
**2011**, 38, 14336–14345. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Particular case of the normalizations η and μ in which the values (A,a,b,B) = (1,6,7,9) have been set. The first graph is the normalization η with the values k

_{1}= k

_{2}= 1, and the second is the normalization η with k

_{1}= k

_{2}= −1. Finally, the third is the normalization ξ.

**Figure 2.**Special case of the behavior of the academic performance indicator P

_{A}by increasing the value of each of its variable by 0.15.

Dimension | Original | Transf. | Ideal | Normalization |
---|---|---|---|---|

Academic | {VL, L, LM, M, MH, H, VH} | [1, 7] | [6, 7] | ${\eta}_{1,6,7,7;1,0}\left(x\right)$ |

Cognitive | {VL, L, LM, M, MH, H, VH} | [1, 7] | [6, 7] | ${\eta}_{1,6,7,7;-1,0}\left(x\right)$ |

Economic | [0, 1] | [0.8, 1] | [0.8, 1] | ${\xi}_{0,0.8,1,1}\left(x\right)$ |

Health | [0, 0.65] | [0, 0.65] | 0.65 | ${\xi}_{0,0.65,0.65,0.65}\left(x\right)$ |

Social | {0.1, 0.5, 0.7, 1} | [0.1, 1] | [0.7, 1] | ${\xi}_{0.1,0.7,1,1}\left(x\right)$ |

Indicator P_{A}(X^{1}, R^{L}, R^{U}) | |
---|---|

Excellence | P_{A} = 1 |

Expected | 0 < P_{A} < 1 |

Insufficient | P_{A} = 0 |

**Table 3.**Average of the a priori, posteriori and academic performance indicator results by semester.

Course | R^{L} | R^{U} | X^{1} | P_{A} |
---|---|---|---|---|

2016_1 | 0.3718 | 0.8540 | 0.8346 | 0.8602 |

2016_2 | 0.4081 | 0.8533 | 0.7913 | 0.7734 |

2017_1 | 0.4225 | 0.8671 | 0.8172 | 0.7814 |

2017_2 | 0.5192 | 0.8879 | 0.7733 | 0.6498 |

2018_1 | 0.5753 | 0.8988 | 0.5868 | 0.2177 |

2018_2 | 0.5080 | 0.8835 | 0.5723 | 0.2795 |

2019_1 | 0.5334 | 0.8868 | 0.8299 | 0.7560 |

Group | P_{A} | Var | N |
---|---|---|---|

M | 0.6211 | 0.1555 | 2040 |

F | 0.6632 | 0.1380 | 935 |

Course | Group | P_{A} | Var |
---|---|---|---|

2016_1 | M | 0.8571 | 0.0681 |

F | 0.8667 | 0.0667 | |

2016_2 | M | 0.7623 | 0.1175 |

F | 0.7949 | 0.0919 | |

2017_1 | M | 0.7708 | 0.1012 |

F | 0.8046 | 0.0854 | |

2017_2 | M | 0.6149 | 0.1454 |

F | 0.7195 | 0.0974 | |

2018_1 | M | 0.2125 | 0.0839 |

F | 0.2297 | 0.0927 | |

2018_2 | M | 0.2661 | 0.0856 |

F | 0.3075 | 0.0796 | |

2019_1 | M | 0.7452 | 0.1095 |

F | 0.7848 | 0.0786 |

Gender | Number of Students | Percentage of Students | ||
---|---|---|---|---|

ER_{l} | ER_{h} | ER_{l} | ER_{h} | |

M | 303 | 1737 | 14.85 | 85.15 |

F | 155 | 780 | 16.58 | 83.42 |

Course | Group | P_{A} | Var |
---|---|---|---|

2016_1 | ER_{l} | 0.8411 | 0.1000 |

ER_{h} | 0.8629 | 0.0631 | |

2016_2 | ER_{l} | 0.8276 | 0.0855 |

ER_{h} | 0.7663 | 0.1116 | |

2017_1 | ER_{l} | 0.8130 | 0.0748 |

ER_{h} | 0.7792 | 0.0979 | |

2017_2 | ER_{l} | 0.7525 | 0.0767 |

ER_{h} | 0.6270 | 0.1412 | |

2018_1 | ER_{l} | 0.4747 | 0.0739 |

ER_{h} | 0.1568 | 0.0702 | |

2018_2 | ER_{l} | 0.4968 | 0.0399 |

ER_{h} | 0.2200 | 0.0795 | |

2019_1 | ER_{l} | 0.9110 | 0.0360 |

ER_{h} | 0.7174 | 0.1101 |

Course | No Participation | Participation | ||
---|---|---|---|---|

P_{A} | Var | P_{A} | Var | |

2016_1 | 0.8977 | 0.0488 | 0.7982 | 0.0927 |

2016_2 | 0.9080 | 0.0386 | 0.6924 | 0.1336 |

2017_1 | 0.8004 | 0.0950 | 0.7641 | 0.0972 |

2017_2 | 0.6509 | 0.1352 | 0.6492 | 0.1297 |

2018_1 | 0.2030 | 0.1262 | 0.2182 | 0.0855 |

2018_2 | 0.6229 | 0.1744 | 0.2654 | 0.0756 |

2019_1 | 0.7020 | 0.1494 | 0.7577 | 0.0999 |

**Table 9.**Percentage of students according to whether or not they have participated in the support program.

Percentage of Students | |||||
---|---|---|---|---|---|

Course | Excellence | Fail | Expected | α ≥ 0.6 | |

No Participation | 2016_1 | 62.62 | 2.49 | 34.89 | 90.97 |

2016_2 | 64.62 | 1.54 | 33.85 | 90.77 | |

2017_1 | 49.62 | 6.77 | 43.61 | 79.32 | |

2017_2 | 28.43 | 14.21 | 57.36 | 62.94 | |

2018_1 | 8.33 | 41.67 | 50.00 | 16.67 | |

2018_2 | 38.46 | 23.08 | 38.46 | 61.54 | |

2019_1 | 46.67 | 13.33 | 40.00 | 66.67 | |

Participation | 2016_1 | 40.21 | 7.22 | 52.58 | 81.44 |

2016_2 | 39.81 | 12.04 | 48.15 | 67.59 | |

2017_1 | 39.38 | 6.16 | 54.45 | 76.37 | |

2017_2 | 27.51 | 13.59 | 58.90 | 64.08 | |

2018_1 | 3.00 | 49.50 | 47.50 | 13.00 | |

2018_2 | 1.26 | 40.69 | 58.04 | 13.56 | |

2019_1 | 38.84 | 8.37 | 52.79 | 77.25 |

**Table 10.**Gender of the students that did not participate in the support program (NP) and the students that did participate, at least once, in the support program (P).

Group | Percentage of Students | ||||
---|---|---|---|---|---|

Excellence | Fail | Expected | α ≥ 0.6 | ||

NP | M | 47.07 | 8.24 | 44.69 | 76.55 |

F | 56.98 | 5.04 | 37.98 | 86.43 | |

P | M | 24.20 | 23.70 | 52.09 | 51.88 |

F | 26.14 | 17.58 | 56.28 | 55.54 |

**Table 11.**Economic risk of the students that have not participated in the support program (NP) and the students that have participated, at least once, in the support program (P).

Percentage of Students | |||||
---|---|---|---|---|---|

Excellence | Fail | Expected | α ≥ 0.6 | ||

NP | ER_{l} | 55.29 | 1.18 | 43.53 | 84.71 |

ER_{h} | 49.38 | 7.96 | 42.66 | 78.86 | |

P | ER_{l} | 32.98 | 3.49 | 63.54 | 61.93 |

ER_{h} | 23.06 | 25.69 | 51.26 | 51.14 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Blasco-Blasco, O.; Liern-García, M.; López-García, A.; Parada-Rico, S.E.
An Academic Performance Indicator Using Flexible Multi-Criteria Methods. *Mathematics* **2021**, *9*, 2396.
https://doi.org/10.3390/math9192396

**AMA Style**

Blasco-Blasco O, Liern-García M, López-García A, Parada-Rico SE.
An Academic Performance Indicator Using Flexible Multi-Criteria Methods. *Mathematics*. 2021; 9(19):2396.
https://doi.org/10.3390/math9192396

**Chicago/Turabian Style**

Blasco-Blasco, Olga, Marina Liern-García, Aarón López-García, and Sandra E. Parada-Rico.
2021. "An Academic Performance Indicator Using Flexible Multi-Criteria Methods" *Mathematics* 9, no. 19: 2396.
https://doi.org/10.3390/math9192396