# Using Data Envelopment Analysis and Multi-Criteria Decision-Making Methods to Evaluate Teacher Performance in Higher Education

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background

#### 2.1. Data Envelopment Analysis

_{ij}, …, y

_{sj}) by using m inputs (x

_{ij}, …, x

_{mj}). The basic output-oriented CCR DEA model is as follows:

_{k}shows relative efficiency of DMU

_{k}, obtained as the maximum possible achievement in comparison with the other DMUs under the evaluation.

_{k}is crucially related to the criteria selected. Jenkins and Anderson [49] claim that the more criteria there are, the less constrained weights are assigned to the criteria and the less discriminating the DEA scores are. The number of criteria may be substantial, and it may not be clear which one to choose. Moreover, the selection of different criteria can lead to different efficiency evaluation results. Of course, it is possible to consider all the criteria for evaluation. Still too many of them may lead to too many efficient units, and it may give rise to difficulties in distinguishing efficient units from inefficient ones. For this reason, the problem of selecting adequate criteria becomes an essential issue for improving the discrimination power of DEA.

#### 2.2. Conjoint Analysis

_{k}levels, model implies that the overall utility of the profile j (j = 1, …, J) for the respondent i (i = 1, …, I) can be expressed as follows [59]:

_{jkl}is a (0,1) variable that equals 1 if profile j contains lth level of attribute k, otherwise it equals 0. β

_{ikl}is respondent i’s utility (part-worth) assigned to the level l of the attribute k; ε

_{ij}is a stochastic error term.

#### 2.3. The Analytic Hierarchy Process (AHP)

## 3. Methodological Framework

_{k}, k = 1, …, K. The starting set of K criteria together with obtained importance values FI

_{k}will be used as inputs for the next step.

_{k}, k = 1, …, K. In the second case, when the number of criteria is not too high, importance values FI

_{k}are used for the weight restrictions to better discriminate between DMUs. This decision leads to the classification of selected criteria into the subsets of m inputs and n outputs depending on their nature, data collection, and DEA model selection. Finally, in this step, the chosen DEA model is solved to obtain the efficiency scores for each DMU in the observing set.

## 4. Empirical Study

#### 4.1. Subjective Assessment of Teacher’s Efficiency

**SCENARIO A: Reducing the number of criteria**

**SCENARIO B: Imposing weights restriction**

- Set f as an index of criterion with the lowest importance FI according to results of the conjoint analysis
- Impose boundaries for all criteria evaluated by the conjoint analysis. AR DEA constraints presented as Equation (2) in Section 2.1, are defined here as follows:$$\frac{F{I}_{r}}{\underset{r}{\mathrm{min}}(F{I}_{r})}\le \frac{{v}_{r}}{{v}_{f}}\le \frac{\underset{r}{\mathrm{max}}(F{I}_{r})}{\underset{r}{\mathrm{min}}(F{I}_{r})},r=1,\dots ,s.$$

_{r}value (Table 1) and varies from 1.12 for C8 to 2.89 for C1. That indicates that the lower bounds are asymmetric.

#### Verification of the DEA Results

#### 4.2. Objective Assessment of Teacher’s Efficiency

#### 4.2.1. Objective Assessment of Teaching Efficiency

- The total number of students registered for the listening subject by each of the selected teachers, over one academic year (I1)
- Annual salary of the teacher (I2)

- Total number of students who passed the exam with the chosen subject teacher in one academic year (O1)
- Average exam grade per subject/teacher (O2)

#### 4.2.2. Assessment of the Research Efficiency

#### 4.3. Aggregated Assessment of Overall Teacher’s Efficiency

_{1}= 0.43 (subjective teachers’ efficiency), w

_{2}= 0.21 (objective teaching efficiency), and w

_{3}= 0.36 (research efficiency). The values obtained for the weight coefficients show that the subjective assessment of teaching has double the significance of the objective assessment of teaching. The teachers also considered that the importance of research work is 36% in the aggregated assessment of efficiency.

_{k}≥ 0.95). Accordingly, the potential for efficiency growth can be found in the improvements of research and publishing quality, which will have a positive impact on the citation. The improvement can be achieved by upgrading subjective teaching efficiency, since it is considered as the most important one (w

_{1}= 0.43). Particular focus should be on a methodical approach to teaching, a more understandable presentation of teaching content, and a slower lecture tempo as criteria of particular relevance to students.

## 5. Conclusions

- It allows subjective and objective efficiency assessment, as well as determining an overall efficiency score by considering the weights associated with the various aspects of efficiency;
- It provides better criteria selection that is well-matched for the stakeholders and allows the selection of different criteria combinations suitable for different objectives and numbers of DMUs;
- It incorporates students’ preferences by selecting a meaningful and desirable set of criteria or imposing weight restrictions;
- It identifies key aspects of teaching that affect student satisfaction;
- It increases the discriminative power of the DEA and thus enables a more realistic ranking of teachers.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Letcher, D.W.; Neves, J.S. Determinant of undergraduate business student satisfaction. Res. High. Educ.
**2010**, 6, 1–26. [Google Scholar] - Venesaar, U.; Ling, H.; Voolaid, K. Evaluation of the Entrepreneurship Education Programme in University: A New Approach. Amfiteatru Econ.
**2011**, 8, 377–391. [Google Scholar] - Johnes, G. Scale and technical efficiency in the production of economic research. Appl. Econ. Lett.
**1995**, 2, 7–11. [Google Scholar] [CrossRef] - Despotis, D.K.; Koronakos, G.; Sotiros, D. A multi-objective programming approach to network DEA with an application to the assessment of the academic research activity. Procedia Comput. Sci.
**2015**, 55, 370–379. [Google Scholar] [CrossRef] [Green Version] - Dommeyer, C.J.; Baum, P.; Chapman, K.; Hanna, R.W. Attitudes of Business Faculty Towards two Methods of Collecting Teaching Evaluations: Paper vs. Online. Assess. Eval. High. Edu.
**2002**, 27, 455–462. [Google Scholar] [CrossRef] - Zabaleta, F. The use and misuse of student evaluation of teaching. Teach. High. Edu.
**2007**, 12, 55–76. [Google Scholar] [CrossRef] - Onwuegbuzie, J.; Daniel, G.; Collins, T. A meta-validation model for assessing the score-validity of student teacher evaluations. Qual. Quant.
**2009**, 43, 197–209. [Google Scholar] [CrossRef] - Mazumder, S.; Kabir, G.; Hasin, M.; Ali, S.M. Productivity Benchmarking Using Analytic Network Process (ANP) and Data Envelopment Analysis (DEA). Big Data Cogn. Comput.
**2018**, 2, 27. [Google Scholar] [CrossRef] [Green Version] - Mulye, R. An empirical comparison of three variants of the AHP and two variants of conjoint analysis. J. Behav. Decis. Mak.
**1998**, 11, 263–280. [Google Scholar] [CrossRef] - Helm, R.; Scholl, A.; Manthey, L.; Steiner, M. Measuring customer preferences in new product development: Comparing compositional and decompositional methods. Int. J. Product Developm.
**2004**, 1, 12–29. [Google Scholar] [CrossRef] - Scholl, A.; Manthey, L.; Helm, R.; Steiner, M. Solving multiattribute design problems with analytic hierarchy process and conjoint analysis: An empirical comparison. Eur. J. Oper. Res.
**2005**, 164, 760–777. [Google Scholar] [CrossRef] - Helm, R.; Steiner, M.; Scholl, A.; Manthey, L. A Comparative Empirical Study on common Methods for Measuring Preferences. Int. J. Manag. Decis. Mak.
**2008**, 9, 242–265. [Google Scholar] [CrossRef] [Green Version] - Ijzerman, M.J.; Van Til, J.A.; Bridges, J.F. A comparison of analytic hierarchy process and conjoint analysis methods in assessing treatment alternatives for stroke rehabilitation. Patient
**2012**, 5, 45–56. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Kallas, Z.; Lambarraa, F.; Gil, J.M. A stated preference analysis comparing the analytical hierarchy process versus choice experiments. Food Qual. Prefer.
**2011**, 22, 181–192. [Google Scholar] [CrossRef] - Danner, M.; Vennedey, V.; Hiligsmann, M.; Fauser, S.; Gross, C.; Stock, S. Comparing Analytic Hierarchy Process and Discrete-Choice Experiment to Elicit Patient Preferences for Treatment Characteristics in Age-Related Macular Degeneration. Value Health
**2017**, 20, 1166–1173. [Google Scholar] [CrossRef] [Green Version] - Popović, M.; Kuzmanović, M.; Savić, G. A comparative empirical study of Analytic Hierarchy Process and Conjoint analysis: Literature review. Decis. Mak. Appl. Manag. Eng.
**2018**, 1, 153–163. [Google Scholar] [CrossRef] - Pakkar, M.S. A hierarchical aggregation approach for indicators based on data envelopment analysis and analytic hierarchy process. Systems
**2016**, 4, 6. [Google Scholar] [CrossRef] [Green Version] - Sinuany-Stern, Z.; Mehrez, A.; Hadad, Y. An AHP/DEA methodology for ranking decision making units. Int. Trans. Oper. Res.
**2000**, 7, 109–124. [Google Scholar] [CrossRef] - Martić, M.; Savić, G. An application of DEA for comparative analysis and ranking of regions in Serbia with regards to social-economic development. Eur. J. Oper. Res.
**2001**, 132, 343–356. [Google Scholar] [CrossRef] - Feng, Y.J.; Lu, H.; Bi, K. An AHP/DEA method for measurement of the efficiency of R&D management activities in university. Int. Trans. Oper. Res.
**2004**, 11, 181–191. [Google Scholar] [CrossRef] - Tseng, Y.F.; Lee, T.Z. Comparing appropriate decision support of human resource practices on organizational performance with DEA/AHP model. Expert Syst. Appl.
**2009**, 36, 6548–6558. [Google Scholar] [CrossRef] - Saen, R.F.; Memariani, A.; Lotfi, F.H. Determining relative efficiency of slightly non-homogeneous decision making units by data envelopment analysis: A case study in IROST. Appl. Math. Comput.
**2005**, 165, 313–328. [Google Scholar] [CrossRef] - Zhu, J. DEA/AR analysis of the 1988–1989 performance of the Nanjing Textiles Corporation. Ann. Oper. Res.
**1996**, 66, 311–335. [Google Scholar] [CrossRef] - Seifert, L.M.; Zhu, J. Identifying excesses and deficits in Chinese industrial productivity (1953–1990): A weighted data envelopment analysis approach. Omega
**1998**, 26, 279–296. [Google Scholar] [CrossRef] - Premachandra, I.M. Controlling factor weights in data envelopment analysis by incorporating decision maker’s value judgement: An approach based on AHP. J. Inf. Manag. Sci.
**2001**, 12, 1–12. [Google Scholar] - Lozano, S.; Villa, G. Multi-objective target setting in data envelopment analysis using AHP. Comp. Operat. Res.
**2009**, 36, 549–564. [Google Scholar] [CrossRef] - Kong, W.; Fu, T. Assessing the Performance of Business Colleges in Taiwan Using Data Envelopment Analysis and Student Based Value-Added Performance Indicators. Omega
**2012**, 40, 541–549. [Google Scholar] [CrossRef] - Korhonen, P.J.; Tainio, R.; Wallenius, J. Value efficiency analysis of academic research. Eur. J. Oper. Res.
**2001**, 130, 121–132. [Google Scholar] [CrossRef] - Cai, Y.Z.; Wu, W.J. Synthetic Financial Evaluation by a Method of Combining DEA with AHP. Int. Trans. Oper. Res.
**2001**, 8, 603–609. [Google Scholar] [CrossRef] - Johnes, J. Data envelopment analysis and its application to the measurement of efficiency in higher education. Econ. Educ. Rev.
**2006**, 25, 273–288. [Google Scholar] [CrossRef] [Green Version] - Yang, T.; Kuo, C.A. A hierarchical AHP/DEA methodology for the facilities layout design problem. Eur. J. Oper. Res.
**2003**, 147, 128–136. [Google Scholar] [CrossRef] - Ertay, T.; Ruan, D.; Tuzkaya, U.R. Integrating data envelopment analysis and analytic hierarchy for the facility layout design in manufacturing systems. Inf. Sci.
**2006**, 176, 237–262. [Google Scholar] [CrossRef] - Ramanathan, R. Data envelopment analysis for weight derivation and aggregation in the analytical hierarchy process. Comput. Oper. Res.
**2006**, 33, 1289–1307. [Google Scholar] [CrossRef] - Korpela, J.; Lehmusvaara, A.; Nisonen, J. Warehouse operator selection by combining AHP and DEA methodologies. Int. J. Prod. Econ.
**2007**, 108, 135–142. [Google Scholar] [CrossRef] - Jyoti, T.; Banwet, D.K.; Deshmukh, S.G. Evaluating performance of national R&D organizations using integrated DEA-AHP technique. Int. J. Product. Perform. Manag.
**2008**, 57, 370–388. [Google Scholar] [CrossRef] - Sueyoshi, T.; Shang, J.; Chiang, W.C. A decision support framework for internal audit prioritization in a rental car company: A combined use between DEA and AHP. Eur. J. Oper. Res.
**2009**, 199, 219–231. [Google Scholar] [CrossRef] - Mohajeri, N.; Amin, G. Railway station site selection using analytical hierarchy process and data envelopment analysis. Comput. Ind. Eng.
**2010**, 59, 107–114. [Google Scholar] [CrossRef] - Azadeh, A.; Ghaderi, S.F.; Mirjalili, M.; Moghaddam, M. Integration of analytic hierarchy process and data envelopment analysis for assessment and optimization of personnel productivity in a large industrial bank. Expert Syst. Appl.
**2011**, 38, 5212–5225. [Google Scholar] [CrossRef] - Raut, R.D. Environmental performance: A hybrid method for supplier selection using AHP-DEA. Int. J. Bus. Insights Transform.
**2011**, 5, 16–29. [Google Scholar] - Thanassoulis, E.; Dey, P.K.; Petridis, K.; Goniadis, I.; Georgiou, A.C. Evaluating higher education teaching performance using combined analytic hierarchy process and data envelopment analysis. J. Oper. Res. Soc.
**2017**, 68, 431–445. [Google Scholar] [CrossRef] - Wang, C.; Nguyen, V.T.; Duong, D.H.; Do, H.T. A Hybrid Fuzzy Analytic Network Process (FANP) and Data Envelopment Analysis (DEA) Approach for Supplier Evaluation and Selection in the Rice Supply Chain. Symmetry
**2018**, 10, 221. [Google Scholar] [CrossRef] [Green Version] - Salhieh, S.M.; All-Harris, M.Y. New product concept selection: An integrated approach using data envelopment analysis (DEA) and conjoint analysis (CA). Int. J. Eng. Technol.
**2014**, 3, 44–55. [Google Scholar] [CrossRef] [Green Version] - Charnes, A.; Cooper, W.W.; Rhodes, E. Measuring Efficiency of Decision Making Units. Eur. J. Oper. Res.
**1978**, 2, 429–444. [Google Scholar] [CrossRef] - Banker, R.D.; Charnes, A.; Cooper, W.W. Some models for estimating technical and scale inefficiencies in data envelopment analysis. Manag. Sci.
**1984**, 30, 1078–1092. [Google Scholar] [CrossRef] [Green Version] - Ahn, T.; Charnes, A.; Cooper, W.W. Efficiency characterizations in different DEA models. Socio-Econ. Plan. Sci.
**1988**, 22, 253–257. [Google Scholar] [CrossRef] - Banker, R.D.; Morey, R.C. The use of categorical variables in data envelopment analysis. Manag. Sci.
**1986**, 32, 1613–1627. [Google Scholar] [CrossRef] - Golany, B.; Roll, Y. An application procedure for DEA. Omega
**1989**, 17, 237–250. [Google Scholar] [CrossRef] - Emrouznejad, A.; Witte, K. COOPER-framework: A unified process for non-parametric projects. Eur. J. Oper. Res.
**2010**, 207, 1573–1586. [Google Scholar] [CrossRef] [Green Version] - Jenkins, L.; Anderson, M. A multivariate statistical approach to reducing the number of variables in data envelopment analysis. Eur. J. Oper. Res.
**2003**, 147, 51–61. [Google Scholar] [CrossRef] - Nunamaker, T.R. Using data envelopment analysis to measure the efficiency of non-profit organizations: A critical evaluation. MDE Manag. Decis. Econ.
**1985**, 6, 50–58. [Google Scholar] [CrossRef] - Morita, H.; Avkiran, K.N. Selecting inputs and outputs in data envelopment analysis by designing statistical experiments. J. Oper. Res. Soc. Jpn.
**2009**, 52, 163–173. [Google Scholar] [CrossRef] [Green Version] - Edirisinghe, N.C.; Zhang, X. Generalized DEA model of fundamental analysis and its application to portfolio optimization. J. Bank. Financ.
**2007**, 31, 3311–3335. [Google Scholar] [CrossRef] - Jablonsky, J. Multicriteria approaches for ranking of efficient units in DEA models. Cent. Eur. J. Oper. Res.
**2011**, 20, 435–449. [Google Scholar] [CrossRef] - Dimitrov, S.; Sutton, W. Promoting symmetric weight selection in data envelopment analysis: A penalty function approach. Eur. J. Oper. Res.
**2010**, 200, 281–288. [Google Scholar] [CrossRef] - Shi, H.; Wang, Y.; Zhang, X. A Cross-Efficiency Evaluation Method Based on Evaluation Criteria Balanced on Interval Weights. Symmetry
**2019**, 11, 1503. [Google Scholar] [CrossRef] [Green Version] - Thompson, R.G.; Singleton, F.D.; Thrall, M.R.; Smith, A.B. Comparative Site Evaluation for Locating a High-Energy Physics Lab in Texas. Interfaces
**1986**, 16, 35–49. [Google Scholar] [CrossRef] - Radojicic, M.; Savic, G.; Jeremic, V. Measuring the efficiency of banks: The bootstrapped I-distance GAR DEA approach. Technol. Econ. Dev. Econ.
**2018**, 24, 1581–1605. [Google Scholar] [CrossRef] - Addelman, S. Symmetrical and asymmetrical fractional factorial plans. Technometrics
**1962**, 4, 47–58. [Google Scholar] [CrossRef] - Popović, M.; Vagić, M.; Kuzmanović, M.; Labrović Anđelković, J. Understanding heterogeneity of students’ preferences towards English medium instruction: A conjoint analysis approach. Yug. J. Op. Res.
**2016**, 26, 91–102. [Google Scholar] [CrossRef] - Kuzmanovic, M.; Makajic-Nikolic, D.; Nikolic, N. Preference Based Portfolio for Private Investors: Discrete Choice Analysis Approach. Mathematics
**2020**, 8, 30. [Google Scholar] [CrossRef] [Green Version] - Saaty, R.W. The analytic hierarchy process—What it is and how it is used. J. Math. Model.
**1987**, 9, 161–176. [Google Scholar] [CrossRef] [Green Version] - Stankovic, M.; Gladovic, P.; Popovic, V. Determining the importance of the criteria of traffic accessibility using fuzzy AHP and rough AHP method. Decis. Mak. Appl. Manag. Eng.
**2019**, 2, 86–104. [Google Scholar] [CrossRef] - Kuzmanović, M.; Savić, G.; Popović, M.; Martić, M. A new approach to evaluation of university teaching considering heterogeneity of students’ preferences. High. Educ.
**2013**, 66, 153–171. [Google Scholar] [CrossRef] - Basso, A.; Funari, S. Introducing weights restrictions in data envelopment analysis models for mutual funds. Mathematics
**2018**, 6, 164. [Google Scholar] [CrossRef] [Green Version] - Buschken, J. When does data envelopment analysis outperform a naive efficiency measurement model? Eur. J. Oper. Res.
**2009**, 192, 647–657. [Google Scholar] [CrossRef] - Mester, G.L. Measurement of results of scientific work. Tehnika
**2015**, 70, 445–454. [Google Scholar] [CrossRef] - Hirsch, J.E. An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. USA
**2005**, 102, 16569–16572. [Google Scholar] [CrossRef] [Green Version] - Available online: https://scholar.google.com/intl/en/scholar/citations.html (accessed on 9 January 2020.).

**Table 1.**A set of criteria considered in the conjoint analysis survey, and the resulting utilities and importance values.

No | Criteria (Attributes) | Attribute Levels | Part-Worths (β) | Relative Importance Values (FI) |
---|---|---|---|---|

C1 | Clear and understandable presentation of teaching content | Yes | 0.865 | 22.98% |

No | −0.865 | |||

C2 | Methodical and systematic approach to teaching | Yes | 0.73 | 18.96% |

No | −0.73 | |||

C3 | Tempo of lectures | Too slow | 0.451 | 14.92% |

Optimal | −0.117 | |||

Too fast | −0.334 | |||

C4 | Preparedness for lectures | Good | 0.266 | 7.96% |

Poor | −0.266 | |||

C5 | Punctuality | On time | 0.303 | 9.00% |

Late | −0.303 | |||

C6 | Encouraging students to actively participate in classes | Yes | 0.28 | 8.14% |

No | −0.28 | |||

C7 | Informing students about their progress | Yes | 0.324 | 9.08% |

No | −0.324 | |||

C8 | Takes into account students’ comments and answers their questions | Yes | 0.293 | 8.95% |

No | −0.293 | |||

Constant | 4.046 | |||

Correlations | ||||

Pearson’s R = 0.966 (sig. = 0.000) | ||||

Kendall’s tau = 0.933 (sig. = 0.000) | ||||

Kendall’s tau (for two holdouts) = 1.000 |

Criteria | Min | Max | Mean | Std. Dev. |
---|---|---|---|---|

C1 | 2.17 | 5.00 | 4.400 | 0.535 |

C2 | 2.22 | 5.00 | 4.369 | 0.559 |

C3 | 2.17 | 5.00 | 4.302 | 0.565 |

C4 | 2.67 | 4.95 | 4.527 | 0.470 |

C5 | 1.72 | 4.92 | 4.380 | 0.652 |

C6 | 2.00 | 4.85 | 4.194 | 0.624 |

C7 | 1.83 | 4.85 | 4.127 | 0.643 |

C8 | 1.78 | 5.00 | 4.375 | 0.632 |

EWSM-Original | WSM-Conjoint | DEA | |
---|---|---|---|

Min | 2.073 | 2.098 | 0.539 |

Max | 4.944 | 4.960 | 1.000 |

Mean | 4.335 | 4.344 | 0.941 |

Std. Dev | 0.544 | 0.541 | 0.090 |

Spearman’s rho correlations | |||

EWSM-Original | 1 | 0.993 ** | 0.809 ** |

WSM-Conjoint | 0.993 ** | 1 | 0.797 ** |

DEA | 0.809 ** | 0.797 ** | 1 |

** Significant at the 0.01 level (2-tailed). | |||

3 efficient teachers,h_{k} ≥ 0.95 for 18 out of 27 DMUs |

DMU | Ranks | DEA | Weights | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|

EWSM-Original | WSM-Conjoint | DEA | θk | C1 | C2 | C3 | C4 | C5 | C6 | C7 | C8 | |

1 | 6 | 6 | 14 | 0.982 | 0 | 0 | 0 | 0.82 | 0 | 0.18 | 0 | 0 |

2 | 13 | 13 | 4 | 0.996 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |

17 | 20 | 20 | 5 | 0.996 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |

Method | DEA | Conjoint & DEA (Scenario A) | Conjoint AR DEA (Scenario B) | ||||||
---|---|---|---|---|---|---|---|---|---|

m + s | 8 | 3 (C1, C2, C3) | 8 | ||||||

Teachers | T | P | T + A | T | P | T + A | T | P | T + A |

No. of DMUs | 27 | 17 | 10 | 27 | 17 | 10 | 27 | 17 | 10 |

Average | 0.941 | 0.955 | 0.943 | 0.895 | 0.914 | 0.900 | 0.884 | 0.909 | 0.895 |

SD | 0.088 | 0.131 | 0.055 | 0.105 | 0.149 | 0.069 | 0.107 | 0.152 | 0.074 |

Max | 1.000 | 0.996 | 1.000 | 1.000 | 0.985 | 1.000 | 1.000 | 0.982 | 1.000 |

Min | 0.539 | 0.539 | 0.866 | 0.445 | 0.445 | 0.824 | 0.425 | 0.425 | 0.747 |

h_{k} = 1 | 3 | 0 | 3 | 1 | 0 | 1 | 1 | 0 | 1 |

h_{k} ≥ 0.95 | 18 | 12 | 6 | 11 | 7 | 4 | 7 | 4 | 3 |

EWSM-Original | WSM-Conjoint | DEA | Conjoint & DEA (Scenario A) | Conjoint AR DEA (Scenario B) | |
---|---|---|---|---|---|

Original | 1.000 | 0.993 | 0.809 | 0.933 | 0.997 |

Conjoint | 1.000 | 0.797 | 0.939 | 0.991 | |

DEA | 1.000 | 0.777 | 0.820 | ||

Conjoint & DEA | 1.000 | 0.941 | |||

Conjoint AR DEA | 1.000 |

Method | DEA | Conjoint & DEA (Scenario A) | Conjoint AR DEA (Scenario B) | |||
---|---|---|---|---|---|---|

m + s | 8 | 3 | 8 | |||

No. of DMUs | 27 | 1000 | 27 | 1000 | 27 | 1000 |

Average | 0.971 | 0.948 | 0.827 | 0.800 | 0.889 | 0.861 |

SD | 0.061 | 0.055 | 0.179 | 0.069 | 0.100 | 0.110 |

Max | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |

Min | 0.760 | 0.637 | 0.390 | 0.577 | 0.672 | 0.507 |

h_{k} = 1 | 23 (85.19%) | 153 (15.3%) | 5 (18.52%) | 3 (0.30%) | 5 (18.52%) | 18 (1.8%) |

h_{k} ≥ 0.95 | 23 (85.19%) | 613 (61.3%) | 9 (33.33%) | 11(1.10%) | 9 (33.33%) | 261 (26.1%) |

Parameters | Scientific Research Costs | Number of Citations | h Index | i10 Index |
---|---|---|---|---|

Min | 58419.00 | 683 | 14 | 17 |

Max | 91666.70 | 20 | 3 | 1 |

Average value | 32153.10 | 205.18 | 6.81 | 5.25 |

Standard deviation | 16368.70 | 161.16 | 2.74 | 4.14 |

Correlation | ||||

Scientific research costs | 1 | 0.646 | 0.624 | 0.616 |

Number of citations | 1 | 0.925 | 0.892 | |

h index | 1 | 0.944 | ||

i10 index | 1 |

DMU | Subjective Teachers’ Efficiency | Objective Teachers’ Efficiency | Overall Teachers’ Efficiency | Rank | |
---|---|---|---|---|---|

Conjoint & DEA (Scenario A) | Teaching | Research | |||

1 | 0.9581 | 0.989 | 0.7941 | 0.9055 | 9 |

2 | 0.9571 | 0.696 | 1 | 0.9177 | 5 |

3 | 0.7913 | 0.991 | 1 | 0.9083 | 8 |

4 | 0.4444 | 0.742 | 0.8037 | 0.6362 | 27 |

5 | 0.8769 | 0.955 | 1 | 0.9376 | 3 |

6 | 0.9857 | 0.755 | 0.6334 | 0.8104 | 22 |

7 | 0.9429 | 0.979 | 0.8478 | 0.9162 | 7 |

8 | 0.8552 | 1 | 0.7262 | 0.8391 | 18 |

9 | 0.9165 | 0.769 | 0.8439 | 0.8593 | 15 |

10 | 0.9765 | 1 | 1 | 0.9898 | 1 |

11 | 0.9586 | 0.739 | 0.7649 | 0.8427 | 16 |

12 | 0.8424 | 0.991 | 0.8889 | 0.8903 | 11 |

13 | 0.95 | 0.692 | 0.6071 | 0.7723 | 26 |

14 | 0.9091 | 0.761 | 0.6524 | 0.7855 | 25 |

15 | 0.8409 | 0.955 | 1 | 0.9221 | 4 |

16 | 0.8667 | 0.937 | 0.6655 | 0.8090 | 23 |

17 | 0.8345 | 0.947 | 0.7697 | 0.8347 | 19 |

18 | 0.95 | 0.767 | 0.6877 | 0.8171 | 21 |

19 | 0.9238 | 0.767 | 0.6877 | 0.8058 | 24 |

20 | 0.86 | 1 | 0.9373 | 0.9172 | 6 |

21 | 0.8211 | 0.992 | 0.8877 | 0.8809 | 14 |

22 | 0.8248 | 1 | 0.9065 | 0.891 | 10 |

23 | 1 | 1 | 0.9683 | 0.9885 | 2 |

24 | 0.8267 | 1 | 0.7692 | 0.8423 | 17 |

25 | 0.9789 | 0.948 | 0.7252 | 0.8810 | 13 |

26 | 0.9733 | 0.767 | 0.6877 | 0.8271 | 20 |

27 | 0.9833 | 0.889 | 0.769 | 0.8863 | 12 |

Average | 0.8907 | 0.8899 | 0.8157 | 0.8635 | |

SD | 0.1092 | 0.1168 | 0.1285 | 0.07245 | |

Max | 1 | 1 | 1 | 0.9898 | |

Min | 0.4444 | 0.692 | 0.6071 | 0.6362 | |

h_{k} = 1 | 1 | 6 | 5 | 0 | |

h_{k} ≥ 0.95 | 11 | 13 | 6 | 2 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Popović, M.; Savić, G.; Kuzmanović, M.; Martić, M.
Using Data Envelopment Analysis and Multi-Criteria Decision-Making Methods to Evaluate Teacher Performance in Higher Education. *Symmetry* **2020**, *12*, 563.
https://doi.org/10.3390/sym12040563

**AMA Style**

Popović M, Savić G, Kuzmanović M, Martić M.
Using Data Envelopment Analysis and Multi-Criteria Decision-Making Methods to Evaluate Teacher Performance in Higher Education. *Symmetry*. 2020; 12(4):563.
https://doi.org/10.3390/sym12040563

**Chicago/Turabian Style**

Popović, Milena, Gordana Savić, Marija Kuzmanović, and Milan Martić.
2020. "Using Data Envelopment Analysis and Multi-Criteria Decision-Making Methods to Evaluate Teacher Performance in Higher Education" *Symmetry* 12, no. 4: 563.
https://doi.org/10.3390/sym12040563