An Integrated Decision-Making Model for Analyzing Key Performance Indicators in University Performance Management

: University performance has an important effect on the social influence of universities. With increasing emphasis placed on higher education, it is important to improve and optimize university performance management. However, the performance of university management is affected by numerous indicators in practice, and it is difficult for administrators to optimize all of them because of resource restriction. To address this concern, in this paper, we design a novel integrated model by combining linguistic hesitant fuzzy sets (LHFSs) with the decision-making trial and evaluation laboratory (DEMATEL) method to identify key performance indicators (KPIs) for improving the level of university performance management. Specifically, the LHFSs are utilized to express the hesitant and vague interrelationship assessment of performance indicators provided by experts. A modified DEMATEL is adopted to visualize the causal relationship between performance indicators and determine critical ones. Moreover, we introduce a gray relation analysis (GRA)-based method to derive experts’ weights when their weight information is unknown. Finally, a comprehensive university in Shanghai, China, is employed as an example to illustrate the practicability and availability of the proposed linguistic hesitant fuzzy DEMATEL model


Introduction
In the past few decades, great changes have taken place in the construction and size of higher education, especially in China [1,2]. With the reform of the national education system, the modification of the modern higher education system, educational models, and teaching methods promoted the diversified development of modern universities [3,4]. To enhance the competitiveness of a university, problems on how to enhance overall strength of the university, improve the teaching quality, and cultivate highly qualified talent have attracted the attention of university administrators [5,6]. As evidence of the achievement of organizational goals, performance measurement has emerged in university performance management [7][8][9]. A growing number of researchers are focusing on university performance measurement [10][11][12]. However, the performance of university management is affected by several indicators, and it is unrealistic to improve them all simultaneously due to the restriction of resources. Hence, it is imperative to identify key performance indicators (KPIs) to measure and evaluate the performance of universities [13].
In prior research, KPIs for university performance management were often identified using prepared questionnaires and expert interviews [14,15]. Moreover, some researchers focused on university performance measurement using the balanced scorecard (BSC) method [16,17]. However, these research methods are vulnerable to the subjectivity of humans and cannot explore the causal interrelationships among university performance indicators. The decision-making trial and evaluation laboratory (DEMATEL) method initiated by Gabus and Fontela [18] is a powerful approach to extract the relationships and the interdependence among elements. It is superior to conventional cause-effect analysis techniques because it visualizes the structure of complex causal relationships through matrices and digraphs [19,20]. Given its capabilities, the DEMETAL method has been broadly used to deal with causal relationships among complex factors in various research fields [21][22][23]. Thus, it is promising to adopt the DEMATEL method to analyze the interrelationships among indicators and identify KPIs for university performance management.
Due to the complex interrelationships of indicators, as well as the ambiguity of human thinking, experts tend to express their opinions with linguistic terms in the process of performance indicator evaluation [24]. Moreover, they usually provide their interrelationship assessment information with hesitancy or anonymity. Linguistic hesitant fuzzy sets (LHFSs) were presented by Meng et al. [25] to deal with complex decision-making information and the hesitancy of decision-makers. This method can not only deal with the qualitative evaluations of experts but also reflect their hesitancy, uncertainty, and fuzziness [26,27]. Thus, LHFSs are able to express the vague information of decisionmakers more accurately in practical situations. Recently, the LHFS method was employed in various different areas to address the qualitative preferences of decision-makers. For example, Dong et al. [28] presented a linguistic hesitant fuzzy (LHF)-VIKOR (in Serbian: ViseKriterijumska Optimizacija I Kompromisno Resenje) method for selecting transportation systems. Wu et al. [29] proposed an integrated model by combining the cloud model with LHFSs for the risk assessment of a seawaterpumped hydro storage project. Meng et al. [30] introduced an extend model using LHFSs for evaluating corporate environmental performance. On the basis of the linguistic hesitant fuzzy rock engineering system connection cloud, Gao et al. [31] established a stability evaluation model of surrounding rock stability.
Considering the advantages of LHFSs and the DEMATEL method in the decision-making process, this paper aims to combine them to develop an integrated model to analyze the interrelationships of performance indicators and identify KPIs for university performance management. In summary, this study makes the following valuable contributions: (1) we introduce the use of LHFSs to deal with the uncertain and vague assessment information provided by experts about the interrelationships of performance indicators; (2) we propose a modified DEMATEL method to investigate the causal relationships of indicators and obtain KPIs for university performance improvement; (3) we construct a gray relation analysis (GRA)-based method to acquire experts' weights when their weight information is unknown. Finally, a practical example of a university in Shanghai, China, is provided to demonstrate the application and effectiveness of the proposed LHF-DEMATEL model.
The remainder of this paper is arranged as follows: previous studies related to university performance management and the DEMATEL method are reviewed in Section 2. In Section 3, a hybrid decision-making model using LHFSs and DEMATEL is put forward to identify KPIs for university performance management. In Section 4, a practical case is presented to validate the effectiveness and advantages of the proposed LHF-DEMATEL model. Finally, Section 5 draws the conclusions of this paper and provides further research directions.

University Performance Management
Over the past few years, performance management in universities has been an important research topic and has received great attention from researchers. For example, Philbin [17] analyzed and identified the KPIs of a multidisciplinary university institute in the United Kingdom using the balanced scorecard method. Acquaah et al. [32] put forward a disproportionate stratified random sampling technique for identifying the KPIs impacting performance management implementation in public universities in Uganda. Chang et al. [33] used the multivariate analysis of covariance to evaluate university students' knowledge management performance. By using the fuzzy analytic hierarchy process (AHP), Wang and Liu [34] assessed the performance of scientific research project management in a university. Amin et al. [35] conducted a questionnaire survey on a public university in Malaysia to measure the impact of human resource management practices on organizational performance. The multicriteria constructivist methodology for decision support was adopted by Valmorbida et al. [13] to monitor and manage the performance of the Federal Technological University of Paraná-Brazil. Karuhanga [36] applied principal component and cluster analysis techniques for the performance measurement of public universities in Uganda. Thanassoulis et al. [37] integrated AHP with data envelopment analysis (DEA) to evaluate higher-education teaching performance. By integrating AHP and a technique for order preference by similarity to an ideal solution (TOPSIS), Xu et al. [38] established a model to assess teaching performance in a smart campus. By combing a standardized u-control chart with the ABC analysis method, Carlucci et al. [39] proposed an integrated approach to evaluate the teaching and course quality in an Italian public university.

The DEMATEL Method
In the literature, the DEMATEL technique has been adopted by researchers to analyze the interrelationships among system elements in various fields [40]. For instance, Tsai [41] utilized the fuzzy DEMATEL method to identify key success factors of the auto lighting aftermarket industry. Ding and Liu [42] presented a two-dimensional uncertain linguistic DEMATEL model to find significant success factors in emergency management. Raj et al. [43] investigated the critical barriers to the adoption of Industry 4.0 technologies in the manufacturing sector by using the gray DEMATEL method. Abdullah et al. [44] analyzed the interrelationships and dependence of criteria in sustainable solid waste management using an interval-valued intuitionistic fuzzy DEMATEL method. Liu and Ming [45] employed the rough DEMATEL method to analyze and evaluate requirements for a smart industrial product service system. Jiang et al. [46] presented a large-group linguistic Z-DEMATEL approach for identifying KPIs in hospital performance management. Chauhan et al. [22] utilized a hybrid model on the basis of interpretive structural modeling (ISM) and DEMATEL to select a sustainable supply chain for agricultural produce in India. A hesitant fuzzy DEMATEL model was used by Kumar et al. [47] to analyze the factors related to the adoption of sustainable supply chain practices. By integrating a DEMATEL-based analytic network process, Hsu et al. [48] investigated the critical selection criteria for local middle and top management in multinational enterprises.
An extensive review of the related literature shows that researchers made great efforts in measuring and improving university performance management. However, no known current study, as per available data, has considered the interaction between performance indicators in university performance management. On the other hand, many fuzzy extensions of the DEMATEL method have been proposed to analyze the interrelationships among performance indicators and determine KPIs in different complex systems. However, these modified DEMATEL methods are not efficient in expressing the hesitant assessment information of decision-makers. In order to address the above issues, we extend the DEMATEL method with LHFSs in this study and develop an LHF-DEMATEL model to evaluate the relationships of performance indicators and identify KPIs for university performance management.

The Proposed LHF-DEMATEL Model
In this section, a hybrid model using LHFSs and DEMATEL is proposed to analyze the interrelationships of performance indicators and to identify KPIs for university performance management. First, the LHFSs are utilized to deal with the fuzzy assessment information of experts regarding the interrelationships of performance indicators. Second, a GRA-based weighting method is adopted to derive the weights of experts. Finally, a modified DEMATEL method is employed to analyze the interrelationships among indicators and identify KPIs. Figure 1 shows the detailed steps of the proposed LHF-DEMATEL approach in the form of a flowchart.  Note that the basic concepts and definitions of LHFSs can be found in [19,25]. The procedure of the proposed LHF-DEMATEL model is introduced below. Stage 1. Determine expert weights on the basis of the GRA method. The GRA method proposed by Deng [49] is an effective method to choose the alternative with the highest gray relational grade to a reference sequence. In this study, the GRA method is adopted to determine the weights of experts when their weight information is unknown.
Step 1: Calculating the distance matrices between where ij d is calculated using Equation (2) according to the LHF distance defined in [28].
indicates p-th possible membership degree of the q-th linguistic term in * ij LH , and p is a position weight related to q p r .
Step 2: Obtaining the gray relation coefficients to the ideal matrix.
The gray relation coefficient matrix of the k-th expert can be calculated by where  is the distinguishing coefficient satisfying Step 3: Determining the weights of experts. The weight of each expert k w is calculated by where k ij  is the average gray relation coefficient of k H and can be computed by In this stage, a modified DEMATEL technique is employed to interpret the interrelationships of performance indicators for identifying KPIs. The application steps of the modified DEMATEL method are listed below.
Step 4: Constructing the group interrelationship evaluation matrix. By using the linguistic hesitant fuzzy weighted averaging (LHFWA) operator [25], the group interrelationship evaluation matrix Step 7: Computing the total relation matrix.
where I is an n n  identify matrix.
Step 8: Construct the causal diagram of indicators. The vectors R and C, which denote the sum of the rows and columns respectively, are acquired using Equations (10) and (11).
where R represents the total influence that indicator i F exerts to the other indicators, while C stands for the total influence that indicator i F receives from the other indicators.
Finally, a causal diagram can be constructed by mapping the ordered pairs of   where the horizontal axis R C  is named "Prominence" and the vertical axis R C  is named "Relation". In the causal diagram, "Prominence" represents the strength of influences that are given and received of the indicators, whereas "Relation" indicates the net effect contributed by the indicators to the system. Generally, if 0 R C   , the indicator should be group under the cause group; if 0 R C   , the indicator should be group under the effect group [40,50].

Case Study
In this section, the performance analysis of a university in Shanghai is provided to illustrate the flexibility and effectiveness of our proposed LHF-DEMATEL model.

Implementation of the Proposed Model
The considered university has 10 academic disciplines of engineering, science, medicine, management, economics, philosophy, humanities, law, education, and arts. Currently, the university registers over 41,000 students and it has 4200 academic staff for teaching or research. Currently, it has 47 programs for academic master's degrees, 18 programs for professional master's degrees, 26 programs for master's degrees in engineering, 30 programs for academic doctoral degrees, three programs for professional doctoral degrees, and 25 postdoctoral research stations. To further improve the efficiency and the quality of management, and to enhance the core competitiveness of the university, it is of the utmost importance for administrators to determine KPIs for university performance management.
Through expert interviews and a literature review [15,37,51] of expert E1 is displayed in Table 2. Below, the implementation steps of our proposed LHF-DEMATEL model are described.
Step 3: Using Equations (4) and (5), the weights of the five experts are derived as Step 5: By Equation (7), the direct relation matrix   14 14 ij Z z   is established as provided in Table 5.
Step 6: Via Equation (8), the normalized direct relation matrix   14 14 ij X x   is constructed as represented in Table 6.
Step 7: With Equation (9), the total relation matrix   14 14 ij T t   is acquired as shown in Table   7.
Step 8: By utilizing Equations (10) and (11), the sums of rows and columns are calculated respectively. The values of R C  and R C  are determined, as displayed in Table 8. Then, the causal diagram of the 14 performance indicators is drawn, as shown in Figure 2. Achievements in the construction of information-based teaching equipment and information resources Teacher satisfaction (F13) The satisfaction of teachers to the management system, curriculum arrangement, welfare Student satisfaction (F14) The satisfaction of students to the infrastructure construction, learning conditions, teaching capacity

Discussion of the Results
As depicted in Figure 2, the 14 indicators can be divided into a cause group and an effect group according to the value of R C  . The indicators contained in the cause group are F1, F2, F3, F4, F5, F7, F8, and F12, and the remaining indicators belong to the effect group. A larger value of R C  denotes the higher influence of that indictor on the others.

Cause Indicator Analysis
For the cause indicators, they have a net influence on the whole university performance management, and their performance can seriously affect the university. F1 has the highest value of R C  , which means that it exerts a more important influence on the university performance than it receives from other indicators. Therefore, F1 is a KPI for the performance management of this university in Shanghai.
With respect to F2, it has the largest value of R C  , and its net effect value R C  is the third highest among all the indicators. These indicate that F2 has a great impact on other indicators in the university performance management. Consequently, the improvement of F2 will greatly enhance the efficiency of whole system. Thus, F2 can be identified as a KPI in the given case. Similarly, F4 and F5 are KPIs for the performance management of this university in Shanghai.
Although the R C  value of F3 ranks second among the indicators, both R and C values are not sufficiently high. The low value of R C  indicates that F3 cannot have a significant impact on the improvement in university performance. Hence, F3 is not a KPI for the performance management of this university in Shanghai.

Effect Indicator Analysis
For the effect indicators, they are impacted by other indicators. The R C  value of F6 ranks third among all the indicators. Moreover, both R and C values of F6 are fairly high, although their R C  value is negative. In other words, the relatively low value of R C  cannot dispute the reality that it has a remarkable impact on the whole system. Thus, F6 can be recognized as a KPI in this considered case. Similarly, F11 is a KPI for the performance management of this university in Shanghai.
In addition, F9 has high value of R C  , but its value of R C  is −0.731, revealing that it has no significant effect on the other indicators. Moreover, the adjustment of other indicators can lead to the improvement of F9. Therefore, F9 is not a KPI for the university performance management. Meanwhile, F13 and F14 are not KPIs for the performance management of this university in Shanghai.

Conclusions
In this study, we proposed a new LHF-DEMATEL model to analyze and identify KPIs for university performance management. In this model, the LHFSs were utilized to deal with the uncertain and vague assessment information provided by experts about the interrelationships of performance indicators, and a modified DEMATEL method was employed to identify KPIs for university performance management. Moreover, a GRA-based method was proposed to derive experts' weights with unknown weight information. Finally, an empirical example was adopted to validate the effectiveness of our proposed LHF-DEMATEL approach. Six KPIs were determined for the considered application, including "quantity and structure of teachers", "professional abilities", "funding input", "number of key disciplines", "quality of newly established specialty", and "teachers' scientific research level".
In future research, we will focus on the following directions: first, other advanced uncertainty theories, such as unbalanced fuzzy linguistic term sets, can be adopted to handle the uncertainties of experts' evaluations in the future. Second, in the process of evaluating the interrelationships among performance indicators, experts' opinions are diverse and may be conflicting. Hence, it is promising to develop a method to eliminate conflicting opinions in university performance management. In addition, the approach developed in this research is general and can be used for identifying KPIs in other fields, and other factors such as intensive human capital losses in the higher education [52,53] can be considered in the university performance management.