Reform of Chinese Universities in the Context of Sustainable Development: Teacher Evaluation and Improvement Based on Hybrid Multiple Criteria Decision-Making Model

Abstract

China is pushing universities to implement reforms in order to achieve the sustainable development goals, but with the development level of teachers becoming the key restricting factor. In this sense, teacher evaluation and improvement act as positive factors for China to achieve the 2030 sustainable development goals. Previous studies on teacher evaluation have usually assumed that the relationship between the evaluation criteria is independent, with the weights of each standard derived from this assumption. However, this assumption is often not in line with the actual situation. Decisions based on these studies are likely to waste resources and may negatively impact the efficiency and effectiveness of teachers’ sustainable development. This study developed an integrated model for the evaluation and improvement of teachers based on the official teacher evaluation criteria of China’s International Scholarly Exchange Curriculum (ISEC) programme and a multiple criteria decision-making methodology. First, a decision-making trial and a laboratory-based analytical network process were used to establish an influential network-relation diagram (INRD) and influential weights under ISEC standards. Next, an important performance analysis was used to integrate the weight and performance of each standard to produce a worst-performance criterion set for each university teacher. Finally, the worst performance set used an INRD to derive an improvement strategy with a cause–effect relationship for each teacher. This study chose a Chinese university that has implemented teaching reform for our case study. The results show that our developed model can assist decision-makers to improve their current evaluations of teachers and to provide a cause–effect improvement strategy for education reform committees and higher education institutions.

1. Introduction

Education is crucial to sustainable development. The action plan entitled ‘Transforming our World: The 2030 Agenda for Sustainable Development’ jointly concluded among 193 countries was released at the World Summit on Sustainable Development in September 2015 [1]. Since then, the goal of sustainable development has become a new goal of global development. One of the goals in the agenda is about ensuring inclusive and equitable quality education and lifelong learning opportunities for all [2,3]. The Talloires Declaration announced in 1990 pioneered the inclusion of sustainable development in higher education, highlighting the role of higher educational institutions in promoting global sustainable development [3]. The higher educational institutions subsequently began to shoulder the task of training human resources with the vision of sustainable development. Following the identification of the Global Education Roadmap 2030 at the World Education Forum 2015, the Global Action Program on Education for Sustainable Development was launched. Sustainable development and education for sustainable development thus gained stronger momentum to be promoted worldwide [4]. According to UNESCO, education for sustainable development can guarantee the future of the economy, the environment, and society. To achieve the goal, universities are undergoing teaching and research reforms [5]. China published the National Plan on Implementation of the 2030 Agenda for Sustainable Development in September 2016. The Plan conveyed China’s resolution to advance the said 2030 Agenda [6]. A sub-plan was proposed to fulfil the educational development goal, further facilitating the reform of China’s higher educational institutions from the perspective of sustainable development. In 2019, China’s Ministry of Education (MOE) came up with the plan to develop nearly 10,000 national top and 10,000 provincial top majors for the undergraduates, thus boosting the reform of teaching activities in higher education institutions [7].

Teachers play a fundamental role in the reforms [3,8]. While higher education institutions are carrying out reforms in response to the goal of sustainable development, sustainability-based teaching activities pose new challenges to teachers at universities. The Teaching Staff Development Plan is a feasible approach to accelerating the integration of education for sustainable development [5,9]. For China’s universities, particularly the regional ones (i.e., universities established by governments under the provincial level), the quality of teachers has become a key obstacle to reform. Therefore, in 2018, the Action Program to Rejuvenate Education for the Teaching Staff 2018–2022 was released by five ministries in China, including the MOE [10]. In this context, regional universities began to invest heavily in the training and re-education of the teaching staff, in the hope that they would stand out in the competition that takes sustainable development as its goal.

However, a key defect with China’s higher education is the lack of, and uneven distribution of sufficient quality education resources [11,12]. At the same time, existing education resources have not been fully leveraged yet [13,14]. Among such resources, there is an important issue of university teacher development wherein massive amounts of invested resources are inefficiently used. Unless addressed properly, this issue would hinder universities in China from realizing the goal of education for sustainable development. If teacher evaluations can identify the core reasons why and where they perform poorly before resources are invested in specific areas, the information can be used to effectively improve the resource investment and use rate in Chinese universities. To solve this problem, a subsidiary department of the Chinese MOE developed the International Scholarly Exchange Curriculum (ISEC) program based on the directive of “globalizing education to deepen reforms in higher education”. University teacher evaluation and improvement is part of the ISEC program, where the assessment criteria of university teachers are also known as the ISEC standard in this study.

Teacher evaluation as a management tool of education aims to facilitate the growth of the teaching staff [15,16,17,18] and are within the scope of multiple criteria decision-making (MCDM). For example, Ghosh [19] combined the analytic hierarchy process (AHP) and the technique for order of preference by similarity to ideal solution (TOPSIS) in order to evaluate faculty performance in engineering education. Pavani et al. [20] developed an expert-based group model for evaluating teacher performance using fuzzy AHP and TOPSIS. Xu et al. [21] evaluated teaching performance on a smart campus. Wang et al. [22] proposed a hybrid model for classroom teaching performance based on TOPSIS and the triangle fuzzy number. These studies provided valuable contributions to teacher evaluations. However, the relationships among the criteria in these models are independent and do not reflect real conditions. To address this issue, the suggestions for teacher improvement provided by these MCDM models are often directed toward the improvement of poor performance, but the factors that lead to poor performance may not always be addressed by these improvement strategies. As the Chinese proverb goes, “treat the head when the head aches, treat the foot when the foot hurts”. Thus, the improvement of the teaching capacity is not satisfactory, resulting in resource wastage and scarcity.

To address this issue, the study combined the MCDM with the teacher evaluation standards adopted by ISEC in order to create a new evaluation model. First, the evaluation model is the use of China’s ISEC standards. These standards in China constitute the practical application of ISEC teachers to choose and improve problems. Based on the ISEC standards, the decision-making trial and evaluation laboratory (DEMATEL)-based analytic network process (ANP; together DANP) was then used to establish an influential network-relation diagram (INRD), and subsequently to obtain the influential weights within ISEC standards. The derived INRD has been proven to be an effective tool to explore the cause and effect relations in many papers [23,24,25,26,27]. Second, the importance-performance analysis (IPA) was used to combine the influential weights and performance of attributes, and to capture a set of criteria for identifying where each teacher performs worst. Finally, decision-makers can use the INRD to focus on the causation of poor performance to determine the actual factors related to each teacher’s performance, and to prioritise a direction for improvement. The integrated model of DEMATEL and IPA methods has been successfully applied to different studies [28,29,30,31,32]. The model focuses on analyzing the factors influencing poor teacher performance according to certain criteria with the intention of using fewer but more focused resources to produce effective improvements. This method provides a new mechanism for the sustainable development of university teachers based on evaluations and also supplements the inadequacies of the existing studies to a certain degree.

In this study, empirical data from 15 domain experts from the ISEC management institute were applied to demonstrate our proposed model. The results show that “Professional ethics and literacy (C₁)” is the primary influential standard and “Teacher ability and development (C₂)” has the highest influential weight. Teachers A and B both performed poorly in “Teaching performance (C₃₁)” and “Research cooperation (C₃₂)”. In other words, teachers A and B must improve their performance in these two standards. According to traditional performance improvement strategies, the ISEC management institute should invest resources in encouraging teachers to publish their teaching results and promote research cooperation among teachers. In practice, teacher A and teacher B enjoy the same training and development resources. However, the essential causes of their under-performance vary, which, accordingly, requires different training resources. This phenomenon reveals the issue of the untargeted allocation of teaching staff development resources in the reform of China’s universities.

The rest of this paper is structured as follows. Section 2 provides a brief introduction of current teacher evaluations and outlines the literature documenting research methods. Section 3 introduces the DANP and IPA methods used in our new model. Section 4 details the implementation of ISEC topics in this model. Section 5 discusses the results and features of the model, and Section 6 summarizes our contributions and directions for future research.

2. Review of University Teacher Evaluation Models

The previous university teacher evaluation models can be roughly divided into three research stages: (1) the selection of appropriate criteria in the evaluation model, (2) building the decision-making model using the MCDM methodology, and (3) building the decision-making model using statistical or data analysis methodologies.

2.1. Selection of Appropriate Criteria in the Evaluation Model

This stage of research is focused on the selection of subjective and objective indicators in an evaluation model. Contradictions and conflicts exist between indicators, such as encouraging teacher vision and personal development versus increasing wages, and between objective evaluation results and critical feedback [33,34,35]. For example, when Mills and Hyles [36] evaluated university teachers at Oklahoma State University’s Stillwater College (Stillwater, OK, USA), the results provided limited feedback and drawbacks included scattered goals, unclear standards, and inconsistencies in management perspectives. Based on this, they used interviews and surveys to understand opinions on teacher evaluations, and their development process established a consolidated solution with a hierarchical architecture that integrated goals, directions, and procedures for annual performance evaluations. For this reason, some studies began to explore the establishment of more reasonable evaluation indicators and weights. For example, Desselle et al. [37] studied university teacher evaluation systems at Duquesne University’s Mylan School of Pharmacy (Stillwater, OK, USA). They used a modified Delphi procedure to confirm 29 teaching activities and 44 academic actions, including their weights, in an evaluation standard. Filipe et al. [38] provided guidelines for avoiding conflict between processes and goals by developing a message management system to evaluate teaching activity—however, they encountered difficulties similar to those of previous studies due to differing interpretations of how to implement and assess teaching. Subjective indicators are an inevitable aspect of the assessment process, creating ambiguity that hinders the transparency and fairness of teaching performance assessments. However, some studies have shifted from subjective indicators that rely on the evaluator (e.g., enthusiasm for one’s work) to relatively objective indicators (e.g., the number of reference papers). Although this can prevent fuzziness due to subjectivity, it results in fewer facets of evaluation that cannot fully reflect a teacher’s true performance level [39,40,41]. Based on these lessons, others have proposed to combine decision analysis models with objective and subjective indicators or mathematical programming models, but they often lack real-time scoring systems based on theoretical rationality or have weight settings that vary depending on the evaluator’s subjectivity. This neglects the principle of value trade-off and results in a total evaluation score that may have no substance or value. Using a methodology for systematic decision modelling to establish the factors and their weights in an evaluation system is a key problem that remains unsolved [42,43,44].

2.2. Building Decision-Making Models Using MCDM Methodology

The MCDM is specifically applied to solve evaluative decision-making problems with multiple criteria, i.e., evaluation, selection, and improvement problems. Some researchers have used MCDM methods to build decision models. For example, Ghosh [29] used AHP and TOPSIS to review teacher evaluations in engineering schools. Filipe et al. [38] developed a multi-criteria information system to review teaching practices. Hein et al. [45] used tools such as consensus theory, information entropy, and TOPSIS to construct a multi-criteria decision analysis method to evaluate 56 university professors. However, people’s representations of their opinions with regard to objects or events, in reality, contain fuzziness. Therefore, some scholars have developed various fuzzy-based MCDM models. For example, Chen et al. [46] proposed a framework for teaching evaluation based on a combination of fuzzy AHP and fuzzy comprehensive evaluation methods. Chang and Wang [47] proposed a type of multi-criteria decision-making model oriented toward teachers in an attempt to solve the issues of pervasive subjectivity, imprecision, and fuzziness within the faculty. Dey Mondal and Ghosh [48] used AHP, fuzzy AHP, complex proportional assessment of alternatives with grey relations (COPRAS), and TOPSIS in combination with game theory and compromise planning methods in MCDM to evaluate the performance level of teachers. These models provide decision-makers with a simple and easy-to-use method for evaluating and selecting university teachers. However, in these models, decision-making is based on independent relationships among criteria that cannot provide decision-makers with suitable systemic improvement strategies for all university teachers.

2.3. Building Decision-Making Models Using Statistical or Data Analysis Methodologies

The last research stage has involved overcoming the fuzzy defects of these MCDM models that use statistical or data analysis methods to construct decision-making models. For example, Nikolaidis and Dimitriadis [49] established a framework based on statistical quality control to use student feedback to a maximum degree. Lyde et al. [50] used a multisource method for evaluation (MME) and improved constraints such as the timing of reflections, accountability from year to year, and mentoring in order to construct a more comprehensive formative teaching assessment tool. Bi [51] evaluated five years of teaching at a management school of a university by creating a mean and standard deviation diagram based on statistical process control theory. Xu et al. [31] used principal component analysis (PCA) to calculate and identify six primary components and then used AHP to calculate the weight of each hierarchy before using grey correlation to improve the TOPSIS target decision analysis algorithm to avoid errors in decision-making due to subjective factors. These models offer a perspective on data behaviour as a basis for decision-making, but they rely on massive amounts of data and are unable to provide decision-makers with causal influence relationships affecting teacher performance.

2.4. Research Gaps in Their Decision-Making Models

Past decision-making models presented different contributions to teacher evaluation, selection, and improvement problems. However, these MCDM models have a major defect in that the relationship between criteria in the evaluation model is independent. Therefore, current MCDM models cannot help decision-makers to obtain a guide for the performance improvement of each teacher. To fill the research gap, this study developed a novel MCDM model that uses the DANP method to construct the INRD and influential weights for criteria and the IPA method to search the worst performances of criteria for each teacher. The detailed modelling process and its corresponding method are described in Section 3. The comparison of the three categories of decision-making models is shown in

Table 1. Comparison with different decision-making models.

Category	Characteristic	Limitations or Current Defects
Selection of appropriate criteria in the evaluation model	The research is focused on the selection of subjective and objective indicators in an evaluation model.	The modelling process of the scoring system is not considered the relationship between indicators. The weight setting of the indicator depends on the evaluator’s subjectivity.
Building the decision-making model using MCDM methodology	The research used MCDM methods to construct various decision-making models for multi-criteria evaluation, selection, and improvement problems.	The decision-making depends on a group of experts’ domain knowledge. The current MCDM models are based on independent relationships among criteria that cannot provide decision-makers with suitable systemic improvement strategies for each teacher.
Building the decision-making model using statistical or data analysis methodologies	The research used statistical or data analysis methods to construct decision-making models for overcoming the fuzzy defects of original MCDM models.	The models are unable to provide decision-makers with causal influence relationships affecting teacher performance.

.

3. Our Proposed Hybrid DANP-IPA Model

This study developed an integrated hybrid MCDM model that combines the DANP method and IPA analysis. The former can be used to derive the influential network-relationship diagram (INRD) and influential weights that can help decision-makers to understand the cause-effect direction based on a systemic perspective. The latter can help decision-makers to easily capture the worst performance of each teacher in all attributes. Finally, the worst performance attributes of each teacher can be based on the INRM to develop a series of the most appropriate improvement strategies. The modelling flow diagram and corresponding methods of this hybrid DANP-IPA model are depicted in Figure 1.

3.1. DANP Method

The DANP method was developed by Lee et al. [52] by combining the DEMATEL technique [53] and the ANP method [54]. The DANP method retains interdependent relationships among criteria and further derives an influential network-relation diagram (INRD) and influential weights for all criteria. The INRD established by the DANP method can help to form decision-making equations for various systemic plans to improve alternative/objective performance after evaluation [55]. Based on this advantage, the method has been applied in many areas, such as public open space development [56], creative communities [57], quality of life [58], supplier management [59], airline performance [60], green buildings [61], and international airports [62]. The detailed steps in the DANP calculation are as follows.

Step 1: Build an initial influence–relationship matrix.

For an evaluation criteria model, respondents assess the degree of influence between criteria using a pairwise comparison based on a five-point Likert scale (ranging from 0 = “no influence” to 4 = “extremely high influence”). Then, the influential matrices of all respondents are integrated into a matrix by averaging to produce the initial influence relationship matrix D. Matrix D represents the actual experience within the group of all respondents:

$$\mathit{D}={[({\displaystyle {\sum }_{\Theta =1}^n}{c}_{ij}^{\Theta })/\alpha ]}_{n\times n}$$

(1)

where $c_{ij}^{\Theta }$ is the result of the respondent $\Theta$, indicating the degree of influence between criteria i and j; $\alpha$ is the total number of respondents, and n is the total number of criteria.

Step 2: Derive a normalized influence–relationship matrix.

The initial–influence relationship matrix D derives a normalized influence-relationship matrix A using Equations (2) and (3), in which all diagonal terms are 0 and the maximum sum of a row or column is 1:

$$\rho =\underset{i,j}{\max }\left[{\max }_i{\displaystyle {\sum }_{j=1}^n{d}_{ij}},{\max }_j{\displaystyle {\sum }_{i=1}^n{d}_{ij}}\right]$$

(2)

$$\mathit{A}=\mathit{C}/\rho$$

(3)

where $\rho$ is the maximum value of the sum of a row or column.

Step 3: Obtain a total influence–relationship matrix.

Matrix A calculates and adds the influence degree of each iteration through the Markov chain process and produces a total influence relation matrix Q, as shown in Equation (4):

$$\mathit{Q}=\mathit{A}+{\mathit{A}}^2+\cdots +{\mathit{A}}^{\delta }=\mathit{A}{(\mathit{I}-\mathit{A})}^{-1},\mathrm{when}{\lim }_{\delta \to \infty }{\mathit{A}}^{\delta }={[0]}_{n\times n}$$

(4)

Step 4. Build an influential network–relationship diagram (INRD).

First, the sum of each row and column can obtain vectors $u_i$ and $v_i$ through Equations (5) and (6). Then, $(u_i+v_i)$ is the total strength of influences given and received, or prominence, as shown in Equation (7). Otherwise, $(u_i-v_i)$ is the net influence degree between given and received influences, also called the cause/effect. A positive cause/effect value indicates that factor i affects other factors and belongs to the cause group; if the value is negative, factor i is affected by other factors and belongs to the effect group. Finally, the INRD is established based on the vectors of prominence and cause/effect:

$$u_i={(u_i)}_{n\times 1}={\left[{\displaystyle {\sum }_{j=1}^n{q}_{ij}}\right]}_{n\times 1}^{}, i\in \{1,2,\dots ,n\}$$

(5)

$$v_i={(v_j)}_{1\times n}^{\prime }={\left[{\displaystyle {\sum }_{i=1}^n{q}_{ij}}\right]}_{1\times n}, j\in \{1,2,\dots ,n\}$$

(6)

where $\prime$ denotes transposition, $u_i$ indicates the sum of direct and indirect effects of the factor $i$ on the other factors, and $v_i$ indicates the sum of direct and indirect effects factor $i$ received from the other factors.

Step 5: Transfer to an unweighted supermatrix.

First, the total influence–relation matrix Q can be divided into two matrices: the attribute level ${\mathit{Q}}_C$ and the dimension level ${\mathit{Q}}_D$. Second, each row within a dimension in the total influence relation matrix ${\mathit{Q}}_C$ uses Equations (7)–(9) to obtain the normalized total influence relation matrix ${\mathit{Q}}_C^{\rho }$, as shown in Equation (7), in which ${\mathit{Q}}_C^{\rho 11}$ is an example to demonstrate the basic concept of normalizing, as shown in Equations (8) and (9).

$${\mathit{Q}}_{\mathit{C}}^{\rho }=\begin{array}{cc}& \begin{array}{c}\begin{array}{ccccc}{\mathit{D}}_1& & {\mathit{D}}_{\mathit{j}}& & {\mathit{D}}_{\mathit{m}}\\ {\mathit{c}}_{11\cdots }{\mathit{c}}_{1{\mathit{m}}_1}& \cdots & {\mathit{c}}_{\mathit{j}1\cdots }{\mathit{c}}_{\mathit{j}{\mathit{m}}_{{}_{\mathit{j}}}}& \cdots & {\mathit{c}}_{\mathit{m}1\cdots }{\mathit{c}}_{\mathit{m}{\mathit{m}}_{\mathit{m}}}\end{array}\end{array}\hfill \\ \hfill \begin{array}{c}\begin{array}{l}{\mathit{D}}_1\\ \\ ⋮\\ \end{array}\\ \begin{array}{l}\\ \\ {\mathit{D}}_i\\ \\ \\ ⋮\end{array}\\ \begin{array}{l}\\ \end{array}\\ {\mathit{D}}_m\end{array}\begin{array}{c}\begin{array}{l}{\mathit{c}}_{11}\\ {\mathit{c}}_{\begin{array}{l}12\\ ⋮\end{array}}\\ {\mathit{c}}_{\begin{array}{l}1{\mathit{m}}_1\\ ⋮\end{array}}\\ {\mathit{c}}_{\mathit{i}1}\\ {\mathit{c}}_{\begin{array}{l}\mathit{i}2\\ ⋮\end{array}}\end{array}\\ \begin{array}{l}{\mathit{c}}_{\begin{array}{l}\mathit{i}{\mathit{m}}_{\mathit{i}}\\ ⋮\end{array}}\\ c_{\mathit{m}1}\\ c_{\mathit{m}2}\\ {\mathit{c}}_{\mathit{m}{\mathit{m}}_{\mathit{m}}}\end{array}\end{array}& {\left[\begin{array}{ccccc}{\mathit{Q}}_{\mathit{c}}^{\rho 11}& \cdots & {\mathit{Q}}_c^{\rho 1\mathit{j}}& \cdots & {\mathit{Q}}_c^{\rho 1m}\\ \\ ⋮& & ⋮& & ⋮\\ \\ {\mathit{Q}}_c^{\rho \mathit{i}1}& \cdots & {\mathit{Q}}_c^{\rho ij}& \cdots & {\mathit{Q}}_{\mathit{c}}^{\rho im}\\ \\ ⋮& & ⋮& & ⋮\\ \\ {\mathit{Q}}_c^{\rho m1}& \cdots & {\mathit{Q}}_c^{\rho mj}& \cdots & {\mathit{Q}}_c^{\rho mm}\end{array}\right]}_{\mathit{n}\times \mathit{n}|\mathit{m}<\mathit{n},{\sum }_{\mathit{j}=1}^{\mathit{m}}{\mathit{m}}_{\mathit{j}}=\mathit{n}}\hfill \end{array}$$

(7)

$$q_i^{11}={\displaystyle {\sum }_{j=1}^{m_1}{q}_{ij}^{11}},i=1,2,\dots ,m_1$$

(8)

$${\mathit{Q}}_C^{\rho 11}=\left[\begin{array}{ccccc}{q}_{11}^{11}/q_1^{11}& \cdots & q_{1j}^{11}/q_1^{11}& \cdots & q_{{}^{1m_1}}^{11}/q_1^{11}\\ ⋮& & ⋮& & ⋮\\ q_{i1}^{11}/q_i^{11}& \cdots & q_{ij}^{11}/q_i^{11}& \cdots & q_{{}^{i{m}_1}}^{11}/q_i^{11}\\ ⋮& & ⋮& & ⋮\\ q_{{}^{m_{1}1}}^{11}/q_{m_1}^{11}& \cdots & q_{{}^{m_{1}j}}^{11}/q_{m_1}^{11}& \cdots & q_{{}^{m_1{m}_1}}^{11}/q_{m_1}^{11}\end{array}\right]=\left[\begin{array}{ccccc}{q}_{11}^{\rho 11}& \cdots & q_{1j}^{\alpha 11}& \cdots & q_{1m_1}^{\alpha 11}\\ ⋮& & ⋮& & ⋮\\ q_{i1}^{\alpha 11}& \cdots & q_{ij}^{\alpha 11}& \cdots & q_{{}^{i{m}_1}}^{\alpha 11}\\ ⋮& & ⋮& & ⋮\\ q_{{}^{m_{1}1}}^{\alpha 11}& \cdots & q_{{}^{m_{1}j}}^{\alpha 11}& \cdots & q_{{}^{m_1{m}_1}}^{\alpha 11}\end{array}\right]$$

(9)

Lastly, the normalized influence relation matrix ${\mathit{Q}}_c^{\rho }$ is transposed to obtain the unweighted supermatrix $\mathit{B}=({\mathit{Q}}_c^{\rho }{)}^{\prime }$, as shown in Equation (10):

$$\mathit{B}={({\mathit{Q}}_{\mathit{C}}^{\rho })}^{\prime }=\begin{array}{cc}& \begin{array}{c}\begin{array}{ccccc}{\mathit{D}}_1& & {\mathit{D}}_{\mathit{i}}& & {\mathit{D}}_{\mathit{m}}\\ {\mathit{c}}_{11\cdots }{\mathit{c}}_{1{\mathit{m}}_1}& \cdots & {\mathit{c}}_{\mathit{i}1\cdots }{\mathit{c}}_{\mathit{i}{\mathit{m}}_{{}_{\mathit{i}}}}& \cdots & {\mathit{c}}_{\mathit{m}1\cdots }{\mathit{c}}_{\mathit{m}{\mathit{m}}_{\mathit{m}}}\end{array}\end{array}\hfill \\ \hfill \begin{array}{c}\begin{array}{l}{\mathit{D}}_1\\ \\ ⋮\\ \end{array}\\ \begin{array}{l}\\ \\ {\mathit{D}}_j\\ \\ \\ ⋮\end{array}\\ \begin{array}{l}\\ \end{array}\\ {\mathit{D}}_m\end{array}\begin{array}{c}\begin{array}{l}{\mathit{c}}_{11}\\ {\mathit{c}}_{\begin{array}{l}12\\ ⋮\end{array}}\\ {\mathit{c}}_{\begin{array}{l}1{\mathit{m}}_1\\ ⋮\end{array}}\\ {\mathit{c}}_{\mathit{j}1}\\ {\mathit{c}}_{\begin{array}{l}\mathit{j}2\\ ⋮\end{array}}\end{array}\\ \begin{array}{l}{\mathit{c}}_{\begin{array}{l}\mathit{j}{\mathit{m}}_{\mathit{j}}\\ ⋮\end{array}}\\ c_{\mathit{i}1}\\ c_{\mathit{i}2}\\ {\mathit{c}}_{\mathit{m}{\mathit{m}}_{\mathit{m}}}\end{array}\end{array}& {\left[\begin{array}{ccccc}{\mathit{B}}^{11}& \cdots & {\mathit{B}}^{i1}& \cdots & {\mathit{B}}^{m1}\\ \\ ⋮& & ⋮& & ⋮\\ \\ {\mathit{B}}^{1j}& \cdots & {\mathit{B}}^{ij}& \cdots & {\mathit{B}}^{mj}\\ \\ ⋮& & ⋮& & ⋮\\ \\ {\mathit{B}}^{1m}& \cdots & {\mathit{B}}^{in}& \cdots & {\mathit{B}}^{mm}\end{array}\right]}_{\mathit{n}\times \mathit{n}|\mathit{m}<\mathit{n},{\sum }_{\mathit{j}=1}^{\mathit{m}}{\mathit{m}}_{\mathit{j}}=\mathit{n}}\hfill \end{array}$$

(10)

Step 6: Obtain a weighted supermatrix.

Each row within the goal in the total influence–relation matrix ${\mathit{Q}}_D$ uses Equations (11)–(13) to obtain the normalized total influence relation matrix ${\mathit{Q}}_D^{\rho }$, as shown in Equation (11). Matrices ${\mathit{Q}}_C^{\rho }$ and ${\mathit{Q}}_D^{\rho }$ produce a new matrix through Equation (14), the weighted supermatrix ${\mathit{B}}^{\Theta }$:

$${\mathit{Q}}_D={\left[\begin{array}{ccccc}{q}_{11}& \cdots & q_{1j}& \cdots & q_{1m}\\ ⋮& & ⋮& & ⋮\\ q_{i1}& \cdots & q_{ij}& \cdots & q_{im}\\ ⋮& & ⋮& & ⋮\\ q_{m1}& \cdots & q_{mj}& \cdots & q_{mm}\end{array}\right]}_{m\times m}$$

(11)

$$d_i={\displaystyle {\sum }_{j=1}^m{q}_D^{ij}},i=1,2,\dots ,m\mathrm{and}{q}_D^{\rho ij}=q_D^{ij}/d_i,j=1,2,\dots ,m$$

(12)

$$\mathit{W}=({\mathit{Q}}_D^{\rho }{)}^{\prime }={\left[\begin{array}{ccccc}{f}_{{}_D}^{11}/d_1& \cdots & f_{{}_D}^{1j}/d_1& \cdots & f_{{}_D}^{1m}/d_1\\ ⋮& & ⋮& & ⋮\\ f_{{}_D}^{i1}/d_i& \cdots & f_{{}_D}^{ij}/d_i& \cdots & f_{{}_D}^{im}/d_i\\ ⋮& & ⋮& & ⋮\\ f_{{}_D}^{m1}/d_m& \cdots & f_{{}_D}^{mj}/d_m& \cdots & f_{{}_D}^{mm}/d_m\end{array}\right]}_{{}_{m\times m}}={\left[\begin{array}{ccccc}{f}_{{}_D}^{\rho 11}& \cdots & f_{{}_D}^{\rho 1j}& \cdots & f_{{}_D}^{\alpha 1m}\\ ⋮& & ⋮& & ⋮\\ f_{{}_D}^{\rho i1}& \cdots & f_{{}_D}^{\rho ij}& \cdots & f_{{}_D}^{\rho im}\\ ⋮& & ⋮& & ⋮\\ f_{{}_D}^{\rho m1}& \cdots & f_{{}_D}^{\rho mj}& \cdots & f_{{}_D}^{\rho mm}\end{array}\right]}_{m\times m}$$

(13)

$${\mathit{B}}^{\Theta }=\mathit{W}\times \mathit{B}=\left[\begin{array}{ccccc}{q}_D^{\rho 11}\times {\mathit{B}}^{11}& \cdots & q_D^{\rho i1}\times {\mathit{B}}^{i1}& \cdots & q_D^{\rho m1}\times {\mathit{B}}^{m1}\\ ⋮& & ⋮& & ⋮\\ q_D^{\rho 1j}\times {\mathit{B}}^{1j}& \cdots & q_D^{\rho ij}\times {\mathit{B}}^{ij}& \cdots & q_D^{\rho mj}\times {\mathit{B}}^{mj}\\ ⋮& & ⋮& & ⋮\\ q_D^{\rho 1m}\times {\mathit{B}}^{1m}& \cdots & q_D^{\rho im}\times {\mathit{B}}^{im}& \cdots & q_D^{\rho mm}\times {\mathit{B}}^{mm}\end{array}\right]$$

(14)

Step 7: Limit the weighted supermatrix and derive the influential weights.

The weighted supermatrix ${\mathit{B}}^{\Theta }$ convergences the influence degree of each time through the Markov chain process and finally obtains the influential weights for all criteria/dimensions, as shown in Equation (15).

$$\underset{\Lambda \to \infty }{\lim }{({\mathit{B}}^{\Theta })}^{\Lambda }$$

(15)

3.2. Importance–Performance Analysis Method

Importance–performance analysis (IPA) is a well-known business management method [63] that was first developed by Martilla and James [64] to identify the critical performance criteria of products or services [65,66]. The method is used to create an IPA matrix or priority map using standard performance and importance scores, which can be divided into four quadrants (Q1–Q4), as shown in Figure 2 [31,63,67]:

(1)

Q1: Keep up the good work, indicating the main strengths and potential competitive advantages of a product or service.

(2)

Q2: Possible overkill, representing that these criteria are a low priority for customers. That is, the organization should reduce resources directed toward these criteria because resources are limited.

(3)

Q3: Low priority, representing criteria that are not important to customers and not performing exceptionally well. The organization should not care too much about these attributes.

(4)

Q4: Concentrate here, representing the service’s primary weaknesses and threats to its competitiveness. For the organization, these criteria have the highest priority in terms of investment.

This approach can help decision-makers to easily understand the performance and importance of criteria. The method is widely used in many different areas, such as supplier management [31], tourism development [66,68], and strategy management [69].

4. Empirical Case

In this section, an empirical study using data from the ISEC in China is presented to illustrate the application of the proposed DANP-IPA model for evaluating and improving the performance of university teachers.

4.1. Case Background Problem Description

Compared with developed countries, China lacks higher education resources and their distribution is imbalanced, especially at regional universities, which constitute the majority of universities in China. To solve this problem, a subsidiary department of the Chinese MOE developed the International Scholarly Exchange Curriculum (ISEC) program based on the directive of “globalizing education to deepen reforms in higher education”. Universities that participated in the program received assistance with curriculum, teaching, and quality assurance reforms—however, these reforms were based on teachers. For this reason, ISEC sought to establish a sustainable development mechanism in order to cultivate quality teaching teams that would autonomously reform curriculums and teaching and achieve the goal of comprehensive teaching reform. This line of thinking is a departure from China’s current method of promoting change from the top downward. The core of this development program involves teachers and the sustainable development of promoting reforms in higher education. The corresponding content includes (1) curriculum systems, (2) support for teacher development systems, and (3) service systems. Currently, the program has been implemented in approximately 30 test universities in nine provinces with more than 1500 teachers listed in ISEC, accumulating practical experience with both successes and failures. These ISEC teachers have met the ISEC inclusion criteria, i.e., age, academic qualifications, foreign study or exchange experience, international curriculum teaching experience, critical thinking, and ISEC mission acceptance. The program prioritizes the development of teachers on the front line as its core. ISEC is the coordinating mechanism and training platform for teacher development, partnering with universities to use global educational resources to support the improvement of teacher abilities. Next, ISEC teachers act as leaders to push teaching reforms throughout the university and faculty. Therefore, establishing a decision-making model with practical value to assist decision-makers with effectively evaluating ISEC teachers and improving their abilities is a critical problem.

With assistance from the ISEC management institute, 15 ISEC domain experts were selected from the ISEC expert database (one American, two Australian, and 12 Chinese). These experts were teachers, administrative staff members, or ISEC teacher representatives (associate professor or above) at renowned universities. The ISEC domain experts represent elite ISEC teachers. As such, this study is based on the ISEC university teacher standard indicator system (

Table 2. University teachers’ evaluation system based on ISEC standards.

Dimension	Criterion	Content
Professional ethics and literacy (C1)	Professional ethics (C11)	According to the Code of Professional Ethics of Teachers in Higher Education formulated by the MOE, teachers are examined for their moral performance in the teaching process.
	Professional literacy (C12)	Evaluation of a teacher’s professionalism, professional sense of belonging, and physical and mental health
Teacher ability and development (C2)	Basic teaching skills (C21)	Evaluation of future planning ability and level of teaching
	Teaching implementation (C22)	Achievement of teaching goals and amount of teaching resources used
	External review (C23)	Evaluation of teaching performance by students, peers, and experts
	Professional development (C24)	Planning and implementation of personal career development
Teacher performance and contributions (C3)	Teaching performance (C31)	Contributions to teaching performance, awards, research standards, and teaching teams
	Research cooperation (C32)	Status of personal and group research results

), and it integrated MCDM and IPA tools to establish a mixed multi-criteria decision-making model. This DANP-IPA model can be used to evaluate and improve ISEC standards for teachers.

4.2. INRD and Influential Weight Using the DANP Method

The degree of influence between standards was calculated using a five-point measurement scale for all experts, and then Equation (1) was used to consolidate and obtain an initial influence–relationship matrix (

Table 3. Initial influence–relation matrix.

Criteria	C11	C12	C21	C22	C23	C24	C31	C32
C11	0.00	3.20	2.67	3.00	2.87	2.93	2.73	2.07
C12	2.87	0.00	2.87	3.00	3.00	2.93	2.87	2.40
C21	1.67	2.13	0.00	3.80	3.73	3.13	3.00	2.33
C22	2.07	2.33	3.00	0.00	4.00	3.13	3.53	2.33
C23	1.80	2.33	3.00	3.20	0.00	3.33	3.47	2.40
C24	2.20	2.60	3.07	3.20	3.20	0.00	3.27	2.93
C31	1.67	2.13	2.93	3.00	3.13	3.27	0.00	3.13
C32	1.73	2.27	1.93	2.13	2.33	3.53	2.87	0.00

). In this matrix, Equations (2)–(4) allowed the inference of a total influence–relationship matrix (

Table 4. Total influence–relation matrix.

Criteria	C11	C12	C21	C22	C23	C24	C31	C32
C11	0.58	0.81	0.90	0.97	1.01	1.00	0.98	0.81
C12	0.70	0.70	0.92	0.99	1.03	1.02	1.00	0.83
C21	0.66	0.78	0.80	1.01	1.05	1.02	1.01	0.83
C22	0.68	0.81	0.94	0.89	1.08	1.05	1.05	0.85
C23	0.65	0.78	0.91	0.98	0.89	1.02	1.01	0.82
C24	0.69	0.82	0.94	1.01	1.05	0.92	1.04	0.87
C31	0.64	0.76	0.89	0.96	1.00	1.00	0.86	0.84
C32	0.58	0.69	0.77	0.83	0.87	0.91	0.88	0.63

). Using Equations (5) and (6), the influence structure of each standard was obtained (

Table 5. Sum of given influence (r_i) and received influence (d_i).

Dimension	ri	di	ri + di	ri − di	Criterion	ri	di	ri + di	ri − di
C1	2.59	2.10	4.69	0.48	C11	7.06	5.18	12.24	1.88
					C12	7.20	6.16	13.36	1.04
C2	2.64	2.86	5.50	−0.22	C21	7.16	7.08	14.25	0.08
					C22	7.35	7.66	15.00	−0.31
					C23	7.07	7.98	15.05	−0.91
					C24	7.35	7.94	15.29	−0.59
C3	2.38	2.64	5.02	−0.27	C31	6.96	7.82	14.77	−0.86
					C32	6.17	6.49	12.65	−0.32

) to draw an INRD (Figure 3).

Figure 3 shows the entire mutual influence network within the ISEC teacher standard evaluation system, where “Professional ethics and literacy (C₁)” is the primary standard influencing “Teacher ability and development (C₂)” and “Teacher performance and contributions (C₃)”—this shows that “Professional ethics and literacy (C₁)” is the basis of two criteria. That is, a teacher’s professional ethics and literacy impact their abilities and future development, which is reflected in performance and teaching contributions. Further analysis showed that “Professional ethics (C₁₁)” and “Teaching implementation (C₂₂)” are causal groups (i.e., r_i − d_i > 0). “Research cooperation (C₃₂)” and “Teaching performance (C₃₁)” are effect groups (i.e., r_i − d_i < 0). Past studies focused on decision-makers investing resources to improve specific standards to correct poor performance. However, this may not address the true cause of the problems as decision-makers may neglect the influence structure of standards being composed of interdependent, not independent, relationships. When decision-makers focus solely on poorly performing standards (effect) and invest massive amounts of resources, they do not recognize that the problem may stem from causal standards. For instance, “Research cooperation (C₃₂)” maybe a teacher’s poorest performing standard and the decision-maker may hope that the teacher can cooperate with other researchers. They then host workshops to provide cooperative opportunities and invest massive amounts of resources to encourage teachers to work together on research projects. However, teachers may want to focus on lectures and teaching due to their own state of “Professional ethics (C₁₁)” and “Professional literacy (C₁₂)”, which is conservative. They may simply lack an assertive attitude toward learning, leading to poor research ability. In other words, various factors can cause “Research cooperation (C₃₂)” to be the area of poorest performance. Here, INRD provides a systemic view that assists decision-makers in understanding the relationship structure influencing the standards to pinpoint each teacher’s problems. The aspect of “Basic teaching skills (C₂₁)” in “Teacher ability and development” is a basic standard in teaching because it influences other standards such as “Teaching implementation (C₂₂)”, “Professional development (C₂₄)”, and “External review (C₂₃)”.

For the weights of the influencing relationships, the total influence–relationship matrix (Table 4) uses Equations (7)–(10) to establish an unweighted supermatrix (

Table 6. Unweighted supermatrix.

Criteria	C11	C12	C21	C22	C23	C24	C31	C32
C11	0.42	0.50	0.46	0.46	0.46	0.46	0.46	0.45
C12	0.58	0.50	0.54	0.54	0.54	0.54	0.54	0.55
C21	0.23	0.23	0.21	0.24	0.24	0.24	0.23	0.23
C22	0.25	0.25	0.26	0.22	0.26	0.26	0.25	0.25
C23	0.26	0.26	0.27	0.27	0.23	0.27	0.26	0.26
C24	0.26	0.26	0.26	0.26	0.27	0.23	0.26	0.27
C31	0.55	0.55	0.55	0.55	0.55	0.54	0.51	0.58
C32	0.45	0.45	0.45	0.45	0.45	0.46	0.49	0.42

). Next, Equations (11)–(14) are used in the matrix to establish a weighted supermatrix (

Table 7. Weighted supermatrix.

Criteria	C11	C12	C21	C22	C23	C24	C31	C32
C11	0.11	0.14	0.13	0.13	0.13	0.13	0.13	0.13
C12	0.16	0.13	0.15	0.15	0.15	0.15	0.15	0.15
C21	0.09	0.09	0.08	0.09	0.09	0.09	0.09	0.09
C22	0.10	0.09	0.10	0.08	0.10	0.10	0.09	0.09
C23	0.10	0.10	0.10	0.10	0.09	0.10	0.10	0.10
C24	0.10	0.10	0.10	0.10	0.10	0.09	0.10	0.10
C31	0.19	0.19	0.19	0.20	0.19	0.19	0.17	0.20
C32	0.16	0.16	0.16	0.16	0.16	0.16	0.17	0.14

). Finally, Equation (15) is used to achieve a stable, extreme supermatrix. The influence weights for each standard are shown in

Table 8. Influence weights for each criterion of ISEC standard.

Dimension	Local Weight	Ranking	Criterion	Local Weight	Ranking	Global Weight	Ranking
C1	0.277	3	C11	0.458	2	0.127	4
			C12	0.542	1	0.150	3
C2	0.376	1	C21	0.231	4	0.087	8
			C22	0.249	3	0.094	7
			C23	0.260	1	0.098	5
			C24	0.260	2	0.098	6
C3	0.347	2	C31	0.545	1	0.189	1
			C32	0.455	2	0.158	2

.

Table 8 shows the influential weight of each standard within the entire system and the degree of influence for each standard, which is beneficial for subsequent teacher evaluation processes, as the performance in each standard considers the degree of influence. Judging from the results, the dimension of “Teacher ability and development (C₂)” has the highest influential weight and reflects teacher ability and development as the most influential relationship in the evaluation system. In addition to being driven by “Professional ethics and literacy (C₁)”, performance is also reflected by “Teacher performance and contributions (C₃)”. A teacher’s long-term planning and development in each period of “Teacher performance and contributions (C₃)” impacts improvements in future “Teacher ability and development (C₂)”, which is why “Teacher ability and development (C₂)” is the most influential dimension. “Teaching performance (C₃₁)” and “Research cooperation (C₃₂)” are the top two criteria in terms of influential weight because they reflect performance in the other dimensions, so if other dimensions perform poorly, this is reflected in “Teaching performance (C₃₁)” and “Teacher ability and development (C₂)”.

4.3. University Teacher Evaluation Using the IPA Method

Based on teacher performance and weights, this section outlines our use of the IPA method to analyze and gather the standard in which each teacher performs the poorest. With the assistance of the ISEC management institute, five members of the review committee and three university teachers participated in this study. All five members had experience in evaluating ISEC teachers for more than one semester; they used a 0–10-point scale to evaluate three university teachers. These scores were averaged and consolidated into a performance score (

Table 9. The average performance of university teacher A.

Criteria	Member_1	Member_2	Member_3	Member_4	Member_5	Average
C11	8	7	8	7	8	7.6
C12	8	7	8	7	8	7.6
C21	7	8	8	6	7	7.2
C22	7	8	7	8	8	7.6
C23	7	8	7	6	7	7
C24	6	6	6	6	6	6
C31	5	5	5	6	5	5.2
C32	5	5	5	5	5	5

,

Table 10. The average performance of university teacher B.

Criteria	Member_1	Member_2	Member_3	Member_4	Member_5	Average
C11	8	7	8	7	8	8
C12	8	7	7	7	8	8
C21	9	7	8	8	8	9
C22	8	7	8	8	7	8
C23	8	8	8	9	8	8
C24	6	6	6	6	6	6
C31	8	8	6	7	7	8
C32	6	6	5	6	6	6

and

Table 11. The average performance of university teacher C.

Criteria	Member_1	Member_2	Member_3	Member_4	Member_5	Average
C11	8	8	8	8	8	8
C12	8	8	8	8	8	8
C21	8	9	8	8	8	8
C22	9	9	9	9	7	9
C23	9	8	9	9	8	9
C24	9	9	9	9	9	9
C31	10	10	10	8	10	10
C32	10	10	10	10	8	10

). Next, the centre values of weights and performance were used as threshold values to separate standards into four groups as follows: Group I—high weights and performance; group II—low weights and high performance; group III—low weights and performance, and group IV—high weights and low performance. Decision-makers must focus on group IV, as standards within the group are categorized as high weight but the performance in these standards is the poorest. The investment of resources should prioritize the standards in this group to effectively improve the performance in the group. The analysis results for the three university teachers are provided in

Table 12. IPA method for university teacher evaluation.

Criterion	Weight	Teacher A		Teacher B		Teacher C
		Performance	Group	Performance	Group	Performance	Group
C11	0.127	7.6	I	7.6	I	8.0	I
C12	0.150	7.6	I	7.4	IV	8.0	I
C21	0.087	7.2	III	8.0	II	8.2	II
C22	0.094	7.6	II	7.6	II	8.6	II
C23	0.098	7.0	III	8.2	II	8.6	II
C24	0.098	6.0	III	6.0	III	9.0	II
C31	0.189	5.2	IV	7.2	IV	9.6	I
C32	0.158	5.0	IV	5.8	IV	9.6	I

Note: Center value as a threshold value (7.5, 0.125). The values of x and y are derived from the central point between the maximum and minimum of performance and weight, respectively.

and Figure 4, Figure 5 and Figure 6.

Figure 4, Figure 5 and Figure 6 show that teacher C performed the best overall. Teachers A and B perform poorest in the standards of “Teaching performance (C₃₁)” and “Research cooperation (C₃₂)”. In other words, teachers A and B must improve their performance in these two standards. Based on traditional performance improvement strategies, the ISEC management institute should invest resources to encourage these teachers to publish their teaching results in order to promote academic cooperation among teachers. However, their problems may not actually stem from these two standards because their performance simply reflects the existence of a problem. To avoid this issue, the cause–effect relationship analysis of INRD (Figure 3) can be used to understand the entire issue and propose appropriate improvement measures for each university professor—this is also the solution that allows both the minimization of resources and the maximization of benefit. This will be discussed in further detail in the next section.

5. Discussion

This section presents the results of the IPA with a cause-and-effect relationship analysis based on the INRD to create plans and strategies for teacher improvement. Finally, the influential weights produced from this study and actual current weights are explored.

5.1. Proposal for Improvement Suggestions Based on the INRD

To understand how INRD is used, this study used the case of teacher B to explain how INRD is used in analyzing and proposing improvement measures. Figure 7 shows that teacher B performed poorly in “Teaching performance (C₃₁)” and “Research cooperation (C₃₂)”. Of the two, “Research cooperation (C₃₂)” was associated with the poorest performance. However, the standards that impact “Teacher performance and contributions (C₃)” are “Teacher ability and development (C₂)” and “Professional ethics and literacy (C₁)”. The teaching-related standards are “Basic teaching skills (C₂₁)”, “Teaching implementation (C₂₂)”, and “External review (C₂₃)”, which all had good scores (7.6 to 8.2). This was also reflected in the “Teaching performance” score (7.2). Of these, “Professional development (C₂₄)” was the poorest performing standard (6.0), which may be due to different factors, such as not listing research cooperation in professional development planning, using the majority of their daily time for teaching or administrative tasks, not having the ability or opportunity to conduct research cooperation, or not having a suitable research budget. All these factors can lead to poor performance in “Research cooperation (C₃₂)”. Based on the above analysis, this study conducted discussions with ISEC management staff and proposed suggestions for improvements based on the perspectives of “Professional ethics and literacy (C₁)” and “Teacher ability and development (C₂)”. The university should reassess teacher B’s professional ethics and literacy, research ability, and time spent teaching, and adjust these three aspects, for instance, by improving a teacher’s professional acknowledgment, adjusting their lecture and teaching time, arranging for them to learn the skills required for research, and arranging opportunities for research cooperation. The ISEC Management Institute could arrange a series of comprehensive courses to improve research capability and post-curricular meetings for research cooperation.

5.2. Comparison of Weights

Next, this section compared the weights obtained by the analysis of ISEC teacher standards using the DANP and AHP methods to actual weights currently being used, as shown in

Table 13. Comparative analysis of weights.

Dimension/Criterion	Real Case		DANP		AHP
	Local Weight	Ranking	Local Weight	Ranking	Local Weight	Ranking
Professional ethics and literacy (C1)	0.2	3	0.277	3	0.459	1
Professional ethics (C11)	0.4	2	0.458	2	0.519	1
Professional literacy (C12)	0.6	1	0.542	1	0.481	2
Teacher ability and development (C2)	0.5	1	0.376	1	0.324	2
Basic teaching skills (C21)	0.2	2	0.231	4	0.237	2
Teaching implementation (C22)	0.3	1	0.249	3	0.220	3
External review (C23)	0.3	1	0.260	1	0.354	1
Professional development (C24)	0.2	2	0.260	2	0.188	4
Teacher performance and contributions (C3)	0.3	2	0.347	2	0.218	3
Teaching performance (C31)	0.7	1	0.545	1	0.816	1
Research cooperation (C32)	0.3	2	0.455	2	0.184	2

. This study discovered the following: (1) actual weights and DANP weights stem from the cumulative practical teaching experience of experts, and the results show that the rankings in both are close. This result shows that the establishment of actual weights contains implicit systemic perspectives. (2) Actual weights do not provide specific values but approximate values. For instance, the standards “Basic teaching skills (C₂₁)” and “Professional development (C₂₄)” are both weighted 0.2, whereas “Teaching implementation (C₂₂)” and “External review (C₂₃)” are both weighted 0.3. The same weight values provide the impression that the standards “Basic teaching skills (C₂₁)” and “Professional development (C₂₄)” are equally important, and “Teaching implementation (C₂₂)” and “External review (C₂₃)” are equally important. However, the DANP weights show that: “External review (C₂₃)” $\succ$ “Professional development (C₂₄)” $\succ$ “Teaching implementation (C₂₂)” $\succ$ “Basic teaching skills (C₂₁)”. By comparison, precise weights allow for precise evaluations along with subsequent use and investment of resources for teacher development. From a practical standpoint, this does not merely involve categorizing the importance of standards. (3) Currently, actual weights do not precisely describe factors. Further, past studies have often used the AHP method as weight analysis of criteria in teacher evaluation models [19,20,29,46,48]. In this study, the ranking of dimensions and criteria between the AHP method and the other two methods is a little different, however, AHP weights can provide specific values for each criterion. However, the AHP method assumes that the relationship between the criteria is independent, which is inconsistent with the operation of the real world. On the contrary, the DANP method used to obtain the weights of the standards from the perspective of systemic influence allows these systemic perspectives to identify cause-and-effect relationships using INRD. This assigns greater significance and high explanatory power to standard weights. Based on this feature, the DANP has been successfully applied to different issues [56,57,58,59,60,61,62]. Compared to the previous model, the DANP method employed in this study can provide more specific information to help decision-makers to obtain a complete systematic solution.

6. Conclusions and Remarks

ISEC teacher evaluation standards are a key aspect of Chinese education reform. In the past, the ISEC management institute has focused on teachers’ poorest performing standards to propose a series of improvement measures. However, these measures did not consider the mutual influence structures amongst the standards. Therefore, the improvements often treat the symptoms but not the source, and resources are not used efficiently for maximum benefit. Standard weights are expressed in integers that cannot effectively differentiate the degree of relative importance among standards and explain the management significance behind each weight. This type of evaluation process does not truly reflect the abilities of each teacher. To solve this issue, this study developed a mixed multi-criteria decision-making (DANP-IPA) model based on ISEC teacher standards.

First, this model provides the INRD and influential weights based on the systemic perspective. INRD assists decision-makers in understanding the influential relationship structure among standards. Influential weights integrate influential perspectives into subsequent processes of teacher evaluation so that improvement measures are based on cause and effect. In the practical case studies used in this research, INRD identified “Professional ethics and literacy (C₁)” as the primary influential standard that impacts “Teacher ability and development (C₂)” and “Teacher performance and contributions (C₃)”. In other words, the basic factors that affect teacher ability and performance are the individual’s own professional ethics and literacy. “Teacher ability and development (C₂)” was shown to have the highest influential weight because it is driven by “Professional ethics and literacy” and is reflected in “Teacher performance and contributions (C₃)”. A teacher’s long-term planning and development and “Teacher performance and contributions (C₃)” in each period were found to be influenced by future improvements in “Teacher ability and development (C₂)”. This is why “Teacher ability and development (C₂)” was found to have the highest influential ranking of all dimensions. “Teaching performance (C₃₁)” and “Research cooperation (C₃₂)” were the two highest in terms of influential weight because they reflect the performance of other standards, meaning that if other standards perform poorly, this is eventually reflected in “Teaching performance (C₃₁)” and “Research cooperation (C₃₂)”. Finally, the IPA performance analysis showed that teacher C had the best overall performance—in comparison, teachers A and B both performed poorly in “Teaching performance (C₃₁)” and “Research cooperation (C₃₂)”. Teachers A and B must improve their performance in these two standards. According to traditional performance improvement strategies, the ISEC management institute should invest resources to encourage teachers to publish their teaching results and promote research cooperation among teachers. However, these problems may not actually stem from these two standards because they are simply a reflection of the problem.

Based on the above analysis, this study conducted discussions with ISEC management staff and provided suggestions for improvements based on the perspectives of “Professional ethics and literacy (C₁)” and “Teacher ability and development (C₂)”. Specifically, the university should reassess teacher B’s professional ethics and literacy, research ability, and time spent teaching and adjust these three aspects, for example, by improving the teacher’s professional acknowledgment, adjusting their lecture and teaching time, arranging for them to learn the skills required for research, and arranging opportunities for research cooperation. The ISEC management institute could arrange a series of comprehensive courses to improve research capability and post-curricular meetings for research cooperation. Therefore, this study contributes to this topic as follows: (1) our method can be combined with INRD (Figure 3) to analyze the cause-effect relationship and understand the entire problem to propose the most suitable improvement measures for each university professor. (2) Our solution both minimizes resources and maximizes benefits to improve the efficiency of resource investment in the development of university teachers. Our method will have a catalyzing effect on the continued development and cultivation of teachers.

Although this study provides a scientific decision-making model, the orientation of future research involves the exploration of multiple facets. This study focused on the establishment of the model and the application of ISEC teacher assessment and improvement. In our current research, the limitations are the ISEC evaluation standards and improvement strategies that must depend on domain-experts’ knowledge (i.e., influential network-relation diagram and influential weights). The data source requires the expertise of a group of ISEC domain experts with practical experience in the issue area. Therefore, the data sources (i.e., DANP and performance) require support from the ISEC management institute. In addition, the student’s perspective in the evaluation process and the subsequent development of the teacher were not included in the scope of this research, which forms another limitation of this study. Based on these limitations, one direction of future studies could involve the consideration of the perspective of students in the evaluation process for university teachers. Another research opportunity could be the integration of research on data exploration and multi-criteria decision-making into the model with the use of big data to analyse and understand the correlations between standards for filtering key standards and their weights—as a way of ultimately proposing improvement measures based on expert knowledge with objective behavioural rules and subjective practical experience. As the Chinese education reform deepens, various problems relating to building decision-making models will require corresponding solutions from future researchers.