Evaluation of Online Teaching Quality: An Extended Linguistic MAGDM Framework Based on Risk Preferences and Unknown Weight Information

Abstract

Evaluation of online teaching quality has become an important issue because many universities are turning to online classes due to the Corona Virus Disease 2019 (COVID-19) pandemic. In this paper, online teaching quality evaluation is considered as a linguistic multi-attribute group decision-making (MAGDM) problem. Generally, the evaluation sematic information can be symmetrically or asymmetrically distributed in linguistic term sets. Thus, an extended linguistic MAGDM framework is proposed for evaluating online teaching quality. As the main contribution, the proposed method takes into account the risk preferences of assessment experts (AEs) and unknown weight information of attributes and sub-attributes. To be specific, the Delphi method is employed to establish a multi-level evaluation indicator system (EIS) of online teaching quality. Then, by introducing the group generalized linguistic term set (GLTS) with two risk preference parameters, a two-stage optimization model is developed to calculate the weights of attributes and sub-attributes objectively. Subsequently, the linguistic MAGDM framework was divided into two stages. The first stage maximizes the group comprehensive rating values of teachers on different attributes to obtain partial ranking results for teachers on each attribute. The latter stage maximizes the group comprehensive rating values of teachers to evaluate the overall quality. Finally, a case study is provided to illustrate how to apply the framework to evaluate online teaching quality.

1. Introduction

Due to the outbreak of Corona Virus Disease 2019 (COVID-19), universities in many countries have adopted online teaching methods to prevent the spread of the epidemic on campus. According to data published by the United Nations Educational, Scientific and Cultural Organization (UNESCO), as of 28 April 2020, 168 countries worldwide had closed their campuses due to the epidemic and have adopted online teaching methods [1]. From the Chinese Ministry of Education, as of 8 May, 1454 colleges and universities nationwide were conducting online teaching, 1.03 million teachers were offering 1.07 million courses online, and a total of 17.75 million college students had participated in online learning. Currently, the global epidemic situation is still grim, and online teaching will continue to be one of the primary forms, especially in universities where students come from a wide variety of regions [2].

Large-scale online teaching has effectively solved the problem of students learning at home, but the changing of teaching environment from offline to online has also posed significant challenges to teaching quality. For instance, the changing scenarios and the lack of a sense of presence make it difficult for teachers and students to interact effectively. Meanwhile, network congestion occurs frequently when a large number of students attend the classes at the same time. Against this backdrop, how to assess and promote the quality of online teaching has become a key issue for education departments and universities. To this end, exploring and constructing scientific and rational indicators and scientific methods for assessing the quality of online teaching plays a crucial role in improving the online teaching quality [1,3].

To the best of knowledge, the complexity of teaching quality assessment leads to an evaluation process involving multiple levels of indicators and numerous assessment experts (AEs). Thus, the issue of assessing the quality of online teaching can also be regarded as a multi-attribute group decision-making (MAGDM) problem. In practical teaching evaluation problems, the choice of evaluation indicator system (EIS) and the evaluation method is related to the quality of evaluation results. The EISs are generally divided into two categories: single-level EISs [4,5,6,7] and multi-level EISs [8,9,10,11,12,13]. Additionally, there are many kinds of evaluation methods, such as the grey relational coefficient embed to the technique for order of preference by similarity to an ideal solution (Grey-TOPSIS) [4], analytic hierarchy process (AHP) [8,10,12,13], fuzzy comprehensive evaluation (FCE) [5,8,11,13], characteristic objects methods [6,7], cloud model [9] and so on. For example, Song and Zheng [4] combined the grey correlation with TOPSIS methods to construct a new model for assessing the quality of teaching in universities. Faizi et al. [6] extended the characteristic objects method to solve MAGDM problem in a hesitant fuzzy environment. Faizi et al. [7] further proposed a new MAGDM method by combining the characteristic object method and triangular intuitionistic fuzzy numbers, which is different from the hesitant fuzzy environment. Chen et al. [8] proposed an evaluation framework based on fuzzy AHP and FCE methods, and established a teaching performance indicator evaluation system to improve teaching performance of teachers, which is the primary means to improving the teaching quality of teachers. Chang and Wang [9] assessed teachers in higher education by constructing the interval cloud evaluation matrix and determining the order of importance of the decision program based on the cloud model. Wang [11] proposed a new data mining algorithm and established an improved FCE method to deal with the evaluation problem of physical education. Obviously, to ensure the accuracy of evaluation results, a growing number of evaluation problems need to be assessed with multi-level EISs.

According to the existing literature on teaching quality evaluation, there are two gaps need to be filled. The first is the lack of a multi-level EIS based on online teaching quality evaluation. The second is that the evaluation processes mentioned above did not take into account the influence of AEs’ risk preferences on linguistic evaluation information.

As to the selection of indicators for teaching quality evaluation, Song and Zheng [4] investigated the indicator system for assessing the quality of teaching in traditional classrooms, which was constructed from the perspectives of teaching plans, teaching methods, communication skills, and teaching professional level, and then assessed the quality of teaching in five universities. Yang et al. [13] evaluated the quality of the basic nursing course at Peking University in terms of teaching attitudes, teaching environment, teaching content, and teaching feedback. Kong et al. [14] constructed indicators for assessing teaching quality from the dimensions of attitude, content, teaching art, classroom management, and teaching effectiveness. Peng and Dai [15] assessed the classroom teaching quality of teachers in terms of the dimensions of teaching capacity, teaching attitude, teaching content, teaching method, and teaching effect. These researches provide a reference for this paper to build a new multi-level EIS of online teaching evaluation.

In addition, in the process of online teaching evaluation, due to the complexity of the evaluation problem and the limited knowledge of AEs, it is easier for them to give evaluation information in the form of linguistic terms on the teachers with regard to each sub-attribute. In order to allow linguistic terms to be aggregated and compared, scholars have carried out semantic representations of linguistic terms. Herrera and Martínez [16] proposed a 2-tuple linguistic representation model to represent semantic values of linguistic terms in a balanced linguistic term set (LTS) and represented the linguistic term as a linguistic 2-tuple. Wang and Hao [17] presented a new proportional 2-tuple linguistic representation model to compute with words in an unbalanced LTS. Dong et al. [18] defined the concept of the numerical scale, and on this basis, extended the 2-tuple linguistic representation model to compute with words in an unbalanced LTS. Subsequently, Rodriguez et al. [19] introduced the concept of a hesitant fuzzy LTS, which provided a linguistic and computational basis to increase the richness of linguistic elicitation. Dong et al. [20] proposed a novel computing with words method to construct the hesitant fuzzy LTS by using an unbalanced LTS and the numerical scale. Li et al. [21] presented a personalized individual semantics model to personalize individual semantics by the 2-tuple linguistic model and an interval numerical scale. Liu et al. [22] proposed a new fuzzy envelope for the hesitant fuzzy LTS in form of type-2 fuzzy sets for representing comparative linguistic expressions in decision contexts, which overcomes the limitation of existing representations in dealing with inherent uncertainties. However, in actual online teaching evaluation, different AEs may have different expectations for semantic values of linguistic terms, which can be described by their risk preferences [23]. Therefore, it is necessary to investigate the linguistic representation model in linguistic MAGDM problems based on risk preferences.

With regard to linguistic MAGDM based on risk preferences, Zhou and Xu [24] extended the sigmoid function and presented the concept of a generalized linguistic term set (GLTS). They applied it to linguistic multi-attribute decision making involving risk preferences. Lin and Wang [23] presented the concept of GLTSs with triangular fuzzy semantic values, and developed a new method for coping with linguistic MADM problems considering decision makers’ risk preferences. Lin and Wang [25] developed a new framework to solve the linguistic MAGDM problem based on risk preferences, where attribute evaluation values are represented by linguistic terms and the attribute weight information is incomplete. Guo and Sun [26] combined single-valued neutrosophic LTSs with prospect theory, and introduced a new comprehensive fuzzy decision approach to deal with linguistic MAGDM. Focusing on decision makers’ risk preferences and regret theory, Ma et al. [27] initiated an approach to solve the behavioral multi-attribute decision-making problem based on probability expressions. Liao et al. [28] launched a Choquet integral-based hesitant fuzzy gained and lost dominance score method based on prospect theory to deal with MAGDM problems with experts’ risk preference psychology. Obviously, the above methods and case studies are all based on a single-level EIS. This paper extends the linguistic MAGDM based on risk preferences to the context of a multi-level EIS.

Different from group decision problems with preference relations [29,30], the MAGDM problems involves the determination of attribute weights. To evaluate the quality of online teaching according to the given EIS, it is an important issue of whether to provide attribute weights. In fact, it is difficult for AEs to give attribute weight information in many cases. Thus, some scholars have conducted studies on this issue. For example, based on the standard deviation and average deviation of the interval type-2 fuzzy set, Gong [31] established an optimization model to calculate attribute weights in linguistic MAGDM, in which linguistic terms are transformed into interval type-2 fuzzy sets. Gupta et al. [32] proposed a new method for solving linguistic MAGDM problems, in which linguistic terms are converted into trapezoidal intuitionistic fuzzy numbers and Shannon’s entropy theory is used to determine attribute weights. Wang and Mendel [33] initiated a new method to deal with linguistic MAGDM problems based on interval type-2 fuzzy sets, where an optimization model is established to determine completely unknown attribute weights. Touqeer et al. [34] used a combination of grey relational analysis (GRA) method and projection method to calculate completely unknown attribute weights in linguistic MAGDM problems. We can see that these literatures are all based on the researches of linguistic MAGDM under the single-level EIS. Therefore, considering multi-level EIS and risk preferences, this paper develops a two-stage optimization model to derive the attribute weights and the sub-attribute weights.

In response to the above research deficiencies, this paper designs an extended framework to assess the quality of online teaching. The novelties of the framework are as follows:

• Taking into account the problem of online teaching quality evaluation, based on the existing traditional classroom teaching quality EISs [4,13,14,15], the Delphi method is employed to construct a brand-new multi-level EIS of online teaching quality, including five attributes and 15 sub-attributes.
• Inspired by the decision method in Lin and Wang [25], this framework extends the linguistic MAGDM method involving risk preferences from the single-level EIS to the multi-level one.
• In teaching quality evaluation problems based on multi-level EISs, many works of literature adopted the AHP approach to derive the weights of attributes and sub-attributes [8,9,10,11,12,13]. Sometimes AEs cannot make pairwise comparisons on the importance of attributes or sub-attributes, that is, they cannot use the AHP method to calculate attribute weights. Thus, based on the principle of maximizing the group comprehensive evaluation value of each alternative, a two-stage attribute and sub-attribute weight solution optimization model is established in this framework.
• The evaluation ranking results obtained by this framework are divided into the overall ranking result and partial ranking results, which are helpful for teachers and universities to understand their own situation and make targeted improvements.

The rest of this paper is structured as follows: Section 2 introduces the basic concepts related to LTSs, group GLTSs with individual risk preferences, the Euclidean distance between any two positive triangular fuzzy numbers, and the score function. Section 3 constructs a multi-level EIS for assessing online teaching quality of teachers, and then establishes a two-stage optimization model to solve the weights of attributes and sub-attributes in the EIS. Section 4 devises an extended evaluation framework and then describes the detailed evaluation process. Section 5 proposes a case study to verify the validity and feasibility of the proposed evaluation framework. Section 6 carries on the practical discussion to the evaluation ranking results including the overall and partial ranking results. Finally, Section 7 concludes this paper.

2. Preliminaries

In this section, we introduce some concepts about LTSs, group GLTSs, the Euclidean distance, and the score function.

2.1. Linguistic Term Set

Definition 1.

Reference [35]. Let $S=\{s_{-{\tau }_1},s_{-{\tau }_1+1},\dots ,s_0,\dots ,s_{{\tau }_2-1},s_{{\tau }_2}\}$ be an LTS, where ${\tau }_1$ and ${\tau }_2$ are two positive integers, ${\tau }_1+{\tau }_2+1$ represents the granularity of the LTS $S$, $s_0$ represents the central linguistic term, such as “medium” and “general”. If ${\tau }_1={\tau }_2$, then this LTS has the following properties:

• The LTS$S$is ordered:$s_i>s_j$if and only if$i>j$;
• There is a negation operator:$Neg(s_i)=s_{-i}$where$Neg(s_0)=s_0$.

If ${\tau }_1={\tau }_2$ and the semantic information is symmetrically and uniformly distributed, then $S$ is called a balanced LTS; otherwise, $S$ is called an unbalanced LTS. For example, an LTS with seven granularities is denoted as follows:

$$S=\left\{\begin{array}{l}{s}_{-3}=verypoor(VP),s_{-2}=poor(P),s_{-1}=slightlypoor(SP),s_0=medium(M),\\ s_1=slightlygood(SG),s_2=good(G),s_3=verygood(VG)\end{array}\right\}.$$

(1)

2.2. Group Generalized Linguistic Term Set

2.2.1. Euclidean Distance

Definition 2.

Reference [36]. Let ${\tilde{v}}_{\alpha }=(v_{\alpha }^L,v_{\alpha }^M,v_{\alpha }^U)$ and ${\tilde{v}}_{\beta }=(v_{\beta }^L,v_{\beta }^M,v_{\beta }^U)$ be any two positive triangular fuzzy numbers, then the Euclidean distance between them is defined as

$$d({\tilde{v}}_{\alpha },{\tilde{v}}_{\beta })=\sqrt{\frac{1}{3}\left({(v_{\alpha }^L-v_{\beta }^L)}^2+{(v_{\alpha }^M-v_{\beta }^M)}^2+{(v_{\alpha }^U-v_{\beta }^U)}^2\right)}.$$

(2)

2.2.2. Score Function

For comparing and ranking triangular fuzzy numbers, assume that $\tilde{v}=(v^L,v^M,v^U)$ is a positive triangular fuzzy number, then its score function could be denoted as [37]

$$S(\tilde{v})=\frac{v^L+2v^M+v^U}{4}.$$

(3)

2.2.3. Group Generalized Linguistic Term Set Based on Risk Preferences

To describe the influence of the evaluation group’s risk preference on the semantic information of linguistic terms in an LTS, Lin and Wang [25] introduced the concept of a group GLTS.

Definition 3.

Reference [25]. Let $S=\{s_{-{\tau }_1},s_{-{\tau }_1+1},\dots ,s_0,\dots ,s_{{\tau }_2-1},s_{{\tau }_2}\}$ be an LTS, then a group GLTS is denoted as follows:

$$\tilde{S}=\left\{\langle s_i,{\tilde{v}}_i\rangle |i=-{\tau }_1,\dots ,0,\dots ,{\tau }_2,{\tilde{v}}_i=(v_i^L,v_i^M,v_i^U)\right\},$$

(4)

where ${\tilde{v}}_i$ is a triangular fuzzy number with group risk preference parameters ${\theta }_1$ and ${\theta }_2$ $({\theta }_1,{\theta }_2>0)$, and represents a fuzzy semantic value of the linguistic term $s_i$ where $i=-{\tau }_1,-{\tau }_1+1,\dots ,{\tau }_2-1,{\tau }_2$. $v_i^L,v_i^M$ and $v_i^U$ are defined as

$$v_i^L=\{\begin{array}{ll}{\left(1+e^{-{\theta }_{2}i}\right)}^{-1},& i=-{\tau }_1\\ {\left(1+e^{-{\theta }_2(i-1)}\right)}^{-1},& i=-{\tau }_1+1,\dots ,-1,0\\ {\left(1+e^{-{\theta }_1(i-1)}\right)}^{-1},& i=1,2,\dots ,{\tau }_2\end{array}$$

(5)

$$v_i^M=\{\begin{array}{ll}{\left(1+e^{-{\theta }_{2}i}\right)}^{-1},& i=-{\tau }_1,\dots ,-1\\ 0.5,& i=0\\ {\left(1+e^{-{\theta }_{1}i}\right)}^{-1},& i=1,2,\dots ,{\tau }_2\end{array}$$

(6)

$$v_i^U=\{\begin{array}{ll}{\left(1+e^{-{\theta }_2(i+1)}\right)}^{-1},& i=-{\tau }_1,\dots ,-1\\ {\left(1+e^{-{\theta }_1(i+1)}\right)}^{-1},& i=0,1,\dots ,{\tau }_2-1\\ {\left(1+e^{-{\theta }_{1}i}\right)}^{-1},& i={\tau }_2\end{array}$$

(7)

Obviously, for any $i=-{\tau }_1,\dots ,0,\dots ,{\tau }_2$, we have $v_{-1}^U=v_0^M=v_1^L=0.5$ and $0<v_i^L,v_i^M,v_i^U<1$. If ${\theta }_1>{\theta }_2$, then the evaluation group tends to make a risk-seeking assessment. If ${\theta }_1^{}={\theta }_2^{}$, then the evaluation group tends to make a risk-neutral assessment. If ${\theta }_1<{\theta }_2$, then the evaluation group tends to make a risk-averse assessment.

As for the symmetry of semantic information in the group GLTS $\tilde{S}$, if ${\tau }_1={\tau }_2>1$ and ${\theta }_1={\theta }_2$, then $d({\tilde{v}}_{-i},0)=d({\tilde{v}}_i,0)$. This means that the semantic information $\tilde{S}$ is symmetrically distributed. Under this circumstance, if $d({\tilde{v}}_i,{\tilde{v}}_{i+1})\approx d({\tilde{v}}_{i-1},{\tilde{v}}_i)$ for $i=-{\tau }_1+1,\dots ,-1,0,1,\dots ,{\tau }_2-1$, then $\tilde{S}$ follows a symmetric and approximately uniform distribution. Besides, if ${\tau }_1={\tau }_2>1$ and ${\theta }_1\ne {\theta }_2$, then $d({\tilde{v}}_{-i},0)\ne d({\tilde{v}}_i,0)$. There also exists $i\in \{-{\tau }_1+1,\dots ,-1,0,1,\dots ,{\tau }_2-1\}$, so that $d({\tilde{v}}_i,{\tilde{v}}_{i+1})\ne d({\tilde{v}}_{i-1},{\tilde{v}}_i)$, which means that $\tilde{S}$ follows an asymmetric and non-uniform distribution.

2.2.4. Determination of Risk Preference Parameters

In order to determine the group GLTS defined above, it is assumed that ${\tilde{V}}^{(k)}=\left\{{\tilde{v}}_{o_{\alpha }}^{(k)},{\tilde{v}}_{o_{\beta }}^{(k)},\dots ,{\tilde{v}}_{o_{\gamma }}^{(k)}\right\}$ is an expected set of triangular fuzzy semantic values offered by the AE $e_k\in E(k=1,2,\dots ,q)$ according to linguistic terms in the LTS $S$. Then, an optimization model is constructed below:

$$\begin{array}{l}\min J={\displaystyle \sum _{k=1}^q{\lambda }_k\left(d({\tilde{v}}_{o_{\alpha }},{\tilde{v}}_{o_{\alpha }}^{(k)})+d({\tilde{v}}_{o_{\beta }},{\tilde{v}}_{o_{\beta }}^{(k)})+\dots +d({\tilde{v}}_{o_{\gamma }},{\tilde{v}}_{o_{\gamma }}^{(k)})\right)}\\ s.t.{\theta }_1>0,{\theta }_2>0\end{array}$$

(8)

where $o_{\alpha },o_{\beta },\dots ,o_{\gamma }\in \{-{\tau }_1,\dots ,-{\tau }_2\}$, ${\theta }_1$ and ${\theta }_2$ are decision variables, and $d(.,.)$ denotes the Euclidean distance defined by (2).

By solving Equation (8), the values of the group risk preference parameters ${\theta }_1$ and ${\theta }_2$ are obtained. Then by plugging them into Equations (4)–(7), the group GLTS is derived.

3. EIS of Online Teaching Quality

3.1. Selection of Indicators for Evaluating the Online Teaching Quality

The selection of appropriate indicators is the fundamental to conducting the evaluation. In order to effectively identify the critical factors affecting the quality of teaching, the literature analysis and Delphi methods were used to construct indicators for evaluating the online teaching quality.

First, analysis of the existing literature reveals that in the traditional evaluation of teaching quality, scholars have focused on indicators of content, teaching methods, and effectiveness. Similarly, these indicators also could be selected for assessing online courses.

Then, through interviews with six teachers and 14 students who participated in online teaching, it was found that all interviewees focused on the classroom atmosphere in the online education process. Besides, the online teaching environment and the teaching platform were also key factors mentioned by the interviewees. Based on literature and interview research, this paper identified five attributes of the online teaching environment, online teaching platform, online teaching content and methods, online classroom climate, and online teaching effectiveness. These attributes were then decomposed to constitute 15 sub-attributes. Based on the literature study and interview research, this paper identified five primary indicators of online teaching environment, online teaching platform, online teaching content and methods, online classroom atmosphere, and online teaching effectiveness. Then, the primary indicators were decomposed to formulate 15 secondary indicators. Finally, the five primary indicators were treated as attributes $a_j(j=1,2,\dots ,5)$ and the secondary indicators were treated as sub-attributes $a_{jl}(j=1,2,\dots ,5;l=1,2,3)$. Sub-attributes represents a specific refinement of the attribute. In other words, the five attributes in this multi-level EIS contained 15 sub-attributes. The multi-level EIS is shown in

Table 1. The multi-level evaluation indicator system (EIS) of online teaching quality.

Attribute	Sub-Attribute
Online teaching environment ($a_1$)	Network environment ($a_{11}$)
	Hardware equipment ($a_{12}$)
	Physical environment ($a_{13}$)
Online teaching platform ($a_2$)	Ease of operation ($a_{21}$)
	Functionality ($a_{22}$)
	Personalized support ($a_{23}$)
Online teaching content and methods ($a_3$)	Teaching content ($a_{31}$)
	Teaching schedule ($a_{32}$)
	Teaching format ($a_{33}$)
Online classroom atmosphere ($a_4$)	Influence of teaching ($a_{41}$)
	Classroom atmosphere ($a_{42}$)
	Students’ attitudes to learning ($a_{43}$)
Online teaching effectiveness ($a_5$)	Level of knowledge acquired ($a_{51}$)
	Level of thought-provoking ($a_{52}$)
	Problem-solving skills ($a_{53}$)

below.

The attributes of the online teaching environment mainly refer to the external environmental requirements that support online education, including the network, hardware equipment, and teaching environment. The network environment requires the instructor to have a smooth network to ensure video and audio transmission when conducting instruction. Hardware equipment mainly refers to the teacher having the equipment to conduct online instruction, including computers, microphones, and cameras. Additionally, the physical environment primarily refers to space where classes are conducted without outside interference.

The online teaching platform requires that the online teaching system used by teachers and students be stable and easy to use. Personalized support requires the system to be able to sign in, take attendance, interact, and submit assignments. Different from the traditional teaching mode, the online platform plays the role of a “classroom” connecting students and teachers. Therefore, the stability of the online platform and personalized support functions are vital guarantees for the effective implementation of online teaching.

Regarding the content and methodological attributes of online teaching, teachers are required to teach comprehensive content and to highlight key points. The combination of theory and practice is also very crucial. Secondly, the teaching time is arranged in such a way that students can easily absorb and digest the course knowledge. It is also necessary to use a variety of teaching formats, such as a combination of live and recorded lectures, monologues and inspirational lectures. Unlike the traditional classroom teaching quality assessment, the content and methods for online teaching put forward higher requirements for universities and teachers. Specifically, teachers need to adjust teaching strategies, improve teaching methods, and optimize instructional design according to the needs of the scenario in order to compensate for the lack of presence and interaction resulting from online teaching.

The online classroom atmosphere contains the evaluation of teachers and students. Among them, the influence of teaching requires teachers to be full of energy, affable, use body language reasonably and appropriately, and teach in a natural and generous manner. The sub-attribute of student’s attitude requires students to have a right attitude towards learning and to pay attention to the class. Besides, the interaction and discussion between teacher and students is also an important basis for assessing the classroom atmosphere. This can be interpreted as a lively classroom atmosphere reflects the positive interaction between the teacher and the students and also makes the teacher more enthusiastic in teaching and helps the students to listen attentively to the lesson.

The teaching effectiveness is the reflection of the quality of instruction and can be used to measure if the teaching objectives are being met. This attribute includes if students master the content of the course in a timely and effective manner and whether the lectures inspire students to think. In addition, fostering creativity and improving problem-solving skills are also incorporated into this attribute.

Based on the above-detailed description of the evaluation indicators, for the sake of more intuitively understanding the MAGDM problems of online teaching quality evaluation, a multi-level hierarchic structure of online teaching quality evaluation composed of an objective, attributes, sub-attributes, and alternatives (teachers) is shown in Figure 1.

3.2. Determination of the Weights of Evaluation Indicators

Since this paper adopted a MAGDM approach to assess the online teaching quality of teachers, the determination of attribute weights was particularly vital. Obviously, the importance weights of different sub-attributes to the same attribute will also be different. Due to the limited knowledge of AEs and the complexity of evaluation problems, it may be difficult for them to give the weights of attributes and sub-attributes, that is, the weights of attributes and sub-attributes are completely unknown. It is supposed that $w={(w_1,w_2\dots ,w_m)}^T$ represents the attribute weight vector containing $m$ attributes. $w_j={(w_{j1},w_{j2},\dots ,w_{jp})}^T$ means the sub-attribute weight vector, which contains $p$ sub-attributes under the attribute $a_j$. ${\displaystyle \sum _{j=1}^m{w}_j}=1$ and $w_j\in [0,1]$, ${\displaystyle \sum _{l=1}^p{w}_{jl}}=1$ and $w_{jl}\in [0,1]$.

When the GLTS $\tilde{S}$ of the evaluation group is determined, the AE $e_k$ gives a linguistic-term-based decision matrix $D^{jk}={(d_{s_{il}}^{jk})}_{n\times p}$ of the alternative $x_i$ over the sub-attribute $a_{jl}$ under the attribute $a_j$ where $i=1,2,\dots ,n$, $l=1,2,\dots ,p$, $j=1,2,\dots ,m$, $k=1,2,\dots ,q$. Subsequently, each linguistic-term-based decision matrix $D^{jk}={(d_{s_{il}}^{jk})}_{n\times p}$ can be converted into a triangular fuzzy decision matrix expressed by

$${\tilde{R}}^{jk}={({\tilde{r}}_{il}^{jk})}_{n\times p}=(r_{il}^{jkL},r_{il}^{jkM},r_{il}^{jkU}{)}_{n\times p},j=1,2,\dots ,m;k=1,2,\dots ,q,$$

(9)

where

$${\tilde{r}}_{il}^{jk}={\tilde{v}}_{u_{il}}^{jk}=(v_{u_{il}}^{jkL},v_{u_{il}}^{jkM},v_{u_{il}}^{jkU}),i=1,2,\dots ,n;l=1,2,\dots ,p;j=1,2,\dots ,m;k=1,2,\dots ,q.$$

(10)

Based upon triangular fuzzy decision matrices ${\tilde{R}}^{jk}(j=1,2,\dots ,m;k=1,2,\dots ,q)$, assuming that the importance weight vector of AEs is $\omega ={({\omega }_1,{\omega }_2,\dots ,{\omega }_q)}^T$, ${\omega }_k\in \left[0,1\right]$ and ${\displaystyle \sum _{k=1}^q{\omega }_k=1}$. Then a triangular fuzzy weighted average operator and the AEs’ weight vector $\omega ={({\omega }_1,{\omega }_2,\dots ,{\omega }_q)}^T$ are utilized to aggregate individual triangular fuzzy decision matrices related to the attribute $a_j$ into a group one as follows:

$${\tilde{R}}^j={({\tilde{r}}_{il}^j)}_{n\times p},j=1,2,\dots ,m,$$

(11)

where

$${\tilde{r}}_{il}^j={\displaystyle \sum _{k=1}^q{\omega }_k{\tilde{r}}_{il}^{jk}}=\left({\displaystyle \sum _{k=1}^q{\omega }_k}{v}_{u_{il}}^{jkL},{\displaystyle \sum _{k=1}^q{\omega }_k}{v}_{u_{il}}^{jkM},{\displaystyle \sum _{k=1}^q{\omega }_k}{v}_{u_{il}}^{jkU}\right).$$

(12)

Next, combining the sub-attribute weight vector $w_j={(w_{j1},w_{j2},\dots ,w_{jp})}^T$ and applying a triangular fuzzy weighted average operator, the triangular fuzzy evaluation values in the elements of the $ith$ row in ${\tilde{R}}^j$ can be aggregated into a group comprehensive evaluation value for the alternative $x_i(i=1,2,\dots ,n)$ over the attribute $a_j$, denoted by ${\tilde{h}}_i^j$, namely:

$$\begin{array}{l}{\tilde{h}}_i^j={\displaystyle \sum _{l=1}^p{w}_{jl}{\tilde{r}}_{il}^j}=\left({\displaystyle \sum _{l=1}^p{w}_{jl}{r}_{il}^{jL}},{\displaystyle \sum _{l=1}^p{w}_{jl}{r}_{il}^{jM}},{\displaystyle \sum _{l=1}^p{w}_{jl}{r}_{il}^{jU}}\right)\\ =\left({\displaystyle \sum _{l=1}^p\left(w_{jl}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkL}}\right),{\displaystyle \sum _{l=1}^p\left(w_{jl}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkM}}\right),{\displaystyle \sum _{l=1}^p\left(w_{jl}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkU}}\right)}}}\right)\end{array}$$

(13)

Once ${\tilde{h}}_i^j$ is obtained, according to the attribute weight vector $w={(w_1,w_2\dots ,w_m)}^T$ and the triangular fuzzy weighted average operator, the group comprehensive evaluation value ${\tilde{h}}_i$ of the alternative $x_i$ can be determined, that is

$${\tilde{h}}_i={\displaystyle \sum _{j=1}^m{w}_j{\tilde{h}}_i^j}={\displaystyle \sum _{j=1}^m\left(w_j{\displaystyle \sum _{l=1}^p{w}_{jl}^{}{\tilde{r}}_{il}^j}\right)},i=1,2,\dots ,n.$$

(14)

Since both $w={(w_1,w_2\dots ,w_m)}^T$ and $w_j={(w_{j1},w_{j2},\dots ,w_{jp})}^T(j=1,2,\dots ,m)$ are completely unknown in this paper, thus, a two-stage optimization model is constructed to determine the weights of attributes and sub-attributes, shown as follows:

Stage 1:

As mentioned in Equation (13), ${\tilde{h}}_i^j$ represents the group comprehensive evaluation value for the alternative $x_i$ with respect to the attribute $a_j$, the larger ${\tilde{h}}_i^j$ represents the better the alternative $x_i$ with regard to the attribute $a_j$, which leads to a better alternative $x_i$. Therefore, based on Equation (3), we first need to find the sub-attribute weight vector $w_j={(w_{j1},w_{j2},\dots ,w_{jp})}^T$ $(j=1,2,\dots ,m)$, so that $S({\tilde{h}}_i^j)$ is maximized for the alternative $x_i$ with respect to the attribute $a_j$ for all $i=1,2,\dots ,n$, $j=1,2,\dots ,m$. Thus, a multi-objective optimization model in this stage is constructed as follows:

$$\begin{array}{l}\max F_{x_i}^j=S({\tilde{h}}_i^j)\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}i=1,2,\dots ,n;j=1,2,\dots ,m\\ s.t.\{\begin{array}{c}{w}_{jl}\ge 0,l=1,2,\dots ,p\\ {\displaystyle \sum _{l=1}^p{w}_{jl}^2}=1.\end{array}\end{array}$$

(15)

As for the attribute $a_j(j=1,2,\dots ,m)$, each alternative $x_i(i=1,2,\dots ,n)$ is feasible and non-inferior, and the optimization model of each alternative has the same constraint conditions. Therefore, the aforementioned multi-objective optimization model can be transformed into an aggregated optimization model by setting the same sub-attribute weights for each alternative under the same attribute as follows:

$$\begin{array}{l}\max F^j=\frac{1}{n}{\displaystyle \sum _{i=1}^{n}S({\tilde{h}}_i^j)}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}j=1,2,\dots ,m\\ s.t.\{\begin{array}{c}{w}_{jl}\ge 0,l=1,2,\dots ,p\\ {\displaystyle \sum _{l=1}^p{w}_{jl}^2}=1.\end{array}\end{array}$$

(16)

Based on Equations (3) and (13), the optimization model (16) can be equivalently converted into

$$\begin{array}{l}\max F^j=\frac{1}{4n}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{l=1}^p\left(r_{il}^{jL}+2r_{il}^{jM}+r_{il}^{jU}\right)w_{jl}}}\hspace{1em}\hspace{1em}\hspace{1em}j=1,2,\dots ,m\\ s.t.\{\begin{array}{c}{w}_{jl}\ge 0,l=1,2,\dots ,p\\ {\displaystyle \sum _{l=1}^p{w}_{jl}^2}=1.\end{array}\end{array}$$

(17)

Solving the above optimization model (17) yields the optimal sub-attribute weight vector $w_j^{*}={(w_{j1}^{\ast },w_{j2}^{\ast },\dots ,w_{jp}^{\ast })}^T$ related to the attribute $w_j(j=1,2,\dots ,m)$. The optimal solution of the model can be expressed as

$$w_{jl}^{\ast }=\frac{{\displaystyle \sum _{i=1}^n(r_{il}^{jL}+2r_{il}^{jM}+r_{il}^{jU})}}{\sqrt{{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{l=1}^p{(r_{il}^{jL}+2r_{il}^{jM}+r_{il}^{jU})}^2}}}}.$$

(18)

Since the weight vector satisfies the constraint condition of unitization, it is also necessary to normalize $w_j^{*}$, namely:

$${\overline{w}}_{jl}^{*}=\frac{w_{jl}^{*}}{{\displaystyle \sum _{l=1}^p{w}_{jl}^{\ast }}}=\frac{{\displaystyle \sum _{i=1}^n(r_{il}^{jL}+2r_{il}^{jM}+r_{il}^{jU})}}{{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{l=1}^p(r_{il}^{jL}+2r_{il}^{jM}+r_{il}^{jU})}}},j=1,2,\dots ,m;l=1,2,\dots ,p.$$

(19)

The optimal sub-attribute weight vector ${\overline{w}}_j^{*}={({\overline{w}}_{j1}^{\ast },{\overline{w}}_{j2}^{\ast },\dots ,{\overline{w}}_{jp}^{\ast })}^T$ after normalization is obtained. Then, substituting ${\overline{w}}_j^{*}$ into Equation (13), the optimal group comprehensive evaluation value ${\tilde{h}}_i^{j*}$ of the alternative $x_i$ over the attribute $w_j$ is obtained by

$$\begin{array}{l}{\tilde{h}}_i^{j*}={\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{\tilde{r}}_{il}^j}=\left({\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{r}_{il}^{jL}},{\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{r}_{il}^{jM}},{\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{r}_{il}^{jU}}\right)\\ =\left({\displaystyle \sum _{l=1}^p\left({\overline{w}}_{jl}^{*}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkL}}\right),{\displaystyle \sum _{l=1}^p\left({\overline{w}}_{jl}^{*}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkM}}\right),{\displaystyle \sum _{l=1}^p\left({\overline{w}}_{jl}^{*}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkU}}\right)}}}\right)\end{array}$$

(20)

Stage 2:

Once ${\tilde{h}}_i^{j*}$ is determined, the larger the triangular fuzzy evaluation value ${\tilde{h}}_i$, the better the alternative $x_i$ is. Similarly, based on Equation (3), we need to derive the attribute weight vector $w={(w_1,w_2\dots ,w_m)}^T$ so that for any $i=1,2,\dots ,n$, $S({\tilde{h}}_i^{})$ is maximized. The following multi-objective optimization model of the second stage is constructed as

$$\begin{array}{l}\max F_{x_i}^{}=S({\tilde{h}}_i^{})\hspace{1em}\hspace{1em}\hspace{1em}i=1,2,\dots ,n\\ s.t.\{\begin{array}{c}{w}_j\ge 0,j=1,2,\dots ,m\\ {\displaystyle \sum _{j=1}^m{w}_j^2}=1.\end{array}\end{array}$$

(21)

In the same way, the above multi-objective optimization model (21) is converted into an aggregated optimization model:

$$\begin{array}{l}\max F=\frac{1}{n}{\displaystyle \sum _{i=1}^{n}S({\tilde{h}}_i^{})}\\ s.t.\{\begin{array}{l}{w}_j\ge 0,j=1,2,\dots ,m\\ {\displaystyle \sum _{j=1}^m{w}_j^2}=1.\end{array}\end{array}$$

(22)

Based upon the score function of triangular fuzzy numbers, we substitute ${\tilde{h}}_i^{j*}$ into Equation (14), and the optimization model (22) can be equivalently converted into:

$$\begin{array}{l}\max F=\frac{1}{4n}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\left(h_i^{j*L}+2h_i^{j*M}+h_i^{j*U}\right)w_j}}\\ s.t.\{\begin{array}{l}{w}_j\ge 0,j=1,2,\dots ,m\\ {\displaystyle \sum _{j=1}^m{w}_j^2}=1.\end{array}\end{array}$$

(23)

Solving the optimization model (23) yields the optimal attribute weight vector $w^{*}={(w_1^{*},w_2^{*}\dots ,w_m^{*})}^T$. Then, utilize Equation (24) to normalize $w_j^{*}(j=1,2,\dots ,m)$, the normalized attribute weight vector ${\overline{w}}^{*}={({\overline{w}}_1^{*},{\overline{w}}_2^{*}\dots ,{\overline{w}}_m^{*})}^T$ is derived by

$${\overline{w}}_j^{*}=\frac{w_j^{*}}{{\displaystyle \sum _{j=1}^m{w}_j^{\ast }}}=\frac{{\displaystyle \sum _{i=1}^n(h_i^{j*L}+2h_i^{j*M}+h_i^{j*U})}}{{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m(h_i^{j*L}+2h_i^{j*M}+h_i^{j*U})}}},j=1,2,\dots ,m.$$

(24)

Subsequently, substituting ${\overline{w}}^{*}$ into Equation (14), and at the same time, according to Equation (20), the group comprehensive evaluation value of the alternative $x_i(i=1,2,\dots ,n)$ is calculated as

$$\begin{array}{l}{\tilde{h}}_i^{*}={\displaystyle \sum _{j=1}^m{\overline{w}}_j^{*}{\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{\tilde{r}}_{il}^j}}=\left({\displaystyle \sum _{j=1}^m{\overline{w}}_j^{*}}{\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{r}_{il}^{jL}},{\displaystyle \sum _{j=1}^m{\overline{w}}_j^{*}}{\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{r}_{il}^{jM}},{\displaystyle \sum _{j=1}^m{\overline{w}}_j^{*}}{\displaystyle \sum _{l=1}^p{\overline{w}}_{jl}^{*}{r}_{il}^{jU}}\right)\\ =\left({\displaystyle \sum _{j=1}^m\left({\overline{w}}_j^{*}{\displaystyle \sum _{l=1}^p\left({\overline{w}}_{jl}^{*}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkL}}\right)}\right)},{\displaystyle \sum _{j=1}^m\left({\overline{w}}_j^{*}{\displaystyle \sum _{l=1}^p\left({\overline{w}}_{jl}^{*}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkM}}\right)}\right),{\displaystyle \sum _{j=1}^m\left({\overline{w}}_j^{*}{\displaystyle \sum _{l=1}^p\left({\overline{w}}_{jl}^{*}{\displaystyle \sum _{k=1}^q{\omega }_k{v}_{u_{il}}^{jkU}}\right)}\right)}}\right)\end{array}$$

(25)

As the summary of this section, the brief flow of the two-stage optimization model used to solve the weights of attributes and sub-attributes is shown in Figure 2.

4. A Method for Linguistic MAGDM Involving Risk Preferences and Completely Unknown Weight Information

Taking into account linguistic MAGDM problems involving decision makers’ risk preferences, let $X=\{x_1,x_2,\dots ,x_n\}$ be a set of alternatives, $A=\{a_1,a_2,\dots ,a_m\}$ be a set of attributes, and $a_j=\{a_{j1},a_{j2},\dots ,a_{jp}\}(j=1,2,\dots ,m)$ be a set of sub-attributes related to the attribute $a_j$. The weights of attributes and sub-attributes are completely unknown. Suppose that $q$ AEs represented by $E=\{e_1,e_2,\dots ,e_q\}$ assess the alternative $x_i\in X$ with regard to the sub-attribute $a_{jl}\in a_j$ to the attribute $a_j\in A$ according to the LTS $S$ defined in Definition 1. Then they offer linguistic-term-based decision matrices as $D^{jk}={(d_{s_{il}}^{jk})}_{n\times p}(j=1,2,\dots ,m;k=1,2,\dots ,q)$, where $d_{s_{il}}^{jk}\in S$ and the weights of AEs are known. Based on the risk preferences of AEs, a method for solving linguistic MAGDM problems with the completely unknown weights of attributes and sub-attributes is provided. The detailed evaluation process is described as follows:

Preprocessing:

Step 1: Based on the LTS $S$, each AE $e_k\in E$ assesses the alternative $x_i\in X$ with regard to the sub-attribute $a_{jl}\in a_j$ relative to the attribute $a_j\in A$. The evaluation process could be expressed by linguistic-term-based decision matrices $D^{jk}={(d_{s_{il}}^{jk})}_{n\times p}(j=1,2,\dots ,m;k=1,2,\dots ,q)$. Then, each AE provides his or her respective set of expected triangular fuzzy semantic values ${\tilde{V}}^{(k)}=\left\{{\tilde{v}}_{o_{\alpha }}^{(k)},{\tilde{v}}_{o_{\beta }}^{(k)},\dots ,{\tilde{v}}_{o_{\gamma }}^{(k)}\right\}$.

Step 2: Compute the group risk preference parameters ${\theta }_1$ and ${\theta }_2$ by solving the optimization model (8), and substitute them into Equations (4)–(7) to obtain the group GLTS $\tilde{S}$.

Step 3: Transform decision matrices $D^{jk}={(d_{s_{il}}^{jk})}_{n\times p}(j=1,2,\dots ,m;k=1,2,\dots ,q)$ into triangular fuzzy decision matrices ${\tilde{R}}^{jk}={\left({\tilde{r}}_{il}^{jk}\right)}_{n\times p}(j=1,2,\dots ,m;k=1,2,\dots ,q)$ according to Equations (9) and (10).

Step 4: Utilize the triangular fuzzy weighted average operator to aggregate individual triangular fuzzy decision matrices ${\tilde{R}}^{jk}={\left({\tilde{r}}_{il}^{jk}\right)}_{n\times p}(j=1,2,\dots ,m;k=1,2,\dots ,q)$ into group triangular fuzzy decision matrices ${\tilde{R}}^j={({\tilde{r}}_{il}^j)}_{n\times p}(j=1,2,\dots ,m)$ according to Equation (12).

Stage 1:

Step 5: Determine the normalized optimal sub-attribute weight vector ${\overline{w}}_j^{*}={({\overline{w}}_{j1}^{\ast },{\overline{w}}_{j2}^{\ast },\dots ,{\overline{w}}_{jp}^{\ast })}^T$ by solving the nonlinear programming model (17) and using Equation (19) for normalization.

Step 6: Compute the optimal group comprehensive evaluation value ${\tilde{h}}_i^{j*}$ $(i=1,2,\dots ,n;j=1,2,\dots ,m)$ for the alternative $x_i$ over the attribute $a_j$ according to Equation (20).

Step 7: Make use of Equation (3) to calculate scores $S({\tilde{h}}_i^{j*})(i=1,2,\dots ,n;j=1,2,\dots ,m)$.

Step 8: Obtain the ranking of all the alternatives over each attribute in descending order according to the size of scores $S({\tilde{h}}_i^{j*})(i=1,2,\dots ,n;j=1,2,\dots ,m)$, and use $x_{i_{\alpha }}\stackrel{a_j}{\succ }{x}_{i_{\beta }}$ to indicate that $x_{i_{\alpha }}$ is superior to $x_{i_{\beta }}$ with respect to the attribute $a_j$.

Stage 2:

Step 9: Get the optimal attribute weight vector $w^{*}={(w_1^{*},w_2^{*}\dots ,w_m^{*})}^T$ by solving the nonlinear programming model (23). Then employ Equation (24) to derive the normalized optimal one denoted by ${\overline{w}}^{*}={({\overline{w}}_1^{*},{\overline{w}}_2^{*}\dots ,{\overline{w}}_m^{*})}^T$.

Step 10: Calculate the optimal group comprehensive evaluation value ${\tilde{h}}_i^{*}(i=1,2,\dots ,n)$ of each alternative in $X$ by making use of Equation (25).

Step 11: Generate scores $S({\tilde{h}}_i^{*})(i=1,2,\dots ,n)$ of the optimal group comprehensive evaluation value for each alternative based on Equation (3).

Step 12: Assess and rank the alternatives $x_i(i=1,2,\dots ,n)$ in descending order according to the size of scores $S({\tilde{h}}_i^{*})(i=1,2,\dots ,n)$, and select the best alternative. Then, make use of $x_{i_{\alpha }}\stackrel{}{\succ }{x}_{i_{\beta }}$ to express that $x_{i_{\alpha }}$ is preferred to $x_{i_{\beta }}$, where $i_{\alpha },i_{\beta }\in \{1,2,\dots ,n\}$.

Based on the detailed description of the above evaluation process, an extended framework of online teaching quality evaluation is devised in Figure 3.

5. A Case Study about Online Teaching Quality Evaluation

This section aims to provide a case to clarify the feasibility and effectiveness of the method and models presented in this paper.

5.1. Problem Description

During the outbreak of COVID-19, almost all universities across the countries postponed the opening of their classes. In order not to delay the progress of classroom teaching, a certain university requires teachers to teach students courses through online teaching. Because the online teaching method is relatively novel and unfamiliar to teachers, it is very crucial to control the quality of online teaching. For the sake of improving the quality of online teaching in this university, three AEs $e_k(k=1,2,3)$ are invited to assess the online teaching quality of the four excellent teachers $x_i(i=1,2,3,4)$ selected by the university. Then the best teacher is chosen to share his or her experience to improve the overall quality of online teaching in this university.

5.2. Evaluation of Online Teaching Quality among Four Teachers

Preprocessing:

Based upon the seven granular LTS $S$ presented in Definition 1 and the EIS of online teaching quality constructed in Table 1, the linguistic-term-based decision matrices $D^{jk}={(d_{s_{il}}^{jk})}_{4\times 3}(j=1,2,3,4,5;k=1,2,3)$ are given by experts who assess four teachers with regard to fifteen sub-attributes by using linguistic terms in $S$. The importance weight vector of experts are $\omega ={(0.35,0.25,0.4)}^T$. The original linguistic-term-based decision matrices they provide are shown as follows:

$$\begin{gathered} D^{11}=\left(\begin{array}{ccc}G& SG& M\\ SP& M& G\\ G& G& P\\ VG& SP& SG\end{array}\right),D^{12}=\left(\begin{array}{ccc}VG& G& P\\ SG& P& G\\ SP& SG& SG\\ M& P& G\end{array}\right),D^{13}=\left(\begin{array}{ccc}SP& SG& M\\ G& SG& G\\ M& G& G\\ P& SG& VG\end{array}\right), \\ {D}^{21}=\left(\begin{array}{ccc}SG& P& P\\ G& G& SG\\ P& SP& VG\\ SG& SG& G\end{array}\right),D^{22}=\left(\begin{array}{ccc}VG& VG& M\\ P& VG& G\\ SP& SG& P\\ P& G& SG\end{array}\right),D^{23}=\left(\begin{array}{ccc}SG& G& G\\ P& G& M\\ SG& SG& G\\ SP& M& SP\end{array}\right), \\ {D}^{31}=\left(\begin{array}{ccc}M& SG& P\\ M& M& SG\\ G& SG& VG\\ G& G& G\end{array}\right),D^{32}=\left(\begin{array}{ccc}P& G& P\\ SG& SG& SG\\ M& VP& VG\\ G& VG& G\end{array}\right),D^{33}=\left(\begin{array}{ccc}SG& G& VG\\ VP& M& SG\\ M& SP& G\\ G& M& SG\end{array}\right), \\ {D}^{41}=\left(\begin{array}{ccc}G& SP& VG\\ M& M& SG\\ SP& P& SG\\ SP& P& G\end{array}\right),D^{42}=\left(\begin{array}{ccc}SG& P& G\\ SP& SG& SG\\ G& VP& SG\\ SG& SP& G\end{array}\right),D^{43}=\left(\begin{array}{ccc}P& P& SG\\ G& SG& SG\\ G& VG& SG\\ M& SP& SP\end{array}\right), \\ {D}^{51}=\left(\begin{array}{ccc}G& P& G\\ G& M& G\\ P& SP& SG\\ SP& SP& SG\end{array}\right),D^{52}=\left(\begin{array}{ccc}SG& VG& SG\\ M& M& G\\ P& VG& G\\ P& P& VG\end{array}\right),D^{53}=\left(\begin{array}{ccc}SP& SP& G\\ SP& P& M\\ SP& VG& G\\ P& G& VG\end{array}\right). \end{gathered}$$

Then, three experts $e_k(k=1,2,3)$ give their own sets of triangular fuzzy semantic values they expected, expressed as follows:

$$\begin{array}{l}{\tilde{V}}^{(1)}=\left\{{\tilde{v}}_{-2}^{(1)}=(0.2,0.3,0.35),{\tilde{v}}_1^{(1)}=(0.55,0.6,0.7),{\tilde{v}}_2^{(1)}=(0.6,0.7,0.85)\right\},\\ {\tilde{V}}^{(2)}=\left\{{\tilde{v}}_{-3}^{(2)}=(0.15,0.15,0.25),{\tilde{v}}_0^{(2)}=(0.5,0.6,0.75),{\tilde{v}}_2^{(2)}=(0.75,0.8,0.85)\right\},\\ {\tilde{V}}^{(3)}=\left\{\begin{array}{l}{\tilde{v}}_{-3}^{(3)}=(0.15,0.15,0.25),{\tilde{v}}_{-2}^{(3)}=(0.15,0.25,0.35),{\tilde{v}}_{-1}^{(3)}=(0.25,0.35,0.45),\\ {\tilde{v}}_0^{(3)}=(0.35,0.45,0.55),{\tilde{v}}_1^{(3)}=(0.45,0.55,0.65),{\tilde{v}}_2^{(3)}=(0.55,0.65,0.75),\\ {\tilde{v}}_3^{(3)}=(0.65,0.75,0.75)\end{array}\right\}.\end{array}$$

As the importance weights of the three experts are 0.35, 0.25, and 0.4, respectively, solving the optimization model (8) yields the group risk preference parameters ${\theta }_1=0.3644,{\theta }_2=0.5664$. Apparently, the group of AEs tend to make a risk-averse decision. Substituting the group risk preference parameters into Equations (4)–(7), the GLTS with seven granularities can be obtained as follows:

$$\tilde{S}=\left\{\begin{array}{l}\langle s_{-3},(0.1546,0.1546,0.2436)\rangle ,\langle s_{-2},(0.1546,0.2436,0.3621)\rangle ,\\ \langle s_{-1},(0.2436,0.3621,0.5000)\rangle ,\langle s_0,(0.3621,0.5000,0.5901)\rangle ,\\ \langle s_1,(0.5000,0.5901,0.6745)\rangle ,\langle s_2,(0.5901,0.6745,0.7490)\rangle ,\\ \langle s_3,(0.6745,0.7490,0.7490)\rangle \end{array}\right\}.$$

According to Equations (9) and (10), the above-mentioned linguistic-term-based decision matrices $D^{jk}$$(j=1,2,3,4,5;k=1,2,3)$ are transformed into triangular fuzzy decision matrices ${\tilde{R}}^{jk}(j=1,2,3,4,5;k=1,2,3)$ as follows:

$$\begin{gathered} {\tilde{R}}^{11}=\left(\begin{array}{ccc}(0.5901,0.6745,0.7490)& (0.5000,0.5901,0.6745)& (0.3621,0.5000,0.5901)\\ (0.2436,0.3621,0.5000)& (0.3621,0.5000,0.5901)& (0.5901,0.6745,0.7490)\\ (0.5901,0.6745,0.7490)& (0.5901,0.6745,0.7490)& (0.1546,0.2436,0.3621)\\ (0.6745,0.7490,0.7490)& (0.2436,0.3621,0.5000)& (0.5000,0.5901,0.6745)\end{array}\right), \\ {\tilde{R}}^{12}=\left(\begin{array}{ccc}(0.6745,0.7490,0.7490)& (0.5901,0.6745,0.7490)& (0.1546,0.2436,0.3621)\\ (0.5000,0.5901,0.6745)& (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)\\ (0.2436,0.3621,0.5000)& (0.5000,0.5901,0.6745)& (0.5000,0.5901,0.6745)\\ (0.3621,0.5000,0.5901)& (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)\end{array}\right), \\ {\tilde{R}}^{13}=\left(\begin{array}{ccc}(0.2436,0.3621,0.5000)& (0.5000,0.5901,0.6745)& (0.3621,0.5000,0.5901)\\ (0.5901,0.6745,0.7490)& (0.5000,0.5901,0.6745)& (0.5901,0.6745,0.7490)\\ (0.3621,0.5000,0.5901)& (0.5901,0.6745,0.7490)& (0.5901,0.6745,0.7490)\\ (0.1546,0.2436,0.3621)& (0.5000,0.5901,0.6745)& (0.6745,0.7490,0.7490)\end{array}\right), \\ {\tilde{R}}^{21}=\left(\begin{array}{ccc}(0.5000,0.5901,0.6745)& (0.1546,0.2436,0.3621)& (0.1546,0.2436,0.3621)\\ (0.5901,0.6745,0.7490)& (0.5901,0.6745,0.7490)& (0.5000,0.5901,0.6745)\\ (0.1546,0.2436,0.3621)& (0.2436,0.3621,0.5000)& (0.6745,0.7490,0.7490)\\ (0.5000,0.5901,0.6745)& (0.5000,0.5901,0.6745)& (0.5901,0.6745,0.7490)\end{array}\right), \\ {\tilde{R}}^{22}=\left(\begin{array}{ccc}(0.6745,0.7490,0.7490)& (0.6745,0.7490,0.7490)& (0.3621,0.5000,0.5901)\\ (0.1546,0.2436,0.3621)& (0.6745,0.7490,0.7490)& (0.5901,0.6745,0.7490)\\ (0.2436,0.3621,0.5000)& (0.5000,0.5901,0.6745)& (0.1546,0.2436,0.3621)\\ (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)& (0.5000,0.5901,0.6745)\end{array}\right), \\ {\tilde{R}}^{23}=\left(\begin{array}{ccc}(0.5000,0.5901,0.6745)& (0.5901,0.6745,0.7490)& (0.5901,0.6745,0.7490)\\ (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)& (0.3621,0.5000,0.5901)\\ (0.5000,0.5901,0.6745)& (0.5000,0.5901,0.6745)& (0.5901,0.6745,0.7490)\\ (0.2436,0.3621,0.5000)& (0.3621,0.5000,0.5901)& (0.2436,0.3621,0.5000)\end{array}\right), \\ {\tilde{R}}^{31}=\left(\begin{array}{ccc}(0.3621,0.5000,0.5901)& (0.5000,0.5901,0.6745)& (0.1546,0.2436,0.3621)\\ (0.3621,0.5000,0.5901)& (0.3621,0.5000,0.5901)& (0.5000,0.5901,0.6745)\\ (0.5901,0.6745,0.7490)& (0.5000,0.5901,0.6745)& (0.6745,0.7490,0.7490)\\ (0.5901,0.6745,0.7490)& (0.5901,0.6745,0.7490)& (0.5901,0.6745,0.7490)\end{array}\right), \\ {\tilde{R}}^{32}=\left(\begin{array}{ccc}(0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)& (0.1546,0.2436,0.3621)\\ (0.5000,0.5901,0.6745)& (0.5000,0.5901,0.6745)& (0.5000,0.5901,0.6745)\\ (0.3621,0.5000,0.5901)& (0.1546,0.1546,0.2436)& (0.6745,0.7490,0.7490)\\ (0.5901,0.6745,0.7490)& (0.6745,0.7490,0.7490)& (0.5901,0.6745,0.7490)\end{array}\right), \\ {\tilde{R}}^{33}=\left(\begin{array}{ccc}(0.5000,0.5901,0.6745)& (0.5901,0.6745,0.7490)& (0.6745,0.7490,0.7490)\\ (0.1546,0.1546,0.2436)& (0.3621,0.5000,0.5901)& (0.5000,0.5901,0.6745)\\ (0.3621,0.5000,0.5901)& (0.2436,0.3621,0.5000)& (0.5901,0.6745,0.7490)\\ (0.5901,0.6745,0.7490)& (0.3621,0.5000,0.5901)& (0.5000,0.5901,0.6745)\end{array}\right), \\ {\tilde{R}}^{41}=\left(\begin{array}{ccc}(0.5901,0.6745,0.7490)& (0.2436,0.3621,0.5000)& (0.6745,0.7490,0.7490)\\ (0.3621,0.5000,0.5901)& (0.3621,0.5000,0.5901)& (0.5000,0.5901,0.6745)\\ (0.2436,0.3621,0.5000)& (0.1546,0.2436,0.3621)& (0.5000,0.5901,0.6745)\\ (0.2436,0.3621,0.5000)& (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)\end{array}\right), \\ {\tilde{R}}^{42}=\left(\begin{array}{ccc}(0.5000,0.5901,0.6745)& (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)\\ (0.2436,0.3621,0.5000)& (0.5000,0.5901,0.6745)& (0.5000,0.5901,0.6745)\\ (0.5901,0.6745,0.7490)& (0.1546,0.1546,0.2436)& (0.5000,0.5901,0.6745)\\ (0.5000,0.5901,0.6745)& (0.2436,0.3621,0.5000)& (0.5901,0.6745,0.7490)\end{array}\right), \\ {\tilde{R}}^{43}=\left(\begin{array}{ccc}(0.1546,0.2436,0.3621)& (0.1546,0.2436,0.3621)& (0.5000,0.5901,0.6745)\\ (0.5901,0.6745,0.7490)& (0.5000,0.5901,0.6745)& (0.5000,0.5901,0.6745)\\ (0.5901,0.6745,0.7490)& (0.6745,0.7490,0.7490)& (0.5000,0.5901,0.6745)\\ (0.3621,0.5000,0.5901)& (0.2436,0.3621,0.5000)& (0.2436,0.3621,0.5000)\end{array}\right), \\ {\tilde{R}}^{51}=\left(\begin{array}{ccc}(0.5901,0.6745,0.7490)& (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)\\ (0.5901,0.6745,0.7490)& (0.3621,0.5000,0.5901)& (0.5901,0.6745,0.7490)\\ (0.1546,0.2436,0.3621)& (0.2436,0.3621,0.5000)& (0.5000,0.5901,0.6745)\\ (0.2436,0.3621,0.5000)& (0.2436,0.3621,0.5000)& (0.5000,0.5901,0.6745)\end{array}\right), \\ {\tilde{R}}^{52}=\left(\begin{array}{ccc}(0.5000,0.5901,0.6745)& (0.6745,0.7490,0.7490)& (0.5000,0.5901,0.6745)\\ (0.3621,0.5000,0.5901)& (0.3621,0.5000,0.5901)& (0.5901,0.6745,0.7490)\\ (0.1546,0.2436,0.3621)& (0.6745,0.7490,0.7490)& (0.5901,0.6745,0.7490)\\ (0.1546,0.2436,0.3621)& (0.1546,0.2436,0.3621)& (0.6745,0.7490,0.7490)\end{array}\right), \\ {\tilde{R}}^{53}=\left(\begin{array}{ccc}(0.2436,0.3621,0.5000)& (0.2436,0.3621,0.5000)& (0.5901,0.6745,0.7490)\\ (0.2436,0.3621,0.5000)& (0.1546,0.2436,0.3621)& (0.3621,0.5000,0.5901)\\ (0.2436,0.3621,0.5000)& (0.6745,0.7490,0.7490)& (0.5901,0.6745,0.7490)\\ (0.1546,0.2436,0.3621)& (0.5901,0.6745,0.7490)& (0.6745,0.7490,0.7490)\end{array}\right). \end{gathered}$$

According to Equations (11) and (12), the individual triangular fuzzy decision matrices ${\tilde{R}}^{jk}(j=1,2,3,4,5;k=1,2,3)$ are aggregated into group ones ${\tilde{R}}^j(j=1,2,3,4,5)$:

$$\begin{gathered} {\tilde{R}}^1=\left(\begin{array}{ccc}(0.4726,0.5682,0.6494)& (0.5225,0.6112,0.6931)& (0.3102,0.4359,0.5331)\\ (0.4463,0.5441,0.6432)& (0.3654,0.4719,0.5669)& (0.5901,0.6745,0.7490)\\ (0.4123,0.5266,0.6232)& (0.5676,0.6534,0.7304)& (0.4152,0.5026,0.5950)\\ (0.3884,0.4846,0.5545)& (0.3239,0.4237,0.5353)& (0.5923,0.6748,0.7229)\end{array}\right), \\ {\tilde{R}}^2=\left(\begin{array}{ccc}(0.5436,0.6298,0.6931)& (0.4588,0.5423,0.6136)& (0.3807,0.4801,0.5739)\\ (0.3070,0.3944,0.4975)& (0.6112,0.6931,0.7490)& (0.4674,0.5752,0.6594)\\ (0.3150,0.4118,0.5215)& (0.4103,0.5103,0.6134)& (0.5108,0.5929,0.6523)\\ (0.3111,0.4123,0.5266)& (0.4674,0.5752,0.6594)& (0.4290,0.5284,0.6308)\end{array}\right), \\ {\tilde{R}}^3=\left(\begin{array}{ccc}(0.3654,0.4719,0.5669)& (0.5586,0.6450,0.7229)& (0.3626,0.4458,0.5169)\\ (0.3136,0.3844,0.4726)& (0.3966,0.5225,0.6112)& (0.5000,0.5901,0.6745)\\ (0.4419,0.5611,0.6457)& (0.3111,0.3900,0.4970)& (0.6407,0.7192,0.7490)\\ (0.5901,0.6745,0.7490)& (0.5200,0.6233,0.6854)& (0.5541,0.6407,0.7192)\end{array}\right), \\ {\tilde{R}}^4=\left(\begin{array}{ccc}(0.3934,0.4810,0.5756)& (0.1858,0.2851,0.4104)& (0.5836,0.6668,0.7192)\\ (0.4237,0.5353,0.6311)& (0.4517,0.5586,0.6450)& (0.5000,0.5901,0.6745)\\ (0.4688,0.5652,0.6619)& (0.3626,0.4235,0.4872)& (0.5000,0.5901,0.6745)\\ (0.3551,0.4743,0.5797)& (0.2125,0.3206,0.4517)& (0.4515,0.5495,0.6494)\end{array}\right), \\ {\tilde{R}}^5=\left(\begin{array}{ccc}(0.4290,0.5284,0.6308)& (0.3202,0.4174,0.5140)& (0.5676,0.6534,0.7304)\\ (0.3945,0.5059,0.6097)& (0.2791,0.3974,0.4989)& (0.4989,0.6047,0.6854)\\ (0.1902,0.2910,0.4173)& (0.5237,0.6136,0.6619)& (0.5586,0.6450,0.7229)\\ (0.1858,0.2851,0.4104)& (0.3600,0.4924,0.5651)& (0.6134,0.6934,0.7229)\end{array}\right). \end{gathered}$$

Stage 1:

Once the group triangular fuzzy decision matrices ${\tilde{R}}^j(j=1,2,3,4,5)$ for the attribute $a_j$ are obtained, we can utilize Equation (19) to calculate the optimal normalized sub-attribute weight vector to the attribute $a_j(j=1,2,3,4,5)$:

$$\begin{array}{l}{\overline{w}}_1^{*}=(0.3227,0.3299,0{.3474)}^T,{\overline{w}}_2^{*}=(0.3552,0.2880,0{.3568)}^T,\\ {\overline{w}}_3^{*}=(0.3143,0.3269,0{.3588)}^T,{\overline{w}}_4^{*}=(0.3399,0.2645,0{.3956)}^T,\\ {\overline{w}}_5^{*}=(0.2665,0.3107,0{.4228)}^T.\end{array}$$

Subsequently, based upon Equation (20), we can calculate the optimal group comprehensive evaluation value ${\tilde{h}}_i^{j*}(j=1,2,3,4,5)$ of the alternative $x_i(i=1,2,3,4)$ over the attribute $a_j(j=1,2,3,4,5)$ as

$$\begin{array}{l}{\tilde{h}}_i^{1*}=\left(\begin{array}{c}(0.4326,0.5364,0.6234)\\ (0.4696,0.5656,0.6548)\\ (0.4645,0.5601,0.6488)\\ (0.4380,0.5306,0.6067)\end{array}\right),{\tilde{h}}_i^{2*}=\left(\begin{array}{c}(0.4611,0.5512,0.6277)\\ (0.4518,0.5449,0.6277)\\ (0.4123,0.5048,0.5946)\\ (0.3982,0.5006,0.6020)\end{array}\right),\\ {\tilde{h}}_i^{3*}=\left(\begin{array}{c}(0.4276,0.5191,0.6000)\\ (0.4076,0.5034,0.5904)\\ (0.4705,0.5619,0.6342)\\ (0.5543,0.6456,0.7175)\end{array}\right),{\tilde{h}}_i^{4*}=\left(\begin{array}{c}(0.4137,0.5027,0.5887)\\ (0.4613,0.5631,0.6519)\\ (0.4531,0.5376,0.6207)\\ (0.3555,0.4634,0.5734)\end{array}\right),\\ {\tilde{h}}_i^{5*}=\left(\begin{array}{c}(0.4538,0.5468,0.6366)\\ (0.4028,0.5151,0.6129)\\ (0.4496,0.5409,0.6225)\\ (0.4207,0.5221,0.5906)\end{array}\right).\end{array}$$

Next, according to Equation (3), the scores $S({\tilde{h}}_i^{j*})(i=1,2,3,4;$ $j=1,2,3,4,5)$ are computed as

$$\begin{array}{l}S({\tilde{h}}_i^{1*})=\left(\begin{array}{c}0.5322\\ 0.5639\\ 0.5584\\ 0.5265\end{array}\right),S({\tilde{h}}_i^{2*})=\left(\begin{array}{c}0.5478\\ 0.5423\\ 0.5041\\ 0.5004\end{array}\right),S({\tilde{h}}_i^{3*})=\left(\begin{array}{c}0.5165\\ 0.5012\\ 0.5571\\ 0.6408\end{array}\right),\\ S({\tilde{h}}_i^{4*})=\left(\begin{array}{c}0.5020\\ 0.5599\\ 0.5373\\ 0.4639\end{array}\right),S({\tilde{h}}_i^{5*})=\left(\begin{array}{c}0.5460\\ 0.5115\\ 0.5385\\ 0.5139\end{array}\right).\end{array}$$

(26)

For the attribute $a_1$, we have $S({\tilde{h}}_2^{1*})>S({\tilde{h}}_3^{1*})>S({\tilde{h}}_1^{1*})>S({\tilde{h}}_4^{1*})$, so the ranking result of teachers’ online teaching quality over the attribute $a_1$ is denoted by $x_2\stackrel{a_1}{\succ }{x}_3\stackrel{a_1}{\succ }{x}_1\stackrel{a_1}{\succ }{x}_4$. In the same way, the ranking results of teachers’ online teaching quality over different attributes are obtained, as shown in

Table 2. Ranking results of teachers’ online teaching quality evaluation over different attributes.

Attribute	Ranking
Online teaching environment $(a_1)$	$x_2\stackrel{a_1}{\succ }{x}_3\stackrel{a_1}{\succ }{x}_1\stackrel{a_1}{\succ }{x}_4$
Online teaching platform $(a_2)$	$x_1\stackrel{a_2}{\succ }{x}_2\stackrel{a_2}{\succ }{x}_3\stackrel{a_2}{\succ }{x}_4$
Online teaching content and methods $(a_3)$	$x_4\stackrel{a_3}{\succ }{x}_3\stackrel{a_3}{\succ }{x}_1\stackrel{a_3}{\succ }{x}_2$
Online classroom atmosphere $(a_4)$	$x_2\stackrel{a_4}{\succ }{x}_3\stackrel{a_4}{\succ }{x}_1\stackrel{a_4}{\succ }{x}_4$
Online teaching effectiveness $(a_5)$	$x_1\stackrel{a_5}{\succ }{x}_3\stackrel{a_5}{\succ }{x}_4\stackrel{a_5}{\succ }{x}_2$

.

Stage 2:

According to scores $S({\tilde{h}}_i^{j*})(i=1,2,3,4;j=1,2,3,4,5)$ of the optimal group comprehensive evaluation value ${\tilde{h}}_i^{j*}(i=1,2,3,4;j=1,2,3,4,5)$ for the alternative $x_i$ with regard to the attribute $a_j$ obtained in Stage 1, the normalized optimal attribute weight vector is derived as ${\overline{w}}^{*}={(0.2045,0.1964,0.2078,0.1935,0.1978)}^T$ by using Equation (24).

Based on Equation (25), we can calculate the optimal group comprehensive evaluation value ${\tilde{h}}_i^{*}(i=1,2,3,4)$ of each alternative as

$${\tilde{h}}_i^{*}=\left(\begin{array}{c}(0.4377,0.5312,0.6153)\\ (0.4384,0.5381,0.6272)\\ (0.4503,0.5415,0.6245)\\ (0.4350,0.5339,0.6192)\end{array}\right)$$

According to Equation (3), we can obtain the scores $S({\tilde{h}}_i^{*})(i=1,2,3,4)$ as

$$S({\tilde{h}}_i^{*})=\left(\begin{array}{c}0.5289\\ 0.5355\\ 0.5395\\ 0.5305\end{array}\right).$$

Because $S({\tilde{h}}_3^{*})>S({\tilde{h}}_2^{*})>S({\tilde{h}}_4^{*})>S({\tilde{h}}_1^{*})$, the overall ranking result of four teachers is $x_3\succ x_2\succ x_4\succ x_1$, which means that the teacher with the best online teaching quality is $x_3$.

From the perspective of Table 2 and the overall ranking result, if the teacher with the best online teaching quality is chosen in terms of a certain attribute, then teacher $x_1$ is the best in aspects of online teaching platform and online teaching effectiveness. The teacher $x_2$ is better than other three teachers in terms of online teaching environment and atmosphere. The teacher $x_4$ has an absolute advantage in terms of online teaching content and methods. Compared with other teachers, although the teacher $x_3$ is not the best in every aspect, it almost always ranks in the second place, which is high and stable. On the contrary, the teacher $x_2$ ranks the worst in aspects of online teaching content and methods and online teaching effectiveness. The teacher $x_4$ has the worst ranking with respect to online teaching environment and platform. Except for the best ranking of the teacher $x_1$ in terms of the above-mentioned attributes, the first teacher ranks next to last for the other attributes, which indicates that teacher $x_1$ is at the lower end of the ranking and not stable in general. Although teachers $x_1$, $x_2$, and $x_4$ have the best online teaching quality among a certain attribute(s), in view of all the evaluation attributes, the teacher $x_3$ is the best.

6. Practical Discussion

The above-simulated case verifies the feasibility and effectiveness of the proposed evaluation framework. On the one hand, several inspirations to AEs are given by the evaluation process and results of the case. Initially, in the actual evaluation, AEs often have different expectations and considerations for semantic values of linguistic terms. Thus, the risk preference is used to describe and depict this phenomenon, which makes the evaluation results more accurate and persuasive. Meanwhile, we can find that the importance weights of attributes and sub-attributes in many multi-level EISs are subjectively given by AEs for previous evaluation researches. But in this paper, the importance weights of attributes and sub-attributes can be objectively derived through the proposed evaluation method. Therefore, it makes the evaluation process fairer and makes teachers more convincing. Next, from the evaluation results of the above case, we can see that there are not only the overall evaluation ranking result, but also partial ranking results of teachers over each attribute. This form of evaluation results can help universities and teachers better improve the quality of online teaching. For universities, they can not only select the best teacher in the overall ranking for teaching experience, but also choose the best one in a certain aspect to impart teaching experience in this aspect. For teachers, they can not only know their ranking in the comprehensive evaluation, but also understand their own deficiencies in some aspects to improve themselves.

On the other hand, the research in this paper also has some limitations, which need to be improved in the future. Firstly, this paper regards online teaching quality evaluation as a linguistic MAGDM problem, assuming that the AE group uses the same granular LTS for evaluation. In real life, because different AEs have different knowledge and different preferences, LTSs with different granularities may be employed by them for evaluation. How to convert LTSs with different granularities into ones with the same granularity under the consideration of individual AEs’ risk preferences is the direction of future research. Secondly, the form of evaluation information given in this paper is single. When different AEs give different types of evaluation information, how to transform and aggregate them is also the direction to be studied in the future. Thirdly, the proposed evaluation framework is applicable to online teaching quality evaluation, but it is also applicable to other application cases, which is worth further research in the future.

Of course, there are some other MAGDM methods that can be used to solve attribute weights, such as TOPSIS, VIKOR and PROMETHEE II methods [38]. Which method is suitable to be used in a realistic assessment environment should be measured by one or some benchmarks. In this paper, the online teaching evaluation process takes into account the risk preferences of AEs. However, the difference in risk preferences will affect the weights of attributes and sub-attributes in the EIS [23]. Thereby, in an evaluation environment that considers risk preferences, the presented attribute and sub-attribute weight solution optimization model and the devised evaluation framework are more suitable and applicable than some other MAGDM approaches.

7. Conclusions

In the current way of epidemic prevention and control, online teaching has gradually transformed into a mainstream learning method in universities. In order to ensure the quality of online teaching in universities, evaluation of online teaching quality is particularly important. By introducing and expanding the linguistic MAGDM method, this paper mainly copes with the evaluation problem of online teaching quality that considers AEs’ risk preferences and the completely unknown weights information in the multi-level EIS. Its contributions and advantages are manifested in four aspects. First of all, we construct a multi-level EIS of online teaching quality based upon the Delphi method, which can be well applied to evaluation of online teaching quality. Then, we extend Lin and Wang’s method [25] to apply to the evaluation problem based on a multi-level EIS which commonly exists in reality. Furthermore, according to the principle of maximizing the group comprehensive evaluation value of each alternative, we put forward the two-stage optimization model that can objectively solve the weights of attributes and sub-attributes. The presented two-stage optimization model has lower requirements for the AEs than other attribute solving models, and is more practical and objective. Lastly, in the process of evaluation, the evaluation results given by our framework include both the overall evaluation result and partial evaluation results. These evaluation results can help teachers and even universities to understand the specific problems existing in some aspects of current online teaching in more detail for timely improvement.