A Consensus-Based 360 Degree Feedback Evaluation Method with Linguistic Distribution Assessments

Abstract

The 360 degree feedback evaluation method is a multidimensional, comprehensive assessment method. Evaluators may hesitate among multiple evaluation values and be simultaneously constrained by the biases and cognitive errors of the evaluators, evaluation results are prone to unfairness and conflicts. To overcome these issues, this paper proposes a consensus-based 360 degree feedback evaluation method with linguistic distribution assessments. Firstly, evaluators provide evaluation information in the form of linguistic distribution. Secondly, utilizing an enhanced ordered weighted averaging (OWA) operator, the model aggregates multi-source evaluation information to handle biased evaluation information effectively. Subsequently, a consensus-reaching process is established to coordinate conflicting viewpoints among the evaluators, and a feedback adjustment mechanism is designed to guide evaluators in refining their evaluation information, facilitating the attainment of a unanimous evaluation outcome. Finally, the improved 360 degree feedback evaluation method was applied to the performance evaluation of the project leaders in company J, thereby validating the effectiveness and rationality of the method.

1. Introduction

The 360 degree feedback evaluation method, also known as comprehensive or multi-source feedback evaluation [1], differs from traditional top-down, unidirectional methods as it gathers evaluation information from multiple stakeholders who interact with the evaluated individuals, providing comprehensive feedback from various perspectives [2,3]. In the early stages of human resource management with enterprises, 360 degree feedback evaluation was primarily employed for developmental assessments of individuals. This approach involved evaluating the personal capabilities and career development status of employees or organizational members [4,5]. Later, 360 degree feedback evaluation was gradually introduced into performance evaluations, assessing the job performance of employees or organizational members. This evaluation served as the basis for salary adjustments and position changes. The indicators used in 360 degree feedback evaluations are primarily qualitative [6], and the evaluation process consists of the preparation phase, training phase, evaluation implementation phase, and feedback coaching phase [7]. As a tool for assessing employee development and performance, 360 degree feedback evaluation has gained favor among numerous benchmark enterprises in human resource management due to its comprehensiveness and relative objectivity. It has been widely applied in corporate management [2,8,9]. Today, 360 degree feedback evaluation has expanded into various domains, including medicine, engineering projects, education, and beyond [10,11,12,13,14,15].

The key to the effective implementation of 360 degree feedback evaluation lies in assessing the evaluated individuals in an objective and fair manner [3,16]. Within this system, multiple evaluators are involved, including superiors, subordinates, peers, customers, and the individuals being assessed [6,17]. By receiving evaluations from multiple sources, individuals gain diverse and valuable perspectives, enabling them to accurately assess themselves and identify their strengths and areas for improvement [3,18]. Traditional 360 degree feedback evaluation methods rely on precise numerical assessments and use subjective weighting to determine indicator weights. By aggregating evaluation information from various evaluators through weighted averaging, these methods provide feedback to the individuals being assessed. This feedback aims to help individuals improve their behavior, enhance their abilities, and boost their performance [18,19,20].

Currently, scholars primarily explore and analyze the reliability and validity of 360 degree feedback from a theoretical perspective [21,22,23]. However, three issues persist in the implementation process. Firstly, the evaluation content is complex and abstract. Due to the qualitative nature of the indicators, evaluators’ limited capabilities, and the uncertainty in judgment, it is challenging for evaluators to provide precise numerical information [24,25,26,27,28]. Secondly, the determination of 360 degree feedback indicator weights often relies solely on experts’ subjective judgment, leading to significant arbitrariness [29]. Lastly, evaluations are influenced by individual cognition, and cognitive errors and biases can lead to inaccuracies, affecting the quality of the results [1,8,30]. Given these factors, 360 degree feedback evaluation may provide organizations and employees a false sense of objectivity and rationality. Addressing these challenges and improving the effectiveness of 360 degree feedback evaluation outcomes holds significant research importance.

Group decision making refers to the process in which multiple decision-makers provide their evaluation perspectives on a given set of options according to specific evaluation rules. These individual perspectives are then aggregated to form a collective evaluation [31,32]. The process of 360 degree feedback evaluation involves multiple evaluators, necessitating the gathering and integration of their evaluation information, making it a typical group decision-making problem [33,34]. Significant progress has been made in the acquisition and aggregation of decision information within group decision making, and it has been widely applied in various fields [35]. This progress holds important guiding significance for management, investment, and other practical problems [36,37,38,39].

In response to the aforementioned issues, scholars have begun employing group decision-making methods to conduct improvement research. For example, Anisseh et al. [40] utilized intuitionistic fuzzy numbers for evaluation information. Espinilla et al. [41] propose using the linguistic two-tuple representation model and its extended approach to address the heterogeneous information involved in 360 degree evaluations. Cheng [42] employed the entropy method to determine objective weights for performance appraisal indicators, thereby avoiding the subjectivity and arbitrariness previously associated with determining these weights. These improved techniques have played a positive role in overcoming the deficiencies of traditional 360 degree feedback evaluation methods. However, to ensure the objectivity, fairness, and effectiveness of 360 degree feedback evaluations, there are still several issues that need further discussion:

(1)

The evaluators hesitate between multiple values. In enterprises that use 360 degree feedback evaluation, evaluators typically assign scores to individuals using precise numerical values, such as a scale from 1 to 10 [43]. However, in many cases, due to the qualitative nature of the evaluation indicators and the inherent complexity of the 360 degree evaluation process, evaluators may encounter hesitation or uncertainty when making their assessments [28]. Some scholars have proposed using linguistic variables to represent evaluation information, but it is rarely used in 360 degree feedback evaluation. It is necessary to explore further applications of linguistic variables in 360 degree feedback evaluation.

(2)

Evaluation information exists with individual biases [44]. The participation of multiple evaluators can ensure relatively objective results [45], but it cannot eliminate subjectivity. During the evaluation process, emotional factors and personal interests can easily infiltrate, leading evaluators to adopt strategic assessments [43]. For example, subordinates may offer excessively high ratings due to interpersonal needs [23], while colleagues might provide extremely low ratings due to competitive relationships [46]. Handling these extreme evaluations and reasonably aggregating them to ensure the results are as objective and fair as possible is key to effective assessment.

(3)

There are discrepancies in the evaluative information from multiple evaluators. Multiple evaluators come from diverse backgrounds with varying knowledge structures, levels of judgment, and familiarity with the evaluated individuals’ work [47]. As a result, discrepancies may arise in evaluators’ preference information and result rankings [9]. The current 360 degree feedback evaluation method aggregates individual evaluations into collective data without ensuring consensus among evaluators. Handling the discrepancies between evaluators’ assessments to ensure that the evaluation results are as acceptable as possible to all evaluators is also an important issue in the evaluation process.

In summary, to address the challenges of 360 degree feedback evaluation in complex environments, this study aims to present a consensus-based 360 degree feedback evaluation method with linguistic distribution assessments. This approach encompasses the following aspects:

(1)

This paper investigates the 360 degree feedback evaluation method within a linguistic context. Considering the uncertainty and hesitation in evaluators’ expressions, it proposes using linguistic distribution assessments to represent evaluation information. The use of linguistic distribution assessments not only aligns with the evaluators’ expression habits but also captures the uncertainty of the evaluation information, thereby helping to obtain results that closely reflect the evaluators’ cognition.

(2)

The enhanced ordered weighted averaging (OWA) operator is utilized to aggregate evaluative information from multiple evaluators, forming a collective assessment. By applying slight value weighting to unfairness parameters, the influence of these parameters on the decision outcome is mitigated. This approach effectively addresses the issue of evaluator biases, ensuring fairness throughout the evaluation process.

(3)

Utilize group consensus decision-making methods to address conflicts in evaluators’ viewpoints during the evaluation process. This paper designs a consensus-reaching process that embeds a new feedback regulation mechanism to guide evaluators in adjusting their viewpoints to achieve consistent evaluation perspectives. This process improves the reliability and validity of the evaluation results and enhances evaluators’ acceptance of the outcomes.

The rest of the paper is organized as follows. Section 2 introduces the foundational knowledge utilized in this study. Section 3 proposes a consensus-based 360 degree feedback evaluation method, elucidating its specific solution steps. Section 4 employs the proposed 360 degree feedback evaluation method to assess the performance of enterprise employees, examining the effectiveness and practicality of this approach. In Section 5, some discussions are conducted, while Section 6 outlines the research conclusions of this paper.

2. Preliminaries

This section introduces several basics involved in constructing a consensus-based 360 degree feedback evaluation method.

2.1. Two-Tuple Linguistic Model

There are some inevitable uncertainties in the evaluation process, which brings additional challenges. Therefore, in decision making, linguistic terms are often more convenient than precise numerical scales. Let $S=\{s_i|i=0,1,2,\dots ,g\}$ be a linguistic term set, where $s_i$ represents a possible linguistic value and $g+1$ represents the granularity of $S$. The following essential characteristics are commonly fulfilled: (1) $S$ is ordered: $s_i>s_j$, if and only if $i>j$. (2) A negation operator is $Neg(s_i)=s_{g-i}$, where $i=0,1,\dots ,g$. For more details, kindly refer to Herrera and Martínez [48].

The two-tuple linguistic model is a symbolic computation model applied to text. The equivalent information of $\beta$ can be derived from the following functions, resulting in a two-tuple representation:

$$\Delta :[0,g]\to S\times [-0.5,0.5)$$

(1)

$$\Delta (\beta )=\left(s_i,\sigma \right), \mathrm{with} \left\{\begin{array}{l}{s}_i,i=\mathrm{round}(\beta )\hfill \\ \sigma =\beta -i,\sigma \in [-0.5,0.5)\hfill \end{array}\right.$$

(2)

where round $(\cdot )$ is the usual round operation, $s_i$ has the closest index label to “$\beta$”, and “$\sigma$” is the value of the symbolic translation.

Let $S=\{s_i|i=0,1,2,\dots ,g\}$ be a linguistic term set and $\left(s_i,\sigma \right)$ be a two-tuple. There is always a ${\mathsf{\Delta }}^{-1}$ function, such that, from a two-tuple it returns its equivalent numerical value $\beta \in [0,\mathrm{g}]\subset R$. Specifically, the following function can be used to convert a two-tuple linguistic term into its equivalent numerical value:

$$\begin{array}{c}{\mathsf{\Delta }}^{-1}:S\times [-0.5,0.5)\to [0,g]\\ {\mathsf{\Delta }}^{-1}(s_i,\sigma )=i+\sigma =\beta \end{array}$$

(3)

Based on the discussion of the two-tuple linguistic model, it is evident that the process of converting a linguistic term into a linguistic two-tuple involves adding a value of 0 as a symbolic translation: $s_i\in S\Rightarrow (s_i,0)$.

In the above linguistic model, $\beta \in [0,g]$ is the result of the element operation in set $S$, $\mathsf{\Delta }$ is a one-to-one mapping function, and ${\mathsf{\Delta }}^{-1}$ is an inverse operator for $\mathsf{\Delta }$. Here, $(s_i,\sigma )$ and $(s_j,\mu )$ are used to express two linguistic two-tuples. When ${\mathsf{\Delta }}^{-1}(s_i,\sigma )>{\mathsf{\Delta }}^{-1}(s_j,\mu )$, then $(s_i,\sigma )$ is larger than $(s_j,\mu )$.

2.2. Numerical Scale Function and Linguistic Distribution Assessments

2.2.1. Numerical Scale Function

The numerical scales of the linguistic terms are used to handle the linguistic distribution assessments and convert linguistic terms into precise numbers.

Definition 1

([49]). Let $S=\{s_i|i=0,1,2,\dots ,g\}$ be as shown above, $R$ be the set of precise numbers. Then, the function $NS:S\to R$ is regarded as a numerical scale of $S$, and $NS(s_t)$ is referred to as the numerical index of term $s_t$. In this paper, for the following use, we assume that $NS(s_t)=t(t=0,1,\dots ,g)$.

2.2.2. Linguistic Distribution Assessments

Linguistic distribution assessments provide symbolic proportion information concerning linguistic terms. The fundamental concept of linguistic distribution is outlined as follows:

Definition 2

([50]). Let $S=\{s_i|i=0,1,2,\dots ,g\}$ be as shown above; $A=\{(s_t,{\beta }_t)|t=0,1,2,\dots ,g\}$ is the linguistic distribution assessments of $S$, where $s_t\in S$, $0\le {\beta }_t\le 1$ is the symbolic proportion of linguistic term as well as ${\displaystyle {\sum }_{t=0}^g{\beta }_t=1}$.

Definition 3

([50]). Let $A=\{(s_t,{\beta }_t)|t=0,1,2,\dots ,g\}$ be as shown above. A linguistic two-tuple $E(A)$ is used to define the expectation of $A$. Afterwards, the expectation of $A$ is

$$E(A)=\mathsf{\Delta }\left({\displaystyle {\sum }_{t=0}^g{\beta }_t}\times NS(s_t)\right)$$

(4)

Obviously, $E(A)\in S$. The comparison operator exists on $S$: $A_1$ and $A_2$ are the two linguistic distribution assessments. The definition of the comparison operator and negation operator for linguistic distributions is as follows:

(1) 

Comparison operator: If $E(A_1)>E(A_2)$, $A_1$ is higher than $A_2$. If $E(A_1)=E(A_2)$, then $A_1$ is equal to $A_2$.

(2) 

Negation operator: $Neg\{(s_t,{\beta }_t)|t=0,1,2,\dots ,g\}=\{(s_t,{\beta }_{g-t})|t=0,1,2,\dots ,g\}$.

Definition 4

([50]). Let $\{A_1,A_2,\dots ,A_n\}$ be a set of $n$ linguistic distribution assessments on $S$, where $A_i=\{(s_t,{\beta }_t^i)|t=0,1,2,\dots g\}$, $i=1,2,\dots ,n$; let $w={(w_1,w_2,\dots ,w_n)}^T$ be the related weight vector satisfying $w_i\ge 0$, ${\displaystyle {\sum }_{i=1}^n{w}_i}=1$. Afterwards the weighted averaging operator of $\{A_1,A_2,\dots ,A_n\}$ is shown as follows:

$$DAWA(A_1,A_2,\dots ,A_n)=\{(s_t,{\beta }_t|t=0,1,2,\dots ,g\}$$

(5)

where ${\beta }_t={\displaystyle {\sum }_{i=1}^n{\omega }_i}\times {\beta }_t^i$.

2.3. OWA Operator

The OWA operator, proposed by Yager [51], involves three main steps: reordering the input data in descending order, determining the weights of the OWA using an appropriate method, and aggregating these reordered data using the OWA weights. The mathematical expression is as follows:

Definition 5

([51]). An OWA operator of dimension n is a mapping, denoted by $h:$ $R^n\to R$.

$$h\left(x_1,x_2,\dots ,x_n\right)={\displaystyle {\sum }_{i=1}^n{\lambda }_i{a}_i}$$

(6)

where $\lambda =({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n)$ represents the weighted vector associated with the function $h$, ${\lambda }_i\in [0,1]$, ${\displaystyle {\sum }_{i=1}^n{\lambda }_i}=1$, $i=1,2,\dots ,n$, and $a_i$ stands for the data in $h(x_1,x_2,\dots ,x_n)$ sorted in the $i$-th position. $R$ represents the entire set of real numbers, and the function $h$ is termed an OWA operator of dimension n. It is important to note that ${\lambda }_i$ is unrelated to $x_i$ and only associated with the $i$-th position during the aggregation process.

Yager introduced two measures of representativeness, namely, the “orness measure” and “dispersion measure” The “orness measure” quantifies the degree of “or” or “and” operations and is defined as follows:

$$orness(\lambda )=\frac{1}{n-1}{\displaystyle {\sum }_{i=1}^n(n-i){\lambda }_i}$$

(7)

The “dispersion measure” is employed to quantify the contribution of each piece of data to the result and is defined as follows:

$$disp(\lambda )=-{\displaystyle {\sum }_{i=1}^n{\lambda }_i\ln {\lambda }_i}$$

(8)

A key step in the ordered weighted averaging (OWA) operator is determining the position weights for each piece of data. The size of these position weights is crucial to the final aggregation result. Wang and Xu [52] discussed the position weights from the perspective of the normal distribution and linked these weights to the decision-making data, proposing a method for assigning weights dependent on the decision data. This paper employs this method to aggregate multi-source evaluation information. The specific implementation steps are as follows:

(1) For a set of decision data denoted as $D={(d_1,d_2,\dots ,d_n)}^T$ with corresponding weights $f=(f_1,f_2,\dots f_n)$, satisfying $f_i\ge 0$, ${\displaystyle {\sum }_{i=1}^m{f}_i}=1$, the mean of this data set is defined as $\overline{d}$, and the standard deviation is denoted as $\sigma$,

$$\overline{d}=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{d}_i}$$

(9)

$$\sigma =\sqrt{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{\left(\overline{d}-d_i\right)}^2}}$$

(10)

(2) Standardize the data to obtain $O={\left(o_1,o_2,\dots ,o_n\right)}^T$, where

$$o_i=\frac{d_i-\overline{d}}{\sigma }$$

(11)

(3) For a continuous random variable $x$, using the function

$$h_{\mu ,\sigma }(x)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{{(x-\mu )}^2}{2{\sigma }^2}}$$

(12)

A continuous distribution serving as the density function is referred to as a normal distribution with parameters $\mu$ and $\sigma$, denoted as $N(\mu ,{\sigma }^2)$. Specifically, when the parameters are 0 and 1, the normal distribution is referred to as the standard normal distribution $N(0,1)$, with a density function denoted as $h(x)$, where

$$h(x)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{x^2}{2}}$$

(13)

Calculate the density function $Q={(q_1,q_2,\dots ,q_n)}^T$ for $O={\left(o_1,o_2,\dots ,o_n\right)}^T$, where

$$q_i=h\left(o_i\right)$$

(14)

(4) Calculate the weight $f=(f_1,f_2,\dots f_n)$ for data $D={(d_1,d_2,\dots ,d_n)}^T$, where

$$f_i=q_i/{\displaystyle {\sum }_{i=1}^n{q}_i}$$

(15)

3. Consensus-Based 360 Degree Feedback Evaluation Method

Based on the analysis conducted in the preceding sections, it is evident that the implementation of 360 degree feedback evaluation has limitations. To address the complexities associated with 360 degree feedback evaluation, a consensus-based 360 degree feedback evaluation method has been developed. The method employs an improved OWA operator to aggregate evaluation information, mitigating the impact of unfair parameters on evaluation outcomes. Additionally, a group consensus decision-making approach is utilized to manage divergent viewpoints among evaluators, aiming to achieve consensus and enhance the fairness and rationality of 360 degree feedback evaluation results.

3.1. Introduction to the Consensus-Based 360 Degree Feedback Evaluation Method

This paper enhances the traditional 360 degree feedback evaluation method to address prevalent issues during the evaluation process. It comprises four specific phases: the preparation phase, training phase, evaluation implementation phase, and feedback coaching phase.

Preparation phase: Establish the objectives of 360 degree feedback evaluation, principles, evaluation indicators, and the corresponding evaluators and evaluated employees. This ensures they understand the purpose of implementing 360 degree feedback evaluation within the enterprise, enabling them to participate in the entire evaluation process fairly, scientifically, rigorously, and reasonably. Let $M=\{M_1,M_2,\dots ,M_m\}$ be the set of $m$ evaluators. Let $I=\{I_1,I_2,\dots ,I_y\}$ be the set of $y$ evaluation indicators, with the corresponding weight vector $w={(w_1,w_2,\dots ,w_y)}^T$, where ${\displaystyle {\sum }_{j=1}^y{w}_j}=1$. Let $X=\{X_1,X_2,\dots ,X_n\}$ be the set of $n$ evaluated employees. The evaluator $M=\{M_1,M_2,\dots ,M_m\}(i=1,2,\dots ,m)$ assesses the evaluated employees $X=\{X_1,X_2,\dots ,X_n\}(k=1,2,\dots ,n)$ based on 360 degree feedback evaluation indicators $I=\{I_1,I_2,\dots ,I_y\}(j=1,2,\dots ,y)$.

Training phase: Establish an evaluation workgroup. The evaluation workgroup consists of relevant individuals, including the superior, subordinates, colleagues, and customers of the evaluated employees (experts may be involved in the review as needed based on specific circumstances). Explain the optimized 360 degree feedback evaluation process to all evaluators and the evaluated employees, ensuring their familiarity and correct utilization of the evaluation tools to provide genuine and objective feedback.

Evaluation implementation phase: The 360 degree feedback evaluation in this study comprises four main stages: (1) Collecting evaluation information from evaluators. (2) Aggregating the information using the enhanced OWA operator. (3) Calculating the weights of the evaluation indicators through a combination weighting method. (4) Establishing a consensus-reaching process to address conflicts among evaluators’ assessments. This process guides evaluators to adjust their evaluation information of the assessed individuals through a feedback regulation mechanism, ultimately leading to satisfactory evaluation results for the assessed employees.

Feedback coaching phase: During the feedback stage, the evaluation results and outcomes are communicated to the evaluated employees, encouraging self-reflection, identification of potential issues, and the active adoption of improvement measures. This continuous process aims to enhance work performance and professional capabilities, further propelling individual career development in the future.

The specific steps of the proposed consensus-based 360 degree feedback evaluation method are illustrated in Figure 1.

3.2. The Construction of the Consensus-Based 360 Degree Feedback Evaluation Method

This section provides an in-depth analysis of the evaluation implementation phase within the consensus-based 360 degree feedback evaluation method. In addressing the challenges of 360 degree feedback evaluation in complex environments, this paper attempts to utilize an improved OWA operator to aggregate evaluative information from multiple evaluators. Additionally, it employs group consensus decision-making methods to handle discrepancies in evaluators’ viewpoints. This approach aims to make the implementation process of 360 degree feedback evaluation more scientific and rational. The method diagram is illustrated as shown in Figure 2.

The crucial steps involve four distinct stages:

1.

Collecting evaluation information from evaluators.

The evaluators employ the linguistic term set $S=\{s_0,s_1,s_2,\dots ,s_g\}$ to assess the performance of the employees being evaluated on each evaluation indicator, as well as obtaining the evaluation matrix $D^k={(d_{ij}^k)}_{m\times y}$, $d_{ij}^k=\{(s_t,{\beta }_{ij,t}^k)|t=0,1,2,\dots ,g\}$, where $s_t\in S$, ${\beta }_{ij,t}^k\ge 0$, ${\displaystyle {\sum }_{t=0}^g{\beta }_{ij,t}^k}=1$, $k=1,2,\dots ,n$.

2.

Aggregating the collected evaluation information to obtain a provisional collective evaluation.

Step 1: Calculate the expectation matrix $E{D}^k$ for the evaluated employees of $X_k\in X$ using Definition 3, satisfying $E{D}^k={(e{d}_{ij}^k)}_{m\times y}$.

$$e{d}_{ij}^k=\mathsf{\Delta }\left({\displaystyle {\sum }_{t=0}^{g}NS\left(s_t\right)}\cdot {\beta }_{ij,t}^k\right)$$

(16)

Step 2: Utilize Equations (9)–(14) to standardize the data in the evaluation matrix and calculate the probability density of the evaluation data as $G^k={(g_{ij}^k)}_{m\times y}$.

Step 3: Utilize Equation (15) to compute the weight matrix for the evaluation data, denoted as $F^k={(f_{ij}^k)}_{m\times y}$, where ${\displaystyle {\sum }_{i=1}^m{f}_{ij}^k}=1$, $k=1,2,\dots ,n,i=1,2,\dots ,m,j=1,2,\dots ,y$. $f_{ij}^k$ represents the weight assigned by the evaluator $M_i$ to the assessed employee $X_k$ for indicator $I_j$ among all evaluation data provided by evaluators.

Step 4: Calculate the comprehensive evaluation $C={(c_{kj}^c)}_{n\times y}$ for the evaluated employee $X_k$ on the evaluation indicator $I_j$, where

$$c_{kj}^c=\mathsf{\Delta }\left({\displaystyle {\sum }_{i=1}^n{f}_{ij}^k}\cdot {\mathsf{\Delta }}^{-1}e{d}_{ij}^k\right)$$

(17)

3.

Calculating the weights of the 360 degree feedback evaluation indicators.

Using the improved analytic hierarchy process [53] and entropy method [42], calculate the subjective weights ${\omega }_j^s$ and objective weights $w_j^o$ of the evaluation indicators, respectively.

Compute the comprehensive weight $w={(w_1,w_2,\dots ,w_y)}^T$ for the evaluation indicator $I_j$, where

$$w_j=\epsilon w_j^s+(1-\epsilon )w_j^o$$

(18)

4.

Consensus reaching process.

Step 1: Consensus measurement

To measure the consistency level among evaluators on the evaluated employees, we initially used the indicator weights from the third stage to consolidate evaluation information for each evaluated individual, producing the integrated evaluation matrix $Z^i={(z_1^i,z_2^i,\dots ,z_n^i)}^T$ for each evaluator, where

$$z_k^i=\mathsf{\Delta }\left({\displaystyle {\sum }_{j=1}^y{\mathsf{\Delta }}^{-1}(e{d}_{ij}^k})\cdot {\omega }_j\right)$$

(19)

Aggregate the evaluation matrices acquired in the second stage to form the comprehensive collective evaluation matrix $Z^c=(z_1^c,z_2^c,\dots ,z_n^c)$, where

$$z_k^c=\mathsf{\Delta }\left({\displaystyle {\sum }_{j=1}^y{\mathsf{\Delta }}^{-1}(c_{kj}^c})\cdot {\omega }_j\right)$$

(20)

Let $O={(o_1,o_2,\dots ,o_n)}^T$ be called a preference ordering, where $o_i(i=1,2,\dots ,n)$ indicates the position of $x_i$ among $X=\{x_1,x_2,\dots ,x_n\}$. Let $O^i={(o_1^i,o_2^i,\dots ,o_n^i)}^T(i=1,2,\dots ,m)$ be a preference ordering of evaluator $M_i$, and this paper calculates the preference orderings using the collective evaluation matrix $Z^i={(z_1^i,z_2^i,\dots ,z_n^i)}^T$. The higher the evaluation values, the lower the positional order $o_k^i$. Likewise, the collective preference orderings $O^c={(o_1^c,o_2^c,\dots ,o_n^c)}^T$ can also be calculated using the integrated collective evaluation matrix $Z^c={(z_1^c,z_2^c,\dots ,z_n^c)}^T$.

Definition 6

([54]). The ordinal consensus degree (OCD) of each evaluator $M_i$, where $i=1,2,\dots ,m$, is defined as follows:

$$OCD\left(M_i\right)=\frac{1}{n^2}{\displaystyle {\sum }_{k=1}^n\left|o_k^i-o_k^c\right|}$$

(21)

Utilize Equations (9)–(15) to compute the position weights for the data $OCD(M_1)$, $OCD(M_2)$, …, $OCD(M_m)$, denoted as $\mu =({\mu }_1,{\mu }_2,\dots ,{\mu }_m)$, where ${\displaystyle {\sum }_{i=1}^m{\mu }_i}=1$, ${\mu }_i\ge 0$; subsequently, the ordinal consensus degree among all the evaluators can be denoted as follows:

$$OCD\{M_1,M_2,\dots ,M_m\}={\displaystyle {\sum }_{i=1}^m{\mu }_i}\cdot OCD(M_i)$$

(22)

If $OCD\{M_1,M_2,\dots ,M_m\}=0$, all evaluators reach a consensus on the evaluation opinion. The higher the $OCD\{M_1,M_2,\dots ,M_m\}$ value is, the lower the degree of consensus, and the greater the conflict between the evaluators.

Step 2: Adjustment of the evaluation information

If a satisfactory consensus is not reached, the evaluators could adjust their evaluation information according to specific rules in order to facilitate achieving the desired consensus level. The adjustment rules are detailed as follows:

Similarly, referring to Dong et al. [55], and based on the linguistic distribution assessments utilized in this study, adjustments to the evaluation information should adhere to the guidelines specified below:

If $e{d}_{ij}^k>c_{kj}^c$, $M_i$ should lower their evaluation information of $X_k$ over $I_j$;

If $e{d}_{ij}^k=c_{kj}^c$, the evaluation information of $X_k$ over $I_j$ should remain the same;

If $e{d}_{ij}^k<c_{kj}^c$, $M_i$ should raise their evaluation information of $X_k$ over $I_j$.

By modifying the new round of evaluation information, a new consensus level among evaluators can be obtained. If the adjusted consensus level is acceptable, the final evaluation result is determined based on this collective agreement. If not, return to the first stage until a satisfactory consensus level is achieved.

Note 1: This paper focuses on proposing a consensus-based 360 degree feedback evaluation method with linguistic distribution assessments. An improved OWA operator is employed to aggregate evaluation information from different evaluators, forming a collective evaluation. A consensus-reaching process is embedded within the evaluation process to help evaluators achieve a consensus on the evaluation. This study focuses on the practical issues associated with 360 degree feedback evaluation, specifically addressing evaluator bias and conflicts in evaluation viewpoints. It emphasizes the application of this method in corporate performance assessment. In future research, we will include axiomatic analysis to explore the theoretical implications of this method in greater depth and propose relevant lemmas to further support and validate our theoretical framework.

4. Case Study

4.1. Background

J company, established in 2011 in Nanjing, Jiangsu Province, China, specializes in the electric power industry. As a high-tech enterprise, it focuses on developing application systems, integrating information systems, conducting data mining and analysis, and applying visualization technology. The company combines research and development, production, sales, and services to provide comprehensive information solutions for the electric power industry. Its business scope includes computer software development, technical services, system integration, and automation control system research and design.

The power dispatch department is the core of the company, responsible for product development, design, and customer technical support. It is divided into four project teams, each consisting of a project leader, development engineer, testing engineer, implementation engineer, and operation and maintenance engineer. The project leader is the central figure, bearing dual responsibilities for leadership and management. They oversee project implementation, track progress, and manage processes, playing an indispensable role. Evaluating the project leader involves scrutinizing their performance. Summarizing and assessing their work helps identify issues, enhance performance, and improve project completion. Therefore, this study uses the consensus-based 360 degree feedback evaluation method to assess the performance of project leaders at J Company, validating the method’s effectiveness and rationality.

4.2. Construction of the 360 Degree Feedback Evaluation Indicators System

In the context of 360 degree feedback evaluation, the accurate assessment of an individual’s work capabilities and development potential relies on scientifically sound performance indicators. When selecting these indicators, it is crucial to consider not only the project leader’s work performance but also their work attitude and abilities. Accordingly, this study identifies work performance, work attitude, and work ability as the first-level indicators, with project schedule, project delivery quality, project standardization, work responsibility, collaboration, organizational discipline, professional ability, team management ability, and decision-making ability as the second-level indicators. The 360 degree performance evaluation indicators system constructed in this study is illustrated in Figure 3.

4.3. Implementation of the Consensus-Based 360 Degree Feedback Evaluation Method

The consensus-based 360 degree feedback evaluation method consists of the following four phases, with specific steps outlined as follows:

Preparation phase. Determine the objectives and principles of the 360 degree feedback evaluation. To enhance the efficiency of project leaders; achieve J Company’s overall operational objectives; and provide accurate bases for salary adjustments, training, and promotions, the 360 degree feedback evaluation is employed. The evaluation process adheres to the following principles:

(1)

The three public principles: Fairness, impartiality, and transparency in the evaluation process and results;

(2)

Objectivity principle: Basing evaluations on facts and avoiding excessive subjective judgments;

(3)

Results-oriented principle: Emphasizing outcomes, key behaviors, and value contributions;

(4)

Performance coaching: Consistently implementing coaching as the core of performance management throughout the process.

Training phase: Form an evaluation working group. The 360 degree feedback evaluation involves six evaluators, namely, the project director, subordinates, customers, and relevant personnel from the project service center and marketing commerce department. These six evaluators assess the performance of four project leaders, denoted as $X=\{X_1,X_2,X_3,X_4\}$. There are three first-level indicators denoted as $I=\{I_1,I_2,I_3\}$, with nine corresponding secondary evaluation indicators, denoted as $I_1=\{I_{1-1},I_{1-2},I_{1-3}\}$, $I_2=\{I_{2-1},I_{2-2},I_{2-3}\}$, $I_3=\{I_{3-1},I_{3-2},I_{3-3}\}$. Evaluators use a five-grade linguistic term set to assess the performance of project leaders across various evaluation indicators, represented as $S=\{s_0=very poor, s_1=poor, s_2=fair, s_3=good, s_4=very good\}$.

Evaluation implementation phase: The specific procedure consists of the following four steps.

1.

Collecting evaluation information from evaluators.

Evaluators use a set of linguistic terms to assess the performance of the project leaders on each evaluation indicator, resulting in an evaluation matrix $D^k={(d_{ij}^k)}_{m\times y}$, as shown in

Table 1. Evaluation information matrix for employee $X_1$.

${\mathit{D}}^1$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\{(s_2,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\{(s_3,1)\}$
$M_2$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$
$M_3$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\{(s_3,1)\}$	$\{(s_4,1)\}$
$M_4$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$
$M_5$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\{(s_2,1)\}$	$\{(s_3,1)\}$
$M_6$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$

,

Table 2. Evaluation information matrix for employee $X_2$.

${\mathit{D}}^2$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\{(s_2,1)\}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$
$M_2$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$
$M_3$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$
$M_4$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_2,1)\}$	$\{(s_4,1)\}$	$\{(s_4,1)\}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\{(s_3,1)\}$
$M_5$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$
$M_6$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_2,1)\}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$

,

Table 3. Evaluation information matrix for employee $X_3$.

${\mathit{D}}^3$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\{(s_4,1)\}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$
$M_2$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_3,1)\}$
$M_3$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$	$\{(s_3,1)\}$
$M_4$	$\{(s_4,1)\}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.9)\\ (s_3,0.1)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$
$M_5$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$
$M_6$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$

and

Table 4. Evaluation information matrix for employee $X_4$.

${\mathit{D}}^4$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_2,1)\}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$
$M_2$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_2,0.9)\\ (s_3,0.1)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$
$M_3$	$\begin{array}{l}\{(s_1,0.1)\\ (s_2,0.9)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_1,0.1)\\ (s_2,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.9)\\ (s_3,0.1)\}\end{array}$
$M_4$	$\begin{array}{l}\{(s_1,0.2)\\ (s_2,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_4,1)\}$	$\{(s_2,1)\}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$
$M_5$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_1,0.1)\\ (s_2,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$
$M_6$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$

.

2.

Evaluation information aggregation.

The OWA operator is used to aggregate evaluation information, specifically as follows:

Step 1: Calculate the expected value matrix $E{D}^k$ of the project leader $X_k\in \{X_1,X_2,X_3,X_4\}$ using Equation (16);

Step 2: Use Equations (9)–(14) to standardize the data in the evaluation matrix, and calculate the probability density of the evaluation data as: $G^k={(g_{ij}^k)}_{m\times y}$;

Step 3: Using Equation (15), the positional weight matrix is denoted as: $F^k={(f_{ij}^k)}_{m\times y}$;

Step 4: Use Equation (17) to calculate the comprehensive evaluation $C={(c_{kj}^c)}_{n\times y}$ of the evaluated employee $X_k$.

3.

Determine the weights of performance evaluation indicators.

Step 1: Objective weights

The objective weight is obtained by entropy method. The objective weights for each indicator are denoted as $w^o={(0.104,0.175,0.110,0.130,0.097,0.131,0.078,0.081,0.095)}^T$.

Step 2: Subjective weights

The subjective weights are determined based on expert opinions. The subjective weights for each indicator are denoted as $w^s=(0.210,0.131,0.094,0.083,0.069,0.049,0.144,$ $0.120,0.100{)}^T$.

Step 3: Combination weights

Based on Equation (18), the combined weights for the 360 degree performance indicators are determined as $w={(0.157,0.153,0.102,0.107,0.083,0.090,0.111,0.100,0.097)}^T$, with $\epsilon$ set to 0.5 in this study.

4.

Consensus reaching process.

Step 1: Consensus measurement

To measure the consensus level among evaluators $M=\{M_1,M_2,\dots ,M_6\}$ on the performance evaluation of the assessed employees $X=\{X_1,X_2,X_3,X_4\}$, the evaluation information is aggregated using Equation (19). This results in the evaluation matrix, as shown in

Table 5. Comprehensive evaluation matrix.

${\mathit{Z}}^{\mathbf{i}}$	${\mathit{X}}_1$	${\mathit{X}}_2$	${\mathit{X}}_3$	${\mathit{X}}_4$
$Z^1$	$(s_3,0.15)$	$(s_3,0.20)$	$(s_3,0.06)$	$(s_3,-0.12)$
$Z^2$	$(s_3,0.06)$	$(s_3,0.41)$	$(s_3,0.15)$	$(s_3,-0.16)$
$Z^3$	$(s_3,0.18)$	$(s_3,0.03)$	$(s_3,0.31)$	$(s_3,-0.08)$
$Z^4$	$(s_3,0.16)$	$(s_3,0.28)$	$(s_3,0.15)$	$(s_3,-0.12)$
$Z^5$	$(s_3,0.06)$	$(s_3,0.43)$	$(s_3,-0.05)$	$(s_3,-0.01)$
$Z^6$	$(s_3,0.05)$	$(s_3,-0.04)$	$(s_3,0.14)$	$(s_3,-0.17)$

.

Then, by utilizing Equation (20), the comprehensive evaluation matrix of the group is obtained, $Z^c=\{(s_3,0.12),(s_3,0.23),(s_3,0.15),(s_3,-0.16)\}$.

Utilizing the definition of preference ordering to obtain the preference ordering vectors for the evaluators and the group, the results are denoted as $O^1={(2,1,3,4)}^T$, $O^2={(3,1,2,4)}^T$, $O^3={(2,3,1,4)}^T$, $O^4={(2,1,3,4)}^T$, $O^5={(2,1,4,3)}^T$, $O^6={(2,3,1,4)}^T$, $O^c={\left(3,1,2,4\right)}^T$.

In this study, the threshold of ordinal consensus degree was set as 0.1. If $OCD\{M_1,M_2,\dots M_m\}>0.1$, then evaluator $M_i$ in with $OCD(M_i)>0.1$ needs to adjust their evaluation information using the feedback adjustment mechanism. Using Equation (21), the $OCD$ for $M_i(i=1,2,\dots ,6)$ is calculated and the results are as follows: $OCD(M_1)=0.13$, $OCD(M_2)=0$, $OCD(M_3)=0.25$, $OCD(M_4)=0.13$, $OCD(M_5)=0.25$, $OCD(M_6)=0.25$. Utilizing Equations (9)–(15), the location weights for the ordered consensus degree for evaluators are calculated, denoted as $\mu ={(0.22,0.06,0.17,0.22,0.17,0.17)}^T$. Based on this, the collective $OCD\{M_1,M_2,\dots ,M_6\}=0.18$ is computed by Equation (22). If an acceptable consensus level has not been reached among the evaluators, evaluators $M_1$, $M_3$, $M_4$, $M_5$, and $M_6$ need to adjust their evaluation information according to the feedback adjustment mechanism. The specific adjustment process is detailed in the second round of the 360 degree performance evaluation process.

Step 2: Adjustment of the evaluation information

The adjusted evaluation matrix is represented as $D^k={(d_{ij}^k)}_{m\times y}$, as shown in

Table 6. Evaluation information matrix for employee $X_1$.

${\mathit{D}}^1$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$
$M_2$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$
$M_3$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$
$M_4$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$
$M_5$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$
$M_6$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$

,

Table 7. Evaluation information matrix for employee $X_2$.

${\mathit{D}}^2$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.6)\\ (s_3,0.4)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$
$M_2$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.1)\\ (s_4,0.9)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$
$M_3$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$
$M_4$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_2,0.6)\\ (s_3,0.4)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$
$M_5$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$
$M_6$	$\begin{array}{l}\{(s_2,0.6)\\ (s_3,0.4)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$

,

Table 8. Evaluation information matrix for employee $X_3$.

${\mathit{D}}^3$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$
$M_2$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_3,1)\}$
$M_3$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$
$M_4$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_2,0.6)\\ (s_3,0.4)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.3)\\ (s_3,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$
$M_5$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$
$M_6$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.1)\\ (s_3,0.9)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_3,1)\}$

and

Table 9. Evaluation information matrix for employee $X_4$.

${\mathit{D}}^4$	${\mathit{I}}_{1-1}$	${\mathit{I}}_{1-2}$	${\mathit{I}}_{1-3}$	${\mathit{I}}_{2-1}$	${\mathit{I}}_{2-2}$	${\mathit{I}}_{2-3}$	${\mathit{I}}_{3-1}$	${\mathit{I}}_{3-2}$	${\mathit{I}}_{3-3}$
$M_1$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$
$M_2$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$	$\{(s_3,1)\}$	$\{(s_4,1)\}$	$\begin{array}{l}\{(s_2,0.9)\\ (s_3,0.1)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$
$M_3$	$\begin{array}{l}\{(s_2,0.9)\\ (s_3,0.1)\}\end{array}$	$\begin{array}{l}\{(s_2,0.2)\\ (s_3,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$
$M_4$	$\begin{array}{l}\{(s_1,0.1)\\ (s_2,0.9)\}\end{array}$	$\begin{array}{l}\{(s_2,0.4)\\ (s_3,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.9)\\ (s_4,0.1)\}\end{array}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_2,0.7)\\ (s_3,0.3)\}\end{array}$
$M_5$	$\{(s_3,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\begin{array}{l}\{(s_3,0.8)\\ (s_4,0.2)\}\end{array}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.3)\\ (s_4,0.7)\}\end{array}$	$\begin{array}{l}\{(s_3,0.4)\\ (s_4,0.6)\}\end{array}$	$\begin{array}{l}\{(s_3,0.6)\\ (s_4,0.4)\}\end{array}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$
$M_6$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_2,0.5)\\ (s_3,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\{(s_2,1)\}$	$\begin{array}{l}\{(s_3,0.2)\\ (s_4,0.8)\}\end{array}$	$\begin{array}{l}\{(s_3,0.5)\\ (s_4,0.5)\}\end{array}$	$\begin{array}{l}\{(s_3,0.7)\\ (s_4,0.3)\}\end{array}$	$\begin{array}{l}\{(s_2,0.8)\\ (s_3,0.2)\}\end{array}$

.

Repeating the previously mentioned 360 degree feedback evaluation process results in the collective evaluation matrix $C={(c_{kj}^c)}_{n\times y}$. The weights for the 360 degree feedback evaluation indicators are denoted as $w=(0.155,0143,0.104,0.121,0.082,0.088,0.110,0.099,$$0.098{)}^T$. The comprehensive evaluation matrix for each evaluated employee is shown in

Table 10. Comprehensive evaluation matrix.

${\mathit{Z}}^{\mathbf{i}}$	${\mathit{X}}_1$	${\mathit{X}}_2$	${\mathit{X}}_3$	${\mathit{X}}_4$
$Z^1$	$(s_3,0.11)$	$(s_3,0.18)$	$(s_3,0.08)$	$(s_3,-0.13)$
$Z^2$	$(s_3,0.08)$	$(s_3,0.40)$	$(s_3,0.15)$	$(s_3,-0.13)$
$Z^3$	$(s_3,0.16)$	$(s_3,0.19)$	$(s_3,0.14)$	$(s_3,-0.09)$
$Z^4$	$(s_3,0.17)$	$(s_3,0.24)$	$(s_3,0.10)$	$(s_3,-0.16)$
$Z^5$	$(s_3,0.06)$	$(s_3,0.27)$	$(s_3,0.01)$	$(s_3,0.01)$
$Z^6$	$(s_3,0.12)$	$(s_3,0.11)$	$(s_3,0.12)$	$(s_3,-0.14)$

.

Equation (20) is utilized to obtain the comprehensive evaluation matrix $Z^c=\{(s_3,0.13),(s_3,0.23),(s_3,0.10),(s_3,-0.13)\}$. $OCD$ for $M_i(i=1,2,\dots ,6)$ is calculated, and the results are as follows: $OCD(M_1)=0$, $OCD(M_2)=0.13$, $OCD(M_3)=0$, $OCD(M_4)=0$, $OCD(M_5)=0.13$, $OCD(M_6)=0.25$. The collective $OCD\{M_1,M_2,\dots ,M_6\}=0.07$ is calculated by Equation (22). At this point, because $OCD\{M_1,M_2,\dots ,M_6\}<0.1$, an acceptable level of consensus has been reached among the evaluators. Therefore, the evaluation information obtained in this consensus round represents the final evaluation results for the project leaders, denoted as $X_2\succ X_1\succ X_3\succ X_4$.

Feedback stage: Finally, the evaluation results are provided as feedback to each project leader. This feedback helps them identify areas of improvement, address some issues, and enhance their performance for the next evaluation cycle.

5. Discussion

In this section, the proposed consensus-based 360 degree feedback evaluation method is compared with related methods to highlight the contribution of this study.

(1)

Evaluation information representation: Evaluation information representation is a foundational task of 360 degree feedback evaluation. Traditional 360 degree feedback evaluation methods often use numerical forms, such as assigning scores within a 1–10 range [43,46,56]. Archer et al. [56] employed a six-point scale to assess the clinical performance of participants. Baker et al. [46] used a five-point rating scale to assess the performance of the evaluated individuals. However, due to the qualitative nature of evaluation criteria, limited capacity of evaluators, and time constraints, evaluators may hesitate among multiple assessment values. Consequently, linguistic assessment approaches have been employed [40,41,57,58]. Andrés et al. [58] presented a multi-granular linguistic 360 degree feedback evaluation model using a symbolic approach based on the fuzzy linguistic two-tuple. Utilizing linguistic variables is a flexible and practical approach for portraying evaluators’ cognitive information. In contrast to these methods, this study employed linguistic distribution assessments to represent evaluators’ evaluation information, providing a more general linguistic model that can effectively represent qualitative or uncertain evaluation data [59].

(2)

Aggregation of evaluation information: Information aggregation refers to consolidating evaluation information from various sources or types into a comprehensive assessment result [35]. In the 360 degree feedback evaluation process, effectively handling the collected multi-source evaluation information and utilizing a specific aggregation mechanism to integrate opinions from multiple evaluators are critical factors for successful implementation of 360 degree feedback evaluation. Traditional 360 degree feedback evaluation methods employ weighted average operators or simple average operators to aggregate evaluation information from multiple evaluators [43,60]. Joshi et al. [60] used the weighted average operator to calculate the total score of the residents. Saberzadeh-Ardestani et al. [43] generated an overall score for each individual in the multi-source feedback (MSF) assessment system by averaging the scores received from all evaluators. This study utilized the improved OWA operator to determine the positional weights for each evaluation information. In this approach, weights are assigned based on evaluation data, giving relatively lower weights to information located at the ends and higher weights to information in the middle [53]. This approach effectively addresses evaluation information with extreme highs or lows that may contain biases, encouraging evaluators to express their opinions more genuinely and objectively.

(3)

Consensus issue: In implementing the 360 degree feedback evaluation method, multiple evaluators may come from different professional backgrounds. Due to variations in their understanding of the individuals being assessed, the evaluation information provided by evaluators may exhibit significant conflicts. Addressing the conflicts among multi-source evaluation information to render the final evaluation results more acceptable to both evaluators and the evaluated individuals is a critical issue in 360 degree feedback evaluation. However, traditional 360 degree feedback evaluation methods merely aggregate individual evaluation information into collective evaluation information to obtain the ranking of evaluation results, without fully addressing the issue of evaluators’ consensus [41,58,60]. In this study, the integration of a consensus-reaching process into the 360 degree feedback evaluation method was developed, and a feedback mediation mechanism was designed to guide evaluators in adjusting their evaluation information. This approach aims to handle conflicts in evaluators’ assessment opinions during the evaluation process, encouraging evaluators to achieve as much consensus as possible. A detailed comparison highlighting the characteristics of the proposed consensus-based 360 degree feedback evaluation method is provided in

Table 11. The comparison between the proposal and some existing 360 degree feedback evaluation methods.

360 Degree Feedback Evaluation Method	Evaluation Information Representation	Evaluator Bias	Consensus Issue
Saberzadeh-Ardestani et al. [43]	Numerical	Not considered	Not considered
Baker et al. [46]	Numerical	Not considered	Not considered
Archer et al. [56]	Numerical	Not considered	Not considered
Joshi et al. [60]	Numerical	Not considered	Not considered
Bing-You et al. [23]	Numerical	Not considered	Not considered
Andrés et al. [58]	Linguistic	Not considered	Not considered
Anisseh et al. [40]	Linguistic	Not considered	Not considered
Espinilla et al. [41]	Linguistic	Not considered	Not considered
The proposed 360 degree feedback evaluation method	Linguistic	Considered in information aggregation	Considered in consensus model

.

6. Conclusions

The primary contribution of this study is the enhancement of the 360 degree feedback evaluation method. A consensus-based 360 degree feedback evaluation method has been developed to elevate the precision and scientific validity of the final evaluation results. Additionally, it is crucial to acknowledge that within the domain of human resource management in enterprises, there are various assessment issues, such as employee recruitment, talent selection, performance appraisal, and diverse decision-making matters associated with personnel management. The improved 360 degree feedback evaluation method proposed in this paper is equally applicable to a range of employee assessment challenges in human resource management.

The primary research contents of this paper are as follows:

(1)

We investigated a 360 degree feedback evaluation method with linguistic distribution assessments. The study utilized linguistic distribution assessments to represent evaluators’ evaluation information, adapting to evaluators’ expression habits while capturing the uncertainty in the evaluation data. Additionally, a combination weighting method was employed to determine the indicator weights, striking a balance between the subjective importance of the indicators and the objective data information.

(2)

We designed a 360 degree feedback evaluation aggregation mechanism based on an improved OWA operator. The improved OWA operator determines the positional weights of each evaluation information, applying lower weights to extreme evaluation information. Research findings indicate that this approach mitigates the significant impact of extreme appraisal information resulting from individual biases, resulting in fairer evaluation outcomes.

(3)

We constructed a consensus-based 360 degree feedback evaluation method. We utilized group consensus decision-making methods to measure the degree of consensus among evaluators and devised a feedback adjustment mechanism to guide evaluators with lower consensus levels within the group to adjust their appraisal perspectives effectively, facilitating the achievement of consensus among evaluators.

We acknowledge several limitations in our proposed 360 degree feedback evaluation method, and these issues warrant resolution in future research.

(1)

In the evaluation process, the subjective factors such as the knowledge, experience, and cognitive levels of the individuals being evaluated, as well as objective factors like the complexity and uncertainty of evaluation issues, can lead different people to have varying interpretations of the same linguistic terms [61]. Research that incorporates personalized individual semantics should also be integrated into the 360 degree feedback evaluation method.

(2)

In the 360 degree feedback evaluation process, the evaluation indicators are not isolated, and there exists interaction among them, thereby influencing the final evaluation results. The presence of such interaction and correlation adds complexity to the evaluation process, potentially necessitating a more careful design of the evaluation indicator system or the adoption of more sophisticated methods or models, such as the Sugeno integral-based ordered weighted maximum operator (OWMax) [62], to handle the interactions among indicators and ensure the accuracy of comprehensive assessments.

(3)

With the development of information network technology, an organization’s human resource management is moving towards digitization and informatization, significantly enhancing the efficiency of human resource management. Information-based human resource management has seen vigorous growth [63]. The 360 degree feedback evaluation consensus model constructed in this paper can be further optimized with the help of information technology to design organizational performance evaluation systems or employee evaluation systems. This can enable networked 360 degree feedback evaluation management, ultimately improving the efficiency of performance evaluation work.