A Cross-Efficiency Evaluation Method Based on Evaluation Criteria Balanced on Interval Weights

Abstract

Cross-efficiency evaluation approaches and common set of weights (CSW) approaches have long been suggested as two of the more important and effective methods for the ranking of decision making units (DMUs) in data envelopment analysis (DEA). The former emphasizes the flexibility of evaluation and its weights are asymmetric, while the latter focuses on the standardization of evaluation and its weights are symmetrical. As a compromise between these two approaches, this paper proposes a cross-efficiency evaluation method that is based on two types of flexible evaluation criteria balanced on interval weights. The evaluation criteria can be regarded as macro policy—or means of regulation—according to the industry’s current situation. Unlike current cross-efficiency evaluation methods, which tend to choose the set of weights for peer evaluation based on certain preferences, the cross-efficiency evaluation method based on evaluation criterion determines one set of input and output weights for each DMU. This is done by minimizing the difference between the weights of the DMU and the evaluation criteria, thus ensuring that the cross-evaluation of all DMUs for evaluating peers is as consistent as possible. This method also eliminates prejudice and arbitrariness from peer evaluations. As a result, the proposed cross-efficiency evaluation method not only looks for non-zero weights, but also ranks efficient DMUs completely. The proposed DEA model can be further extended to seek a common set of weights for all DMUs. Numerical examples are provided to illustrate the applications of the cross-efficiency evaluation method based on evaluation criterion in DEA ranking.

1. Introduction

Data envelopment analysis (DEA) is a practical methodology originally proposed by Charnes et al. [1]. Since that time, DEA has been widely studied and applied all over the world, and the method has been further developed and expanded by many scholars. In recent years, the outstanding studies in DEA can be found in the literature [2,3,4,5,6,7,8]. DEA is used to evaluate the performance of a group of decision making units (DMUs) using multiple inputs to produce multiple outputs. DEA method requires each decision-making unit to evaluate its efficiency and assign the most favorable weight to itself. In addition, efficiency is the optimistic efficiency of DMU, which should not be greater than 1. If the efficiency value of the decision unit is equal to 1, the DMU is called a DEA efficient DMU. Otherwise, DMU is considered to be a non DEA efficient DMU.

The traditional DEA method has two major drawbacks. The first is a lack of discrimination, and the second is the existence of unrealistic weights. The DEA method allows each DMU to evaluate its efficiency with the most favorable weights. In this way, more than one DMU is often evaluated as DEA efficient, and these DMUs cannot be further distinguished. Therefore, the lack of discrimination is one of the main defects of the DEA method. This also leads to another important problem. The input and output that is beneficial to a particular DMU will be weighted heavily, while the input and output that is unfavorable to the DMU will be weighted lightly, or even ignored. As a result, weighting for self-assessment can sometimes be unrealistic.

Studies to overcome the weakness of DEA’s discrimination power are grouped into two trends. One remedy is the cross-efficiency method suggested by Sexton et al. [9], which introduces a secondary goal. The most commonly used methods include the benevolent and aggressive cross-efficiency assessment proposed by Doyle and Green [10], both of which are calculated using the weights that are benevolent or aggressive to peers. Wang & Chin [11] proposed a neutral cross-efficiency evaluation method, in which the attitude of decision makers is neutral and there is no need to make a choice between benevolent and aggressive formulas. Liang et al. [12] put forward the game cross-efficiency evaluation method. Using the idea of game, each DMU is regarded as an independent player, and bargains between the optimistic efficiency. In addition, Jahanshahloo et al. [13] proposed symmetrical weight distribution technology. This technique rewards the DMU with symmetrical selection weights. Wu et al. [6] proposed a DEA model with balanced weight. The second goal is to reduce the number of zero weights and the large differences in weighted data. Ruiz [14] proposed the cross-efficiency evaluation of direction distance function for fractional programming. Cook and Zhu [15] proposed a unit invariant multiplicative DEA model, which can directly obtain the maximum and unique cross-efficiency scores of each DMU. Wu [7] proposed to use the target recognition model as a means to obtain the reachable targets of all DMUs. Several secondary objective models are proposed for weight selection. These models consider the expected and unexpected cross-efficiency goals of all DMUs. Other cross-efficiency evaluation methods are discussed in Wu and Chu [8], Oral et al. [16], Oukil [17], Carrillo [18], and Shi et al. [19].

Another remedy is the common set of weights (CSW) approach in DEA, which was first suggested by Cook et al. [20]. This method utilizes the idea of common weights to measure the relative efficiency of highway maintenance patrols. Years later, this study was further developed by Jahanshahloo et al. [21], Kao & Hung [22], and Liu & Peng [23]. In more recent studies, Amir et al. [24] proposes a novel TCO-based model in which a common set of weights imprecise DEA (CSW-IDEA) is used to address the managerial and technical issues of handling weighting schemes and imprecise data. Hossein et al. [25] suggests a novel method for determining the CSWs in a multi-period DEA. The CSWs problem is formulated as a multi-objective fractional programming problem. Then, a multi-period form of the problem is formulated, and the mean efficiency of the DMUs is maximized, while their efficiency variances are minimized. The CSW approaches have been developed to find a common set of weights for all DMUs, in order to overcome the shortcomings of the weights flexibility method, where each DMU can take its own most desirable weight.

From the literature review above, all the cross-efficiency evaluation methods are formulated so that each DMU chooses one set of weights determined by the CCR model (self-evaluation model proposed by Charnes, Cooper and Rhodes) that has alternate optima solutions. When a DMU evaluates its peers, the DMU selects one set of ideal weights from alternative weights by means of the quadratic optimization of the target function from various angles. In other words, cross-evaluation is an evaluation method, which determines a set of weights for each DMU, in order to rate itself and its peers in consideration of the diversity of DMUs. The common weight evaluation method, which determines a common set of weights as the common evaluation criteria, is used to evaluate each DMU, without considering the flexibility of the DMUs, and belongs to a non-differentiating evaluation method.

In terms of practical applications, differences between homogeneous DMUs still exist, such as scale, history, culture, and region. Common weight evaluation is obviously unfair, because this method does not take into account the differences of the DMUs. However, there are also shortcomings in cross-evaluation, such as the fact that each DMU excessively enlarges the weights of its own superiority indicators and ignores the importance of input–output indicators, thus forming unrealistic and subjective evaluation conclusions. The combination of the two methods discussed above is more meaningful. That is, each DMU can take its own desirable weight, which is obtained under the constraints of an objective criterion. Therefore, we propose the cross-evaluation method based on evaluation criteria in this paper.

In this paper, we propose a series of DEA models for cross-efficiency evaluation based on evaluation criteria. The evaluation criteria may be formed based on the overall situation of the industry, or the performance of some representative enterprises. Accordingly, two evaluation criteria are proposed, which are balanced on the interval weights of input–output variables. One is based on the eclectic decision-making method, which takes the aggregation of the minimum of upper limit of interval weights and maximum of lower limit of interval weights. The harmonic coefficient $\alpha$ is introduced into the eclectic decision-making method to increase the flexibility of evaluation criteria. The other is based on weighted mathematical expectation. Because the importance of each DMU is different in cross-evaluation, we introduce the parameter $p_j(j=1,2,\dots ,n)$ into the evaluation criteria as a weight that reflects the position of a DMU. Mathematical expectations are weighted and added by $p_j$ to form the evaluation criteria. Then, the proposed method based on an evaluation criterion determines one set of input and output weights for each DMU. This is done by minimizing the deviations of input and output weights for peer evaluation from a standard criterion. In this way, aside from reducing zero weights, the weights for peer-evaluation (which are closer and more concentrated) are more realistic to peers.

The rest of the paper is organized as follows. Section 2 describes the cross-efficiency evaluations, mainly including aggressive and benevolent formulations. The evaluation criterion balanced on interval weights is developed in Section 3. The DEA models for cross-efficiency evaluation based on evaluation criteria are extended in Section 4. Numerical examples are demonstrated in Section 5. Conclusions are offered in Section 6.

2. The Efficiency Evaluation

Suppose there are n DMUs to be evaluated against m inputs and s outputs. Denote by $x_{ij}$ ($i=1,\dots ,m$) and $y_{rj}$ ($r=1,\dots ,s$) the input and output values of DMU_j ($j=1,\dots ,n$), whose efficiencies are defined as follows. Consider a DMU, say, DMU_k, $k\in \{1,\dots ,n\}$, whose efficiency relative to the other DMUs can be measured by the following CCR model (Charnes et al. [1]):

$$\mathrm{Maximize} {\theta }_{kk}=\frac{{\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rk}}}{{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ik}}},$$

(1)

$$\mathrm{subject} \mathrm{to} {\theta }_{jk}=\frac{{\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rj}}}{{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ij}}}\le 1, j=1,\dots ,n,$$

$$u_{rk}\ge 0, r=1,\dots ,s,$$

$$v_{ik}\ge 0, i=1,\dots ,m,$$

which aims to find a set of input and output weights that are most favourable to DMU_k. The Charnes and Cooper transformation can be equivalently transformed into the linear program (LP) below for the solution:

$$\mathrm{Maximize} {\theta }_{kk}={\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rk}},$$

(2)

$$\mathrm{subject} \mathrm{to} {\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ik}}=1,$$

$${\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rj}}-{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ij}}\le 0, j=1,\dots ,n,$$

$$u_{rk}\ge 0, r=1,\dots ,s,$$

$$v_{ik}\ge 0, i=1,\dots ,m.$$

Let $u_{rk}^{\ast }$ ($r=1,\dots ,s$) and $v_{ik}^{\ast }$ ($i=1,\dots ,m$) be the optimal solution to the above model. Then, ${\theta }_{kk}^{\ast }={\displaystyle {\sum }_{r=1}^s{u}_{rk}^{\ast }{y}_{rk}}$ is referred to as the CCR-efficiency of DMU_k, which is the best relative efficiency that DMU_k can achieve, and reflects the self-evaluated efficiency of DMU_k. As such, ${\theta }_{jk}={\displaystyle {\sum }_{r=1}^s{u}_{rk}^{\ast }{y}_{rj}}/{\displaystyle {\sum }_{i=1}^m{v}_{ik}^{\ast }{x}_{ik}}$ is referred to as a cross-efficiency of DMU_j and reflects the peer evaluation of DMU_k to DMU_j ($j=1,\dots ,n; j\ne k$).

Model (2) is solved n times, with a different DMU being solved each time. As a result, n DMUs will have n sets of input and output weights, and each DMU will have (n − 1) cross-efficiency, plus a CCR-efficiency, which together form a cross-efficiency matrix. The average cross-efficiency for DMU_k is $\frac{1}{n}{\displaystyle \sum _{k=1}^n{\theta }_{jk}}$ ($j=1,\dots ,n$), where ${\theta }_{jk}$ ($k=1,\dots ,n;j=k$) are the CCR-efficiencies of the n DMUs, that is, ${\theta }_{jk}={\theta }_{kk}^{\ast },j=k$.

Note that model (2) may have multiple optimal solutions. If the input and output weights are not unique, the use of cross efficiency evaluation will be destroyed. In order to solve this problem, a remedy proposed by Sexton et al. [9] is to introduce a secondary goal, one which optimizes the input and output weights while maintaining the CCR-efficiency determined by model (2). Doyle and green [10] proposed the most commonly used secondary goals, as follows:

$$\mathrm{Minimize} {\displaystyle \sum _{r=1}^s{u}_{rk}\left({\displaystyle \sum _{j=1,j\ne k}^n{y}_{rj}}\right)},$$

(3)

$$\mathrm{subject} \mathrm{to} {\displaystyle \sum _{i=1}^m{v}_{ik}\left({\displaystyle \sum _{j=1,j\ne k}^n{x}_{ij}}\right)} = 1,$$

$${\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rk}}-{\theta }_{kk}^{\ast }{\displaystyle \sum _{i=1}^m{v}_i{x}_{ik}}=0,$$

$${\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rj}}-{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ij}}\le 0, j=1,\dots ,n; j\ne k,$$

$$u_{rk}\ge 0, r=1,\dots ,s,v_{ik}\ge 0, i=1,\dots ,m,$$

and

$$\mathrm{Maximize} {\displaystyle \sum _{r=1}^s{u}_{rk}\left({\displaystyle \sum _{j=1,j\ne k}^n{y}_{rj}}\right)},$$

(4)

$$\begin{gathered} \mathrm{subject} \mathrm{to} {\displaystyle \sum _{i=1}^m{v}_{ik}\left({\displaystyle \sum _{j=1,j\ne k}^n{x}_{ij}}\right)} = 1, \\ {\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rk}}-{\theta }_{kk}^{\ast }{\displaystyle \sum _{i=1}^m{v}_i{x}_{ik}}=0, \\ {\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rj}}-{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ij}}\le 0,j=1,\dots ,n; j \ne k, \\ {u}_{rk}\ge 0, r=1,\dots ,s, \\ {v}_{ik}\ge 0, i=1,\dots ,m. \end{gathered}$$

Model (3) is called the aggressive formula of cross-efficiency evaluation, which aims to minimize the cross-efficiency of peers in some way. Instead, model (4) is called the benevolence formula for cross-efficiency evaluation. To some extent, the model improves the cross-efficiency of other EMUs. These two models optimize input and output weights in two different ways. Therefore, there is no guarantee that they can lead to the same efficiency ranking or conclusion of N DMUs.

In addition, other secondary goals or models are mentioned in the DEA literature. For interested readers, please refer to Sexton et al. [9], Liang et al. [12], Wang et al. [26,27], and Jahanshahloo et al. [13]. These secondary goals focus on how to uniquely determine input and output weights. In the next section, we focus on the diversity of input and output weights, and develop some alternative DEA models to minimize the weights differences used to evaluate peers. This enables cros-efficiency to be evaluated with more reasonable input and output weights.

3. Evaluation Criteria Balanced on Interval Weights of N DMUs

From our perspective, when a DMU is given an opportunity to unilaterally decide upon a set of input and output weights for evaluating peers, in addition to being as favourable as possible to itself, the DMU tends have specific preferences when choosing the set of weights. This preferential choice of weights leads to an unfair and arbitrary situation for peers. Therefore, we need to establish some evaluation criteria to ensure the relative consistency of cross-evaluation for peers by eliminating prejudice. That is, we not only look for non-zero weights, but we also seek to ensure that weights for evaluating peers are as close as possible by taking a certain evaluation criterion as a reference point. That means we seek to minimize the difference between the weights of each DMU and the evaluation criteria. Two evaluation criteria are proposed as follows:

3.1. The DEA Modes of Interval Weights

For a DMU, we get a set of maximum weights from among the alternate optima in the CCR model by maximizing the weight of each variable (including input and output variables). By contrast, a set of minimum weights is obtained by minimizing the weight of each variable. Consider an efficient DMU, say, DMU_k. The maximum attainable value of $u_{rk}$ or $v_{ik}$ of the DMU_k can be obtained by solving the following LP model:

$$\mathrm{Maximize} {\delta }_1{u}_{rk}+{\delta }_2{v}_{ik},$$

(5)

$$\begin{gathered} \mathrm{subject} \mathrm{to} {\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ik}}=1, {\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rk}}-{\theta }_{kk}^{\ast }{\displaystyle \sum _{i=1}^m{v}_i{x}_{ik}}=0, \\ {\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rj}}-{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ij}}\le 0, j=1,\dots ,n; j \ne k, \\ {u}_{rk}\ge 0, r=1,\dots ,s, \\ {v}_{ik}\ge 0, i=1,\dots ,m. \\ {\delta }_1+{\delta }_2=1, \\ {\delta }_1,{\delta }_2=0\mathrm{or}1, \end{gathered}$$

where ${\delta }_1,{\delta }_2$ are a type of Boolean and ${\delta }_1+{\delta }_2=1$. When ${\delta }_1=1$, the optimization goal is $u_{rk}$, and when ${\delta }_2=1$, the optimization goal is $v_{ik}$. Let $u_{rk}^{+}$ ($r=1,\dots ,s$) and $v_{ik}^{+}$ ($i=1,\dots ,m$) be the maximum attainable value (upper bound) obtained by the above model. To obtain the minimum attainable value (lower boundary) of $u_{rk}^{}$ and $v_{ik}^{}$, one simply changes the objective function to minimizing $u_{rk} or v_{ik}$, and $u_{rk}^{-}$ ($r=1,\dots ,s$) and $v_{ik}^{-}$ ($i=1,\dots ,m$) is obtained as the minimum attainable value. For $u_{rk}^{}$ (and $v_{ik}^{}$) of a DMU_k, this method leads to an interval weight $\left[u_{rk}^{-},u_{rk}^{+}\right]$ $\left(or\left[v_{ik}^{-},v_{ik}^{+}\right]\right)$, and then n DMUs lead to n interval weights $\left[u_{r1}^{-},u_{r1}^{+}\right],\dots \left[u_{rj}^{-},u_{rj}^{+}\right],\dots ,\left[u_{rn}^{-},u_{rn}^{+}\right]$ for $u_r^{}\left(r=1,2,\dots ,s\right)$, and interval weights of $v_i^{}\left(i=1,2,\dots ,m\right)$ for the same thing.

3.2. Evaluation Criteria Based on the Eclectic Decision-Making Method

There are multiple methods in the criteria for evaluating peers based on the interval weights. The optimistic decision method would consider the upper limit of the interval weights as a peer evaluation criterion. The pessimistic decision method can also take the lower limit as a peer evaluation criterion. However, the decision makers cannot be absolutely optimistic or pessimistic; they are more likely to be between the two. Therefore, the eclectic decision-making method is introduced as peer evaluation criteria. We can use $u_r^{+\min }$ to express the minimum of the upper limit, and $u_r^{-\max }$ to denote the maximum value of the lower limit. Both $u_r^{+\min }$ and $u_r^{-\max }$ reflect the idea of the eclectic decision-making method, and their formula is as follows:

$$\begin{gathered} u_r^{+\min }=\min \left(u_{rj}^{+}\right),j=1,2,\dots ,n, \\ {u}_r^{-\max }=\max \left(u_{rj}^{-}\right),j=1,2,\dots ,n. \end{gathered}$$

To make the evaluation criteria more realistic and flexible in terms of weights, a harmonic coefficient, namely $\alpha$, is introduced to combine $u_r^{+\min }$ and $u_r^{-\max }$. That is, the parameter $\alpha$ displays a preference for $u_r^{+\min }$, and $1-\alpha$ is introduced as a damping coefficient reflecting the preference for $u_r^{-\max }$. Then, we use the ${\overline{u}}_r$ to express the evaluation criterion of n DMUs in $u_{rj}^{}(j=1,2,\dots ,n)$. The same procedures are used in $v_{ij}^{}\left(j=1,2,\dots ,n\right)$, where ${\overline{v}}_i$ is denoted as the evaluation criterion. In this way, one set of weights (${\overline{u}}_r$, ${\overline{v}}_i$) (as shown in Formula (6a) and (6b)) is obtained as an evaluation criterion, which in turn is a balance between the maximum and the minimum attainable values of n DMUs in $u_{rj}^{}$ and $v_{ij}^{}$. This ensures the evaluation criterion meets a variety of application requirements. This is the first evaluation criterion, which is based on the eclectic decision-making method, or ECED for short:

$${\overline{u}}_r=u_r^{+\min }\alpha +u_r^{-\max }\left(1-\alpha \right),\alpha \le 1;r=1,2,\dots s,$$

(6a)

$${\overline{v}}_i=v_i^{+\min }\alpha +v_i^{-\max }\left(1-\alpha \right),\alpha \le 1;i=1,2,\dots ,m.$$

(6b)

3.3. Evaluation Criterion Based on Weighted Mathematical Expectation

If we denote the optimal solution of the above model (5) by $\left[u_{1j}^{-},u_{1j}^{+}\right],\dots ,\left[u_{sj}^{-},u_{sj}^{+}\right],\left[v_{1j}^{-},v_{1j}^{+}\right],\dots ,\left[v_{mj}^{-},v_{mj}^{+}\right]$ for corresponding DMU_j $,j=1,2,\dots ,n$, then solving model (5) n times would lead to n sets of optimal solutions available for n DMUs, which in turn would form an interval weight matrix (IWM). An IWM is shown as follows:

$$IWM=\left[\begin{array}{cccccc}\left[u_{11}^{-},u_{11}^{+}\right]& ,\dots ,& \left[u_{s1}^{-},u_{s1}^{+}\right],& \left[v_{11}^{-},v_{11}^{+}\right]& ,\dots ,& \left[v_{m1}^{-},v_{m1}^{+}\right]\\ \left[u_{12}^{-},u_{12}^{+}\right]& ,\dots ,& \left[u_{s2}^{-},u_{s2}^{+}\right],& \left[v_{12}^{-},v_{12}^{+}\right]& ,\dots ,& \left[v_{m2}^{-},v_{m2}^{+}\right]\\ ⋮& ,\dots ,& ⋮& ⋮& ,\dots ,& ⋮\\ \left[u_{1n}^{-},u_{1n}^{+}\right]& ,\dots ,& \left[u_{sn}^{-},u_{sn}^{+}\right],& \left[v_{1n}^{-},v_{1n}^{+}\right]& ,\dots ,& \left[v_{mn}^{-},v_{mn}^{+}\right]\end{array}\right]$$

In practical applications, it is unfair and subjective to use the upper or lower bounds of the interval weight by the criterion for peer evaluating. Rather, an application should consider the level that most DMUs can reach. However, we cannot obtain sufficient knowledge of the weight distribution information in the interval. The term probability distribution refers to the probability rule used to express the value of random variables. We may assume that the weights satisfy one or other forms of probability distribution (such as normal distribution or uniform distribution). The mathematical expectation of probability distribution can best represent the interval weight used as a criterion for peer evaluating when that weight obeys the probability distribution. According to the central limit theorem, the random variables approximately obey the normal distribution when the sample size is large enough. Supposing that weights satisfy the standard normal distribution in this paper, the mathematical expectation matrix-based interval weight matrix (IWM) is as follows:

$$\left(\begin{array}{l}{\overline{\mu }}_{11},\dots ,{\overline{\mu }}_{s1},{\overline{\eta }}_{11},\dots ,{\overline{\eta }}_{m1}\\ {\overline{\mu }}_{12},\dots ,{\overline{\mu }}_{s2},{\overline{\eta }}_{12},\dots ,{\overline{\eta }}_{m2}\\ \begin{array}{cccccc}⋮& ,\dots ,& ⋮& ⋮& ,\dots ,& ⋮\end{array}\\ {\overline{\mu }}_{1n},\dots ,{\overline{\mu }}_{sn},{\overline{\eta }}_{1n},\dots ,{\overline{\eta }}_{mn}\end{array}\right),$$

where ${\overline{\mu }}_{rj}(r=1,2,\dots ,s;j=1,2,\dots ,n)$ is the mathematical expectation of the interval weights $\left[u_{rj}^{-},u_{rj}^{+}\right]$ $(s=1,2,\dots ,s;j=1,2,\dots ,n)$, and ${\overline{\eta }}_{ij}(i=1,2,\dots ,m;j=1,2,\dots ,n)$ is the mathematical expectation of interval weights $\left[v_{ij}^{-},v_{ij}^{+}\right]$ $(i=1,2,\dots ,m;j=1,2,\dots ,n)$. Where

$${\overline{\mu }}_{rj}=\frac{u_{rj}^{-}+u_{rj}^{+}}{2},$$

$${\overline{\eta }}_{ij}=\frac{v_{ij}^{-}+v_{ij}^{+}}{2}.$$

For DMU_j, ${\overline{\mu }}_{rj}$ and ${\overline{\eta }}_{ij}$ may not be a real weight, and this represents the compromise decision for the decision-makers, as an objective evaluation criterion. There are n sets of mathematical expectations representing the evaluation criteria for n DMUs. As the importance of each DMU is different in cross-evaluation, let $p_j$ be the weight of DMU_j, which embodies the position of DMU_j. Therefore, the evaluation criterion based on weighted mathematical expectation (ECWME) is calculated as follows:

$${\overline{u}}_r={\overline{u}}_{r1}{p}_1+{\overline{u}}_{r2}{p}_2+\dots +{\overline{u}}_{rn}{p}_n={\displaystyle \sum _{j=1}^n{\overline{u}}_{rj}{p}_j},r=1,2,\dots ,s;$$

(7a)

$${\overline{v}}_r={\overline{\eta }}_{i1}{p}_1+{\overline{\eta }}_{i2}{p}_2+\dots +{\overline{\eta }}_{in}{p}_n={\displaystyle \sum _{j=1}^n{\overline{\eta }}_{ij}{p}_j},i=1,2,\dots ,m,$$

(7b)

where

$$1= p_1 + p_1 +\dots +p_n.$$

4. DEA Models for Cross-Efficiency Evaluation Based on Evaluation Criteria

It is well known that each DMU personally chooses the profile of weights to be used in the cross-efficiency evaluation. Therefore, the DMU’s choice is often prejudiced. One DMU’s attitude towards its peers may be aggressive, benevolent, indifferent, or something else. Those prejudicial attitudes of DMUs to peers need to be avoided in many applications. This is why the weights that are chosen by each DMU for peer evaluations should be based on an evaluation criterion, as stated in part 3. We propose a method that makes a selection between alternate optima of CCR. This is done by, to the greatest extent possible, reducing the degree of deviation of weights for peer evaluation from the evaluation criteria. In other words, our purpose is to look for the profiles of DMU weights that are closest to the evaluation criterion. To do this, a nonlinear programming model is proposed as follows:

$$\mathrm{Minimize} {\displaystyle \sum _{r=1}^s\left|u_{rk}-{\overline{u}}_r\right|y_{rk}+}{\displaystyle \sum _{i=1}^m\left|v_{ik}-{\overline{v}}_i\right|}{x}_{ik},$$

(8)

$$\mathrm{subject} \mathrm{to} {\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ik}}=1,$$

$${\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rk}}-{\theta }_{kk}^{\ast }{\displaystyle \sum _{i=1}^m{v}_i{x}_{ik}}=0,$$

$${\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rj}}-{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ij}}\le 0, j=1,\dots ,n;j\ne k,$$

$$u_{rk}\ge 0, r=1,\dots ,s,$$

$$v_{ik}\ge 0, i=1,\dots ,m,$$

where $({\overline{u}}_1,{\overline{u}}_2,\dots ,{\overline{u}}_s,{\overline{v}}_1,{\overline{v}}_2,\dots ,{\overline{v}}_m)$ are obtained from Formula (6a) and (6b) or Formula (7a) and (7b), which is the evaluation criterion for the evaluating peers. Also, $(u_{1k}^{\ast },u_{2k}^{\ast },\dots ,u_{sk}^{\ast },v_{1k}^{\ast },v_{2k}^{\ast },\dots ,v_{mk}^{\ast })$ are the variables that need to be solved. The above model needs to be solved n times, one time for each DMU. The purpose of the model is to minimize the deviation of input and output weights from the evaluation criterion. In other words, each DMU obtains one set of weights that is favorable to that DMU and is also as close to the evaluation criterion of the evaluating peers as possible.

Model (8) is a form of nonlinear programming. To make this nonlinear model (8) become capable of linear programming, we introduce the new decision variables ${\varphi }_r,{\delta }_i\ge 0,$ $r=1,2,\dots ,s,i=1,2,\dots ,m$, and we add $v_{ik}{x}_{ik}-{\delta }_i\le {\overline{v}}_i{x}_{ik},$ $v_{ik}{x}_{ik}+{\delta }_i\ge {\overline{v}}_i{x}_{ik}$, $u_{rk}{y}_{rk}-{\varphi }_r\le {\overline{u}}_r{y}_{rk},$ $u_{rk}{y}_{rk}+{\varphi }_r\ge {\overline{u}}_r{y}_{rk}$ to the set of constraints. Thus, we minimize the linear objective function ${\displaystyle \sum _{r=1}^s{\varphi }_r+{\displaystyle \sum _{i=1}^m{\delta }_i}}$.

$$\mathrm{Minimize} {\displaystyle \sum _{r=1}^s{\varphi }_r+{\displaystyle \sum _{i=1}^m{\delta }_i}},$$

(9)

$$\mathrm{subject} \mathrm{to} {\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ik}}=1,$$

$${\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rk}}-{\theta }_{kk}^{\ast }{\displaystyle \sum _{i=1}^m{v}_i{x}_{ik}}=0,$$

$${\displaystyle \sum _{r=1}^s{u}_{rk}{y}_{rj}}-{\displaystyle \sum _{i=1}^m{v}_{ik}{x}_{ij}}\le 0, j=1,\dots ,n;j\ne k,$$

$$v_{ik}{x}_{ik}-{\delta }_i\le {\overline{v}}_i{x}_{ik},v_{ik}{x}_{ik}+{\delta }_i\ge {\overline{v}}_i{x}_{ik}$$

$$u_{rk}{y}_{rk}-{\varphi }_r\le {\overline{u}}_r{y}_{rk},u_{rk}{y}_{rk}+{\varphi }_r\ge {\overline{u}}_r{y}_{rk},$$

$${\varphi }_r,{\delta }_i\ge 0,$$

$$u_{rk}\ge 0, r=1,\dots ,s,$$

$$v_{ik}\ge 0, i=1,\dots ,m.$$

If we denote the optimal solution of model (9) by $(u_{1k}^{\ast },u_{2k}^{\ast },\dots ,u_{sk}^{\ast },v_{1k}^{\ast },v_{2k}^{\ast },\dots ,v_{mk}^{\ast })$ for the corresponding DMU_k, then the cross-efficiency of a given DMU_j with the profile of weights provided by DMU_k will be obtained as follows:

$$E_{kj}=\frac{{\displaystyle \sum _{r=1}^s{u}_{rk}^{\ast }{y}_{rj}}}{{\displaystyle \sum _{i=1}^m{v}_{ik}^{\ast }{x}_{ij}}}.$$

Therefore, the cross-efficiency score of DMU_j is the average of these cross-efficiencies:

$${\overline{E}}_j=\frac{1}{n}{\displaystyle \sum _{k=1}^n{E}_{kj}}.$$

Besides cross-evaluation, the weights solved by model (9) are similar to their evaluation criteria derived from formulae (67). The weights are relatively concentrated, and their coefficient of variation is small, which can further be extended to be a common set of weights (CSW) based on evaluation criteria for all DMUs. The extended CSW deviates little from the cross weights solved by model (9), which are easily accepted by each DMU.

Let $(u_{1k}^{\ast },u_{2k}^{\ast },\dots ,u_{sk}^{\ast },v_{1k}^{\ast },v_{2k}^{\ast },\dots ,v_{mk}^{\ast })$, solved by model (9), be the optimal weights of efficient DMU_k, and let $(u_{1t}^{CCR},u_{2t}^{CCR},\dots ,u_{st}^{CCR},v_{1t}^{CCR},v_{2t}^{CCR},\dots ,v_{mt}^{CCR})$, solved by model (2), be the optimal weights of inefficient DMU_t. Suppose there are $n1$ efficient DMUs and $n2$ inefficient DMUs. The CSW based on evaluation criteria is obtained as follows:

$$u_r^{CSW}=\frac{1}{n1+n2}\left({\displaystyle \sum _{j=1}^{n1}{u}_{rj}^{\ast }}+{\displaystyle \sum _{j=1}^{n2}{u}_{rj}^{CCR}}\right),$$

(10a)

$$v_i^{CSW}=\frac{1}{n1+n2}\left({\displaystyle \sum _{j=1}^{n1}{v}_{ij}^{\ast }}+{\displaystyle \sum _{j=1}^{n2}{v}_{ij}^{CCR}}\right).$$

(10b)

We consider the inefficient DMU and efficient DMU separately, because, in the case of low efficiency DMU, the weights’ distribution provided by the proposed method is the only optimal solution to the weights in the CCR model. This same train of thought can be seen in Nuria Ramón (2011). This paper, however, differs from the idea proposed in Nuria Ramón (2011). The cross-evaluation proposed in this paper allows for the inefficient DMUs that keep the weights in the CCR model. For efficient DMUs, the weights for evaluating peers are obtained by minimizing deviation from the evaluation criterion, rather than reducing the differences between the weights of any two DMUs. In addition, the model proposed by Nuria Ramón needs to be solved $\raisebox{1ex}{$n^2$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$ times, and this approach is not suitable for a large number of DMUs. The measure of our approach is different. We focus attention on the deviation of input and output weights from the evaluation criterion. The model in this paper only needs to be solved n times and needs to be more practical.

5. Numerical Examples

Example 1. 

In this section, we provide one numerical example to illustrate the proposed methods detailed above. We consider the numerical examples with the data presented in

Table 1. Data and efficiency of Example 1.

	x1	x2	x3	y1	y2	y3	CCR-Efficiency
DMU1	12	400	20	60	35	17	1
DMU2	19	750	70	139	41	40	1
DMU3	42	1500	70	225	68	75	1
DMU4	15	600	100	90	12	17	0.819
DMU5	45	2000	250	253	145	130	1
DMU6	19	730	50	132	45	45	1
DMU7	41	2350	600	305	159	97	1

. The case of seven academic departments in a university is presented in Table 1, with three inputs and three outputs.

• Input 1: Total number of academic staff (x₁)
• Input 2: Academic staff salaries in thousands of pounds (x₂)
• Input 3: Support staff salaries in thousands of pounds (x₃)
• Output 1: Total number of undergraduate students (y₁)
• Output 2: Total number of postgraduate students (y₂)
• Output 3: Total number of research papers (y₃)

For a DMU, the minimum output and input weights can be obtained by solving model (5). The result is taken as the upper bound of the interval weights (UBIW). The maximum output and input weights can also be obtained by solving model (5). This result is taken as the lower bound of the interval weights (LBIW). Therefore, each DMU will get a set of interval weights, as shown in

Table 2. Weights of decision-making units (DMUs), solved by model (5).

	v1	v2	v3	u1	u2	u3
DMU1	(0, 79.59)	(0, 2.50)	(0, 50.00)	(0, 15.58)	(1.84, 28.57)	(0, 39.71)
DMU2	(0, 52.33)	(0, 1.33)	(0, 2.52)	(5.24, 7.19)	(0, 6.21)	(0, 6.79)
DMU3	(0, 11.60)	(0, 0.37)	(6.27, 14.28)	(0, 4.44)	(0, 3.01)	(0, 13.33)
DMU5	(0, 22.22)	(0, 0.50)	(0, 1.69)	(0, 1.63)	(0, 6.89)	(0, 7.69)
DMU6	(0, 52.22)	(0, 1.37)	(0, 14.27)	(0, 7.57)	(0, 10.99)	(0, 22.22)
DMU7	(7.32, 24.39)	(0, 0.29)	(0, 0.53)	(0, 3.28)	(0, 6.29)	(0, 5.41)

.

Firstly, the cross-evaluation efficiency is discussed, based on eclectic decision-making evaluation criterion (ECED). The UBIW and LBIW are obtained by Formulaes (6a) and (6b), as shown in the first and second row of

Table 3. Eclectic decision-making evaluation criterion (ECED) of all DMUs. UBIW, upper bound of the interval weights; LBIW, lower bound of the interval weights.

	v1	v2	v3	u1	u2	u3
Minimum of UBIW	11.602	0.3	0.53	1.63	3.01	5.41
Maximum of LBIW	7.32	0	6.28	5.24	1.85	0
Cross-evaluation criterion	9.46	0.15	3.40	3.43	2.43	2.71

. To make sure the evaluation criteria fall into the interval weights of all DMUs (as much as possible), the ECED is calculated with α = 0.5, as can be seen in the last row of Table 3. Each DMU attempts to obtain a set of weights that is as close (or as similar as possible) to the cross-evaluation criterion. This is done by minimizing deviation from the cross-evaluation criterion. When the evaluation criterion is the ECED, the cross-evaluation weights of each DMU solved by model (9) are shown in

Table 4. Cross evaluation weights of each DMU, solved by model (9), based on the ECED.

	v1	v2	v3	u1	u2	u3
DMU1	0.94603	0.20587	0.31493	0.34347	2.13693	0.27055
DMU2	4.79055	0.00002	0.12812	0.61358	0.09490	0.27055
DMU3	0.94603	0.00557	0.74154	0.34347	0.03571	0.27055
DMU4	6.4150	0.0062	0	0.910	0	0
DMU5	2.06488	0.00001	0.02828	0.15625	0.17446	0.27055
DMU6	4.43379	0.00002	0.31493	0.34347	0.09490	1.11982
DMU7	2.12799	0.00001	0.02124	0.22803	0.09490	0.15837

.

Secondly, another criterion is proposed, which is based on weighted mathematical expectation. There is a mathematical expectation regarding the arbitrary weight interval. Then, n sets of mathematical expectations are computed from Table 2 for n DMUs. These are seen in the second to seventh lines in

Table 5. Expectation of each DMU and the evaluation criterion based on weighted mathematical expectation (ECWME) of all DMUs.

	v1	v2	v3	u1	u2	u3
DMU1	39.795	1.25	25	7.79	15.205	19.855
DMU2	26.165	0.665	1.26	6.215	3.105	3.395
DMU3	5.8	0.185	10.275	2.22	1.505	6.665
DMU5	11.11	0.25	0.845	0.815	3.445	3.845
DMU6	26.11	0.685	7.135	3.785	5.495	11.11
DMU7	15.855	0.145	0.265	1.64	3.145	2.705
evaluation criterion	20.81	0.53	7.46	3.74	5.32	7.93

. In order to be comparable to the evaluation criterion based on eclectic decision-making (ECED), we assume that each DMU has the same status, that is, $p_1=p_2=\dots =p_7$. Then, the evaluation criterion based on weighted mathematical expectation is shown in the last line of Table 5. In

Table 6. Evaluation weights of each DMU solved by model (9), based on the ECWME.

	v1	v2	v3	u1	u2	u3
DMU1	2.08100	0.15027	0.74600	0.37400	1.83083	0.79300
DMU2	4.93131	0.00002	0.08991	0.56205	0.53200	0.00157
DMU3	1.13735	0.00001	0.74600	0.36162	0.00002	0.24848
DMU4	6.4150	0.0062	0	0.910	0	0
DMU5	1.62016	0.00001	0.10837	0	0.53200	0.17585
DMU6	2.08100	0.03173	0.74600	0.30587	0.53200	0.79300
DMU7	2.08100	0.00001	0.02445	0	0.53200	0.15889

, we show the weights for peer evaluation as solved by model (9) for the seven academic departments, which are based on the ECWME.

When comparing Table 4 and Table 6, we can see that the weights of DMU4 have not changed. This is because DMU4, which is an inefficient DMU, retains the weights in the CCR model in the cross-evaluation of this paper. In the coming section, we illustrate how the performance of the proposed approach is better than the classic cross-efficiency evaluation method in reducing zero weights. In

Table 7. Weights of each DMU solved by CCR model.

	v1	v2	v3	u1	u2	u3
DMU1	0	2.5	0	0	28.5714	0
DMU2	0	1.3333	0	7.1942	0	0
DMU3	0	0.3738	6.2762	4.4444	0	0
DMU4	64.1504	0.0629	0	9.1082	0	0
DMU5	0	0.5	0	0	4.3165	2.8777
DMU6	0	1.2756	1.3759	7.5758	0	0
DMU7	9.9403	0.2521	0	0	6.2893	0

, we show the weights solved by DEA model (2); that is, the CCR model. Cross-evaluation weights under the proposed method are limited by the constraints of the criteria, so the weights in Table 6 fluctuate around the evaluation criteria. The weights of the CCR model have no restriction of evaluation criteria. Each DMU only considers whether the weights are favourable or not, and does not need to consider the reality of the weights. So a large number of zero-value weights often appear in Table 7, which is easy to find by comparing Table 6 and Table 7. In

Table 8. Weights of each DMU solved by benevolent cross evaluation.

	v1	v2	v3	u1	u2	u3
DMU1	1.99	0.08	0	0.33	0.91	0.28
DMU2	2.87	0.07	0	0.56	0.65	0
DMU3	0	0.03	0.74	0	0.24	1.04
DMU4	5.38	0	0	0.76	0	0
DMU5	2.47	0.10	0	0.41	1.12	0.35
DMU6	2.07	0.08	0	0.34	0.94	0.30
DMU7	2.53	0.10	0	0.42	1.15	0.36

and

Table 9. Weights of each DMU solved by aggressive cross evaluation.

	v1	v2	v3	u1	u2	u3
DMU1	0	0	0.88	0	0.50	0
DMU2	0	0.11	0.12	0.68	0	0
DMU3	0	0	0.92	0.29	0	0
DMU4	4.98	0.01	0	0.76	0	0
DMU5	4.29	0	0.40	0	0	2.26
DMU6	0.97	0	0.75	0	0	1.24
DMU7	6.58	0	0	0	1.70	0

, we show the weights solved by aggressive and benevolent cross evaluation. It is particularly noticeable that the number of zero weights is sharply reduced in the proposed ECED and ECWME-based approach when comparing Table 4, Table 6, Table 7, Table 8 and Table 9.

The cross-efficiency scores and the rankings based on all the methods mentioned in this paper are provided in

Table 10. Cross-efficiency scores and rankings based on all the methods mentioned in this paper.

	ECWME	Rank	ECED (α = 0.5)	Rank	CCR	Rank	Benevolent	Rank	Aggressive	Rank
DMU1	0.91	3	0.85	3	1	1	0.97	2	0.97	1
DMU2	0.94	2	0.91	2	1	1	0.93	3	0.72	4
DMU3	0.81	5	0.79	5	1	1	0.8	6	0.77	3
DMU4	0.59	7	0.57	7	0.89	7	0.58	7	0.39	7
DMU5	0.84	4	0.82	4	1	1	0.91	4	0.66	5
DMU6	0.99	1	0.97	1	1	1	0.99	1	0.84	2
DMU7	0.79	6	0.75	6	1	1	0.9	5	0.52	6

. We find that the proposed ECED and ECWME-based method has more discrimination power than CCR-efficiency, and basically provides different ranking with benevolent and aggressive cross-evaluation. This indicates that the proposed approaches in this paper represent a new method, one which can achieve effective ranking for DMUs. In addition, cross-efficiency scores and the rankings of the proposed approach based on ECED are shown in

Table 11. Scores and rankings based on ECED when α = 0.2, α = 0.5, and α = 0.8.

	Score (α = 0.2)	Rank	Score (α = 0.5)	Rank	Score (α = 0.8)	Rank
DMU1	0.88	3	0.81	4	0.77	6
DMU2	0.93	2	0.92	2	0.94	2
DMU3	0.80	5	0.79	5	0.80	5
DMU4	0.58	7	0.59	7	0.62	7
DMU5	0.82	4	0.82	3	0.87	3
DMU6	0.99	1	0.97	1	0.99	1
DMU7	0.77	6	0.77	6	0.83	4

when harmonic coefficient α = 0.2, α = 0.5, and α = 0.8. The proposed approach’s economic meaning is that each DMU has its own inputs and outputs, which are different from others. This leads to different DMU rankings under different evaluation criteria. This may become a macro policy whose value is high or low, according to the evaluation need, by adjusting α = 0.2, 0.5, or any value between 0 and 1.

Evaluation criteria are often formulated according to the present industry, which is not the integration of all enterprises, but the embodiment of some representative enterprises. That is, each DMU has a different role in the formation of evaluation criteria. Therefore, we consider two ECWME situations in this example, when p₁ = p₂ = p₃ = p₄ = p₅ = p₆ = p₇ and 3p₁ = 2 p₂ = 6p₃ = 6p₄ = 6 p₅ = 6 p₆ = 6 p₇. The cross-efficiency scores and rankings based on ECWME under the two combinations of p are shown in

Table 12. Scores and rankings based on ECWME using two combinations of p.

	p1 = p2 = , …, = p7		p1 = 0.2, p2 = 0.3, p3 = …, = p7 = 0.1
	Score	Rank	Score	Rank
DMU1	0.89	2	0.92	3
DMU2	0.84	3	0.93	2
DMU3	0.76	5	0.80	5
DMU4	0.45	7	0.56	7
DMU5	0.82	4	0.83	4
DMU6	0.93	1	0.99	1
DMU7	0.72	6	0.79	6

.

The coefficient of variation (CV) of the weights of each variable is computed and compared with the CCR model. The proposed approach is based on ECED and ECWME, and the result can be seen in

Table 13. Coefficient of variation (CV) of multiple methods.

	CV Based on ECED			CV Based on ECWME		CCR
	α = 0.8	α = 0.5	α = 0.2	p1 = , …, = p7	p1 = 0.2, p2 = 0.3, p3 = …, = p7 = 0.1
DMU1	0.86	1.11	1.70	0.76	1.74	2.26
DMU2	0.07	0.20	0.001	0.12	0.0004	0.96
DMU3	0.43	0.28	0.34	0.33	0.31	2.14
DMU5	0.10	0.07	0.08	0.19	0.09	1
DMU6	1.43	1.58	0.76	0.57	0.63	1.87
DMU7	0.36	0.33	0.23	0.33	0.57	2.65
Mean of CV	0.54	0.60	0.52	0.38	0.56	1.81
sum of CV	3.25	3.57	3.12	2.3	3.34	10.87

. As can be seen, the mean of CV of the weights provided by the proposed approach is smaller than that of the weights provided by the CCR model. Also, the proposed ECWME-based approach is smaller than the proposed approach based on ECED. The CV of weights based on ECED and ECWME changes in line with changes in coefficient α and combinations of p.

Obviously, the CV of the weights of the two methods is very small in any case. This means that the weights of each DMU are very similar. These similar weights are further extended to be CSW, under which the score and rank based on ECED and ECWME are shown in

Table 14. Score and rank under the common set of weights (CSW) based on ECED and ECWME.

	Score and Rank Under ECWME
	α = 0.8		α = 0.5		α = 0.2		p1 = , …, = p7		p1 = 0.2, p2 = 0.3, p3 = …, = p7 = 0.1
	Score	Rank	Score	Rank	Score	Rank	Score	Rank	Score	Rank
DMU1	0.75	5	0.78	5	0.78	5	0.88	2	0.80	5
DMU2	0.94	2	0.96	2	0.96	2	0.86	3	0.94	2
DMU3	0.74	6	0.76	6	0.76	6	0.76	5	0.81	4
DMU4	0.67	7	0.67	7	0.67	7	0.47	7	0.61	7
DMU5	0.78	4	0.79	3	0.79	3	0.78	4	0.82	3
DMU6	0.95	1	0.97	1	0.97	1	0.94	1	0.99	1
DMU7	0.79	3	0.78	4	0.78	4	0.58	6	0.77	6

. From the results shown in Table 14, the CSW in any case can be used to achieve the effective ranking of DMUs, and the rankings are different from each other.

Example 2: 

In order to illustrate the rationality and feasibility of the method, data pertaining to 27 innovative machinery manufacturing enterprises in Fujian Province in 2015 are collected and evaluated in terms of four inputs and four outputs, which are defined below:

• x₁: R&D (research and development) personnel (ratio of R&D personnel to total personnel);
• x₂: Total expenditure on scientific and technological activities in the current year (in increments of $10,000);
• x₃: Total expenditure on R&D of enterprises (in increments of $10,000);
• x₄: Number of senior technicians and technicians at the end of the year (persons);
• y₁: Sales revenue of new products (services or processes) of enterprises in this year (in increments of 10,000 yuan);
• y₂: Added value of enterprises in this year (in increments of 10,000 yuan);
• y₃: Total profits realized by enterprises in this year (in increments of 10,000 yuan);
• y₄: Total labor productivity of enterprises in this year (in increments of 10,000 yuan/person).

The letters in the enterprise number are the initials of the name of the area in which the enterprise is located. For example, “PT” in “PT 1” stands for Pu Tian City of Fujian Province.

The application example involves 27 innovative machinery manufacturing enterprises in Fujian Province, located in different cities in Fujian Province. The CCR-efficiency of each enterprise is shown in last column of

Table 15. Application example.

Enterprise Number	Inputs				Outputs				CCR Efficiency
	x1	x2	x3	x4	y1	y2	y3	y4
PT1 (DMU1)	15	1361	222	27	3012	926.51	89	4.2	0.44
SM4 (DMU2)	12.2	520	435	205	2144	1146	318	5.6	0.46
QZ15 (DMU3)	16.54	226.31	226.31	7	2799.76	1118.97	158.63	8.4	0.67
LY1 (DMU4)	40	595	74.16	7	198	1554.96	220.35	11.5	1
SM1 (DMU5)	11.18	0.396	317.3	9	3100	603.6	107.37	3.7	1
ND1 (DMU6)	12.61	224.6	224.8	119	3436.8	581.6	1177	4.9	1
NP1 (DMU7)	4.91	349	187	6	3801	1404	193	4.3	1
PT6 (DMU8)	100	273	285	198	2533	1716	167	8.7	0.69
QZ1 (DMU9)	12.22	398.89	398.89	8	5192.85	1955.78	125.79	8.8	0.78
QZ8 (DMU10)	11	532.86	532.86	391	3472.98	3550.89	528.22	9.1	0.90
FZ14(DMU11)	11.87	566	566	2	2937	5197	400	0.2	1
QZ22 (DMU12)	15.72	788.73	488.73	15	13,287.54	2517.07	1682.58	11	0.98
ND1 (DMU13)	10	100	485	15	12,932	1688	483	5.6	1
PT7 (DMU14)	14.66	1044.5	961.62	232	11,142	4362	827.14	18.8	1
FZ7 (DMU15)	16.28	886	886	8	15,344	8327	1442	10.2	1
LY1 (DMU16)	11.48	173.71	173.7	4	1975	1508.96	43.8	12.4	1
SM2 (DMU17)	24.56	346.04	223.3	11	2605.94	771	173.67	13.5	0.80
LY4 (DMU18)	21.18	179.39	179.39	2	1858.3	504	70	5.9	0.88
LY3 (DMU19)	18.18	275	275	0	2841	364	78	3.3	1
ZZ4 (DMU20)	10.32	454.71	454.71	31	3300.33	1382.91	532.36	4.9	0.60
QZ9 (DMU21)	14.62	275.7	252.6	57	4435.23	1301	111.55	7.6	0.73
ND94 (DMU22)	5.33	328	41	4	5549.02	368	987	2.5	1
ND14 (DMU23)	24.64	462	461	18	6610	1748	129	6.3	0.56
QZ35 (DMU24)	10.78	547.99	518.68	11	5837	2619.39	285.45	6.4	0.69
QZ30 (DMU25)	15.48	613.72	573.81	13	4820.71	2621.4	404.94	10.4	0.67
QZ36 (DMU26)	10.76	825.62	825.62	16	10502	5389.64	717.38	10.5	1
QZ12 (DMU27)	12.55	615.88	316.79	11	9018.71	384.22	64.99	1.5	0.66

. According to CCR-efficiency, 12 out of the 27 sampled enterprises are DEA effective. The interval weights of these 12 DEA-effective enterprises are obtained by solving model (5), as shown in

Table A1. Weights of 12 efficient machinery manufacturing enterprises.

DMU	v1	v2	v3	v4	u1	u2	u3	u4
DMU4	(0, 0.0824)	(0, 0.0058)	(0.029, 0.082)	(0, 0.55915)	(0, 0.0003)	(0, 0.0032)	(0, 0.00185)	(0, 0.4348)
DMU5	(0, 0.4453)	(0, 12.676)	(0, 0.0157)	(0, 0.55335)	(0, 0.0016)	(0, 0.0083)	(0, 0.04655)	(0, 1.35135)
DMU6	(0, 0.2795)	(0.013,0.027)	(0, 0.0125)	(0, 0.01635)	(0, 0.0003)	(0, 0.0018)	(0.0048, 0.00665)	(0, 0.3408)
DMU7	(1.116, 1.40)	(0, 0.0016)	(0.008, 0.016)	(0, 0.0995)	(0.0005, 0.0008)	(0, 0.00115)	(0, 0.0008)	(0.982, 1.263)
DMU11	(0, 0.38425)	(0, 0.0086)	(0, 0.0084)	(0.076, 2.424)	(0, 0.00075)	(0.0005, 0.0012)	(0, 0.0092)	(0, 0.5782)
DMU13	(0, 0.5)	(0, 0.04005)	(0, 0.0094)	(0, 0.30675)	(0, 0.0004)	(0, 0.00295)	(0, 0.01035)	(0, 0.5804)
DMU14	(0.332, 0.507)	(0, 0.0023)	(0, 0.0025)	(0, 0.00485)	(0, 0.0002)	(0, 0.0004)	(0, 0.00135)	(0.276, 0.404)
DMU15	(0, 0.30715)	(0, 0.0053)	(0, 0.0054)	(0, 0.56935)	(0, 0.00035)	(0, 0.0006)	(0, 0.0022)	(0, 0.2597)
DMU16	(0, 0.4337)	(0, 0.025)	(0, 0.0248)	(0, 1.04525)	(0, 0.0009)	(0, 0.003)	(0, 0.00775)	(0.072, 0.439)
DMU19	(0, 0.27505)	(0, 0.0182)	(0, 0.0182)	(19.93, 20.03)	(0, 0.00175)	(0, 0.0138)	(0, 0.0641)	(0, 1.515)
DMU22	(0, 0.9381)	(0, 0.0149)	(0, 0.12195)	(0, 1.235)	(0, 0.0009)	(0, 0.01005)	(0, 0.00505)	(0, 1.465)
DMU26	(0.444, 0.687)	(0, 0.003)	(0, 0.003)	(0, 0.1071)	(0, 0.0003)	(0, 0.0009)	(0, 0.002)	(0, 0.418)

of the Appendix A. On the basis of the interval weights (Table A1 of the Appendix A), 10 ECEDs are denoted by A, B, C, D, E, F, G, H, I, and J, when the values of α were 1, 0.9, 0.8, 0.7, 0.6, 0.5, 0.3, 0.2, and 0.1, respectively. With any decrease of α, the value of ECED increases gradually. That is, the ECED is the minimum of the 10 evaluation criteria when α = 1 and the highest evaluation criteria when α = 0.1. The 10 ECED are shown in

Table A2. Eclectic decision-making evaluation criterion (ECED) under different values of parameter α.

ECED	v1	v2	v3	v4	u1	u2	u3	u4
A (α = 1)	0.1648	0.0032	0.0054	0.0097	0.0004	0.0008	0.0016	0.5194
B (α = 0.9)	0.2599	0.01288	0.00779	2.01145	0.00041	0.00077	0.00192	0.56565
C (α = 0.8)	0.355	0.02256	0.01018	4.0132	0.00042	0.00074	0.00224	0.6119
D (α = 0.7)	0.4501	0.03224	0.01257	6.01495	0.00043	0.00071	0.00256	0.65815
E (α = 0.6)	0.5452	0.04192	0.01496	8.0167	0.00044	0.00068	0.00288	0.7044
F (α = 0.5)	0.6403	0.0516	0.01735	10.01845	0.00045	0.00065	0.0032	0.75065
G (α = 0.4)	0.7354	0.06128	0.01974	12.0202	0.00046	0.00062	0.00352	0.7969
H (α = 0.3)	0.8305	0.07096	0.02213	14.02195	0.00047	0.00059	0.00384	0.84315
I (α = 0.2)	0.9256	0.08064	0.02452	16.0237	0.00048	0.00056	0.00416	0.8894
J (α = 0.1)	1.0207	0.09032	0.02691	18.02545	0.00049	0.00053	0.00448	0.93565

of the Appendix A. For the 27 enterprises, if they are evaluated in a non-differentiated way (such as using the common weight evaluation method), then it is not conducive to encouraging enterprises in different regions to carry out innovative activities according to local conditions. However, evaluating these enterprises in a differentiated way (such as using the traditional cross-efficiency evaluation method) allows each enterprise to avoid its own weaknesses and emphasize its own advantages. In this case, the evaluation results will inevitably be unfair. Therefore, differentiated evaluation under evaluation criteria is carried out for these 27 enterprises, that is, cross-efficiency evaluation under the 10 criteria (Table A2 of the Appendix A). The comparison of the evaluation results is shown in Figure 1 and Figure 2.

We can see in Figure 1 that the degree of cross-efficiency of the 27 enterprises varies under different criteria, that is, the cross-efficiency of 27 enterprises is generally higher under the evaluation criterion (minimum criterion) when α = 1, and lower under the evaluation criterion (highest criterion) when α = 0.1. This result shows that the evaluation criteria have a supervisory and regulatory effect on the cross-efficiency of enterprises. When the evaluation criteria are lower, the weights space available for enterprises is larger, and the enterprises can select more favorable weights for themselves. Therefore, the cross-efficiency is generally higher. Conversely, when the weights space is smaller, the cross-efficiency is smaller. This conclusion is consistent with the application. The lower the evaluation criteria, the higher the performance of the enterprises will be.

According to Figure 2, the evaluation criteria also have an impact on the rankings of the 27 enterprises, and the rankings under almost every criterion are different. At the same time, the ranking of the 27 enterprises under different criteria has not changed to any significant degree. In particular, the ranking of the top enterprises and the bottom enterprises is relatively stable. This result shows that the rankings obtained under the proposed method can basically reflect the basic strength of enterprises.

By introducing the parameter p, the ECWME above takes into account the importance of DMUs in the criterion formulation. In order to highlight the changing regularity of cross-efficiency under different ECWME, we try to simplify the evaluation criteria. That is, one evaluation criterion only considers the importance of one DEA-effective enterprise. Therefore, the weighted mathematical expectation of 12 DEA-effective enterprises forms 12 evaluation criteria by adjusting the parameter p (see

Table A3. Combinations of p-value.

Evaluation Criterion	p4	p5	p6	p7	p11	p13	p14	p15	p16	p19	p22	p26
DMU-4	1	0	0	0	0	0	0	0	0	0	0	0
DMU-5	0	1	0	0	0	0	0	0	0	0	0	0
DMU-6	0	0	1	0	0	0	0	0	0	0	0	0
DMU-7	0	0	0	1	0	0	0	0	0	0	0	0
DMU-11	0	0	0	0	1	0	0	0	0	0	0	0
DMU-13	0	0	0	0	0	1	0	0	0	0	0	0
DMU-14	0	0	0	0	0	0	1	0	0	0	0	0
DMU-15	0	0	0	0	0	0	0	1	0	0	0	0
DMU-16	0	0	0	0	0	0	0	0	1	0	0	0
DMU-19	0	0	0	0	0	0	0	0	0	1	0	0
DMU-22	0	0	0	0	0	0	0	0	0	0	1	0
DMU-26	0	0	0	0	0	0	0	0	0	0	0	1

of the Appendix A). Also, 12 ECWMEs are shown in

Table A4. Under different combinations of p in Table A3.

ECWME	v1	v2	v3	v4	u1	u2	u3	u4
DMU-4	0.0824	0.0058	0.08205	0.55915	0.0003	0.0032	0.00185	0.4348
DMU-5	0.4453	12.6763	0.0157	0.55335	0.0016	0.0083	0.04655	1.35135
DMU-6	0.2795	0.0274	0.0125	0.01635	0.0003	0.0018	0.00665	0.3408
DMU-7	1.39955	0.0016	0.01605	0.0995	0.0008	0.00115	0.0008	1.26305
DMU-11	0.38425	0.0086	0.0084	2.42355	0.00075	0.0012	0.0092	0.5782
DMU-13	0.5	0.04005	0.0094	0.30675	0.0004	0.00295	0.01035	0.58035
DMU-14	0.50725	0.0023	0.0025	0.00485	0.0002	0.0004	0.00135	0.4041
DMU-15	0.30715	0.0053	0.0054	0.56935	0.00035	0.0006	0.0022	0.2597
DMU-16	0.4337	0.025	0.0248	1.04525	0.0009	0.003	0.00775	0.4391
DMU-19	0.27505	0.0182	0.0182	20.0272	0.00175	0.01375	0.0641	1.51515
DMU-22	0.9381	0.01485	0.12195	1.23525	0.0009	0.01005	0.00505	1.4647
DMU-26	0.6865	0.003	0.00305	0.1071	0.0003	0.0009	0.002	0.41845

of the Appendix A. The cross-efficiency comparison chart of the 27 sampled machinery manufacturing enterprises based on 12 ECWME is shown in Figure 3, and the ranking comparison chart is shown in Figure 4. In Figure 3, the cross-efficiency of each DMU changes to different degrees under different standards, but the trend is basically the same. This shows that the evaluation criteria have a regulatory effect on the cross-efficiency. However, their decisive effect on the cross efficiency is the enterprise’s own performance. In addition, in Figure 3, when DMU-14 is used as the evaluation criterion, the cross efficiency of each DMU is higher. Combined with DMU-14 (see Appendix A Table A4 for bid winning), its value is lower than other evaluation criteria; it can be seen that the lower the evaluation criterion, the higher the cross efficiency of the evaluated DMU. In Figure 4, we use red dots to mark the ranking of 12 DEA-effective enterprises under 12 ECWMEs (see Table A4 of the Appendix A). For example, a small red dot is used to mark the ranking of a DEA-effective enterprise denoted as DMU5 under ECWME, denoted as DMU-5. This only considers the importance of DMU5. From the red dot distribution in Figure 4, it is not difficult to find that the ranking of DMU5 under ECWME DMU-5 is higher than the ranking under other criteria. The conclusion is also valid for other DEA-efficient enterprises, such as DMU6, DMU7, DMU11, and so on. This conclusion is reached because the weights for the enterprises (DMU5, DMU6, DMU7, DMU11, and so on) to be selected are subject to less conditional constraints (under ECWME, such as DMU-5, DMU-6, DMU-7, DMU-11, and so on) in cross-evaluation. That is, there is more weights room for the enterprises to choose, and each enterprise can obtain more conducive weights for themselves. This result is also consistent with the actual application. That is, the enterprise that is used as the evaluation criterion has more advantages in the comprehensive evaluation. Of course, few evaluation criteria in practical application only consider the importance of a certain enterprise. Often, some mainstream enterprises or representative enterprises are taken as evaluation criteria, in order to enhance the objectivity of the evaluation results and avoid the issue of evaluation subjectivity.

From the analysis of the results of these examples, we can see that, firstly, the level of evaluation criteria will affect the ranking of an enterprise’s performance. Low criteria will help to implement enterprise incentive policies, and high criteria will help to macro-control enterprises. Secondly, enterprises ranked higher when considered by evaluation criteria. Taking multiple enterprises as evaluation criteria is conducive to setting up market benchmarks, and taking the vast majority of enterprises as evaluation criteria is conducive to creating a fair and free competitive market environment.

6. Conclusions

Cross-efficiency evaluation is an important method for ranking DMUs. Existing DEA models for cross-efficiency evaluation tend to choose the set of weights for peer evaluating using subjective attitudes, without an objective evaluation criterion as a reference point for peer evaluation. This makes it difficult for the DM to make a subjective DEA ranking.

To resolve these problems, we have proposed in this paper a cross-efficiency evaluation method based on evaluation criterion balanced on interval weights. The DEA model determines one set of weights for each DMU to evaluate its peers. This is done by minimizing the distance from the CCR-weights of the DMU to the evaluation criteria, which is balanced on interval weights. The criteria standard is different from the original intention of the evaluation, and the corresponding ranking based on cross-efficiency evaluation is also different. On the basis of the interval weights, this paper proposes two types of flexible evaluation criteria. One is based on the evaluation criteria that, in turn, are based on the eclectic decision-making method. Also, changing the harmonic coefficient $\alpha$ can be done to adjust the evaluation criteria. The other criterion is based on the evaluation criteria of mathematical expectation, taking into account the importance of each DMU in the standard formulation by introducing the parameter P. As a result, the cross-efficiencies computed using this method are more objective and flexible, thus meeting the requirements of macro regulation. We have also extended the DEA model and proposed a cross-weight evaluation, which seeks a common set of weights for all DMUs. This method’s usefulness has been illustrated with numerical examples.

From the results of the illustrative example, it is particularly noticeable that the number of zero weights is sharply reduced and using cross-evaluation weights is more objective. This method avoids the situation where decision-makers try, for some purpose, to choose weights that are too subjective. In addition, the proposed approach can lead to different DMU rankings under different evaluation criteria and has more discrimination power than the CCR-efficiency method. The cross-evaluation criteria in the manuscript can be regarded as a means of macro-control, which is derived from the real market situation, and which is applied to macro-control market trends at the same time. Under the market environment, whether an industry or a chain enterprise, there should be established industry evaluation criteria or enterprise management objectives, and performance evaluation can be carried out by reference to the evaluation criteria or management objectives. This paper effectively solves these two problems. Firstly, the two evaluation criteria proposed in this paper provide feasible means and methods for industry criteria or enterprise management objectives; then, the cross-evaluation method based on evaluation criteria provides methods and theoretical support for performance evaluation, taking industry criteria or enterprise management objectives as references. Therefore, the proposed approach can be effectively applied to different evaluation problems, such as enterprise performance evaluation, school management, and the macro-control of banks. This work has several limitations that should be improved in future research. Firstly, the proposed approach assumes that DMUs are homogeneous, which limits the method’s application scope. Secondly, the criteria for macro-control should be based on a large sample, but this paper uses a small sample. Therefore, readers interested in this research can expand the approach by combining DEA and statistical methods that consider the heterogeneity of decision-making units (DMUs).