Identifying Critical Indicators in the Evaluation of Third-Party Reverse Logistics Provider Using Best–Worst Method

Abstract

Evaluation and selection of a third-party reverse logistics provider (3PRLP) is an important tool for enterprises to improve the level of reverse logistics management. The identification of critical indicators plays a crucial role in the evaluation process. Firstly, on the basis of fully considering the characteristics of 3PRLP evaluation and selection, we summarized 27 evaluation indicators from 5 dimensions—overall operation level, management service level, information technology level, social and ecological benefits, and strategic alliance. Secondly, we adopted the Delphi method to determine the formal evaluation index system, and the best–worst method (BWM) to calculate the weight of each indicator. We determined the critical evaluation indicators on the basis of the weights. Finally, based on our results, corresponding countermeasures and implications regarding the process of 3PRLP evaluation were put forward. The research results show that the critical indicators include transportation allocation capacity, network coverage, service price level, service response speed, recovery efficiency, and service flexibility level.

1. Introduction

Reverse logistics was first proposed by the American scholar Stock in 1992. It includes logistics activities such as resource conservation, recycling, replacement, material reuse, and waste disposal. Reverse logistics has the characteristics of uncertainty, complexity of processing systems, and high cost of operation [1]. With the increasingly prominent ecological and environmental issues, countries around the world attach great importance to the research and practice of reverse logistics. Nowadays, reverse logistics has become a research hotspot in the field of logistics and supply chains. It is not only an important foundation and condition for promoting circular economy and achieving sustainable development, but also a powerful guarantee for promoting limited resource circulation and green development of enterprises. For enterprises, strengthening the management of reverse logistics can not only save resources and alleviate environmental pollution issues, but also reduce logistics costs, enhance their competitive advantage, and thus improve economic efficiency [2].

The self-operated reverse logistics model facilitates enterprises to take the initiative to control and manage their supply chains. However, due to the complexity of reverse logistics systems, their operations require specialized reverse logistics information tracking equipment and facilities. Therefore, many enterprises with limited resources often choose to outsource their reverse logistics operations to third-party reverse logistics provider (3PRLP). Third-party reverse logistics corresponds to self-operated reverse logistics. Its supplier has a powerful network information query system, which can provide fast and accurate data support and personalized reverse logistics services to the entrusted enterprises. Compared to self-operated reverse logistics, choosing to outsource reverse logistics activities can give full play to the intensive advantages of 3PRLP. It can also reduce the logistics costs and risks of enterprises and improve the efficiency of reverse logistics [3,4]. Therefore, for the entrusted enterprises, how to select the appropriate 3PRLP has become a crucial step in reverse logistics decision-making.

Evaluating 3PRLP is an important means for enterprises to select suitable suppliers. 3PRLP is a service-oriented supplier, while the traditional supplier evaluation index system in the supply chain cannot be directly used to evaluate service-oriented suppliers. Therefore, the service characteristics of 3PRLP should be reflected in the evaluation process [5,6]. The evaluation and selection of 3PRLP involves many factors and often faces the problem of redundant evaluation indicators. Therefore, identifying critical indicators in the evaluation process of 3PRLP can help enterprises identify key value drivers, thereby promoting the realization of decision-making objectives.

The identification of critical indicators in the evaluation of 3PRLP is a typical multi-criteria decision-making (MCDM) problem. The essence of MCDM problems is to rank and analyze the indicators in the set of indicators under consideration through specific mathematical tools and logical preferences [7]. Some traditional methods of weight calculation include the analytic hierarchy process (AHP) method, the analytic network process (ANP) method, entropy method, and the decision-making trial and evaluation laboratory (DEMATEL), among others. These methods can all be used to rank the importance of indicators. However, they are all based on matrix analysis, which requires a large amount of data collection during the research process and involves complex calculations. For example, the AHP method requires $n\left(n-1\right)/2$ pairs of comparisons during the calculation process, and involves fractional calculations. These will greatly increase the complexity of the calculation and may affect the consistency of the final results.

In our study, we explore the answers to the following research questions: What dimensions and indicators should be considered in the evaluation of 3PRLP? Which are the critical indicators? Furthermore, is the BWM method appropriate for this type of problem and how should it be applied? Answering these questions has certain theoretical and practical implications. Theoretically, we sort out the performance evaluation indicators of 3PRLP systematically and comprehensively with the BWM method, which furtherly expands the application of BWM. Practically, our study reduced the complexity for enterprises to identify the critical indicators.

The remainder of this paper is organized as follows: Section 2 introduces related research results on the evaluation of 3PRLP. Section 3 introduces the methods used in the study. In Section 4, we present the empirical research, which comprehensively uses the Delphi and BWM method to identify and analyze the critical evaluation indicators of 3PRLP. Section 5 discusses managerial implications and outcomes for the evaluation of 3PRLP. Section 6 concludes the paper.

2. Related Literature Review

Third-party reverse logistics emerged with the sustainable and cyclical development of economic models. It is an important means to strengthen the core competitiveness of enterprises, reduce operational costs and risks, and improve customer satisfaction. The evaluation of 3PRLP is a powerful guarantee for ensuring the effectiveness and rationality of the entire third-party reverse logistics operation process. Third party reverse logistics has always been a hot topic in the field of green logistics research. With the promulgation of national green legislation, the concept of sustainable development has taken root in the hearts of the public, and the issue of evaluating 3PRLP is also being studied by scholars. Currently, the research results on the evaluation and selection of 3PRLP mainly focus on the construction of evaluation index system and the selection of evaluation methods.

2.1. Analysis of Evaluation Indicators for 3PRLP

Many scholars have begun to pay attention to the selection of evaluation indicators for 3PRLP and have conducted research from different perspectives. One focus of research is the service level of 3PRLP. From this perspective, scholars believe that when establishing a 3PRLP evaluation index system, in addition to meeting the integrity, flexibility, and objectivity of the general evaluation index system, it is also necessary to reflect the service-oriented characteristics of 3PRLP. Li and Jin constructed an evaluation index system of four aspects by comparing 3PRLP vertically and horizontally. In order to achieve the purpose of controlling management and services, they subdivided the management service quality dimension into secondary indicators such as response speed, service breadth and depth, etc. [8]. Wu, Ma, and Xin believed that service level indicators are key elements that customers face directly, which can reflect the maximum affordability of reverse logistics. They established an evaluation index system of five aspects: management level, service level, technical operation level, informatization level, and social ecological benefits, in order to objectively reflect the operation and service level of reverse logistics providers [9,10,11]. Liu et al. used five aspects as the selection criteria for 3PRLP: total assets, transport cost, customer satisfaction, personalized service, and technology level. Among them, customer satisfaction and personalized service were used to measure the service level of suppliers [12].

Another focus of research is the strategic alliance relationship between 3PRLP and entrusted enterprises, which can comprehensively evaluate their strategic compatibility and stability of the alliance relationship. Li and Zhao embodied the contractual oriented strategic alliance relationship between 3PRLP and entrusted enterprises through four aspects: historical cooperation status, sharing of benefits and risks, core capabilities, and compatibility of strategic concepts [13]. Li and Wang, and Senthil et al. divided the evaluation index system of 3PRLP into four dimensions: service quality, technical level, operational ability, and strategic alliance. Additionally, they further subdivided strategic alliance into secondary indicators such as cultural compatibility, strategic goal compatibility, and sharing of benefits and risks [14,15]. Wu and Yu, Guo et al., and Song et al. constructed an evaluation index system based on the characteristics of 3PRLP selection from four dimensions: service capability, operational capability, cost factor, and cooperative alliance. They measured the level of cooperative alliances through secondary indicators such as the compatibility of strategic concepts, the sharing of benefits and risks, and corporate reputation [16,17,18].

As the research content continues to deepen, extensive research has also been carried out on various evaluation indicators of 3PRLP, and on refining evaluation indicator systems. In a study on evaluation indicator segmentation, Dickson used 23 indicators, such as service quality, production capacity, and technical level, as evaluation indicators for reverse logistics suppliers [19]. Wen comprehensively considered economic, social, and ecological benefits, and constructed a green evaluation indicator system for 3PRLP selection based on three aspects: internal evaluation, external evaluation, and comprehensive evaluation. Their evaluation index system included 9 first-level indicators and 33 secondary indicators [20]. Wang took into account that the existing evaluation index system rarely considered deficiencies in ecological benefits, technological innovation capabilities, waste disposal capabilities, and other aspects. Therefore, he comprehensively constructed 24 indicators including 7 aspects: service capacity, cost, transportation capacity, technical strength, informatization level, ecological benefits, and comprehensive strength of the enterprise [21]. Yang summarized the existing research achievement indicator system of 3PRLP, and finally created an evaluation system including 18 secondary indicators from 6 aspects: logistics indicators, hardware indicators, service indicators, energy consumption indicators, management indicators, and carbon emission reduction indicators [22]. Zarbakhshnia et al. constructed an evaluation system consisting of 16 indicators mainly from 4 aspects: economic standards, environmental standards, social standards, and risk standards [23]. Liu et al. divided the overall goal into 4 first-level indicators and 10 secondary indicators based on the principle of 3PRLP evaluation indicators. The first-level indicators included technical level, service level, management level, and financial level [24]. Liu and Wu constructed an evaluation index system from the four dimensions of cost, technology, service quality, and resources. They further subdivided these indicators into 15 secondary indicators to study the selection and evaluation of third-party reverse logistics service providers [25].

Based on the above literature analysis, we have summarized the indicators involved in previous studies into five dimensions: overall operational level, management service level, information technology level, social and ecological benefits, and strategic alliance, as shown in Table 1.

Table 1. The initial evaluation index system of 3PRLP.

Dimension	Indicator	Indicator Description	References
Overall operation level	Inventory turnover rate	Measure the processing efficiency of reverse logistics systems and enterprise inventory management capability	[20,22]
	Network coverage	Coverage of logistics node configuration and route arrangement	[13,14,21,22,25]
	Transportation allocation capacity	Ability to transport and allocate vehicles, the sign of whether an enterprise can achieve efficient operation	[8,15,16,17,19,25]
	Average processing time	Time spent processing a reverse logistics event (including returns, repairs, recalls, etc.)	[12,14]
	Service price level	Price level of services provided by 3PRLP	[12,13,17,20,21,23,25]
	Resource recovery cost	Expenses consumed in enterprise resource recovery	[17,21,23]
Management service level	Service response speed	The time from accepting the customer’s reverse logistics business to the customer’s resolution	[8,9,10,11,15,17]
	Recovery efficiency	Degree to which the expected effect of resource recovery is achieved within a certain period of time	[9,10,20]
	Industry experience	Time spent in the reverse logistics industry	[9,10,13]
	Service flexibility level	Ability to react and coordinate with changes in reverse logistics service volume or processing time	[11,26]
	Coordination and communication ability	Coordinated operation ability and efficiency of reverse logistics system	[9,10,17]
Information technology level	Cost of information	The sum of various costs incurred by enterprises to obtain or reset information	[17,25]
	Information integration and sharing capability	Information sharing between enterprises and reverse logistics suppliers, avoid shortages and bullwhip effects	[8,14,26]
	Degree of information standardization	Standardization degree of each module of reverse logistics information system	[9,10]
	Rate of R&D investment	Research and development of new technologies and materials	[9,10,14]
	Proportion of professional	Personnel specialized in scientific research and technical research	[12,14,21,24]
	Technical innovation capability	Ability to improve and enhance existing services	[8,9,10,11,14,22,24,25]
Social and ecological benefits	Rate of resource utilization	Degree of recycling of resources	[9,10,22]
	Investment rate of environmental protection funds	Investment in environmental protection, reflect the environmental protection level of the enterprise	[21,26]
	Comprehensive degree of environmental pollution	Impact of implementing reverse logistics activities on the environment	[10,21]
	Enterprise environmental protection awareness	Recognition of green concepts such as “green win-win and sustainable development”	[8,17,27]
	Level of recyclability	Recycling level of waste and other resources	[8,9,10,20]
Strategic alliance	Compatibility of strategic objectives	Compatibility with the strategic planning of the entrusted enterprise	[13,14,16,18]
	Compatibility of corporate culture	Coordination and adaptive compatibility with the environment of the entrusted enterprise	[14,15,17]
	Compatibility of management system	Degree of coordination with the structure and composition of the entrusted enterprise management system	[14,18]
	Sharing of benefits and risks	Sharing benefits and risks with entrusted enterprise	[13,14,15,16]
	Historical cooperation status	Whether there is a long-term and stable cooperative relationship with the entrusted enterprise	[13,16,21]

2.2. Analysis of Evaluation Methods for 3PRLP

The evaluation of 3PRLP is a typical MCDM problem. Currently, typical methods to address this kind of problem include the analytic hierarchy process (AHP) method, the step-wise weight assessment ratio analysis (SWARA) method, the decision-making trial and evaluation laboratory (DEMATEL), the entropy weight method, the technique for order of preference by similarity to ideal solution (TOPSIS) method, the quality function deployment (QFD) method, and the combined application of the above methods.

For example, Zarbakhshnia et al. used the fuzzy SWARA method to weight the evaluation criteria, and then proposed a fuzzy complex proportional assessment (fuzzy COPRAS) method to rank and select 3PRLP with risk factors [23]. Li and Pei proposed a grey TOPSIS method based on rough sets to solve problems in the evaluation process of 3PRLP, such as the need for direct weighting in determining indicator weights and information uncertainty. The application of this method made the evaluation results more scientific and reasonable [26,27]. Liu et al. established an evaluation model for G1 combination weighting based on entropy correction, which was used to perform combination weighting on the indicator system. This model realized the effective combination of subjectivity and objectivity, and provided a decision-making basis for enterprises to choose 3PRLP [28]. Pan used entropy weight method to determine the weight of indicators based on the MCDM method, and then used TOPSIS and gray correlation analysis to obtain the final ranking of indicators. Finally, she proved the feasibility and effectiveness of the research model based on the empirical research of a Chinese automobile manufacturer [29]. Li and Wang constructed a customer-demand-oriented evaluation index system and proposed an evaluation method based on fuzzy QFD. They used a deblurring method based on α-cut set to solve the fuzzy weights. The application of this method compensated for the absence of subjective and objective information in the weight calculation process [30]. Wu, Zhou, Sheng et al., and Guo constructed a 3PRLP evaluation index system based on the characteristics of third-party reverse logistics, and used the AHP method to determine the weight of each evaluation index [31,32,33,34]. Wang et al. exploited fuzzy AHP and fuzzy VIKOR methods to evaluate and select the most efficient 3PLs [35]. Wang et al. developed a decision support system to assist businesses in the selection and evaluation of different 3PRLPs using a hybrid fuzzy MCDM approach [36]. Zarbakhshnia et al. used fuzzy step-wise weight assessment ratio analysis to weight the evaluation criteria and applied a developed fuzzy complex proportional assessment of alternatives to rank and select the sustainable 3PRLPs [23]. Prakash et al. used fuzzy analytic hierarchy process for evaluation and prioritization of selection criteria and fuzzy technique for order performance by similarity to ideal solution for the selection and development of reverse logistics partner [37].

The above evaluation methods have been applied to qualitative analysis, quantitative analysis, and qualitative and quantitative combination analysis. However, some methods, such as the entropy weight method, rely excessively on practical data, and sometimes lead to conclusions that are diametrically opposed to common sense. Methods such as fuzzy QFD are limited to customer-demand-oriented perspectives, so they cannot be widely used. Other methods, such as the AHP method, ANP method, and TOPSIS method, have a cumbersome calculation process and risk failing the consistency test.

To overcome the shortcomings of the above methods, we proposed a 3PRLP evaluation method based on BWM. The best–worst method is an MCDM method based on comparison, which is used to calculate the weight of indicators. Compared to other methods, BWM not only has fewer paired comparison data and inputs, but also has better consistency and robustness.

2.3. Research Gap

We used the above research results on 3PRLP evaluations to guide our study. We combined the actual requirements of the 3PRLP evaluation process, integrating and optimizing them on the basis of previous research to form a 3PRLP evaluation index system. The aforementioned studies have proposed some factors for analyzing the evaluation of 3PRLP. However, it is unrealistic for enterprises to manage all evaluation indicators one by one due to their limited resources. Therefore, it is necessary to identify the critical indicators for evaluation, so as to allow managers to focus on research. To do so, the AHP method, SWARA method, entropy weight method, TOPSIS method, and QFD method were used for qualitative and quantitative combination analysis. However, these methods share a common problem; that is, the calculation process is cumbersome and inconsistent, which may affect the final result judgment. Compared with other methods, BWM has a simpler calculation process and more reliable weight distribution results. In this study, we use the BWM method to identify critical indicators. Compared to other MCDM methods, BWM is based on vector analysis, with a concise calculation process and significantly simplified data collection, so the calculation results have good consistency [38]. Therefore, based on the unique advantages of BWM, we achieve the identification of critical indicators for the evaluation of 3PRLP.

3. Methodologies

3.1. Delphi Method

The Delphi method was proposed by the RAND Corporation in the 1950s. It is a forecasting and information gathering process which, instead of physically bringing people together, uses written responses from participants who remain anonymous to each other. The objective of the Delphi method is to obtain the most reliable consensus of a group of experts.

It is characterized by anonymity, feedback, and statistics [39]. The characteristic of anonymity enables experts to make independent judgments on decision-making issues. The characteristic of feedback can ensure the achievement of consensus by overcoming the subjective differences caused by different expert fields, experiences, and personal cognition of each expert. The characteristic of statistics indicates that certain statistical indicators are needed in the decision process. In our study, a consensus deviation index (CDI) is adopted to indicate the degree of expert consensus. The CDI value is equal to the standard deviation divided by the mean. The larger the CDI, the weaker the expert consensus. In our paper, we use 0.1 as the basis for judgment of the CDI. Figure 1 shows the process of the Delphi method.

Figure 1. Implementation process of the Delphi method.

3.2. Best–Worst Method

The best–worst method is a new MCDM method proposed by the Dutch scholar Rezaei to define the index weight coefficient. This method quantifies qualitative factors by expressing and processing people’s subjective judgments in quantitative form. To some extent, it weakens the subjective impact, improves the accuracy of indicator weights, and makes the evaluation process more scientific. The BWM is similar to the AHP method in that they are both based on pairwise comparisons between indicators. However, the BWM constructs a structured way to compare, simplifying the steps that require pairwise comparisons of all indicators in the AHP method. In addition, a small number of paired comparisons eliminate inconsistencies in the indicator comparison process, making the calculation process simpler [40]. Therefore, the BWM is suitable for the evaluation of 3PRLP, which involves many factors and complex operations. Applying the BWM to the study of this problem can play a good simplification role, and the results obtained will be more reliable.

When using the BWM for evaluation, we first conduct in-depth analysis of complex decision-making issues. Then, we use less quantitative information to mathematically model the decision-making process. Finally, we construct a mathematical programming problem and solve it to obtain the optimal weight of each indicator. In some cases, the BWM can produce multiple optimal solutions, which means that solving the problem will result in different weight sets of decision indicators. With the rapid development of BWM, relevant theories such as interval weights, nonlinear models, and linear models have emerged. Among them, the process of finding the optimal solution through a linear model is concise, and the results are unique and reasonable. Therefore, the linear solution model of BWM has been widely used in various evaluation fields [41,42]. In our study, we adopt the linear BWM model for identifying the critical evaluation indicators of 3PRLP. The specific operation steps are as follows:

Step 1: Determine a set of decision-making indicators C by summarizing and sorting out existing literature and research results. C = {$C_1$,$C_2$,…,$C_n$}, n represents the number of indicators in the decision indicator set.

Step 2: In the determined set of indicators, determine the best (most important) indicator $C_B$ and the worst (least important) indicator $C_W$ based on expert opinions.

Step 3: Determine the preference of the best indicator over all the other indicators. We assumed that there are m experts and n evaluation indicators in the study. Experts compared the relative importance of the best indicator with other indicators, using a number between 1 and 9 in Table 2. The resulting best-to-others (BO) vector is shown as

$$A_{Bj}^k =({\alpha }_{B1}^k, {\alpha }_{B2}^k,\dots ,{\alpha }_{Bn}^k).$$

where ${\alpha }_{Bj}^k$ represents the preference of expert k for the best indicator $C_B$ over the other indicators $C_j$.

Table 2. The evaluation scale for pairwise comparison of BWM indicators.

Preferences	Linguistic Terms
1	Indicates that $C_i$ is as important as $C_j$
2	Indicates that $C_i$ is between equally important and slightly important compared to $C_j$
3	Indicates that $C_i$ is slightly important compared to $C_j$
4	Indicates that $C_i$ is between slightly important and highly important compared to $C_j$
5	Indicates that $C_i$ is highly important compared to $C_j$
6	Indicates that $C_i$ is between highly important and very important compared to $C_j$
7	Indicates that $C_i$ is very important compared to $C_j$
8	Indicates that $C_i$ is between very important and completely important compared to $C_j$
9	Indicates that $C_i$ is completely important compared to $C_j$

Step 4: Determine the preference of all the other indicators over the worst indicator. We assumed that there are m experts and n evaluation indicators in the study. Experts compared the relative importance of the other indicators with the worst indicator, using a number between 1 and 9 in Table 2. The resulting others-to-worst (OW) vector is shown as

$$A_{jW}^k =({\alpha }_{1W}^k, {\alpha }_{2W}^k,\dots ,{\alpha }_{nW}^k)$$

where $A_{jW}^k$ represents the preference of expert k for the other indicators $C_j$ over the worst indicator $C_W$.

Step 5: Construct a linear BWM model to calculate the unique optimal weights $W_j^{*}{}^{}$=($w_1^{*}$,$w_2^{*}$,$\dots$,$w_n^{*}$). Given that the weights and sum of weights are non-negative, the model is as shown in Formula (1).

$$\begin{array}{l}\min {\max }_j\left\{{}^{}\left|w_B-a_{Bj}{w}_j\right|,\left|w_j-a_{jW}{w}_W\right|\right\}\\ s.t.\{\begin{array}{l}{\displaystyle \sum _{j=1}^n{w}_j=1}\\ w_j\ge 0,\forall j\end{array}\end{array}$$

(1)

Formula (1) is equivalent to the following Model (2):

$$\begin{array}{l}\min {\xi }^L\\ s.t.\{\begin{array}{l}\left|w_B-a_{Bj}{w}_j\right|\le {\xi }^L,\forall j\\ \left|w_j-a_{jW}{w}_W\right|\le {\xi }^L,\forall j\\ {\displaystyle \sum _{j=1}^n{w}_j=1}\\ w_j\ge 0,\forall j\end{array}\end{array}$$

(2)

The above model is a linear problem, which has a unique solution, where $w_B$ represents the weight of the best indicator $C_B$, $w_W$ represents the weight of the worst indicator $C_W$, and $w_j$ represents the weight of indicator $C_j$. By solving model (2), the optimal weights and ${\mathsf{\xi }}^{L*}$ are obtained.

Step 6: Conduct consistency check. To check the consistency of the comparison, the consistency ratio (CR) is calculated as follows:

$$CR=\frac{{\xi }^{L*}}{Consistency Index}$$

(3)

The consistency index (CI) is shown in Table 3.

Table 3. Consistency index table.

${\alpha }_{BW}$	2	3	4	5	6	7	8	9
Consistency Index	0.44	1.00	1.63	2.30	3.00	3.73	4.47	5.23

The CR values close to 0 show a high level of consistency.

The operation process of BWM is shown in Figure 2.

Figure 2. Implementation steps of the BWM.

4. Empirical Study

4.1. Establishing the Formal Decision Structure Based on the Delphi Method

Through a literature review, the initial set of 3PRLP evaluation indicators in this paper included 5 first-level indicators and 27 second-level indicators, as shown in Table 1.

In order to verify the scientificity and rationality of the evaluation index, it is necessary to use the Delphi method to seek expert opinions on the initial evaluation index system. We invited five experts from the field of third-party reverse logistics with rich practical experience and professional theoretical background to form an expert group, as shown in Table 4.

Table 4. Professional backgrounds of the five experts for Delphi survey.

Expert	Title	Gender	Age	Field of Expertise	Seniority
A	Associate professor	Male	61	Third-party logistics	30~35
B	Associate professor	Male	52	Supply chain management	25~30
C	Associate professor	Male	35	Supply chain and logistics management	10~15
D	Associate research fellow	Male	43	Green logistics and supply chain	15~20
E	Senior manager	Male	50	Enterprise management	20~30

We used points between 0 and 10 to measure the necessity of each evaluation indicator, and then judged the necessity of the evaluation index according to the average score and CDI [43]. The specific rules for this study are as follows. First, we considered the average score of each indicator. If the average score is too low, it indicates that the indicator is relatively unnecessary and needs to be removed. Second, we considered the CDI value of each indicator. If the CDI value is too high, it indicates that the experts have differing opinions on the indicator, and no consensus has been reached. The next round of scoring is thus required until the expert group agrees. We took 6 points as the critical value of necessity and 0.1 as the critical value of CDI.

In the first round of the Delphi survey, we provided the experts with the initial evaluation index system as shown in Table 1. Experts examined the description of the indicators and gave a preliminary judgment on the rationality of the initial indicators.

In the second round of the Delphi survey, the experts rated each indicator in 3PRLP evaluation. They scored from 0 to 10, where 0 indicated absolutely unnecessary, while 10 indicated absolutely necessary. The rating scores are shown in Table 4. The CDI values of the 4 indicators (average processing time, coordination and communication ability, proportion of professionals, and enterprise environmental protection awareness) were lower than 0.1, indicating that experts reached consensus on the above-mentioned 4 indicators. Meanwhile, the average scores of the above-mentioned 4 indicators were lower than 6, indicating that experts agreed that these four indicator elements were unnecessary and could be directly eliminated. In addition, the average scores of 13 indicators were higher than 6 and the CDI values were lower than 0.1, indicating that these indicators were necessary for our 3PRLP research, and experts had reached consensus. As shown in Table 5, there were still 11 indicators whose average scores were greater than 6 and CDI values were greater than 0.1. Therefore, we conducted the third-round survey.

Table 5. Necessity analysis of indicators in the second round of Delphi questionnaire.

Dimension	Indicator	Necessity Scoring					Mean Value	Standard Deviation	CDI	Whether to Eliminate
		A	B	C	D	E
Overall operation level	Inventory turnover rate	8	5	7	9	7	7.2000	1.4832	0.2060	No
	Network coverage	7	8	8	7	8	7.6000	0.5477	0.0721	No
	Transportation allocation capacity	9	8	10	8	9	8.8000	0.8367	0.0951	No
	Average processing time	5	5	5	4	5	4.8000	0.4472	0.0932	Yes
	Service price level	6	7	7	6	6	6.4000	0.5477	0.0856	No
	Resource recovery cost	5	6	5	7	4	5.4000	1.1402	0.2111	No
Management service level	Service response speed	9	8	9	8	10	8.8000	0.8367	0.0951	No
	Recovery efficiency	7	9	6	5	7	6.8000	1.4832	0.2181	No
	Industry experience	4	5	3	4	5	4.2000	0.8367	0.1992	No
	Service flexibility level	7	8	7	7	8	7.4000	0.5477	0.0740	No
	Coordination and communication ability	5	6	5	5	5	5.2000	0.4472	0.0860	Yes
Information technology level	Cost of information	7	5	3	6	5	5.2000	1.4832	0.2852	No
	Information integration and sharing capability	8	9	8	9	8	8.4000	0.5477	0.0652	No
	Degree of information standardization	6	6	8	5	7	6.4000	1.1402	0.1782	No
	Rate of R&D investment	3	5	7	4	6	5.0000	1.5811	0.3162	No
	Proportion of professionals	5	5	4	5	5	4.8000	0.4472	0.0932	Yes
	Technical innovation capability	8	7	8	9	8	8.0000	0.7071	0.0884	No
Social and ecological benefits	Rate of resource utilization	4	5	3	3	5	4.0000	1.0000	0.2500	No
	Investment rate of environmental protection funds	8	7	8	9	8	8.0000	0.7071	0.0884	No
	Comprehensive degree of environmental pollution	6	7	6	5	6	6.0000	0.7071	0.1179	No
	Enterprise environmental protection awareness	5	6	5	6	6	5.6000	0.5477	0.0978	Yes
	Level of recyclability	10	8	9	9	8	8.8000	0.8367	0.0951	No
Strategic alliance	Compatibility of strategic objectives	8	9	9	8	8	8.4000	0.5477	0.0652	No
	Compatibility of corporate culture	6	10	7	8	6	7.4000	1.6733	0.2261	No
	Compatibility of management system	6	4	5	6	5	5.2000	0.8367	0.1609	No
	Sharing of benefits and risks	7	7	6	7	7	6.8000	0.4472	0.0658	No
	Historical cooperation status	7	6	7	7	6	6.6000	0.5477	0.0830	No

In the third round, the summarized results formed in the second-round survey were provided to the experts for reference. The scores for the third round are shown in Table 6. After statistical analysis of the questionnaire survey results, it was found that the CDI values of all indicators were less than 0.1, indicating that the evaluation indicators had passed the expert consistency test. Among them, the average scores of 6 secondary indicators were lower than 6 points. According to the principle of judging the necessity of the above indicators, the 6 evaluation indicators were eliminated, including resource recovery cost, industry experience, cost of information, rate of R&D investment, rate of resource utilization, and compatibility of management system.

Table 6. Necessity analysis of indicators in the third round of Delphi questionnaire.

Dimension	Indicator	Necessity Scoring					Mean Value	Standard Deviation	CDI	Whether to Eliminate
		A	B	C	D	E
Overall operation level	Inventory turnover rate	7	8	7	8	7	7.4000	0.5477	0.0740	No
	Network coverage	7	8	8	7	8	7.6000	0.5477	0.0721	No
	Transportation allocation capacity	9	8	10	8	9	8.8000	0.8367	0.0951	No
	Average processing time	5	5	5	4	5	4.8000	0.4472	0.0932	Yes
	Service price level	6	7	7	6	6	6.4000	0.5477	0.0856	No
	Resource recovery cost	5	6	5	5	5	5.2000	0.4472	0.0860	Yes
Management service level	Service response speed	9	8	9	8	10	8.8000	0.8367	0.0951	No
	Recovery efficiency	7	8	7	7	8	7.4000	0.5477	0.0740	No
	Industry experience	4	5	5	5	5	4.8000	0.4472	0.0932	Yes
	Service flexibility level	7	8	7	7	8	7.4000	0.5477	0.0740	No
	Coordination and communication ability	5	6	5	5	5	5.2000	0.4472	0.0860	Yes
Information technology level	Cost of information	5	5	4	5	5	4.8000	0.4472	0.0932	Yes
	Information integration and sharing capability	8	9	8	9	8	8.4000	0.5477	0.0652	No
	Degree of information standardization	6	6	7	6	7	6.4000	0.5477	0.0856	No
	Rate of R&D investment	5	5	6	5	5	5.2000	0.4472	0.0860	Yes
	Proportion of professionals	5	5	4	5	5	4.8000	0.4472	0.0932	Yes
	Technical innovation capability	8	7	8	9	8	8.0000	0.7071	0.0884	No
Social and ecological benefits	Rate of resource utilization	5	4	4	3	5	4.2000	0.8367	0.1992	Yes
	Investment rate of environmental protection funds	8	7	8	9	8	8.0000	0.7071	0.0884	No
	Comprehensive degree of environmental pollution	6	7	6	7	6	6.4000	0.5477	0.0856	No
	Enterprise environmental protection awareness	5	6	5	6	6	5.6000	0.5477	0.0978	Yes
	Level of recyclability	10	8	9	9	8	8.8000	0.8367	0.0951	No
Strategic alliance	Compatibility of strategic objectives	8	9	9	8	8	8.4000	0.5477	0.0652	No
	Compatibility of corporate culture	8	7	7	8	7	7.4000	0.5477	0.0740	No
	Compatibility of management system	6	6	5	6	5	5.6000	0.5477	0.0978	Yes
	Sharing of benefits and risks	7	7	6	7	7	6.8000	0.4472	0.0658	No
	Historical cooperation status	7	6	7	7	6	6.6000	0.5477	0.0830	No

After the above three rounds of Delphi questionnaire surveys, the final indicator system for evaluation of 3PRLP was obtained as shown in Table 7.

Table 7. Final indicator system for 3PRLP evaluation.

Dimension	Code	Indicator	Code
Overall operation level	C1	Inventory turnover rate	C11
		Network coverage	C12
		Transportation allocation capacity	C13
		Service price level	C14
Management service level	C2	Service response speed	C21
		Recovery efficiency	C22
		Service flexibility level	C23
Information technology level	C3	Information integration and sharing capability	C31
		Degree of information standardization	C32
		Technical innovation capability	C33
Social and ecological benefits	C4	Investment rate of environmental protection funds	C41
		Comprehensive degree of environmental pollution	C42
		Level of recyclability	C43
Strategic alliance	C5	Compatibility of strategic objectives	C51
		Compatibility of corporate culture	C52
		Sharing of benefits and risks	C53
		Historical cooperation status	C54

4.2. Identification of Critical Evaluation Indicators Based on BWM

The BWM method was used to identify the critical indicators in 3PRLP evaluation. The preference of pairwise comparison between indicators was obtained through a questionnaire. There were 38 questions in the questionnaire, including the selection of 12 best and worst indicators and 26 paired comparative questions. The questionnaire was completed by five experts in the field of third-party reverse logistics using a 1–9 scoring scale. The question was “Select the best and worst indicators from the following indicators. Preference of optimal indicators compared to other indicators. Preferences of other indicators compared to the worst indicators”.

Firstly, we determined a set of 3PRLP evaluation index systems, as shown in Table 5.

Secondly, experts determined the best and worst indicators for each dimension based on their own knowledge and experience. Differences in professional fields could lead to differences in the best and worst indicators determined. If there were multiple best and worst criteria, only one of them was selected as a representative.

Thirdly, experts compared the determined best indicator with other indicators in pairs, and compared the other indicators with the determined worst indicator. We took the opinion of expert A in the comparison of first-level indicators as an example, who selected C1 as the best indicator and C5 as the worst indicator. This expert used the pairwise comparison between the best indicator (C1) and other indicators to obtain the best-to-others vector $A_{B1}$ = (1, 2, 2, 3, 8), and used the pairwise comparison between other indicators and the worst indicator (C5) to obtain the others-to-worst vector $A_{W1}$ = (8, 5, 5, 4, 1). Similarly, according to the BWM operation steps 2–4, the best-to-others vector and the others-to-worst vector for each dimension and each secondary indicator could be obtained in turn. The experts’ preference scores for pairwise comparison between indicators are shown in Table 8.

Table 8. Comparative preference scores for each dimension/secondary indicator.

Dimension
Best-to-Others	Expert Preferences (1–9 scale)					Others-to-Worst	Expert Preferences (1–9 scale)
Ⅰ	Ⅱ	Ⅲ	Ⅳ	Ⅴ		Ⅰ	Ⅱ		Ⅲ	Ⅳ	Ⅴ
$\mathrm{The} \mathrm{best} \mathrm{indicator} {\mathrm{C}}_{\mathrm{B}}$	C1	C1	C1	C1	C1	The worst indicator ${\mathrm{C}}_{\mathrm{W}}$	C5	C5	C3	C3	C3
C1	1	1	1	1	1	C1	8	8	5	7	8
C2	2	3	2	2	2	C2	5	2	4	6	4
C3	2	4	5	7	8	C3	5	3	1	1	1
C4	3	5	3	6	4	C4	4	4	2	2	2
C5	8	8	4	4	5	C5	1	1	2	3	2
Secondary indicator (Overall operation level)
$\mathrm{The} \mathrm{best} \mathrm{indicator} {\mathrm{C}}_{\mathrm{B}}$	C13	C13	C12	C14	C13	The worst indicator ${\mathrm{C}}_{\mathrm{W}}$	C14	C11	C11	C11	C11
C11	2	7	7	8	7	C11	4	1	1	1	1
C12	2	3	1	5	3	C12	3	4	7	2	4
C13	1	1	5	5	1	C13	5	7	2	2	7
C14	5	4	4	1	2	C14	1	2	3	8	4
Secondary indicator (Management service level)
$\mathrm{The} \mathrm{best} \mathrm{indicator} {\mathrm{C}}_{\mathrm{B}}$	C22	C21	C22	C21	C23	The worst indicator ${\mathrm{C}}_{\mathrm{W}}$	C23	C22	C23	C22	C22
C21	3	1	5	1	3	C21	3	6	2	9	3
C22	1	6	1	9	6	C22	6	1	7	1	1
C23	6	2	7	2	1	C23	1	5	1	8	6
Secondary indicator (Information technology level)
$\mathrm{The} \mathrm{best} \mathrm{indicator} {\mathrm{C}}_{\mathrm{B}}$	C31	C31	C32	C31	C31	The worst indicator ${\mathrm{C}}_{\mathrm{W}}$	C33	C32	C32	C32	C33
C31	1	1	4	1	1	C31	6	5	2	6	8
C32	5	5	1	6	5	C32	2	1	6	1	2
C33	6	3	6	3	8	C33	1	2	1	3	1
Secondary indicator (Social and ecological benefits)
$\mathrm{The} \mathrm{best} \mathrm{indicator} {\mathrm{C}}_{\mathrm{B}}$	C43	C43	C43	C41	C43	The worst indicator ${\mathrm{C}}_{\mathrm{W}}$	C41	C42	C41	C42	C41
C41	4	3	7	1	6	C41	1	4	1	3	1
C42	2	7	3	3	3	C42	3	1	3	1	2
C43	1	1	1	2	1	C43	4	7	7	2	6
Secondary indicator (Strategic alliance)
$\mathrm{The} \mathrm{best} \mathrm{indicator} {\mathrm{C}}_{\mathrm{B}}$	C53	C53	C51	C53	C51	The worst indicator ${\mathrm{C}}_{\mathrm{W}}$	C52	C54	C52	C54	C54
C51	2	3	1	3	1	C51	3	3	8	3	8
C52	5	2	8	3	3	C52	1	4	1	3	3
C53	1	1	5	1	5	C53	5	8	2	7	2
C54	3	8	4	7	8	C54	2	1	3	1	1

According to the analysis of experts’ preferences in the dimensions in Table 6, we found that all experts considered the overall operation level (C1) to be the best indicator dimension. While two experts (I, II) considered strategic alliance (C5) to be the worst indicator dimension, three experts (III, IV, V) considered the information technology level (C3) to be the worst indicator dimension, and no expert believed that the level of management service (C2) and social and ecological benefits (C4) were the best or worst indicator dimensions. In addition, most indicators were identified by experts as the best or worst. However, no experts identified five indicators, such as inventory turnover rate (C11), technical innovation capability (C33), and comprehensive degree of environmental pollution (C42) as the best indicators, and no experts identified seven indicators, such as network coverage (C12), transportation allocation capacity (C13), and service response speed (C21) as the worst indicators.

Finally, we substituted the above scoring results into the linear model proposed in step 5 of the BWM operation. After solving the model, we obtained the optimal weights and ${\mathsf{\xi }}^{L*}$ of five experts for each dimension, as shown in Table 9.

Table 9. Optimal weights and consistency test of each dimension.

Dimension	Optimal Weights					Mean of Weight
	Ⅰ	Ⅱ	Ⅲ	Ⅳ	Ⅴ
C1	0.3894	0.4963	0.4169	0.4594	0.4754	0.4475
C2	0.2124	0.1898	0.2357	0.2645	0.2459	0.2297
C3	0.2124	0.1460	0.0725	0.0557	0.0574	0.1088
C4	0.1416	0.1168	0.1571	0.0882	0.1229	0.1253
C5	0.0442	0.0511	0.1178	0.1322	0.0984	0.0887
${\mathsf{\xi }}^{L*}$	0.0354	0.0876	0.0544	0.0696	0.0164	Results are reliable.

Similarly, by solving the model, we obtained the optimal weights and ${\mathsf{\xi }}^{L*}$ of each secondary indicator as shown in Table 10.

Table 10. Optimal weights and consistency test of secondary indicators.

Dimension	Indicator	Optimal Weights					Mean of Weight
		Ⅰ	Ⅱ	Ⅲ	Ⅳ	Ⅴ
C1	C11	0.2453	0.0702	0.0785	0.0787	0.0606	0.1067
	C12	0.2453	0.2105	0.6152	0.1348	0.1818	0.2775
	C13	0.4340	0.5614	0.1361	0.1348	0.4849	0.3502
	C14	0.0754	0.1579	0.1702	0.6517	0.2727	0.2656
${\mathsf{\xi }}^{L*}$		0.0566	0.0702	0.0785	0.0787	0.0606	Results are reliable.
C2	C21	0.2400	0.5833	0.1571	0.5972	0.2400	0.3635
	C22	0.6600	0.0833	0.7429	0.0556	0.1000	0.3284
	C23	0.1000	0.3334	0.1000	0.3472	0.6600	0.3081
${\mathsf{\xi }}^{L*}$		0.0600	0.0833	0.0429	0.0972	0.0600	Results are reliable.
C3	C31	0.7302	0.6500	0.1852	0.6600	0.7533	0.5957
	C32	0.1587	0.1250	0.7037	0.1000	0.1558	0.2486
	C33	0.1111	0.2250	0.1111	0.2400	0.0909	0.1556
		0.0635	0.0250	0.0370	0.0600	0.0260	Results are reliable.
C4	C41	0.1250	0.2500	0.0909	0.5416	0.1111	0.2237
	C42	0.3125	0.0833	0.2364	0.1667	0.2222	0.2042
	C43	0.5625	0.6667	0.6727	0.2917	0.6667	0.5721
${\mathsf{\xi }}^{L*}$		0.0625	0.0833	0.0364	0.0417	0.0000	Results are reliable.
C5	C51	0.2539	0.1728	0.6250	0.1912	0.5979	0.3682
	C52	0.0923	0.2593	0.0721	0.1912	0.2062	0.1642
	C53	0.4846	0.5062	0.1346	0.5441	0.1237	0.3586
	C54	0.1692	0.0617	0.1683	0.0735	0.0722	0.1090
${\mathsf{\xi }}^{L*}$		0.0231	0.0123	0.0481	0.0294	0.0206	Results are reliable.

According to the results in Table 9 and Table 10, we can derive that all the CR values are close to 0, indicating that the optimal weights are reliable. Considering that the 3PRLP index system is a multi-layer structure, we multiplied the weight of each dimension by the weight of the secondary indicators under that dimension to obtain the global weight and ranking of each secondary indicator [43,44]. The results are shown in Table 11.

Table 11. Global weight and ranking of indicators.

Dimension	Weight of Dimension	Sorting	Indicator	Local Weight	Global Weight	Comprehensive Sorting
Overall operation level C1	0.4475	1	Inventory turnover rate C11	0.1067	0.0477	9
			Network coverage C12	0.2775	0.1242	2
			Transportation allocation capacity C13	0.3502	0.1567	1
			Service price level C14	0.2656	0.1188	3
Management service level C2	0.2297	2	Service response speed C21	0.3635	0.0835	4
			Recovery efficiency C22	0.3284	0.0754	5
			Service flexibility level C23	0.3081	0.0708	7
Information technology level C3	0.1088	4	Information integration and sharing capability C31	0.5957	0.0648	8
			Degree of information standardization C32	0.2487	0.0271	13
			Technical innovation capability C33	0.1556	0.0169	15
Social and ecological benefits C4	0.1253	3	Investment rate of environmental protection funds C41	0.2237	0.0280	12
			Comprehensive degree of environmental pollution C42	0.2042	0.0256	14
			Level of recyclability C43	0.5721	0.0717	6
Strategic alliance C5	0.0887	5	Compatibility of strategic objectives C51	0.3682	0.0327	10
			Compatibility of corporate culture C52	0.1642	0.0146	16
			Sharing of benefits and risks C53	0.3586	0.0318	11
			Historical cooperation status C54	0.1090	0.0097	17

According to the analysis of the above results, among the evaluation dimensions of 3PRLP, the overall operational level was determined as the most critical indicator dimension, with an importance of 0.4475. The management service level and social ecological benefits were identified as the next most important dimensions, with their relative weights of 0.2297 and 0.1253, respectively. According to the partial weights of each indicator in Figure 3, “transportation allocation capacity” was determined as the optimal indicator for the overall operational level, followed only by “network coverage” and “service price level”. In addition, the three indicators related to management service level (service response speed, recovery efficiency, and service flexibility level) were equally important, indicating that these three indicators were important indicators for evaluating management service level. “Information integration and sharing capability” and “Level of recyclability“ were the most important indicators for the information technology dimension and the social and ecological benefits dimension, respectively. The indicators of high importance in strategic alliance were “compatibility of strategic objectives” and “sharing of benefits and risks”. Overall, based on global weights, the six most critical indicators in the evaluation of 3PRLP were as follows: transportation allocation capacity, network coverage, service price level, service response speed, recovery efficiency, and service flexibility level.

Figure 3. Local weights of each dimension/sub indicators.

5. Discussion and Implications

As we can see in the empirical results, the overall operational level (C1) and management service level (C2) were the most important dimensions of 3PRLP evaluations, and social and ecological benefits (C4) were important dimensions for evaluations. Critical evaluation indicators included transportation allocation capacity (C13), network coverage (C12), service price level (C14), service response speed (C21), recovery efficiency (C22), and service flexibility level (C23).

In order to verify the effectiveness and sensitivity of the BWM model, we conducted a comparative analysis with the AHP model. The result was as shown in Table 12. We can see that the weights of each indicator derived by the BWM model and the AHP model vary slightly. The sorting of the six critical indicators was basically consistent. This means that the BWM model is robust. As for the complexity, the AHP model needed 31 paired comparisons while the BWM model needed just 26 paired comparisons. Therefore, the BWM method has significantly higher efficiency.

Table 12. Comparative analysis with AHP model.

Code	Indicator	BWM Model		AHP Model
		Weight	Sorting	Weight	Sorting
C11	Inventory turnover rate	0.0574	9	0.0517	9
C12	Network coverage	0.1173	2	0.1230	2
C13	Transportation allocation capacity	0.1449	1	0.1740	1
C14	Service price level	0.1145	3	0.1034	4
C21	Service response speed	0.0915	4	0.1162	3
C22	Recovery efficiency	0.0827	5	0.0922	5
C23	Service flexibility level	0.0775	6	0.0732	6
C31	Information integration and sharing capability	0.0626	8	0.0622	7
C32	Degree of information standardization	0.0256	13	0.0261	10
C33	Technical innovation capability	0.0168	15	0.0164	15
C41	Investment rate of environmental protection funds	0.0277	12	0.0254	11
C42	Comprehensive degree of environmental pollution	0.0242	14	0.0222	12
C43	Level of recyclability	0.0709	7	0.0581	8
C51	Compatibility of strategic objectives	0.0334	10	0.0201	13
C52	Compatibility of corporate culture	0.0138	16	0.0108	16
C53	Sharing of benefits and risks	0.0312	11	0.0181	14
C54	Historical cooperation status	0.0080	17	0.0069	17

Based on the above analysis results, the following implications for 3PRLP evaluations were put forward:

(1)

The overall operation level (C1) and management service level (C2) are the most critical evaluation dimensions in 3PRLP evaluation. This result has also been supported by other studies. Pei evaluated candidate 3PRLP based on the G2-weighted TOPSIS method. The result showed that the transportation capacity in the operation level was the key indicator, and the degree of informatization in the operation level was an important indicator [27]. Liu and Wu used the fuzzy analytic hierarchy process (FAHP) to study the selection of 3PRLP, and the results showed that service quality was an important evaluation dimension [25]. The overall operation level involves the operational efficiency, quality, reliability, and other aspects of the entire reverse logistics supply chain, which directly affects the reputation and customer satisfaction of 3PRLP. The management service level is related to whether a 3PRLP can establish a stable long-term cooperative relationship with the entrusted enterprise. Moreover, enterprises with high management service levels can not only ensure that their operational processes are fully optimized and monitored, but also quickly respond to service issues. Therefore, the entrusted enterprise should pay special attention to the importance of the overall operation level and management service level during the evaluation process. The evaluation of these dimensions can more comprehensively reflect the overall strength and business level of reverse logistics suppliers, which is of great significance for entrusted enterprises to formulate appropriate cooperation strategies and decisions.

(2)

Transportation allocation capability (C13), network coverage (C12), and service price level (C14) are critical factors in 3PRLP evaluation. These three indicators are also the most important indicators under the overall operational level dimension. Among them, transportation allocation capacity is the most important of all evaluation indicators. It is the key to reflecting the timeliness, safety, and efficiency of cargo transportation, as well as an important factor affecting customer satisfaction. This result is consistent with the discovery of Li [26]. He determined the weight of evaluation indicators based on the grey TOPSIS method of rough sets, and found that transportation and quality were the most important indicators. In addition, network coverage is also a very important indicator for evaluating 3PRLP. Extensive network coverage can better meet customer needs and better achieve resource sharing, thereby improving operational efficiency. Service price level is an important indicator for evaluation, but it needs to be comprehensively considered in combination with other indicator elements. Under the same conditions, selecting suppliers with high service quality and efficiency may be relatively expensive. However, in the long run, high-quality service and efficient operation may bring more benefits to the enterprise. Therefore, when evaluating 3PRLP, the entrusted enterprise should take transportation and allocation capabilities as the core criteria for selecting third-party suppliers, while seeking a balance between cost and benefit based on comprehensive consideration of network coverage and service price level.

(3)

Service response speed (C21), recovery efficiency (C22), and service flexibility level (C23) are important indicators for 3PRLP evaluation. These three indicators are all secondary indicators under the management service dimension, with similar importance, indicating that these three indicators are closely related under this dimension. First of all, good service response speed can solve customer problems from the source, laying the foundation for in-depth service development. It is the key to carrying out recycling and flexible adjustment. Secondly, recovery efficiency can ensure that customers receive timely, accurate, and high-quality services. It is the fundamental guarantee for improving service response speed and flexibility. Finally, the level of service flexibility reflects the coordinated response ability of suppliers to handle various uncertain and unexpected situations. It is a comprehensive reflection of service response speed and recovery efficiency. Therefore, it is necessary for the entrusted enterprise to focus on the development of third-party suppliers in these three aspects to ensure the efficiency, quality, and flexibility of reverse logistics services.

(4)

Social ecological benefit (C4) is an important evaluation dimension for evaluation. In previous studies, Yang constructed a six-dimensional indicator system and assigned the weight of evaluation indicators based on the information entropy method [22]. The results showed that the carbon emission reduction indicator dimension had the highest importance. Similar to the above research, with the high attention paid to sustainable development issues, we all emphasized the importance of social and ecological benefits in 3PRLP evaluation. However, in contrast, this study does not identify social and ecological benefits as the most important indicator dimension. We believe that brand image and social responsibility are one of the important factors for enterprises to choose suitable suppliers, but not the critical factors. As a service-oriented enterprise, the value of 3PRLP should be reflected in their services and operations. The level of recyclability (C43), as the most important secondary indicator in the social and ecological benefits dimension, reflects the level of resource recycling by third-party suppliers. It has a significant impact on the sustainable development and environmental image of enterprises. Entrusted enterprises should pay attention to the importance of social and ecological benefits while measuring economic benefits. Therefore, in the process of 3PRLP evaluation, it is necessary not only to evaluate the reverse logistics capabilities of third-party suppliers, but also to pay attention to their recycling and reuse levels.

(5)

The indicators related to the level of information technology (C3) and strategic alliance (C5) are also indicators for 3PRLP evaluation. Among them, the weight of information integration and sharing capability (C31) in the information technology dimension is as high as 0.5963. It is the most important indicator under this dimension, but its global weight is not high. This is because the global weight of secondary indicators is affected by the weight of the first indicators, resulting in a lower final ranking. In addition, from the empirical research results, other secondary indicators under these two dimensions are not critical indicators, and their comprehensive rankings are not high [45]. In previous studies, Song et al. studied the selection of 3PRLP based on the intuitionistic fuzzy MCDM method. They ultimately determined that customer satisfaction and corporate reputation in cooperative alliances were the two most important criteria for entrusted enterprises to select third-party suppliers [18]. Liu et al. constructed a 3PRLP evaluation model based on entropy correction G1 combination weighting, and the final result showed that the technical indicator dimension had the highest importance [28]. As the elements of innovation and development, information technology level and strategic alliance have a certain degree of impact on the evaluation of third-party reverse logistics, but they will not play a decisive role in the evaluation results. 3PRLP is a service-oriented supplier, and efficient operational processes and high-quality service quality are more important and practical choices for entrusted enterprises.

6. Conclusions

Based on a summary of the existing literature, we constructed a 3PRLP evaluation index system, and optimized and adjusted it on this basis to ensure the comprehensiveness and rationality of the evaluation process. Then, we applied BWM to the identification of critical indicators for 3PRLP evaluation. Finally, based on the empirical research results, we discussed and proposed corresponding countermeasures. The conclusions were as follows:

Firstly, we reviewed the related literature to form an initial indicator set, and on this basis, we optimized and collated the final indicator system for 3PRLP evaluation. We extracted 27 initial indicator elements that affect the evaluation and selection of 3PRLP from 5 dimensions: overall operation level, management service level, information technology level, social and ecological benefits, and strategic alliance. The formal indicator set was determined based on the Delphi survey. By calculating the average scores and CDI values of each indicator, 10 unnecessary indicator elements were discarded. After three rounds of Delphi questionnaire surveys, a 3PRLP evaluation index system with 17 secondary indicators in 5 dimensions was finally obtained.

Then, we used BWM to calculate the weights and comprehensive ranking of dimensions and secondary indicators, and determined the critical evaluation indicators. Five experts and scholars with rich practical experience and professional theoretical background in the field of third-party reverse logistics were invited to fill in the questionnaire. The comparison vector was determined through pairwise comparison between indicators. By solving the BWM linear model and calculating the average values, the local weights of each index were obtained. The global weights and comprehensive ranking of each index were obtained by further solving. The six critical evaluation indicators finally determined included transportation allocation capacity, network coverage, service price level, service response speed, recovery efficiency, and service flexibility level.

Finally, we proposed some managerial implications according to the analysis results, regarding the process of evaluating 3PRLP.

The proposed final evaluation indicator system combined qualitative and quantitative indicators which affected the performance of 3PRLP. Additionally, we took into account the combination of long-term and short-term indicators. Indicators such as strategic alliance and social ecological benefits focus on long-term performance, while indicators such as overall operation level and management service level focus on recent performance. Furthermore, the connotation of the indicators is clear, and the data of indicators is easy to obtain in practice. During the process of pair comparisons through questionnaire surveys, the computational complexity of the BWM method is greatly reduced. All in all, the proposed method in this study is easy to apply and it is appropriate for both short-term and medium-to-long-term evaluation of 3PRLP.

There were some limitations to this study. Although we divided 27 evaluation indicators into 5 dimensions in detail, we did not consider a more exhaustive indicator set and did not conduct a one-by-one inspection for all indicators. At the same time, with the innovative development in the field of third-party reverse logistics and the publication of relevant research, the selection of evaluation indicator elements should be further expanded. In addition, we did not consider ambiguity and uncertainty in the rating process of BWM. In future research, we will consider the fuzzy condition. Finally, we identified key evaluation indicators based on the BWM, but we did not explore the interaction between the indicators and the occurrence of multiple best and worst indicators at the same time. Therefore, considering combining BWM with other methods and improving traditional BWM to consider the issue of data uncertainty and inaccuracy will be an interesting and promising topic in future research.