A Large-Scale Group Decision-Making Approach to Assess Water Resource Sustainability with Double-Level Linguistic Preference Relation

Abstract

With the rapid development of science and technology and the continuous progress of society, water resource sustainability has attracted much attention. The assessment process of water resource sustainability has become a hot topic. Because professional models cannot ensure the accuracy of evaluation results, expert judgment techniques are used to perform the assessment process. Meanwhile, to eliminate the biases and consider people’s cognitive processes in complex decision making, this study utilizes a large group of experts to evaluate the sustainability of water resources, which is called a large-scale group water resource sustainability assessment (LGWRSA). This study proposes the double-level linguistic preference relation (DLLPR) to enable experts to present more reasonable and intuitive linguistic opinions. Based on the DLLPR, this study presents a clustering method, a weighting method, and a comprehensive adjustment factor determination method. To prevent minority opinions and non-cooperative behaviors from interfering with the decision-making process and to make the decision-making process develop in an accurate and objective direction, a consensus model is developed to modify minority opinions and suggestions and manage non-cooperative behaviors. After a sufficient number of experts reach a consensus through the method proposed in this article, a more accurate evaluation result can be obtained. Finally, an actual LGWRSA problem is established to derive water resource sustainability expectations for four provinces, which validates the effectiveness of the proposed method. A comparative analysis is performed to illustrate the benefits of these methods and present their shortcomings.

1. Introduction

Water plays an essential role in our lives. Water is the source of life and one of the essential material resources for human survival and development. Although water resources are renewable resources, their spatial and temporal distribution is uneven, their changes are uncertain, and they are susceptible to pollution. The progress of human society and economic development, the continuous expansion of industry, agriculture, and cities, especially the sharp increase in the world population, and uncontrolled human activities have led to environmental degradation, water pollution, and severe waste, forcing the world’s water resources to become increasingly scarce. Therefore, the sustainable utilization of water resources has become an increasingly hot topic. The sustainable utilization of water resources can meet the needs of contemporary economic and social development and is the only choice to seek the sustainable development path of mutual coordination among the economy, society, and the environment. To achieve the sustainable usage of water resources, we must investigate the assessment method of water resource sustainability, and the sustainability evaluation of water resources becomes particularly crucial.

Over the decades, from the perspective of relevant research, many scholars have proposed several evaluation methods for water resource sustainability, which can be simply divided into two categories: expert evaluation methods [1] and professional models [2]. The professional models, e.g., the blue water footprint, the pressure–state–response model, and the matter–element extension method, usually need sufficient data to ensure the accuracy of evaluation results [3]. But, sustainability is a complicated and comprehensive issue, and whether professional models are suitable for a comprehensive assessment is still a challenge in a large-scale water resource system involving multiple elements [4]. In contrast, the expert evaluation methods depend on the experts’ knowledge and experience to perform the sustainability assessment. Therefore, under scenarios of inadequate data, scholars usually prefer expert evaluation methods to perform the assessment. However, a single expert or a small number of experts can produce subjective biases and result in decision-making results deviating from the truth. Therefore, we invite multiple experts with different backgrounds to participate in the evaluation. When the number of experts exceeds 11, the water resource sustainability assessment is called a large-scale group water resource sustainability assessment (LGWRSA) problem [5].

Multi-criteria decision making (MCDM) is a specific expert evaluation method. In the MCDM problem, the water resource sustainability data of one category of indicators are available from public departments, e.g., industrial water and sewage coverage. Based on the above data, these indicators, called objective criteria, can be objectively evaluated. In contrast, it is hard to evaluate the other category of indicators, called subjective criteria, due to a lack of data, e.g., surface water quality. To evaluate subjective criteria, experts’ opinions are collected, clustered, and modified to reach a consensus on the LGWRSA problem. Therefore, the final evaluation results are derived from a combination of subjective opinions and objective data. Until now, most studies of the LGWRSA focus on the following three aspects:

(1) Clustering: Clustering is used to improve decision-making efficiency. The purpose of the clustering algorithms is to divide a large number of experts into several subgroups, and each subgroup agrees for later calculation. At present, there are many kinds of research on clustering methods. Zhou et al. proposed a DBSCAN clustering algorithm for identifying experts with outlier ratings, reducing time consumption and iteration in the decision-making process, and maximizing group decision satisfaction [6]. Yang et al. used an FCM (Fuzzy c-Method) clustering method to specify the best partitions for simultaneous object construction and to attribute variables into isomorphic blocks, which reduced the size of the original scene and could be used to solve large-scale clustering problems [7].

(2) Consensus building process (CRP): The CRP is used to help experts reach as much agreement as possible. Due to different knowledge backgrounds and living environments, experts have conflicts and differences in the decision-making process, so the evaluation information provided by experts is also diversified. To achieve consistency, many studies have proposed consensus degrees and measures to improve consensus degrees. A consensus degree is used to express the degree of consistency between evaluation information. Rodriguez et al. proposed a fuzzy consensus model to deal with the LGWRSA, which can reduce the time cost of the CRP [8]. Liu et al. developed a method for experts to use the hesitation fuzzy preference relationship to solve the hesitation fuzzy LGWRSA problem of their evaluation information [9]. Li et al. used a social network-based subgroup detection algorithm for the CRP and proposed an estimation method based on a collaborative filtering algorithm to estimate the missing preference information of opinion leaders in each subgroup [10]. Gao et al. established an opinion leader recognition method based on k-kernel decomposition and a consensus model based on clustering [11]. Based on the uncertainty of the comparative language expression modeling of hesitant fuzzy language term set (HFLTS), Rodriguez et al. proposed a new HFLTS cohesion measure based on the restricted equivalent function, which is used to measure the cohesion of expert subgroups, to jointly promote the consensus process of the majority [12]. To balance the clustering analysis progress (CAP) and CRP, Wu et al. proposed the dynamic clustering analysis process based on a consensus evolutionary network [13].

(3) Evaluation information: The evaluation information is used to express the preference for the evaluation object. Experts are more inclined to express their preferences in different ways due to their different educational backgrounds and life experiences. In the LGWRSA, what kind of language system to choose is a significant problem. We all know that natural language (very low, just right, and slightly high) is more in accord with people’s real ideas because human beings tend to use natural language. However, natural language sometimes cannot accurately and comprehensively describe some complex language terms. Rodriguez et al. proposed a concept of a hesitant fuzzy linguistic term set, which promotes the elicitation of language information when experts hesitate in several language terms to express their preferences in language decision making [14]. Based on this, Gou et al. defined a double hierarchy linguistic term set (DHLTS) [15], which divides complex language terms into two levels: the first level is the main language level, and the second level is the degree description of the first level. Xiao et al. proposed a method based on knowledge matching degree, which calculates the reliability of evaluation information according to the estimation of the matching level between the individual evaluation value and the group evaluation value. Compared with the traditional methods, this method can improve the accuracy and reliability of the missing evaluation information [16]. Yang et al. constructed a new hybrid bilateral matching method and a large-scale group decision-making method considering social network analysis. The decision-making information can be expressed more objectively and reasonably [17].

In the actual CRP, we encounter two special subgroups: (1) Non-cooperative subgroups: some individuals or subgroups are unwilling to change their preferences, which are not their real thoughts but may be beneficial to them. (2) Minority subgroups: this subgroup contains only a few experts. Although non-cooperative behaviors and minority opinions account for a small proportion of the LGWRSA, they may determine the direction of decision-making problems. We can provide appropriate adjustments and calibrations to ensure more reasonable consensus results.

According to the above literature review, the existing studies have the following limitations:

(1) It is difficult for the expert evaluation method to avoid experts adding their preferences to the evaluation, resulting in the final result being not thoroughly accurate, i.e., the biases caused by subjective factors.

(2) The existing double hierarchy linguistic term set (DHLTS) [15] can only express the quality of a certain index. However, when expressing their opinions, experts often like to compare pairwise alternatives to provide their preferences, which reflects the good and bad relationship between different alternatives.

To overcome the aforementioned limitations, Li et al. [18] proposed a dynamic relationship network analysis method based on the Leuven algorithm, which dynamically changes individual opinions by defining a correction index to eliminate subjective factors. Motivated by the research of [18], this study develops a new clustering algorithm that considers the elimination of subjective factors. In addition, inspired by the preference relation, this study defines the concept of the double-level linguistic preference relation (DLLPR) and uses the DLLPR to present the evaluation opinions of experts more reasonably. This study divides the criteria of water resource sustainability assessment into subjective criteria and objective criteria, giving corresponding weights, respectively. The concept of the double-level linguistic preference relationship (DLLPR) is proposed to reflect the good and bad relationship between different alternatives. And based on the DLLPR, the corresponding adjustment coefficient is proposed to modify the expert opinions and reduce the subjective impact. Finally, the subjective criteria and objective criteria are integrated to obtain the final decision-making results.

The novelties of this study are below:

(1) The concept of the DLLPR is proposed for experts to compare and judge pairwise objects together to express the difference in their approval of the pairwise schemes/objects. Based on the DLLPR, a new clustering method based on the DLLPR is proposed.

(2) Two consensus methods have been developed in the LGWRSA to identify, manage, and correct minority opinions and non-cooperative behaviors, respectively, and the corresponding consistency algorithm is used to improve the consensus degree of each subgroup.

The Jinsha River is the upper reaches of the Yangtze River, China’s largest river. In this study, the proposed method is used to evaluate the water resource sustainability of the Jinsha River, and the effectiveness of the proposed method is verified by the actual LGWRSA problem, and the water resource sustainability expectations of the four provinces are given.

The rest of this study is organized as follows. Section 2 analyzes DHLTS and non-cooperative behaviors and the minority opinions and proposes the definition of the DLLPR. Section 3 develops a consensus model. Section 4 proposes some methods to manage minority opinions and non-cooperative behaviors and calculate expectations. Section 5 solves a real LGWRSA problem using the proposed method. Section 6 makes some comparative analyses. Some remarks are concluded in Section 7.

2. Preliminaries

This section introduces concepts related to HLTS and DHLTS.

2.1. HLTS and DHLTS

A linguistic term is an approximation that expresses qualitative linguistic values through linguistic variables [19,20]. Assuming $S=\left\{s_{\alpha }\right|\alpha =-t;\dots ,-1, 0, 1,\dots ;t\}$ is an ordered discrete itemset, where $s_{\alpha }$ represents the possible value of a language variable, a set of nine $S$ items might, for example, look like this: S = {s₋₄ = extremely bad; s₋₃ = very bad; s₋₂ = slightly bad; s₋₁ = little bad; s₋₀ = just right; s₁ = little good; s₂ = slightly good; s₃ = very good; s₄ = extremely good}

The above language model has some limitations. We cannot use them to express only a slight amount of good. Each complex linguistic term is divided into two parts, “adverb + adjective”, which are expressed in different linguistic terms. Wang proposed the concept of language terms that weaken hedges and considered “adverbs” as a minority of weakened hedges expressed by other language labels [21].

Let $S=\left\{s_t|t=-\tau ,\dots -1,0,1,. .,\tau \right\}$ be the first-level linguistic term set (LTS) and $O=\left\{o_k|k=-\zeta ,\dots -1,0,1,\dots ,\zeta \right\}$ be the second-level LTS.

$$s_o=\left\{s_t<o_k>|t=-\tau ,\dots -1,0,1,. .,\tau ;k=-\zeta ,\dots -1,0,1,\dots ,\zeta \right\}$$

(1)

where $s_t<o_k>$ is called a double hierarchy linguistic term (DHLT) [22].

When the first-level language item is $s_t$, $o_k$ denotes the second-level language item, assuming $S=\{ s_{-2}=very bad,s_{-1}=bad,s_0=medium,s_1=excellent,s_2=very excellent\}$ and $O=\{ o_{-2}=slightly, o_{-1}=a bit,o_0=just right,o_1=very,o_2=unbelievable\}$ are the first-level LTS and the second-level LTS, respectively. When the teacher describes the student’s academic performance and the student’s academic performance is slightly good, $s_1<o_{-1}>$; when the student’s academic performance is slightly worse, $s_{-1}<o_{-2}>$; and when the student’s academic performance is incredibly good, $s_1<o_2>$.

2.2. Minority Opinions and Non-Cooperative Behavior

In the CRP of the LGWRSA, there is usually an obvious problem in such a large group, and their behavior is not conducive to reaching a consensus because they do not adjust their preferences to reach a consensus [23]. In large teams, there are usually several groups or coalitions of experts with similar interests. Some of these groups tend to modify their preferences to reach an agreement, while others do not modify their preferences or even modify their preferences contrary to what other experts do. These individuals and subgroups who are unwilling to cooperate are called non-cooperative behaviors. There are four main types of experts with minority views in large groups [24]: (1) leaders who have a forward-looking and unique perspective and who have the authority to determine the final decision; (2) experts who generally have a good understanding of decision-making issues and can offer professional and authoritative opinions; (3) young, aggressive experts who have relatively extreme views, usually personal, and are less influenced by the views of others; and (4) independent experts of note, who offer a view that is often unusual and does not usually follow the herd mentality.

As far as we know, for leaders and experienced experts, such as experienced professors, the preferences they provide are generally positive. In contrast, the other two decision support systems are often inexperienced or extreme, such as new employees and employees with different minds than the average employee. Because these two decision support systems are often inexperienced or difficult to understand, the preferences they provide should be carefully considered. Therefore, special attention should be paid to the preferences provided by the first two, and caution should be applied to the latter.

Although disagreements or minority preferences are often considered barriers to decision making, the results are more reasonable and accurate if handled appropriately. There are some gaps in the existing studies on minority opinions and non-cooperative behaviors. First of all, some concepts have only been partially studied, only deal with non-cooperative behavior or discuss minority views [25,26]. Therefore, the information processing is not reasonable enough. On this basis, Xu et al. proposed an improved consensus model for large-scale group emergency decision making and introduced a management method for minority opinions and non-cooperative behaviors based on a minimum group consensus threshold and a maximum number of iterations [27]. Based on the above analysis, meaningful work will consist of developing a consensus-building model, which is based on the DLLPR proposed later to deal with LGWRSA problems and make the CRP more reasonable.

2.3. DLLPR

As mentioned above, the double hierarchy linguistic term set (DHLTS) can only represent the strength or weakness of an indicator; this study defines a double-level linguistic preference relation (DLLPR) that reflects the strength or weakness of different alternatives.

To facilitate the representation of the DLLPR, we define a matrix as follows:

$$A=\left(\begin{array}{ccc}{a}_{11}& \dots & a_{1n}\\ ⋮& \ddots & ⋮\\ a_{n1}& \cdots & a_{nn}\end{array}\right)$$

(2)

where $n$ is the number of objects the expert evaluates, $a_{ij}=s_t<o_k>,\left(i,j=1,2, \dots ,n\right)$. Used to illustrate the relative superiority of two evaluation objects, i and j, in one respect, it has the following properties:

(1)

Elements on the diagonal of the matrix $a_{ii}=s_0<o_0>(i=1,2\dots n)$;

(2)

If $a_{ij}=s_t<o_k>$, then the element is in its symmetric position $a_{ji}=s_{-t}<o_{-k}>$.

To facilitate subsequent adjustments, we define addition and scale change operations for preference matrix elements as follows:

(1)

$s_{t1}<o_{k1}>+s_{t2}<o_{k2}>=s_{t1+t2}<o_{k1+k2}>$;

(2)

$\lambda s_t<o_k>=s_{\lambda t}<o_{\lambda k}>;(0<\lambda <1)$.

Example 1:

Suppose that the first-level LTS and the second-level LTS$S=\{s_{-2}=very bad,s_{-1}=bad, s_0=medium,s_1=excellent,s_2=very excellent\}$ and $O=\{ o_{-2}=slightly, o_{-1}=a bit,o_0=just right,o_1=very,o_2=unbelievable\}$, respectively. When an expert evaluates the quality of groundwater in areas A, B, and C, he or she can give the following matrix:$A=\left(\begin{array}{ccc}{s}_0<o_0>& s_1<o_{-2}>& s_{-1}<o_{-2}>\\ s_{-1}<o_2>& s_0<o_0>& s_{-1}<o_0>\\ s_1<o_2>& s_1<o_0>& s_0<o_0>\end{array}\right)$. It can be obtained that, in terms of groundwater quality, Area A is slightly better than Area B, Area A is slightly worse than Area C, and Area B is worse than Area C. This meets the preference of experts who prefer to compare the two alternatives to provide their own opinions.

3. Clustering, Weight Calculation, and Consensus Level Determination Methods

This section describes solutions to the LGWRSA problem based on the DLLPR. Here are some basic points discussed.

(1) Clustering: Experts are divided into subgroups based on their opinions. Experts with smaller differences of opinion tend to be placed in a subgroup. This study considers subgroups with the fewest experts as minorities.

(2) Weight determination method: Subgroup weight is very important in the CRP. The weight of each subgroup was obtained by establishing some weight determination methods and weight adjustment methods.

(3) Consensus measures: Through several consensus measures, it is possible to determine whether the consensus within all subgroups has reached the given consensus threshold.

3.1. Key Information on the LGWRSA Issue of DLLPR

Based on the DLLPR LGWRSA problem, the solution process is consistent with the classic LGWRSA problem process: the experts give their opinions on different scenarios, we cluster the experts, analyze and adjust the opinions within each subgroup, multiply the opinions of different subgroups by related weights, and finally integrate the final decision results. The special point is that the DLLPR is the form in which experts give their opinions.

(1) Suppose $P=\left\{p_1, p_2,\dots , p_m\right\}$ is a limited set of solutions that represent all possible solutions to the LGWRSA problem;

(2) Suppose $S=\left\{s_1, s_2,\dots , s_n\right\}$ is the set of experts who provide preferences. $\mathrm{\Omega }={\left({\omega }_1, {\omega }_2,\dots {\omega }_n\right)}^T$ is the weight vector for all experts, where $0\le {\omega }_a\le 1,\left(a=1,2,\dots , n\right)$ and ${\displaystyle \sum _{a=1}^n{\omega }_a=1}$;

(3) The views of all experts are given in the form of the DLLPR, m × m-size matrixes

$$\left\{A_1, A_2,\dots , A_n\right\}:A_k=\left(\begin{array}{ccc}{a}_{11}& \dots & a_{1m}\\ ⋮& \ddots & ⋮\\ a_{m1}& \cdots & a_{mm}\end{array}\right), k=1,2,\dots , n$$

3.2. Clustering Method Based on DLLPR

For the clustering method, Gao et al. established a method for identifying opinion leaders based on k-kernel decomposition and a clustering method based on similarity [7]. Tang et al. used a fuzzy c-means clustering algorithm (FCM) to cluster experts into subgroups and developed conflict detection and weight determination methods between subgroups [28]. However, k-means clustering will be affected by initial values and outliers. The clustering result of the k-means method may not be globally optimal, but only locally optimal. K values generally need to be selected by experience. However, FCM cluster calculation is very heavy. For each subgroup to be classified, the distance from it to all known elements is needed to find its K nearest neighbors. Therefore, a DLLPR-based clustering method is developed in this study.

Step 1: Establish the total distance matrix. Based on the above formula, the total distance matrix associated with all experts’ pairs is obtained as $O={\left(o^{ab}\right)}_{n\times n},\left(a,b=1,2,\dots ,n\right)$.

$$d(A_a,A_b)={\left(\frac{2}{m(m-1)}{\displaystyle \sum _{i=j}^m{\displaystyle \sum _{i<j}^m{\left(f\left(a_{ija}\right)-f\left(a_{ijb}\right)\right)}^2}}\right)}^{\frac{1}{2}}$$

(3)

$$f\left(s_t<o_k>\right)=\frac{k+(\tau +t)\zeta }{2\tau \zeta }$$

(4)

The $a_{ija}$ in Formula (3) represents the $a_{ij}$ element in the matrix $A_a$, and $f\left(a_{ija}\right)$ is calculated from Formula (4). It is easy to obtain that the diagonal elements of the matrix $O$ are all zero, and the elements symmetrical to the diagonal are equal, so we only need to take out the upper triangle matrix of the matrix $O$ for the next calculation.

Step 2: Select a clustering threshold. All the different elements (except the diagonal elements) of the upper triangular matrix of the matrix $O$, arranged in ascending order, are represented as ${\beta }_1<{\beta }_2<\dots <{\beta }_p<\dots <{\beta }_q$, where ${\beta }_p$ is the p-th smallest value, and $q\le n(n-1)/2$.

Step 3: Obtain the optimal clustering threshold ${\beta }^{\ast }$, $s{d}_p$ which is the threshold change rate.

$$s{d}_p=\frac{{\beta }_p-{\beta }_{p-1}}{m_p-m_{p-1}}$$

(5)

where $m_p$ and $m_p{}_{-1}$ are the numbers of experts in categories $p$ and $p-1$, respectively. When $m_p=m$, all experts are considered, the computation process ends, and all experts are collected. If $s{d}_{\mu }=ma{x}_p\left\{s{d}_p\right\}$, then take the sub-clustering threshold $\mu -th$, called the optimal clustering threshold, ${\beta }^{\ast }={\beta }^{\mu }$.

Step 4: Obtain the clustering results. First, in the total distance matrix $O={\left(o^{ab}\right)}_{n\times n}$, if $o^{ab}\le {\beta }^{\ast }$, then group, $s_a$ and $s_b$. Finally, all experts are grouped into the following general subgroups: $g_1,g_2,\dots , g_{\zeta }$. If $g_i\cap g_j=\varnothing \left(i\ne j; i,j=1,2,\dots ,\zeta \right)$, i.e., when all subgroups do not intersect, you obtain the final clustering result $G=\left\{ g_1,g_2,\dots , g_{\zeta }\right\}$.

Example 2:

Now let five experts be $\left\{s_1,s_2,s_3,s_4,s_5\right\}$. Give their DLLPRs on n scenarios and calculate the total distance matrix for the five experts as follows:

$$O={\left(o^{ab}\right)}_{5\times 5}=\left(\begin{array}{cc}\begin{array}{cccc}0& 0.445& 0.132& 0.647\\ 0.445& 0& 0.399& 0.346\\ 0.132& 0.399& 0& 0.145\\ 0.647& 0.346& 0.145& 0\end{array}& \begin{array}{c}0.478\\ 0.157\\ 0.586\\ 0.384\end{array}\\ \begin{array}{cccc}0.478& 0.157& 0.586& 0.384\end{array}& 0\end{array}\right)$$

Arrange all the different elements of the upper triangular matrix $O$ in ascending order as follows: $0.132<0.145<0.157<0.346<0.384<0.399<0.445<0.478<0.586<0.647$. Calculate $s{d}_{\mu }=ma{x}_p\left\{s{d}_p\right\}=s{d}_3=0.189$, optimal clustering threshold ${\beta }^{\ast }={\beta }^3=0.157$, then $o^{13}=0.132\le {\beta }^{\ast }$, $o^{25}=0.145\le {\beta }^{\ast }$, and $o^{34}=0.157\le {\beta }^{\ast }$. We know $\left\{s_1,s_3\right\}$ can group, $\left\{s_2, s_5\right\}$ can group, and $\left\{s_3,s_4\right\}$ can group. As seen in Figure 1, the final classification result is $G=\left\{g_1,g_2\right\}$, where $g_1=\left\{s_1,s_3,s_4\right\}$, and $g_2=\left\{s_2,s_5\right\}$.

3.3. Weight Determination Method

Using the above clustering method, experts are divided into $\hspace{0.17em}\{g_1,g_2,\dots , g_{\zeta }\}$ subgroups; the weight of each subgroup is also a crucial element. Based on the analysis above, experienced experts should be assigned greater weight. Young and vibrant experts should be assigned less weight. Subgroups with a large number of experts should be assigned larger weights. Based on these three principles, suppose $g_{\lambda }$, a subgroup, contains $n$ experts $n_{\lambda }$. The sum of the number of all experts is $n$, so that the experts $s_a$ can be in the subgroup $g_{\lambda }$. The weights are determined as follows:

$${\omega }_{\lambda }^a=\frac{1}{n_{\lambda }};(a=1,2,\dots ,n_{\lambda };\lambda =1,2,\dots ,\zeta )$$

(6)

In addition, based on the number of experts in each subgroup, the weights for each subgroup are derived as

$${\omega }_{\lambda }=\frac{n_{\lambda }}{n};(\lambda =1,2,\dots ,\zeta )$$

(7)

We know that each expert’s weight among all experts can be based on ${\omega }^a={\omega }_{\lambda }{\omega }_{\lambda }^a$.

Consensus is used to assess the agreement of a group of experts. The guidance recommendation system, combined with the consensus model as a feedback mechanism, is based on a set of recommendation rules to help experts adjust their opinions and find the correct direction to achieve the highest consensus possible [29]. Based on the degree of consensus above, it is important to set a corresponding consensus threshold when its overall consensus level reaches or exceeds a given threshold $\xi$, when one may consider that the CRP has ended. The consensus threshold is usually set to around 0.9 [30].

Supposing $A=\left\{A_1, A_2,\dots , A_n\right\}$ is the DLLPRs provided by $s_a$, all DLLPRs can then be aggregated into a preference relationship, expressed as $A^{\ast }={(a^{\ast }{}_{ij})}_{m\times m}$, the elements in it being $a^{\ast }{}_{ij}$:

$$a_{ij}^{\ast }={\displaystyle \sum _{a=1}^n{\rho }_a{a}_{ij}^a}$$

(8)

where $\rho ={({\rho }_1,{\rho }_2,\dots ,{\rho }_n)}^T$ is the weight vector for experts.

So for a subgroup $g_{\lambda }$, its group preference matrix $A^g{}^{\lambda }={(a^g{{}^{\lambda }}_{ij})}_{m\times m}$, where $a^g{{}^{\lambda }}_{ij}={\displaystyle \sum _{a=1}^{n_{\lambda }}{\omega }_{\lambda }^a{a}_{ij}^a}$; overall preference matrix for all experts $A^{\ast }={\left(a^{\ast }{}_{ij}\right)}_{m\times m}$, where $a^{\ast }{}_{ij}={\displaystyle \sum _{a=1}^n{\omega }_{\lambda }^a{a}_{ij}^a}$; and consistency ($C$) of group preference matrix $A^g{}^{\lambda }$ and general preference matrix $A^{\ast }$ is defined as follows:

$$C\left(A^g{}^{\lambda }\right)=1-d\left(A^g{}^{\lambda },A^{\ast }\right)$$

(9)

where $d$ is the distance defined by formula (3). The global consistency ($GC$) is then obtained as follows:

$$GC={\displaystyle \sum _{\lambda =1}^{\xi }\frac{C\left(A^g{}^{\lambda }\right)}{\xi }}$$

(10)

where $0\le GC\le 1$; the greater the value $GC$, the greater the consistency among all experts. If $GC\ge \xi$, the level of consensus among all experts is high enough for the CRP to end. Otherwise, you need to make some changes to your preferences or weights to increase the degree of consensus and reach the given consensus threshold.

4. Consensus Reaching Process and Scoring Calculation

This section focuses on developing a method for determining CRP parameters, and identifying minority opinions and non-cooperative behaviors for management and adjustment. A subjective and objective adjustment factor is developed to deal with the LGWRSA problem of the DLLPR.

4.1. Comprehensive Adjustment Factor

A.

Subjective adjustment factor:

During the CRP, if $GC<\xi$, a consensus has not yet been reached. Supposing that $g_{\lambda }$ is the least consistent of all subgroups, $C\left(A^{g\lambda }\right)=min\left\{C\left(A^{gi}\right)\right|i=1,2,\dots ,\xi \}$. To improve the group consensus level, the subgroup discusses an adjustment factor for itself:${\epsilon }_{g\lambda }^s(0\le {\epsilon }_{g\lambda }^s\le 1)$. The larger the adjustment factor, the more likely the subgroup is to modify its own opinion.

B.

Objective adjustment factor:

Generally speaking, for $g_{\lambda }$, the greater the difference between the subgroup and the whole, the greater the subgroup should improve to reach the consensus threshold $\xi$. In other words, the lower the consistency of the subgroup $g_{\lambda }$, the more the subgroup members’ opinions need to be adjusted. The objective adjustment factor is calculated by the following formula:

$${\epsilon }_{g\lambda }^o=\frac{1-\xi }{1-C\left(A^{g\lambda }\right)}$$

(11)

where $0\le {\epsilon }_{g\lambda }^o\le 1$, and it can be seen from the formula that the higher the given consistency threshold, the more experts need to adjust their opinions.

C.

Comprehensive adjustment factor:

Considering both the subjective and objective adjustment factors, a comprehensive adjustment factor reflecting the degree to which the final group opinion needs to be adjusted can be obtained by the following:

$${\epsilon }_{g\lambda }=\left\{\begin{array}{ll}{\epsilon }_{g\lambda }^s,\hfill & {\epsilon }_{g\lambda }^o\ge {\epsilon }_{g\lambda }^s\hfill \\ \sigma {\epsilon }_{g\lambda }^s+(1-\sigma ){\epsilon }_{g\lambda }^o,\hfill & {\epsilon }_{g\lambda }^o<{\epsilon }_{g\lambda }^s\hfill \end{array}\right.$$

(12)

where $0\le {\epsilon }_{g\lambda }\le 1$, and the parameters $\sigma (0\le \sigma \le 1)$ are used to reflect the importance of the subjective adjustment factor and can be chosen by yourself.

After calculating the comprehensive adjustment factor, the adjusted preference matrix of $g_{\lambda }$ can be calculated according to the following formula:

$$A^{g\lambda \ast }={\epsilon }_{g\lambda }{A}^{\ast }+(1-{\epsilon }_{g\lambda })A^{g\lambda }$$

(13)

After this calculation process, the consistency of the population is improved, and it is easier to reach the given consistency threshold.

4.2. Managing Minority Opinions

A.

Identify minority views

A subgroup $g_{\lambda }$ is considered a minority when meeting the following two conditions.

(1)

The subgroup should have the least degree of consensus $C\left(A^{g\lambda }\right)$

(2)

The number of members in this subgroup is less than the average number of members per subgroup.

When $C\left(A^{g\lambda }\right)=min\left\{C\left(A^{gi}\right)\right|i=1,2,\dots ,\zeta \}$ and $n^{g\lambda }<n/\zeta$, it can be considered that subgroup $g_{\lambda }$ is a minority group, where $n^{g\lambda }$ is the number of members of the subgroup $g_{\lambda }$, $n$ is the sum of the number of all experts, and $\zeta$ is the number of a subgroup.

B.

Judge whether the opinion of this subgroup is worth adopting

First, let the minority opinion groups explain the rationality of their opinions, and then discuss the minority opinion groups in the remaining other groups and express their attitudes and opinions.

If over half of the subgroups believe the subgroup $g_{\lambda }$’s opinion is worth considering, i.e., $n^{\ast }>(\zeta -1)/2$, then the weight of this subgroup ${\omega }_{\lambda }$ should be improved to enhance the importance. At the same time, the degree of adjustment should be based on the number of subgroups supporting the subgroup $n^{\ast }$. The more subgroups that support minority opinions, the higher the weight of the subgroup should be. If the number of subgroups supporting the opinions of the subgroup $g_{\lambda }$ is less than half, which means a majority of experts have different opinions on the rationality of the opinions given by the minority subgroup, then step C can be skipped, and the weight of this subgroup is not increased.

C.

Adjust the weight of this minority subgroup

Suppose that $\mathrm{\Omega }={({\omega }_1, {\omega }_2,\dots {\omega }_{\zeta })}^T$ is the weight vector of all subgroups and arrange them in ascending order as $\mathrm{\Omega }={({\omega }_{(1)}, {\omega }_{(2)},\dots {\omega }_{(}{}_{\zeta )})}^T$. Let the weight of the minority opinion subgroup $g_{\lambda }$ be ${\omega }_{(w)}$, i.e., the subgroup with the smallest weight. The weight adjustment coefficient ${\Delta }_{\lambda }$ can be obtained by calculating the number of support subgroups $n^{\ast }$ and half of the number of the remaining subgroups $(\zeta -1)/2$:

$${\Delta }_{\lambda }=\left\{\begin{array}{ll}{n}^{\ast }-(\zeta -1)/2,\hfill & \zeta =2k-1;(k=1,2,\dots )\hfill \\ n^{\ast }-\zeta /2,\hfill & \zeta =2k;(k=1,2,\dots )\hfill \end{array}\right.$$

(14)

Based on the above weight adjustment coefficient ${\Delta }_{\lambda }$, the weight of the minority opinion subgroup $g_{\lambda }$ can be adjusted as follows:

$${\omega }_{\lambda }^{\ast }=\min \left\{{\omega }_{(\zeta )},{\omega }_{(w+{\Delta }_{\lambda })}\right\}$$

(15)

where ${\omega }_{\lambda }^{\ast }$ is the adjusted weight of the subgroup $g_{\lambda }$.

Repeat the consistency measurement of the above method until all minority opinions are considered, then proceed to the next step.

4.3. Dealing with Non-Cooperative Behavior

A.

Identify non-cooperative groups:

For a subgroup $g_{\lambda }$, its objective adjustment coefficient is ${\epsilon }_{g\lambda }^o$, and other subgroups are required to give the degree to which the subgroup $g_{\lambda }$’s opinions need to be modified, expressed as ${\epsilon }_{g\lambda }^{\ast }$. If the subjective adjustment coefficient of this subgroup ${\epsilon }_{g\lambda }^s$ meets the condition ${\epsilon }_{g\lambda }^s\le \max \left\{{\epsilon }_{g\lambda }^o,{\epsilon }_{g\lambda }^{\ast }\right\}$, the subgroup $g_{\lambda }$ can be regarded as a non-cooperative group. After identifying the non-cooperative group, we can proceed to the next step of the calculation.

B.

Measure the degree of non-cooperation:

The degree of non-cooperation ${\psi }_{\lambda }$ is used to indicate the degree to which the subgroup $g_{\lambda }$ is unwilling to adjust its views. The greater the degree of non-cooperation, the more stubborn the members of the subgroup are and the more difficult it is to modify. Based on this definition, we use the difference between the subjective adjustment coefficient ${\epsilon }_{g\lambda }^s$ and the comprehensive adjustment coefficient ${\epsilon }_{g\lambda }$ of the subgroup $g_{\lambda }$ to define the degree of non-cooperation ${\psi }_{\lambda }$, which can be calculated by the following formula:

$${\psi }_{\lambda }=\frac{{\epsilon }_{g\lambda }-{\epsilon }_{g\lambda }^s}{{\epsilon }_{g\lambda }}$$

(16)

After calculating the degree of non-cooperation ${\psi }_{\lambda }$, the weight of the non-cooperative group can be appropriately reduced according to it, i.e., the content in step C.

C.

Adjust the weight of non-cooperative groups:

The degree of non-cooperation ${\psi }_{\lambda }$ indicates the degree to which the subgroup $g_{\lambda }$ is unwilling to adjust its views. Therefore, the greater the ${\psi }_{\lambda }$, the greater the degree to which its weight should be reduced.

(1)

If ${\psi }_{\lambda }=0$, then the subgroup $g_{\lambda }$ can be considered as a fully cooperative group. There is no need to change the weight of $g_{\lambda }$; directly use the comprehensive adjustment coefficient to adjust their preferences;

(2)

If ${\psi }_{\lambda }=1$, then the subgroup $g_{\lambda }$ can be considered as a completely uncooperative group. The extreme opinions of this subgroup do not have any positive impact on our decision making, so the best choice is to delete them;

(3)

If $0<{\psi }_{\lambda }<1$, then the subgroup $g_{\lambda }$ can be considered as a partially uncooperative group. Therefore, it should adjust their weights to improve their opinions and then use the comprehensive adjustment coefficient to adjust their preferences.

The modified weight is as follows:

$${\omega }_{\lambda }^{\ast }=\left\{\begin{array}{ll}\begin{array}{c}\begin{array}{c}{\omega }_{\lambda },\hfill \\ {\omega }_{\lambda }\times 0.9,\hfill \\ {\omega }_{\lambda }\times 0.8,\hfill \\ {\omega }_{\lambda }\times 0.7,\hfill \end{array}\\ \begin{array}{c}{\omega }_{\lambda }\times 0.6,\\ {\omega }_{\lambda }\times 0.5,\\ {\omega }_{\lambda }\times 0.4,\end{array}\\ \begin{array}{c}{\omega }_{\lambda }\times 0.3,\hfill \\ {\omega }_{\lambda }\times 0.2,\hfill \\ {\omega }_{\lambda }\times 0.1,\hfill \\ 0,\hfill \end{array}\end{array}\hfill & \begin{array}{c}\begin{array}{c}{\psi }_{\lambda }\in [0,0.1)\hfill \\ {\psi }_{\lambda }\in [0.1,0.2)\hfill \\ {\psi }_{\lambda }\in [0.2,0.3)\hfill \\ {\psi }_{\lambda }\in [0.3,0.4)\hfill \end{array}\\ \begin{array}{c}{\psi }_{\lambda }\in [0.4,0.5)\\ {\psi }_{\lambda }\in [0.5,0.6)\\ {\psi }_{\lambda }\in [0.6,0.7)\end{array}\\ \begin{array}{c}{\psi }_{\lambda }\in [0.7,0.8)\hfill \\ {\psi }_{\lambda }\in [0.8,0.9)\hfill \\ {\psi }_{\lambda }\in [0.9,1)\hfill \\ {\psi }_{\lambda }=1\hfill \end{array}\end{array}\hfill \end{array}\right.$$

(17)

where ${\omega }_{\lambda }^{\ast }$ is the adjusted weight of the subgroup $g_{\lambda }$. Then, repeat the above method until the subgroup no longer contains non-cooperative groups. At this point, the CRP ends.

4.4. Scoring Calculation

After the above steps, the opinions of experts have been reasonably classified and adjusted, and we finally obtain the adjusted overall preference matrix $A^{\ast }$, which can be used to calculate the scoring value of each scheme with the following formula:

$$E\left(p_i\right)={\displaystyle \sum _{j=1}^m\frac{f\left(a_{ij}^{\ast }\right)}{m},\left(i=1,2,\dots m\right)}$$

(18)

4.5. Summarize this Algorithm

With the above method, an algorithm for processing the LGWRSA with the DLLPR is established, and the detailed steps are described in Figure 2:

Step 1: Cluster experts into $\zeta$ subgroups as follows: $G=\{ g_1,g_2,\dots , g_{\zeta }\}$. Calculate the weight vectors for each subgroup according to Formula (7), and then calculate the group preference matrix for each subgroup according to Formula (8): $A^{\lambda }={(a_{ij}^{\lambda })}_{m\times m}$.

Step 2: Aggregate all group preference matrices into a global preference matrix based on Formula (8): $A^{\ast }={(a_{ij}^{\ast })}_{m\times m}$.

Step 3: Calculate the degree of consensus $C\left(A^{g\lambda }\right)$ among subgroups based on Formula (9); the consensus degree $GC$ is obtained from Formula (10). If $GC\ge \xi$, jump to Step 5; otherwise, execute Step 4.

Step 4: Consensus improvement procedure

(1) Identify and manage groups with fewer opinions to determine if their weights need to be fixed. If so, we use Formula (15) to modify it and go back to Step 2; otherwise, execute Step 4.

(2) Determine whether there are non-cooperative groups by method 2. If there is, first reduce its weight according to Formula (17), then calculate the comprehensive adjustment factor according to Formula (12), and use it to adjust preference information. Otherwise, only the comprehensive adjustment factor can be calculated and selected. Go back to Step 2.

Step 5: Output group preference matrix $A^{\lambda }$ for each subgroup and overall preference matrix $A^{\ast }$.

Step 6: Sum all preference results for each row $A^{\ast }$ and find the expected values for each option.

The working flow diagram of the algorithm is illustrated in Figure 2.

5. Case Study

5.1. Subjective Assessment Results

This section applies the proposed algorithm to an LGWRSA problem on water resource sustainability assessment, requiring some experts to express their views on water resource sustainability in the four regions $\left\{ Tibet, Sichuan, Yunnan, Qinghai \right\}$. The expectations for water sustainability in these four regions are calculated in the end.

Supposing that the first-level LTS and the second-level LTS are S = {s₋₃ = very bad, s₋₂ = bad, s₋₁ = slightly bad, s₀ = medium, s₁ = slightly good, s₂ = good, s₃ = very good}; O = {o₋₃ = scarcely, o₋₂ = only a little, o₋₁ = a little, o₀ = just right, o₁ = much, o₂ = very much, o₃ = extremely much}, respectively, we asked 15 experts to give their preference matrix $\left(A_1~A_{15}\right)$:

$$A_1=\left(\begin{array}{cc}\begin{array}{ccc}{s}_0<o_0>& s_1<o_2>& s_3<o_1>\\ s_{-1}<o_{-2}>& s_0<o_0>& s_0<o_2>\\ s_{-3}<o_{-1}>& s_0<o_{-2}>& s_0<o_0>\end{array}& \begin{array}{c}{s}_1<o_{-2}>\\ s_2<o_3>\\ s_{-1}<o_1>\end{array}\\ \begin{array}{ccc}{s}_{-1}<o_2>& s_{-2}<o_{-3}>& s_1<o_{-1}>\end{array}& s_0<o_0>\end{array}\right)$$

$$A_2=\left(\begin{array}{cccc}{s}_0<o_0>& s_2<o_{-2}>& s_2<o_2>& s_1<o_{-2}>\\ s_{-2}<o_2>& s_0<o_0>& s_0<o_2>& s_2<o_3>\\ s_{-2}<o_{-2}>& s_0<o_{-2}>& s_0<o_0>& s_{-1}<o_1>\\ s_{-1}<o_2>& s_{-2}<o_{-3}>& s_1<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_3=\left(\begin{array}{cccc}{s}_0<o_0>& s_1<o_3>& s_3<o_0>& s_1<o_{-2}>\\ s_{-1}<o_{-3}>& s_0<o_0>& s_1<o_2>& s_2<o_2>\\ s_{-3}<o_0>& s_1<o_{-2}>& s_0<o_0>& s_{-1}<o_1>\\ s_{-1}<o_2>& s_{-2}<o_{-2}>& s_1<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_4=\left(\begin{array}{cccc}{s}_0<o_0>& s_2<o_{-2}>& s_2<o_3>& s_1<o_{-2}>\\ s_{-2}<o_2>& s_0<o_0>& s_0<o_1>& s_1<o_1>\\ s_{-2}<o_{-3}>& s_0<o_{-1}>& s_0<o_0>& s_{-2}<o_1>\\ s_{-1}<o_2>& s_{-1}<o_{-1}>& s_2<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_5=\left(\begin{array}{cccc}{s}_0<o_0>& s_{-3}<o_2>& s_1<o_0>& s_2<o_{-1}>\\ s_3<o_{-2}>& s_0<o_0>& s_0<o_2>& s_2<o_{-1}>\\ s_{-1}<o_0>& s_0<o_{-2}>& s_0<o_0>& s_{-1}<o_2>\\ s_{-2}<o_1>& s_{-2}<o_1>& s_1<o_{-2}>& s_0<o_0>\end{array}\right)$$

$$A_6=\left(\begin{array}{cccc}{s}_0<o_0>& s_0<o_0>& s_1<o_{-1}>& s_0<o_{-2}>\\ s_0<o_0>& s_0<o_0>& s_0<o_2>& s_1<o_1>\\ s_{-1}<o_1>& s_0<o_{-2}>& s_0<o_0>& s_{-1}<o_2>\\ s_0<o_2>& s_{-1}<o_{-1}>& s_1<o_{-2}>& s_0<o_0>\end{array}\right)$$

$$A_7=\left(\begin{array}{cccc}{s}_0<o_0>& s_1<o_2>& s_3<o_2>& s_1<o_{-2}>\\ s_{-1}<o_{-2}>& s_0<o_0>& s_{-1}<o_2>& s_2<o_3>\\ s_{-3}<o_{-2}>& s_1<o_{-2}>& s_0<o_0>& s_1<o_1>\\ s_{-1}<o_2>& s_{-2}<o_{-3}>& s_{-1}<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_8=\left(\begin{array}{cccc}{s}_0<o_0>& s_1<o_2>& s_2<o_2>& s_1<o_{-2}>\\ s_{-1}<o_{-2}>& s_0<o_0>& s_0<o_2>& s_1<o_3>\\ s_{-2}<o_{-2}>& s_0<o_{-2}>& s_0<o_0>& s_{-1}<o_1>\\ s_{-1}<o_2>& s_1<o_{-3}>& s_1<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_9=\left(\begin{array}{cccc}{s}_0<o_0>& s_{-2}<o_2>& s_1<o_1>& s_2<o_{-1}>\\ s_2<o_{-2}>& s_0<o_0>& s_0<o_2>& s_2<o_1>\\ s_{-2}<o_{-1}>& s_0<o_{-2}>& s_0<o_0>& s_0<o_2>\\ s_{-2}<o_1>& s_{-2}<o_{-1}>& s_0<o_{-2}>& s_0<o_0>\end{array}\right)$$

$$A_{10}=\left(\begin{array}{cccc}{s}_0<o_0>& s_1<o_2>& s_3<o_1>& s_1<o_{-2}>\\ s_{-1}<o_{-2}>& s_0<o_0>& s_0<o_2>& s_2<o_3>\\ s_{-3}<o_{-1}>& s_0<o_{-2}>& s_0<o_0>& s_{-1}<o_1>\\ s_{-1}<o_2>& s_{-2}<o_{-3}>& s_1<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_{11}=\left(\begin{array}{cccc}{s}_0<o_0>& s_2<o_{-2}>& s_2<o_2>& s_1<o_{-2}>\\ s_{-2}<o_2>& s_0<o_0>& s_0<o_0>& s_1<o_{-1}>\\ s_{-2}<o_{-2}>& s_0<o_0>& s_0<o_0>& s_{-2}<o_1>\\ s_{-1}<o_2>& s_{-1}<o_1>& s_2<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_{12}=\left(\begin{array}{cccc}{s}_0<o_0>& s_0<o_0>& s_1<o_{-2}>& s_0<o_{-1}>\\ s_0<o_0>& s_0<o_0>& s_0<o_2>& s_1<o_1>\\ s_1<o_2>& s_0<o_{-2}>& s_0<o_0>& s_{-1}<o_1>\\ s_0<o_1>& s_{-1}<o_{-1}>& s_1<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_{13}=\left(\begin{array}{cccc}{s}_0<o_0>& s_2<o_{-3}>& s_1<o_2>& s_1<o_{-2}>\\ s_{-2}<o_3>& s_0<o_0>& s_0<o_1>& s_1<o_0>\\ s_{-1}<o_{-2}>& s_0<o_{-1}>& s_0<o_0>& s_1<o_1>\\ s_{-1}<o_2>& s_{-1}<o_0>& s_{-1}<o_{-1}>& s_0<o_0>\end{array}\right)$$

$$A_{14}=\left(\begin{array}{cccc}{s}_0<o_0>& s_{-1}<o_2>& s_1<o_{-1}>& s_2<o_{-2}>\\ s_1<o_{-2}>& s_0<o_0>& s_0<o_2>& s_2<o_{-1}>\\ s_{-1}<o_1>& s_0<o_{-2}>& s_0<o_0>& s_0<o_2>\\ s_{-2}<o_2>& s_{-2}<o_1>& s_0<o_{-2}>& s_0<o_0>\end{array}\right)$$

$$A_{15}=\left(\begin{array}{cccc}{s}_0<o_0>& s_{-2}<o_2>& s_1<o_{-1}>& s_2<o_{-1}>\\ s_2<o_{-2}>& s_0<o_0>& s_0<o_2>& s_2<o_1>\\ s_{-1}<o_1>& s_0<o_{-2}>& s_0<o_0>& s_0<o_1>\\ s_{-2}<o_1>& s_{-2}<o_{-1}>& s_0<o_{-1}>& s_0<o_0>\end{array}\right)$$

Given that the consensus threshold is $\xi =0.9$, the clustering result is $G=\left\{g_1,g_2,g_3,g_4\right\}$, where $g_1=\left\{s_1,s_3,s_7,s_8,s_{10}\right\}$, $g_2=\left\{s_2,s_4,s_{11},s_{13}\right\}$, $g_3=\left\{s_5,s_9,s_{14},s_{15}\right\}$, and $g_4=\left\{s_6,s_{12}\right\}$. The clustering results can be seen in

Table 1. Clustering results and all subgroups’ weights.

Subgroup	Members	$\mathbf{Weight} {\mathit{\omega }}_{\mathit{\lambda }}$	$\mathbf{Consensus} \mathbf{Degree} \mathit{C}\left({\mathit{A}}^{\mathit{g}\mathit{\lambda }}\right)$
$g_1$	$s_1,s_3,s_7,s_8,s_{10}$	0.3333	0.9172
$g_2$	$s_2,s_4,s_{11},s_{13}$	0.2667	0.8611
$g_3$	$s_5,s_9,s_{14},s_{15}$	0.2667	0.8889
$g_4$	$s_6,s_{12}$	0.1333	0.7704

.

As seen in Figure 3, the overall consensus $GC=0.8594$, the threshold $0.9$ is not reached, and the CRP process is required.

First, identify minorities; we know subgroup $g_4$ meets the definition of minority and has a low degree of consensus, so a subgroup $g_4$ is defined as a minority. Let other subgroups express their views on the opinions of a subgroup $g_4$. The result is that two subgroups support the opinions of the subgroup $g_4$, and the opinions of the subgroup $g_4$ are worth adopting. ${\Delta }_{\lambda }=1$; change the weight of the $g_4$ subgroup to $0.2667$. After normalization, the following

Table 2. The weight and consensus degrees of each subgroup after the first round.

Subgroup	${\mathit{g}}_1$	${\mathit{g}}_2$	${\mathit{g}}_3$	${\mathit{g}}_4$
${\omega }_{\lambda }$	0.2941	0.2353	0.2353	0.2353
$C(A^{g\lambda })$	0.8979	0.8519	0.8904	0.8604
$GC$	0.8751

is obtained:

As seen in Figure 4, the overall consensus $GC=0.8704$, the threshold $0.9$ is not reached, and the CRP process is required.

At this time, the subgroup with the lowest degree of consensus is $g_2$. Let them give their subjective adjustment coefficient ${\epsilon }_{g2}^s=0.35$, and then calculate the objective adjustment coefficient of this subgroup ${\epsilon }_{g2}^o=0.3248$; we know that ${\epsilon }_{g2}^o<{\epsilon }_{g2}^s$, the subgroup $g_2$ is very willing to adjust their own opinions, comprehensive adjustment coefficient ${\epsilon }_{g2}={\epsilon }_{g2}^s=0.35$, and degree of non-cooperation ${\psi }_2=0$; the subgroup can be regarded as a fully cooperative subgroup without changing the weight of $g_2$. The calculation process based on Formula (13) directly uses the comprehensive adjustment coefficient ${\epsilon }_{g2}=0.35$ to adjust the preference for $g_2$, as seen in

Table 3. The weight and consensus degrees of each subgroup after the second round.

Subgroup	${\mathit{g}}_1$	${\mathit{g}}_2$	${\mathit{g}}_3$	${\mathit{g}}_4$
${\omega }_{\lambda }$	0.2941	0.2353	0.2353	0.2353
$C(A^{g\lambda })$	0.9079	0.9114	0.8853	0.8553
$GC$	0.8751

.

As seen in Figure 5, the overall consensus $GC=0.8898$, the threshold $0.9$ is not reached, and the CRP process is continued.

At this point, the subgroup with the lowest degree of consensus is $g_4$, they are asked to give their own subjective adjustment factor ${\epsilon }_{g4}^s=0.25$, and then the objective adjustment factor for the subgroup is calculated, ${\epsilon }_{g4}^o=0.3089$, ${\epsilon }_{g4}^o>{\epsilon }_{g4}^s$. We can see that the $g_4$ subgroup is not very willing to adjust their opinions, the comprehensive adjustment factor ${\epsilon }_{g4}=0.5{\epsilon }_{g4}^o+0.5{\epsilon }_{g4}^s=0.279$, and degree of non-cooperation ${\psi }_4=0.103$; consider it as a partially uncooperative subgroup, change the weight of $g_4$ based on Formula (13) to ${\omega }_{g4}^{\ast }=0.2118$, and normalize the weight vector $\mathrm{\Omega }={\left(0.3012, 0.2409, 0.2409, 0.2169\right)}^T$. Then, use the comprehensive adjustment factor ${\epsilon }_{g\lambda }=0.279$ based on formula (13) to fix the preference for $g_4$, as seen in

Table 4. The weight and consensus degrees of each subgroup after the third round.

Subgroup	${\mathit{g}}_1$	${\mathit{g}}_2$	${\mathit{g}}_3$	${\mathit{g}}_4$
${\omega }_{\lambda }$	0.3012	0.2409	0.2409	0.2169
$C(A^{g\lambda })$	0.9037	0.9146	0.8793	0.9251
$GC$	0.9057

.

As seen in Figure 6, the overall consensus $GC=0.9057$ reaches the threshold of $\xi =0.9$, and the CRP ends.

The final $\left\{ Sichuan, Yunnan, Qinghai, Tibet \right\}$ water resource sustainability expectations can be calculated based on Formula (18). The normalized results of subjective scoring are as follows: $E\left(Sichuan\right)=0.6312\hspace{0.17em}$, $E\left(Yunnan\right)=0.5271\hspace{0.17em}$, $E\left(Qinghai\right)=0.4461$, and $E\left(Tibet\right)=0.4871$.

5.2. Overall Assessment Results

The above process involves large-scale group decision making that provides the evaluation rankings concerning the subjective indicators, while the assessment process should also involve the evaluation scores of objective indicators. In this specific LSWRSA problem, there are four indicators, i.e., environmental conservation, maintenance capacity, social equity, and economic efficiency, of which economic efficiency is a subjective indicator, and the remaining indicators are objective ones. After sufficient discussion, the 15 experts involved agree that economic efficiency has a weight of 0.4, and the remaining indicators have a total weight of 0.6. We provide the original data of objective indicators for the water resource sustainability assessment (in the year 2021) in the supplement file. Therefore, the subjective scores, objective scores, and overall scores are computed and shown in

Table 5. The scores of four provinces.

Area	Sichuan	Yunnan	Qinghai	Tibet
Subjective score	0.6312	0.5271	0.4461	0.4871
Objective score	0.7241	0.6563	0.5415	0.6255
Overall score	0.6869	0.6046	0.5033	0.5701

.

6. Discussion

6.1. Analyzing the Results of the Case Study

Based on the above results, we can know that the result of the water resource sustainability assessment is Sichuan > Yunnan > Tibet > Qinghai, i.e., Sichuan Province has the highest score on water resource sustainability. According to the survey, the reasons can be summarized as follows:

(1) Since 2018, to strengthen and standardize the unified management of water resources in the province, Sichuan Province has formulated a water resources management method. Unified regulation and allocation of water resources in a catchment or a catchment surrounded by surface and groundwater dividers.

(2) Sichuan Province ensures that in the process of water resources development, the waste and pollution of water resources are effectively reduced, and the sustainable use of water resources is guaranteed.

(3) As far as sewage treatment is concerned, Sichuan Province actively uses advanced equipment and efficient purification equipment to purify sewage for secondary use.

Therefore, the highest score of water resource sustainability obtained in Sichuan Province is the result of the active formulation and implementation of water resource protection and development measures from the leadership to the people at the grassroots of Sichuan Province. Scientific methods and active implementation are the keys to the winning road.

6.2. Comparative Analysis

Concerning water resource sustainability assessment, many methods have been proposed: Yu et al., based on the driver–pressure–state–impact–response model [31], conducted a comprehensive evaluation of water resource sustainability changes in Beijing from 2008 to 2018. The model considers regional, social, economic, and environmental water systems and climate change, but its criteria have strong limitations. Once you want to add an indicator to a region’s water resources, you need to modify the model substantially, which is not stable enough. If an index needs to be added to the model in this study, it is only necessary to judge whether the objective data of the index are accurate and easy to find. If so, it is directly evaluated by the objective data. If not, ask several experts in the field to give the DLLPR and ultimately calculate the expectations for the indicator. It can be seen that the water resource sustainability assessment model proposed in this study is very stable, and it is easy to add, delete, or change the assessment criteria.

In addition, for the linguistic models, Gou et al. [15] proposed a CRP for the LGWRSA with a double hierarchy hesitant fuzzy linguistic preference relation. Using Gou et al.’s method, the results of clustering and decision making are similar to those proposed, but the focus of the results is different. The clustering technique proposed by Gou et al. is based on similarity and provides a CRP with a feedback mechanism. However, this study discusses clustering methods based directly on distance measures and omits the process of calculating similarity measures. Therefore, it has fewer steps than Gou et al.’s method. In addition, Gou et al. do not recognize minority opinions and non-cooperative behaviors, which may result in some information being ignored in the analysis. In summary, the methods presented in this study deal with non-cooperative behavior and minority opinions, and the CRP process is more elaborate and complex, considering more details. A consensus approach by LGWRSA is useful for an LGWRSA with a variety of types of decision information, and the detailed comparative results with the related consensus models are shown in

Table 6. Comparative results with the related consensus models.

Consensus Models	Considering Minority Opinions	Considering Non-Cooperative Behaviors	Considering Linguistic Preference Relation
Li et al. [32]	Not considered	Confidence and social network-based feedback mechanism	Not considered
Gai et al. [33]	Not considered	Decentralized feedback mechanism	Not considered
Liang et al. [34]	Not considered	Social network-based feedback mechanism	Not considered
Zhang et al. [35]	Not considered	Fuzzy rules and optimal allocation-based feedback mechanism	Not considered
Liang et al. [36]	Not considered	Two-stage consensus feedback mechanism	Not considered
Wu et al. [37].	Group polarization model considering minority rights and satisfying the majority’s requirements	Not considered	Not considered
Ren et al. [38]	Social network-based feedback mechanism	Not considered	Hesitant fuzzy hesitant preference relations
Xiao et al. [39]	Extended MULTIMOORA method protecting minority opinions	Not considered	Not considered
Zhou et al. [23]	Feedback mechanism	Feedback mechanism	Not considered
Xu et al. [27]	Feedback mechanism	Feedback mechanism	Not considered
Liu et al. [40]	Not considered	Social network-based feedback mechanism	Linguistic preference relation
Zheng et al. [5]	Not considered	Compatibility adjusting process-based feedback mechanism	Hesitant fuzzy linguistic preference relations
Zhou et al. [41]	Not considered	Credibility measure-based feedback mechanism	Hesitant fuzzy linguistic preference relations
Wan et al. [42]	Not considered	Personalized individual semantics-based feedback mechanism	Probabilistic linguistic preference relations
Liao et al. [43]	Not considered	Social network-based feedback mechanism	probabilistic linguistic preference relations
Our proposal	Feedback mechanism considering comprehensive adjustment	Feedback mechanism considering comprehensive adjustment	Double-level linguistic preference relation (DLLPR)

.

Seeing Table 6, we can conclude that only a few studies in the literature consider both minority opinions and non-cooperative behaviors simultaneously. Although Zhou et al. [23] and Xu et al. [27] consider both aspects, they do not handle the linguistic preference relations among alternatives, which is not suitable for real LGWRSA problems because experts cannot always provide their preference evaluation information in a non-pairwise manner. For the other consensus models managing non-cooperative behaviors, the social network-based feedback mechanism is frequently used to modify the opinions or weights of non-cooperative subgroups [32,34,38,40,43]. But, in the decision-making process, the experts could not always trust each other, which makes the social network-based mechanism function improperly. Other consensus models use the feedback mechanism based on various measures to adjust the non-cooperative subgroups or minority opinions, but they do not consider the comprehensive adjustment factor reflecting the degree to which the final group opinion needs to be adjusted [5,33,35,36,39,41,42]. In contrast, the proposed consensus model is more comprehensive than the existing methods because it adjusts minority opinions and manages non-cooperative behavior. Particularly, the consensus method to adjust the weight of non-cooperative behavior can reflect the degree of non-cooperation of the team in more detail [44,45,46]. In all, many existing methods are compared synthetically, and the proposed methods have the following two advantages:

(1) The approach proposed in this study focuses on managing minority opinions and non-cooperative behaviors to reach a final consensus, so we only need to cluster decision-making groups at the beginning. However, many existing methods require clustering experts in each iteration, which wastes a lot of time and makes the CRP complex and cumbersome;

(2) There are some deficiencies in the existing studies on a few opinions and uncooperative behaviors: some studies only study a part of them, resulting in incomplete information processing. Clustering methods involve too many human factors and lose a lot of original decision information when dealing only with non-cooperative behavior or minority views.

6.3. Comparative Experiments for the Clustering

To verify the practicability and stability of the proposed clustering method, we perform the following comparative experiments: 10, 15, and 20 experts are asked to give their opinions in the form of the DLLPR, respectively, and the expert subgroups are clustered using the K-means [7] clustering method, the FCM clustering method [28], and the DLLPR-based clustering method proposed in this paper, and the results obtained are shown in the following Figure 7, Figure 8 and Figure 9.

It can be seen that using the clustering method proposed in this paper has obtained the same results as the K-means clustering method and the FCM clustering method, but does not have the aforementioned disadvantages of the latter two methods. This experiment can prove that the proposed clustering method has practical practicality and reliability. This approach fits people’s intuitive perceptions and can be applied to clustering a large number of decision makers into several subgroups.

7. Conclusions

This study presents a complete approach to the LGWRSA. By establishing a consensus model to manage non-cooperative behaviors and a few suggestions in decision making in a double-level linguistic preference relation environment, a clustering method, weight determination method, and consistency measurement method based on the DLLPR are presented. In addition, this study gives the CRP in the LGWRSA, including how to determine the comprehensive adjustment factor, as well as how to adjust minority opinions and manage non-cooperative behavior. In addition, we apply the algorithm to a real LGWRSA problem, evaluate the water resource sustainability of four provinces, Sichuan, Yunnan, Tibet, and Qinghai, and make a detailed comparative analysis. The same results reflected by the objective data are obtained, which proves the rationality and practicability of the method. Some advantages of the proposed method are summarized below.

(1) The proposed clustering method omits the unnecessary steps of traditional clustering methods and makes the clustering steps simpler;

(2) The proposed LGWRSA model is more comprehensive than the existing methods because it adjusts minority opinions and manages non-cooperative behavior;

(3) The proposed method to adjust the weight of non-cooperative behavior can reflect the degree of non-cooperation of the team in more detail;

(4) The comprehensive adjustment factor in the CRP considers both subjective and objective adjustment.

However, this method still has shortcomings. If a large number of experts intentionally give incorrect opinions for the sake of profit, this method cannot identify them and will affect the final evaluation results. So, if using this method, try to find experts who are not motivated by their own profit for evaluation.

In future research, we will focus on more language models and more consensus-building approaches. In addition, using the proposed method can solve some more practical LGWRSA problems, which is very satisfying.