Consensus-Based Automatic Group Decision-Making Method with Reliability and Subjectivity Measures Based on Sentiment Analysis

Abstract

The use of informal language on social media and the sheer volume of information make it difficult for a computer system to analyse it automatically. The aim of this work is to design a new group decision-making method that applies two new consensus methods based on sentiment analysis. This method is designed for application in the analysis of texts on social media. To test the method, we will use posts from the so called social network X. The proposed model differs from previous work in this field by defining a new degree of subjectivity and a new degree of reliability associated with user opinions. This work also presents two new consensus measures, one focused on measuring the number of words classified as positive and negative and the other on analysing the percentage of occurrence of those words. Our method allows us to automatically extract preferences from the transcription of the texts used in the debate, avoiding the need for users to explicitly indicate their preferences. The application to a real case of public investment demonstrates the effectiveness of the approach in collaborative contexts that used natural language.

1. Introduction

Group decision-making is a process that human beings must carry out on a daily basis. We can define a group decision-making process as the selection of one option from a set of options. We call these the alternatives [1,2]. In real life, decisions that can affect a large group of people are made, making it is necessary to have the support of computerised systems to assist group decision-making processes and the application of methods that allow an objective, organised and fair process to be carried out [1]. Opinions are fundamental to almost all human activities because they are key factors that influence our behaviours. The acquisition of public and consumer opinions has long been a big business in itself for marketing, public relations and political campaigning companies [3].

Social networks represent a rich source of data that can be accessed for study and analysis. Individuals and organisations are increasingly using its content to carry out decision-making processes. Among other things, new automatic tools need to be developed to extract information from the networks, analyse it and enable the system to understand the course of the debate. One of the major stumbling blocks in this process is that the information represented in these media uses natural language, which means that special methods are needed for its analysis.

Current decision-making methods that extract information from networks work with sentiment analysis methods. In this way, by analysing experts’ contributions to the debate, it is possible to determine the degree of an expert’s preference for an alternative. However, no measure of control over how that result has been extracted and no information other than the calculated preference is established. This can be a problem, as it is impossible to know whether the preference extracted is real or whether the text being analysed has been misinterpreted. Therefore, establishing measures of error or reliability of the information with which these types of automatic preference extraction methods work is very important if we want to be able to trust the results they provide. Within a group decision-making process, it is important that the group of experts obtain a high degree of consensus prior to the application of the selection process [4]. In order to obtain a high level of consensus among the experts, it is advisable to provide the whole group of experts with some advice (feedback) on how far the group is from consensus, what are the most controversial issues (alternatives), what are the major disagreements with the rest of the group and how their change could influence the overall degree of consensus [5].

In this article, we present a new group decision-making method that works on debates held on social networks. Our objective is to create a new group decision-making method that, based on a debate carried out in a social network, extracts the information from the debate, analyses it, obtains the preferences of the experts on the alternatives and, subsequently, determines the final decision reached. In addition, our method will be able to calculate the consensus reached among the participants. The extraction of preferences is done by using sentiment analysis techniques. This type of technique works on written texts using unstructured natural language. We will perform a dichotomous analysis of the texts (positive sentiments and negative sentiments), of the texts associated with each alternative mentioned by the expert. For the calculation of consensus we propose two new measures of consensus. One is based on the degree of subjectivity of the natural language expressions provided by the experts in the debate. The other measure will compare the preference values extracted for each expert and each alternative. Our method will also include the measurement of opinion reliability. This is calculated as the ratio of positive and negative words used in the discussion. For instance, if an expert uses both positive and negative words in the description of an alternative, then the preference extracted is not as clear as if only positive or negative words were used. In order to take this measure into consideration, we will use Z-numbers [6] to represent the preference values, as this type of representation allows us to represent the reliability of a piece of data.

The article is organised as follows. Section 2 details the basic concepts necessary to understand the method presented. In Section 3, the proposed method is presented. In Section 4, an applied example of the proposal is proposed using a real example in the social network X. In Section 5, we make a comparison of our method with similar methods already published. We will end by showing some conclusions in Section 6.

2. Preliminaries

In this section, the basics necessary to understand the proposed method are presented. In Section 2.1, the basics of group decision-making are presented. In Section 2.2, the sentiment analysis procedure is explained. Finally, in Section 2.3, Z-numbers are presented, which we will use to establish the reliability of the preferences extracted from the discussion.

2.1. Group Decision-Making

In this section we describe the basic concepts of group decision-making processes. In these processes, two or more experts express their preferences over a set of alternatives in order to rank them according to their suitability for solving a given problem. A classic way for experts to provide preferences to the system is through the use of preference relations. A group decision-making problem using preference relations can be defined as follows: Starting from a set of preferences [7], $X=\left\{X_1,X_2,\dots ,X_n\right\}$, and a group of experts $E=\left\{e_1,e_2,\dots ,e_n\right\}$, each expert, $e_i$, provide their preferences over X using a preference relation, $P_{e_i}\subset X\times X$, where ${\mu }_{P_{e_i}}\left(x_j,x_k\right)=p_i^{jk}$ is the preference degree of alternative $x_j$ over $x_k$ for $e_i$ [8]. A group decision-making problem consists on sorting X by using the preference values $P_k,\forall k\in \left[1,n\right]$, that experts have provided to the system.

Within group decision-making processes, two processes are of great importance: consensus and selection. The consensus process allows the experts to reach a final solution that is relatively agreed upon by all, i.e., that has an adequate level of agreement among all experts. The use of consensus measures turns the group decision-making process into a dynamic and iterative process, which may be composed of several rounds where experts express themselves, discuss the different alternatives and try to bring positions closer together. The selection process, on the other hand, uses all individual preferences to obtain a final ranking of alternatives. The alternative that comes first is the most appropriate solution to the problem.

Generally, to solve the formulated problem, most of the methods in the literature follow these steps [2]:

• Providing preferences: First, experts discuss the available alternatives. Next, the experts are asked to provide their preferences [9].
• Aggregating the information: All expert preferences are aggregated into a single piece of collective preference information representing the overall expert opinion [10]. There are a multitude of techniques such as, for example, additive weighted aggregation (AWA) [11]. This operator allows us to derive the ranking of the given alternatives in order to find the most desirable one. Another example is the ordered weighted average (OWA) family of operators [12].
• Calculating consensus: Calculating consensus measures can help us to know whether the experts have reached an agreement or, on the contrary, whether they need to discuss the issue further to reach a reasonable level of consensus. It is important, for the sake of improving the reliability of decision-making results, to allow experts to conduct a thorough discussion before reaching a final decision. Therefore, if the consensus is lower than desired, the experts may be asked to conduct a further round of discussion to bring them closer together. Generally, a maximum number of rounds is also established in case the experts do not reach consensus within a reasonable period of time [13].
• Calculating alternatives ranking: Using the collective preference matrix as a basis, the alternatives are ranked according to the level of preference provided by the experts [10]. This step returns a ranking of alternatives ordered according to the experts’ preferences.

In Figure 1, we can see a graphical representation of the defined process. Group decision-making processes have a strong presence in the current scientific literature. For example, in [14], a consensus process for large-scale group decision-making is demonstrated, relying on limited trust and social networks. In [15], a new model of expressed and private opinions is developed in a scenario where individual private views are updated by reasoning based on the neighbors’ private opinions derived from their actions, while the expressed opinions are influenced by social pressure. In [16], a consensus cost-oriented method for group decision-making is created, which utilises incomplete probabilistic linguistic preference relations. In [17], a group decision-making method driven by dual consistency relying on fuzzy preference relations among individuals and groups is introduced. In [18], the authors present a new decision-making method where sentiment analysis is used for the detection of aggressive behaviour. Finally, in [19], the authors present a new decision-making method that aims to determine the best hotel selection based on user reviews.

2.2. Sentiment Analysis

Traditionally, the most common way to reveal emotions to others was through speech and facial expressions. But with the advancement of technology and social media, people express their emotions through text messages and emoticons on social media. Although the advancement of technology allows users of social media platforms to show their emotions through multimedia content, text is still the most common form used for communication. People profess their emotions through their messages on them (e.g., statuses, comments, blogs, microblogs, …) [20].

Sentiment analysis methods [21] are concerned with identifying how feelings are expressed in texts. We can identify all kinds of concrete feelings: anger, surprise, sadness, joy, etc. We can also make a more general analysis. For example, if in a text we look for expressions that indicate positive (favourable) or negative (unfavourable) reactions towards the subject, we can study the positivity and negativity associated with the subject that the author is dealing with. Specifically, sentiment analysis involves the identification of the following:

• Sentiment expressions.
• Expressions polarity and strength.
• Topic relation.

Polarity refers to the overall tone of the expression, that is, positive, negative or neutral. Thanks to the polarity analysis of the expression, it is possible to know what is the opinion and sentiment that the writer has on the commented topic. Concretely, it is possible to know how the topic make them feel. The strength of the sentiment allows us to know how much importance the writer attaches to the subject matter.

Thanks to this analysis, we can extract information from free text. For example, we can analyse the opinion of users of online documents such as web pages, chat rooms and news articles, instead of doing special surveys with questionnaires that make the user have to provide information explicitly. The result of the sentiment analysis process is a confidence value and a label, such as positive, negative, mixed, neutral or unknown [22]. Sentiment analysis methods provide a degree of occurrence of the sentiment being analysed, which allows us to identify what the author’s goal or sentiment was when writing the text.

In general, sentiment analysis has been investigated mainly at three levels: at document level, at sentence level and at aspect and entity level. Depending on the context and the topic under discussion, we can distinguish between two types of sentences [3]:

• Descriptive sentences: In this case, the user is describing a certain topic or element. Therefore, we can apply sentiment analysis to identify a sentiment or several sentiments based on what is stated.
• Comparative sentences: This type of sentence compares several elements with each other. In this case, sentiment analysis must be able to identify the elements to be compared and look for comparative expressions that relate them. In this way, it is possible to identify which elements the user prefers.

Some of the most frequently applied techniques in the area are as follows [23]:

• Sentiment classification: This primarily involves categorizing opinions into three key groups: positive, negative or neutral. The result of many methods is typically a value that falls within a specific range. Various scales can be utilised to assess an opinion, such as the interval [−1, 1], with −1 representing the highest level of negativity and 1 denoting the highest level of positivity.
• Subjectivity classification: This mainly consists of detecting whether a given sentence is subjective or not. The higher the percentage of positive and negative words used in the text compared to the total number of words, the higher the degree of subjectivity in the text.
• Summary of opinions: It is especially focused on extracting the main features of a given topic from a set of texts. It also identifies the feelings associated with such texts.
• Opinion mining: Efforts to obtain documents that convey a viewpoint on a specified query. In this kind of system, two evaluations are necessary for every document: the relevance to the query and the perspective on the query. Both are typically employed to assess the ranking of documents.
• Sarcasm and irony: It focuses on detecting ironic and sarcastic content on the texts. There is great difficulty in defining reliable techniques that allow the computer to detect them [24].

The following is a typical procedure for sentiment analysis in social networks [2]:

• We select the target feeling; for example, satisfaction, adoration, sadness.
• We generate the list of words associated to the sentiment: $lw=\left\{w_1,\dots ,w_n\right\}$.
• We extract the texts to be analysed from the social networks. To select the texts, we can use keywords. In this way, we can identify if a text is relevant to the topic or if, on the contrary, it is irrelevant and not worth analysing.
• The obtained texts, $T=\left\{t_1,\dots ,t_m\right\}$, are analysed.
• Final results are shown. It is important to notice that each text can be related with more than one sentiment.

The application of sentiment analysis has generated very good results in fields such as marketing, psychology and information extraction. For example, in [25], the authors present a new method based on sentiment analysis that allows measuring people’s perceptions by combining Neurolinguistic Programming (NLP) dictionaries together with machine learning. In [26], the authors apply a business context-aware decision-making approach to select the most appropriate sentiment analysis technique in e-marketing situations. In [27], the authors identify the different sentiment components that are vital to extract topics, sentiments and opinions from the analysed text. In [28], they develop a transformer architecture for sentiment anlysis and clasiffications of hotels opinions. In [29], authors present a review on prompt-based sentiment analysis methods. In [30], they present a novel sentiment analysis approach that uses knowledge-aware dependency graph networks.

2.3. Z-Numbers

Much of the decisions that humans make are based on imprecise information or on the experience of people who express themselves using inexact concepts rather than numerical data. Conceptual information cannot be directly processed by the computer. Therefore, we need to use information representation models that can represent imprecise information and that, in turn, can be managed by a computational system. One of the methods used to represent imprecise information are the Z-numbers [31].

Formally, Z can be represented as $Z=(A,B)$. The first component, A, is a restriction on the values that a real-valued fuzzy variable can take, X. The second component, B, is a measure of reliability or certainty of the first component. For our method, we decided to use the Z-numbers as they allow us to use the reliability component to represent the level of confidence we have in the preferences extracted from the discussion transcripts.

The presence of Z-numbers in scientific articles has grown exponentially in recent years. For example, in [32], a review of the state of the art on z-numbers in group decision-making methods is presented. In [33], authors develop a preference-based regret three-way decision method on multiple decision information systems that uses fuzzy numbers for information representation. Finally, in [34], authors present a hybrid decision approach based on Z-numbers for evaluating blockchain implementation solutions in the sustainable supply chain.

3. Group Decision-Making with Consensus Based on Subjectivity and Sentiment Analysis

The newly designed group decision-making process and the associated consensus measures presented in this paper are shown in this section. Our method automatically extracts experts’ preferences by analysing the transcripts of the discussion conducted. For this purpose, we use sentiment analysis. Then, using the preferences and the information provided by the sentiment analysis process, we identify the degree of consensus among opinions. We can also measure the degree of subjectivity that experts have when giving their opinions. Finally, Z-numbers allow us to calculate a degree of reliability on the extracted preference value. Thus, if an expert provides both positive and negative expressions about a given alternative, then the degree of reliability will be low. If the expressions associated with an alternative are all positive or negative, then the degree of reliability will be high. The method developed follows these steps:

• Decision-making process initialization: First, it is necessary to define the parameters of the decision-making process to be carried out. To do this, the topic of discussion, the set of possible alternatives, the participants, and the posts to be extracted are defined.
• Selection of the data to be analysed: Next, we extract the data from social network X. We must identify which alternatives each text is referring to. To do this, we can make use of hashtags. Each text has the alternative and the expert it refers to assigned.
• Preference extraction: Using sentiment analysis techniques, we extract the experts’ preferences from the texts. For this, we use two sets of words, one with positive and accepting expressions and the other with negative or rejecting expressions. We also use Z-numbers to obtain a measure of the reliability of the information obtained.
• Consensus calculation: Based on the text provided by the experts, the results of the sentiment analysis process and the preferences obtained, we calculate the consensus reached among the experts. For this purpose, we propose two measures, one that calculates the consensus based on the preferences obtained and another that is based on the subjectivity of the comments provided to the system.
• Final ranking of alternatives calculation: Once the consensus reached is high, we aggregate the preferences and, using the selection measures, the final ranking of the alternatives is calculated.

The proposed method has been implemented in R and tested using real data obtained from the social network X using the X API. X is a very interesting social network for this type of analysis as each post is limited to 140 characters. Thanks to this, users of the network are forced to be specific and concise about their contributions. In Figure 2, we can see an outline of the process that the method carries out. In the following subsections, we will describe each of the mentioned steps in detail.

3.1. Decision-Making Process Initialization

First of all, we must define the decision environment in which we are going to work. To do so, we will follow these steps:

• Defining the topics for discussion: The first thing to do is to define the use of a mechanism to obtain the experts’ texts from the social network. The social network X uses hashtags to identify different topics and elements to discuss. It is very common for users to engage in conversations on a particular topic identified by a particular hashtag. Each hashtag can also represent an alternative. In Figure 3 we can see some selected texts by the hashtag #ClaroEcua.

  $$ht=\left\{h{t}_1,\dots ,h{t}_n\right\}$$

  (1)
• Set the number of participants: The number of participants in a given social media discussion can be fixed or open. The more experts participate in a discussion, the more points of view it will have and, therefore, the more interesting the result will be. In the developed model, we do not limit the number of participants. Once the debate is over, a list of hashtags is made with the information they used, which will be original and unique for each of the alternatives to be analysed. We define the set of experts as $E=e_1,\dots ,e_n$.
  It is possible to assign different weights to the experts, depending on their degree of knowledge or experience on the topic. In our proposed model, we adopt the same weight for all experts.
• Establish the number of posts: The number of posts obtained depends on the input provided by experts on the proposed topics. They are collected for further analysis and use in the decision-making process.

  $$Nt=\left\{N{t}_0,\dots ,N{t}_n\right\}$$

  (2)

3.2. Selection of the Data to Be Analysed

A post is a complex object (see Figure 4 and Figure 5), with a lot of associated properties [35] such as Text, Username, Mention, Replies, Follower, RePost, Hashtag and Privacy. For the analysis of the proposed group decision-making process, we focused on the written text, as this is where the user presented the arguments for or against the different alternatives.

For the implementation and testing of the proposed method, we have used the R Software and the X social network API. The X API allows us to obtain the posts published by users and their corresponding metadata [35,36]. This API facilitates the extraction of data in a structured format that can be easily analysed. The R software contains a library that allows us to extract information from the X API and carry out the analysis process.

Before being able to extract preferences and carry out the group decision-making process with consensus, it is important to debug the text to remove all those elements that may hinder the analysis. Therefore we have to remove hashtags, connectors, conjunctions, words without semantic relevance, hyperlinks, the X(@) handle and URLs from the tags. Once this process is finished, the text is reduced to the most relevant set of words. This will make the extraction of preferences more efficient and reliable, as there is less text to analyse and the analysed text is relevant.

As we have already mentioned, hashtags allow us to identify which are the alternatives that experts are describing in their posts. Moreover, as each post is assigned to a user account, we can determine which post belongs to which user. The X API gives us access to this information in an organised way through a set of queries to a database.

Once the texts assigned to each alternative and expert have been obtained, in the next section we will see how to extract the preferences for subsequent use in the group decision-making process and the calculation of consensus and subjectivity.

3.3. Preference Extraction

After initialising the decision-making process and obtaining the data, our method must apply sentiment analysis to obtain the experts’ preferences for each of the alternatives. The reliability of the obtained preferences will also be calculated. This whole process is carried out through the following steps:

• Obtain the sentiment score: Once we have selected the relevant texts from each of the experts, sentiment analysis must be applied for each of them. To do this, we will use functions from the R Syuzhet package [37]. This package provides four sentiment dictionaries and a method to access the robust sentiment extraction tool developed in the Stanford NLP group. In addition, it allows the use of a dictionary that associates words with ten different emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, trust, negative and positive) (see Figure 6). For our research we will only consider positive and negative, since our aim is to determine whether the expert is for or against each of the alternatives.
• Obtain the polarity of the post: In this step, the method obtains the polarity of a post $tw$, i.e., the positive or negative degree $S_{tw}$ obtained from expert survey from the alternative of $tw$ sensitive. The following expression is used:

  $$S_{tw}={\displaystyle \frac{N_{ptw}-N_{ntw}}{N_{ptw}+N_{ntw}}}$$

  (3)

  where $N_{ptw}$ refers to the number of positive words contained in $tw$ and $N_{ntw}$ the number of negative words.
  The result of this process is a numerical value in the range [−1,1]. The closer the result is to the value 1, the more evident the positivity of the post will be. Conversely, the closer the result is to −1, the greater the disagreement with the alternative described. Finally, all those values close to 0 would imply that the expert has a neutral attitude towards the alternative, neither in favour nor against it. In Figure 7, a post analysis example result is shown graphically. The score for each post, as mentioned above, is in the range of [−1,1], where 1 represents very positive polarity and −1 very negative.
  Once the preference has been extracted, we change the domain of the value from the interval [−1,1] to the interval [0,1]. To do this, we apply the following expression:

  $$P_{tw}={\displaystyle \frac{S_{tw}+1}{2}}$$

  (4)
• Calculating the reliability of preferences: Once the preference value has been obtained, we can calculate its associated reliability. For this, as we have already mentioned, we will analyse the number of positive and negative words associated with each alternative. Therefore, if the alternative has many more positive words than negative words associated with it or vice versa, this will mean that the reliability is high. Conversely, if the expert rates the alternative with a similar amount of positivity and negativity, then the expert’s opinion regarding the alternative is unclear. Once the preference and its associated reliability value are obtained, we can create the Z-number representing the preference. For a clearer and more user-friendly representation of preferences that allows a more convenient analysis of expert opinion, we will use linguistic labels to represent each calculated preference value. We will associate each preference value with a linguistic label using the conversion described in the

  Table 1. Linguistic variable definition.

  Value			Linguistic Label
  0.75	1.00	1.00	Very high
  0.50	0.75	1.00	High
  0.25	0.50	0.75	Medium
  0.00	0.25	0.50	Low
  0.00	0.00	0.25	Very Low

  . Once the linguistic label representing the preference value has been obtained, the associated reliability value is calculated. To accomplish this, we use the following expression:

  $$Z=\left|{\displaystyle \frac{N_{ptw}-N_{ntw}}{N_{ttw}}}\right|$$

  (5)

  where $N_{ptw}$ is the number of positive words, $N_{ntw}$ the number of negative words and $N_{ttw}=N_{ptw}+N_{ntw}$.

The more extreme, i.e., the closer it is to 1/−1, either positive or negative, the more polarised the expression is and, therefore, the clearer the expert’s positivity and negativity. Conversely, the less difference there is between positive and negative words, the less reliable the preference is and the less clear the expert’s opinion of the alternative is.

3.4. Group Decision-Making with Consensus Measures

Once the preferences have been calculated from the transcripts of the discussion text, the group decision-making process with consensus can begin. To do this, the method will follow the steps described below:

• Preferences calculation. In the previous section, we used sentiment analysis on the debate transcripts to calculate the preferences of each expert on each alternative. To do this, we created sets of texts for each alternative and expert and extracted the preferences using a sentiment analysis process. The great advantage of this process is that the experts do not have to explicitly provide the preferences to the system. In the following, we will use this information to initiate a decision-making process. We will represent the expert’s preference $e_i$ over the alternative $x_j$ as $p_{ij}$. $p_{ij}$ is a value located in the interval [0,1], with 0 indicating a total disagreement with the alternative and 1 a total agreement with it.
• Consensus calculation. Once the alternatives have been defined, it is possible to calculate the consensus associated with the round of the decision-making process. To do this, we will measure the closeness of the experts’ opinions to obtain a degree of consensus. As we know, in practice, it is sometimes not possible to reach a full and unanimous consensus, but it is important to try to get experts to debate and bring positions closer together. The goal of consensus measures is to help decision-makers reach an agreement on the best solution to a decision-making problem. Therefore their role is increasingly important in solving group decision-making problems. In this article, we propose two different measures of consensus. One measures the closeness of opinions and the other compares experts according to the subjectivity of their opinions. To calculate consensus based on opinion, we use the following expression:

  $$\widetilde{CA}={\displaystyle \frac{{\sum }_{j}CA}{n}}$$

  (6)

  where:

  $$C{A}_j={\displaystyle \frac{{\sum }_i{p}_{ij}}{m}}$$

  (7)

  where n is the total number of alternatives and m is the number of experts.
  For the calculation of the consensus based on the subjectivity of their opinions, we can use the following expression:

  $$P_{+ei}={\displaystyle \frac{N_{ptw}}{N_{ttw}}}$$

  (8)

  $$P_{-ei}={\displaystyle \frac{N_{ntw}}{N_{ttw}}}$$

  (9)
  Expression (8) is for positive words and Expression (9) is for negative ones. The measure of subjectivity in the debate is a key detail that can help us determine which experts show a more vehement attitude in the discussion and which, on the contrary, are more passive or indifferent to the subject under debate.
  Once the consensus has been calculated, the following two actions can be carried out:
  • Invite those experts who have a view that is far from the rest to join the consensus.
  • Try to motivate experts who are more passive in the debate to be more participatory.
  Thanks to these two actions, we try to involve all experts as much as possible in the debate. It is very important that they try to discuss and bring positions closer together in order for the final decision to be as consensual as possible. It is also important that all experts are involved in reaching it. To motivate experts to try to reach consensus, it is possible to set a threshold above which, if not exceeded, experts are allowed to conduct a longer discussion. Our method allows thresholds to be set for both measures. However, for the sake of brevity, we will set only a consensus threshold of $\gamma$ for the opinion-based consensus measure.
• Calculation of the collective preference matrix and final ranking. The preference values calculated for each of the experts are aggregated to calculate the collective preference value specific to each alternative. In this way, we can determine the overall expert opinion. Formally, we aggregate all the $p_{ij}$ values using Expression (7).
  As the values of $p_{ij}$ lie between [0,1], the resulting collective preference values also lie in the same interval. With the $C{A}_i$ values it is possible to score each of the alternatives, order them and generate the ranking that allows us to determine which solution are the best. It is important to remember that for our study, all experts are considered to be of equal importance in the analysis process.

A complete description of the presented method is shown in Algorithm 1.

Algorithm 1 Sentiment-based GDM with Consensus and Reliability Measures (SGDM-CR)

Require: Alternatives A, Posts T, Sentiment Dictionaries D, Threshold $\gamma$ Ensure: Final ranking, Consensus level, Expert feedback   1: Initialise decision-making process (hashtags, experts, post collection)   2: Clean and preprocess posts (remove hashtags, links, stopwords)   3: for each post $t\in T$ do   4:     Identify associated alternative $a_t$ and expert $e_t$   5:     Compute sentiment score: $S_t={\displaystyle \frac{N_{pos}\left(t\right)-N_{neg}\left(t\right)}{N_{pos}\left(t\right)+N_{neg}\left(t\right)}}$   6:     Normalise polarity: $P_t={\displaystyle \frac{S_t+1}{2}}$   7:     Compute reliability: $Z_t=\left|{\displaystyle \frac{N_{pos}\left(t\right)-N_{neg}\left(t\right)}{N_{total}\left(t\right)}}\right|$   8: end for   9: Aggregate $P_t$ values to get $P_{e,a}$ 10: Build preference matrix P and reliability matrix Z 11: Compute consensus $C_A={\displaystyle \frac{1}{n}}{\sum }_{j=1}^n\left({\displaystyle \frac{1}{m}}{\sum }_{i=1}^m{p}_{ij}\right)$ 12: Compute subjectivity per expert: $Sub{j}_e={\displaystyle \frac{N_{pos}\left(e\right)+N_{neg}\left(e\right)}{N_{total}\left(e\right)}}$ 13: if $C_A\ge \gamma$ then 14:     Compute collective preference $C_i={\displaystyle \frac{1}{m}}{\sum }_{e=1}^m{p}_{ei}$ 15:     Rank alternatives by descending $C_i$ 16: else 17:     Provide expert feedback and restart from preprocessing 18: end if 19: return Ranking, $C_A$, subjectivity and reliability feedback

4. Illustrative Example

To show how the method described above works, an example of application is presented in this section. The Prefecture of the Province of Cotopaxi has the objective of improving the standard of living of the inhabitants. To this end, it convenes 5 experts ($e_1,\dots ,e_5$) to determine the best investment in the following:

• Water treatment plant.
• Improve electrical installations.
• Extend Internet coverage.
• Repair the road axis.

The proposed model is applied and the threshold of $\gamma =0.75$ is established as the level of consensus for the value of the preferences. In the following subsections we will develop how our method would solve this decision-making problem.

4.1. Group Decision-Making Process: Initialisation

• Defining the issues for discussion and identifying alternatives: The following topics are identified using hashtags:

  $$ht=\left\{\#WaterEc,\#EnergyEc,\#InternetEc,\#TracksEc\right\}$$
• Establishing the number of experts participating in the process: Five experts are identified, $E=\left\{e_1,\dots ,e_5\right\}$.
• Establishing the number of posts to analyse: The number of short messages is not limited, therefore, the experts are free to express themselves in as many posts as they see fit. To exemplify the model, in this specific case, 133 posts were obtained ($Nt\left(max\right)=133$). A more detailed record can be seen in

  Table 2. Hashtags and number of posts retrieved for each alternative.

  Investment Alternatives	Hashtag	Posts
  Water treatment plant	#WaterEc	40
  Improve electrical facilities	#EnergyEc	30
  Expanding Internet coverage	#InternetEc	38
  Repair the road	#TracksEc	25

  . Each expert has the possibility to express themselves freely, the only restriction is that they must know the hashtags that will be used during the process.

4.2. Data Collection

To obtain the dataset, we used the advantages of the API of the social network X. It allows us to obtain the posts published by the experts and their corresponding metadata.

The credentials (Consumer key, Consumer secret, Access Token and Access Token Secret) provided by the API for secure access were configured. The library associated with R allows us to obtain the short texts written by the experts. Once the texts are obtained, we can apply sentiment analysis to know the polarity of what they wrote, perform the analysis and obtain the preferences.

4.3. Polarity Analysis of Comments Through Sentiment Analysis

The following steps are performed:

• Obtaining the sentiment score: We obtain the texts from the experts and proceed to classify them according to the feelings found in each of these texts. We will focus on detecting only positive and negative sentiments. For this process the expert’s opinion and the topic (hashtag) to which it refers are used.
• Obtaining the polarity of the post: To calculate the score associated with the sentiment, we identified the sentiments associated with each expert, and alternative hashtags related texts that were previously defined. Then, we classify each of them according to whether their sentiment polarity is positive, negative or neutral. To do so, we apply Expression (3). Obtained results for each expert and alternative can be seen on

  Table 3. Experts’ preferences for the alternative #WaterEc.

  E	pwcountij	nwcountij	csij [−1,1]	Assesment [0,1]
  $e_1$	30	2	0.88	0.94
  $e_2$	15	1	0.88	0.94
  $e_3$	24	2	0.85	0.92
  $e_4$	25	2	0.82	0.91
  $e_5$	15	3	0.67	0.83

  ,

  Table 4. Experts’ preferences for the alternative #EnergyEc.

  E	pwcountij	nwcountij	csij [−1,1]	Assesment [0,1]
  $e_1$	20	4	0.67	0.83
  $e_2$	15	3	0.67	0.83
  $e_3$	12	3	0.60	0.80
  $e_4$	10	2	0.67	0.83
  $e_5$	15	2	0.76	0.88

  ,

  Table 5. Experts’ preferences for the alternative #InternetEc.

  E	pwcountij	nwcountij	csij [−1,1]	Assesment [0,1]
  $e_1$	17	3	0.70	0.85
  $e_2$	16	21	−0.14	0.43
  $e_3$	10	15	−0.20	0.40
  $e_4$	20	7	0.48	0.74
  $e_5$	13	6	0.37	0.68

  and

  Table 6. Experts’ preferences for the alternative #TracksEc.

  E	pwcountij	nwcountij	csij [−1,1]	Assesment [0,1]
  $e_1$	10	3	0.54	0.77
  $e_2$	6	3	0.33	0.67
  $e_3$	8	2	0.60	0.80
  $e_4$	8	2	0.60	0.80
  $e_5$	6	1	0.71	0.86

  . Once the values are caclulated, we apply the conversion expression in order to express all the values in the rank [0,1].
  For $e_1$ and the hashtag #WaterEc, we can see the carried out operations in detail below:

  $$S_{tw[-1,1]}={\displaystyle \frac{N_{ptw}-N_{ntw}}{N_{ptw}+N_{ntw}}}$$

  $$S_{tw[-1,1]}={\displaystyle \frac{30-2}{30+2}}={\displaystyle \frac{28}{30+2}}=0.88$$

  $${\mathrm{CS}}_{ij}^{\left[0,1\right]}={\displaystyle \frac{{\mathrm{CS}}_{ij}^{\left[-1,1\right]}+1}{2}}$$

  $${\mathrm{CS}}_{ij}^{\left[0,1\right]}={\displaystyle \frac{0.88+1}{2}}={\displaystyle \frac{1.88}{2}}=0.94$$

Finally, we will calculate the reliability value for each of the obtained preferences. To do this, we will apply Expression (5). We can see the result of this calculation for all experts and alternatives in Table 3, Table 4, Table 5 and Table 6. For $e_1$ and the alternative #WaterEc, results are shown below:

$$Z=\left|{\displaystyle \frac{N_{ptw}-N_{ntw}}{N_{ttw}}}\right|$$

$$Z=\left|{\displaystyle \frac{30-2}{32}}\right|=0.875$$

Note that the greater the difference between the number of positive or negative words in a text, the greater the reliability of the positivity or negativity associated with the text.

4.4. Group Decision-Making Process

• Preference among alternatives. Once the preferences of each expert for each alternative have been calculated, we aggregate them to obtain the overall level of preference for each of the alternatives. The closer the value is to 1, the more popular that alternative is among the experts. We can see the result of this calculation on

  Table 7. Mean preference for each of the alternatives.

  E	#WaterEc	#EnergyEc	#InternetEc	#TracksEc	Mean
  $e_1$	0.94	0.83	0.85	0.77	0.85
  $e_2$	0.94	0.83	0.43	0.67	0.72
  $e_3$	0.92	0.80	0.40	0.80	0.73
  $e_4$	0.91	0.83	0.74	0.80	0.82
  $e_5$	0.83	0.88	0.68	0.86	0.81
  Mean value	0.91	0.84	0.62	0.78

  .
• Calculation of the degree of subjectivity. The degree of subjectivity in the opinions given in the debate, both positive and negative, can be calculated using Expressions (8) and (9). Subjectivity allows us to extract more information about how the experts are performing in the debate and allows us to go beyond the simple opinion they provide. If we compare, for example, the preference values provided by $e_1$ to the ones provided by $e_2$, it is possible to see, in Table 7, that, in both cases, the obtained value is 0.94. However, when applying the calculation of the subjectivity, which is proposed as a novelty in this paper, of each of them, in relation to positivity, it can be seen that $e_2$ obtained a higher value (9.31%) despite making fewer contributions. Consequently, the implication of $e_2$ in the debate is higher than $e_1$. Moreover, it is possible to conclude that the degree of positivity of $e_2$ is higher than the one of $e_1$. In terms of negativity, the value obtained is similar. In

  Table 8. Calculation of the degree of subjectivity for the alternative #WaterEc.

  #WaterEc	${\mathit{e}}_1$	${\mathit{e}}_2$	${\mathit{e}}_3$	${\mathit{e}}_4$	${\mathit{e}}_5$
  word count	62	26	50	36	30
  Total+	30	15	24	20	15
  Total −	2	1	2	2	3
  %+	48.39%	57.69%	48.00%	55.56%	50.00%
  %−	3.23%	3.85%	4.00%	5.56%	10.00%

  , it is possible to see the subjetivity values for the alternative #WaterEc.
• Consensus. To obtain the consensus associated with the group decision-making process we are carrying out, we can apply Expression (7). The value obtained measures the consensus reached by all experts during the discussion.

  Table 9. Consensus for each pair of experts.

  Experts	${\mathit{e}}_1$	${\mathit{e}}_2$	${\mathit{e}}_3$	${\mathit{e}}_4$	${\mathit{e}}_5$
  $e_1$	-	0.13	0.12	0.03	0.03
  $e_2$	0.13	-	0.01	0.10	0.10
  $e_3$	0.12	0.01	-	0.09	0.08
  $e_4$	0.03	0.10	0.09	-	0.01
  $e_5$	0.03	0.10	0.08	0.01	-

  shows the consensus reached by each pair of experts. In

  Table 10. Consensus for each alternative.

  Experts	#A	#E	#I	#V
  $e_1-e_2$	1.00	1.00	0.58	0.90
  $e_1-e_3$	0.99	0.97	0.55	0.97
  $e_1-e_4$	0.97	1.00	0.89	0.97
  $e_1-e_5$	0.90	0.95	0.83	0.91
  $e_2-e_3$	0.99	0.97	0.97	0.87
  $e_2-e_4$	0.97	1.00	0.69	0.87
  $e_2-e_5$	0.90	0.95	0.75	0.81
  $e_3-e_4$	0.99	0.97	0.66	1.00
  $e_3-e_5$	0.91	0.92	0.72	0.94
  $e_4-e_5$	0.92	0.95	0.94	0.94
  Consensus	0.95	0.97	0.76	0.92

  the calculation of the consensus reached for each alternative can be seen. The higher the value obtained, the greater the agreement reached among all experts on the alternative. The calculation of the overall consensus can be seen below:

  $$\widetilde{CS}={\displaystyle \frac{\sum CS}{n}}$$

  $$\widetilde{CS}={\displaystyle \frac{3.6}{4}}=0.9$$
  As the overall consensus reaches a value of 0.9, the threshold of $\gamma =0.75$ set at the beginning of the decision-making process is exceeded. Therefore, it can be determined that there is consensus among the experts and, consequently, the final ranking of alternatives can be calculated.
• Collective preference matrix and final ranking calculation. To obtain the ranking we first obtain the collective preference matrix. Once obtained, we will use Expression (7) to calculate the final ranking of alternatives. We can see the results of these calculations in

  Table 11. Final ranking of alternatives.

  	#WaterEc	#EnergyEc	#InternetEc	#TracksEc
  Mean values	0.95	0.97	0.76	0.92
  Ranking	Second	First	Fourth	Third

  . Therefore, the alternatives are ranked as shown:

  $$R=\{\#EnergyEC,\#WaterEc,\#TracksEC,\#InternetEC\}$$

5. Discussion and State-of-the-Art Comparison

In this paper, we have presented a new group decision-making method that uses sentiment analysis to automatically extract the system preferences. In order to measure the consensus in the process, we present two different measures. One is based on the extracted preferences and the other is based on the subjectivity that experts show in the discussion. In addition, to measure the reliability of the preferences extracted from the debate, we use Z-numbers. The main advantages provided by the presented method are the following:

• Preference extraction method based on sentiment analysis: Preference information is obtained directly from the texts that experts provide to the system during the discussion. This approach has two advantages. On the one hand, experts do not have to provide preferences to the system. On the other hand, the preferences are extracted using all the information provided by the experts during the discussion as a source. When we ask the expert to provide preferences, in the end, we are working with summarised information. However, sentiment analysis methods allow us to extract the information from the discussion so that everything that has been discussed in the discussion is taken into account.
• Using Z-numbers to measure the reliability of the extracted information: Our paper uses Z-numbers to establish a measure of the reliability of the information provided by the experts in the debate. Thus, if an expert provides both positive and negative expressions about an alternative, then the extracted preference will be unreliable. Conversely, if the expert only uses positive or negative expressions to define an alternative, then the reliability of the extracted preference will be high.
• Consensus-taking methods calculated directly from the text of the debate: Analysis of the debate makes it possible to establish measures of expert participation in the debate. Thus, the proposed method proposes consensus measures that take into account the involvement of the experts in the debate. That is, if an expert is very clear in their opinion and provides a high percentage of positive expressions depending on the text analysed, then they can be considered to be very vehement in their opinion. On the contrary, if this percentage is low, then it can be stated that the expert is not very involved or acts neutral. Our method uses this measure of subjectivity in opinion to create a new measure of consensus which, together with the measure of consensus calculated from preferences, gives us a lot of information about how the debate is developing and what the predominant positions are.
• Calculation of the ranking of alternatives based on the information from the debate: Our method converts the information provided by the experts into preferences and calculates the ranking by using selection methods over the obtained preferences. Thanks to this process, our method is capable of determining which alternatives are preferred by the experts. As the extraction of preferences is automatic, this method works even in decision environments where there is a high number of experts and alternatives. Therefore, it is ideal for application in discussions conducted om social networks.
• Applicable to any field: Our proposed model can extract preferences in any field of application, such as tourism, medicine, politics or any other topic where knowledge of users’ opinions and emotions can inform decisions. Although the method was initially designed for use in social networks, there is no reason why it cannot be applied in other types of environments, such as written transcripts of debates, closed forums or other types of decision support systems.

When there is a large amount of text provided by experts and these texts contain words that allow the positivity or negativity of each expert’s discourse to be established, then the proposed method works adequately. However, there may be cases where some experts have provided very little information to the system or where the texts they share are very neutral. In these cases, it is difficult to automatically extract their opinion. The method will assign that expert a neutral opinion value for the alternative, neither positive nor negative. To solve this, it is possible to ask the expert to provide new information to the system and suggest that they express their opinion more directly. The positivity and negativity of opinions is linked to experts using words collected in the dictionaries used for sentiment detection. There may be cases in which the dictionary used does not include the terminology associated with a particular topic or the information does not match to how a particular expert expresses themself. It should be noted that these cases will be minimal, as these dictionaries contain terms that are commonly used by everyone, regardless of the topic being discussed. However, if this occurs, specific dictionaries with the terms used in the topic being discussed or by the expert should be used. Another possible solution is to expand existing dictionaries with new information. Similarly, if we want to work with texts that are not in English or that use a specific tone, it is necessary to include dictionaries that contain the positive, negative and neutral terms for the required cases.

In recent state of the art on the topic, there are few papers that include consensus in a sentiment-analysis-related group decision-making process. For example, in [38], the authors present a novel group decision-making method that uses a sentiment analysis approach to extract preferences from texts and, afterwards, apply consensus measures over the results. For representing the information, they employ distributed linguistic preference relations (DLPRs) for representing the information once it is extracted from the texts. A sentiment analysis approach using dictionaries is used for extracting the preferences. On [39], authors study the phenomenon of echo chambers in group decision-making problems by applying sentiment analysis and consensus measures. They extract opinions from social media and calculate a sentiment polarity matrix. By the use of Aspect-Based Sentiment analysis (ABSA), they use a polarity analysis to calculate consensus and detect echo chambers. In [18], authors present a novel group decision-making process that uses sentiment analysis for detecting aggressiveness and positivity in experts’ comments. Since the method is centred in dealing with a high amount of information, they use clusters of experts in order to be able to handle it.

Although there are methods like the ones mentioned above that use consensus measures and sentiment analysis in group decision-making methods, they do not include any reliability or subjectivity measures. Therefore, they extract less information from the debate and have less means to improve the decision process performance.

In [40], authors use sentiment analysis to detect neutral opinions and improve sentiment analysis procedures in order to carry out a better automatic preference extraction process. They employ the Induced Ordered Weighted Averaging (IOWA) operator for aggregating the information. In order to calculate consensus, they employ the proximity function to the neutral point. This paper already worked on detecting neutrality and identifies the concepts of subjectivity and reliability. Nevertheless, the paper is centred on detecting and filtering all those neutrality contributions. In our paper, we developed two novel measures in order to detect and work with those two cases separately. Instead of filtering unreliable texts, we give the experts a chance to modify their way of expressing themselves in order to be clearer. In the case of experts being too neutral, we can recommend to them that they be more involved in the process.

In [19], authors resolve the problem of hotel selection by using a hotel database review. They apply sentiment analysis to extract the preferences and group decision-making methods to select the best hotel. The information extracted is represented by using fuzzy preference relation matrices. Also, the consensus measures are applied over the preferences extracted. This paper is centred on solving one specific problem using a specific review database, while our proposed method is prepared to work over social media discussions. Therefore, it is capable of extracting the information in real time from social media and applying the group decision-making process.

In [26], the authors use group decision-making techniques to identify the best sentiment analysis technique to apply in a given situation. They use a hierarchical Analytic Hierarchy Process (AHP) approach with an automatised learning method. Therefore, they do not use sentiment analysis for the same purposes as in our paper. Also, they do not provide consensus measures over the opinions found.

When searching in the literature, we could not find a single article that applies Z-numbers to sentiment-analysis-related group decision-making processes. There are several articles, such as [41,42], that apply Z-numbers to represent preferences in group decision-making methods. However, their reasons for its use differ from the ones proposed in our article. For consensus calculation in [41], they employ an aggregation of preferences based on the extended OWA operator. On the contrary, in [42], they employ the Z-number weighted Bonferroni mean (OZWBM).

In conclusion, the mentioned articles are the closest ones to the research carried out in this article that we could find in research databases. Therefore, we could not find any research that directly deals with the reliability and neutrality of experts’ contributions in group decision-making processes. The goal of this paper, therefore, has been to design novel consensus measures that provide a solution to this issue.

All the methods analysed are designed to be applied in different scenarios and cover different needs, making it difficult to make a fair comparison between them. Depending on the context and needs, some will be more suitable than others. For example, when dealing with large amounts of information, the method in [18] may be the most suitable, because it implements tools that allow for the processing of large amounts of information. When detecting different patterns such as neutrality in opinions or the echo chamber effect, the most appropriate methods to use are [39,40], respectively. If we are interested in carrying out the decision-making process and obtaining measures of subjectivity and reliability in the extraction of comments, our method is the most appropriate, as it includes these measures.

With regard to the efficiency of the methods, it should be noted that the method used to represent preferences has a significant impact, since the simpler the representation of preferences, the faster the final calculation. Also, aggregation operations provide faster results. However, these faster methods end up providing less information about the decision-making process, since the more complex the representation of the information, the more details are represented and the more information we obtain at the end of the process. The use of Z-numbers means that the processes that handle them take longer to process the information. Specifically, it takes twice as long, because two values per preference have to be processed instead of one. However, their use allows the level of reliability in the extraction of the information provided to be identified and stored, which can be very useful when studying the reliability of a preference that has been automatically generated from a text.

Table 12. Group decision-making-related methods analysis.

	Entry Type	Consensus	Key Points	Limitations
[OUR]	Debate transcription from social media.	Based on sentiment analysis similarity.	Based on the subjectivity of language. Uses Z-numbers to represent preferences with a reliability measure based on text polarity. Inclusion of subjectivity and reliability in automatic analysis.	Limited to binary analysis. Cannot detect sarcasm and irony.
[18]	Natural language comments.	Optimisation of consensus between clusters.	Analysis of aggressiveness, non-aggressiveness, positivity and negativity.	Dependency on manual classification.
[19]	Fuzzy preference matrix.	Consensus over preferences with feedback to experts.	Includes weighting operators and subjective weighting.	With large numbers of members, heterogeneous granularity does not matter.
[26]	Internet reviews.		Uses hierarchical AHP with AutoML.	Does not employ group consensus measures.
[39]	Opinions from social media.	Calculated over the aspect sentiment polarity matrix.	Uses Aspect-Based Sentiment Analysis (ABSA). Uses sentiment analysis, polarity analysis and consensus to detect echo chambers.	Does not analyse other subjectivity-related aspects.
[40]	Tweets and reviews.	Uses proximity function (NPF) to the neutral point.	Uses the Induced Ordered Weighted Averaging (IOWA) operator for aggregation.	Needs further work in testing the robustness of the measures.
[41]	Z-numbers.	Uses extended OWA.	Includes information from Z-numbers in calculations. Uses OWA operator for aggregation.	Computational complexity of Z-numbers.
[42]	Z-numbers.	Uses overlap function-based Z-number weighted Bonferroni mean (OZWBM).	With overlapping functions and aggregation using OZWBM.	Functions can be difficult to interpret by experts.

shows a comparison of the characteristics of all the models analysed. Finally, in

Table 13. Methods compared with related articles in the literature.

Method	Calculate Ranking	Consensus	Automatic Preferences Obtention	Reliability	Neutrality
[OUR]	Yes	Yes	Yes	Yes	Yes
[18]	Yes	Yes	Yes	Yes	No
[19]	Yes	Yes	Yes	No	Yes
[26]	Yes	No	Yes	Yes	No
[38]	Yes	Yes	Yes	Yes	No
[39]	No	Yes	Yes	No	Yes
[40]	No	Yes	Yes	No	Yes
[41]	Yes	Yes	No	Yes	Yes
[42]	Yes	Yes	No	Yes	Yes

, the methods analysed in this section are compared based on a series of characteristics.

The presented method has some limitations that we will improve in the future. One of the main objectives of the method presented is to be able to carry out a group decision-making process based on information provided in social networks. For this reason, and given that we are interested in knowing whether a user is for or against an issue, we have opted for a dichotomous analysis of emotions. In this way, we determine whether the user is for or against the alternative being discussed. However, it is possible, in future work, to try to perform a deeper analysis of certain emotions that are common in texts written on social networks. For instance, joy, sadness, anger or surprise could be detected. Another limitation of the method is that it does not detect irony and sarcasm. However, as it is a scalable method, it is possible to include a phase in which an existing sarcasm and irony analyser is applied, for example [43,44], to detect these situations and be able to deal with them appropriately. Finally, in order to carry out an adequate analysis, it is necessary that the texts provided by the users are sufficiently informative, i.e., that they contain enough words considered positive and negative. Otherwise, the method considers the text to be neutral and it becomes very difficult to really understand how the user feels about the topic. In this case, a possible solution is to communicate to the user that it is not possible to extract their preferences and encourage them to provide longer and more informative texts to the system. Without sufficient information about what a user thinks about an alternative, the method cannot perform adequately. It is possible to determine whether a user has provided enough information by setting a minimum number of $N_{ttw}$. In this way, if the user’s texts about the alternative do not reach the required minimum, it is possible to stop the analysis and ask for more information.

6. Conclusions

In this paper, we have designed a new group decision-making method that works directly on the transcript of the experts’ discussion of the system alternatives. Thus, experts do not have to explicitly provide their preferences of alternatives, as these are automatically extracted through a sentiment analysis process.

The proposed method analyses more comprehensively the debate carried out by experts in the decision-making process. Specifically, it is able to calculate the level of subjectivity of the expert’s opinion as well, as to measure the reliability of the preference obtained. For subjectivity, the proportion of words that emphasise agreement or disagreement with one of the alternatives is analysed. For reliability, the ratio of positive and negative descriptors associated with each alternative is studied. Thus, if an alternative has a much higher ratio of positive than negative words associated with it, then the reliability in indicating that the expert likes that alternative is high.

Finally, our decision-making method implements a consensus method that allows the experts conducting the debate to try to bring positions closer together in order to reach a decision that is as consensual as possible.