Assessment-Driven Judgment Aggregation Within a Group of Peers

Abstract

Group decision-making sometimes involves evaluating the decision-makers themselves, e.g., selecting the best expert or assigning rewards within a team. In such cases, all participants should be involved, but their influence should reflect their competence or contribution. This article proposes two new opinion aggregation models in which a person’s assessment weight depends on their ranking, preventing low-performing members from exerting the same influence and promoting respected experts. The proposed aggregation methods uphold the principle of distributive justice by ensuring that individual contributions are proportional to the rewards they receive. In addition to formulating new methods for aggregating results, we presented several of their formal properties and indicated practical ways to calculate the results. For one of the methods, which is more challenging to compute, we conducted a Monte Carlo experiment demonstrating the practical feasibility of computing the aggregated weight vector.

1. Introduction

From time to time, we see discussions about which expert is more competent, which of specialist is more professional, which of specialist of merit to the country is more distinguished, etc. The reason for such discussions is most often the need for an award, a diploma of recognition, or a medal in recognition of service to the country or work for an organization. Sometimes, the award is financial and given to everyone who has earned it. In this case, the issue to be resolved is the award’s distribution among all concerned. A specific group of people with expertise in the subject being evaluated selects the best candidate or decides on the award distribution. Having such expertise, however, means that these individuals may be the subject of the evaluation. A similar situation occurs with the distribution of awards for various other team achievements, such as grants, organizational projects, or software projects in IT companies. The solution proposed in this paper is based on the aggregation procedure used in the group decision-making (GDM) process, which is used in the pairwise comparison method. This approach has also resulted in several studies in group decision-making. Traditionally, the Analytic Hierarchy Process (AHP) builds on two opinion aggregation procedures: aggregation of individual judgments (AIJ) and aggregation of individual priorities (AIP) [1,2,3,4]. In the AIJ procedure, experts first combine their pairwise comparison (PC) matrices into a collective judgment matrix and then derive a group priority vector. In contrast, in the AIP procedure, they first compute individual priority vectors and subsequently aggregate them into a collective priority structure. Section 2.2.3 presents the latter approach in greater detail. The literature widely accepts and applies both procedures for aggregating preferences expressed in the form of PC matrices (see Section 2.1); however, Forman and Peniwati [4] recommend AIJ in situations where experts agree in advance on a common assessment framework, as this agreement facilitates coherent judgment aggregation at the group level. Ramanathan and Ganesh [5] advocate aggregating individual priorities (AIP), showing that this procedure satisfies the Pareto principle when decision-makers derive priorities using the eigenvector method (EVM) introduced by Saaty [1]. In contrast, their analysis demonstrates that aggregation of individual judgments (AIJ) may violate the Pareto principle under the same prioritization scheme, highlighting a fundamental theoretical distinction between the two aggregation approaches and informing the choice of aggregation procedure in AHP-based group decision-making contexts. Both methods are equivalent when using the geometric mean method (GMM) or logarithmic least square method (LLSM) (both GMM and LLSM are equivalent for complete and incomplete PC matrices [6]) for the ranking calculation [7]. In addition to the two methods mentioned above, other methods of aggregating preferences have emerged. Blagojević et al. [8] introduce a GED-based aggregation method for AHP group decision-making, employing simulated annealing to heuristically minimize the distance between individual and group judgments. While effective in handling complex preference structures, the approach is computationally demanding and lacks explicit consensus and consistency in modeling. Dong et al. [9] propose a consensus model for AHP group decision=making in which individual preference matrices are iteratively adjusted to minimize a geometric consensual consistency index (GCCI), showing that this process improves group agreement under the row geometric mean method; nevertheless, the approach is developed for static expert settings and fixed consensus parameters, leaving room for later extensions addressing its adaptability and scalability. Altuzarra et al. propose the Bayesian prioritization procedure (BPP), allowing for the aggregation procedure to be extended to preferences (utilities, priorities) among alternatives, which are presented as probability distributions [10]. Escobar and Moreno-Jimenez [11] propose the Aggregation of Individual Preference Structures (AIPS) method for AHP group decision-making, demonstrating that aggregating individual preference structures can better preserve decision-makers’ preference information; nevertheless, the method does not explicitly incorporate consensus control or a means of inconsistency reduction, which is addressed in later studies. Lin et al. [12] propose an aggregation framework for AHP-based group decision-making in which individual pairwise comparison matrices are first transformed into their nearest consistent matrices and then aggregated under an acceptable-consensus constraint. The study demonstrates that incorporating a consensus adjustment improves group agreement while preserving individual consistency properties; however, the proposed approach assumes fixed consensus thresholds and focuses on static group settings, without addressing computational scalability or the dynamic evolution of opinions, which motivates subsequent research on adaptive, computationally efficient, consensus-driven aggregation methods.

Researchers have extended classical aggregation procedures to decision models that represent preferences using interval-valued judgments [13] and fuzzy sets [14], thereby capturing the uncertainty and imprecision inherent in group assessments. Building on this line of research, Mateos et al. [15] introduce a group decision-making (GDM) methodology for settings with incomplete individual beliefs, in which the voting weight of political representatives reflects the electoral support they receive. This approach illustrates how aggregation mechanisms can incorporate contextual information beyond preference judgments; however, it also raises concerns regarding the strategic manipulation of expert influence. Addressing this issue, Kułakowski et al. [16] propose a resilient heuristic aggregation framework that adaptively adjusts expert weights to mitigate manipulation attempts, thereby enhancing the robustness of opinion aggregation in pairwise comparison-based decision models. Zhao et al. [17] focus on individual participation and satisfaction in group decision-making and explicitly incorporate fairness and tolerance into the consensus-formation process. They propose a consensus model that maximizes aggregated utility expressed through an additive Choquet integral, which captures interactions among decision-makers’ preferences rather than treating them as independent. Their results show that the model can balance collective agreement with individual utility and satisfaction; however, the approach relies on eliciting interaction indices and assumes a static preference structure, which may limit its applicability in large-scale or dynamically evolving decision environments, and motivates further research on scalable, adaptive utility-based consensus models.

Decision-making methods are also used in situations where the interested parties (evaluators) evaluate each other. James et al. [18] address the peer assessment problem as a specific instance of group decision-making and focus on improving the reliability of aggregated evaluations. They propose a moderation-oriented framework that identifies potentially unreliable assessors by combining reliability indices with a consensus analysis of peer scores. Their numerical experiments show that this two-angle approach can effectively flag a subset of peer assessments for expert moderation, thereby increasing scoring accuracy while reducing the overall workload of subject experts. However, the proposed methodology targets static assessment settings. It assumes access to sufficient peer-evaluation data, which limits its applicability in highly dynamic environments or in cases with sparse or strategically biased assessments, thereby motivating further research on adaptive and manipulation-resistant moderation mechanisms within GDM frameworks. Luo et al. [19] investigate peer grading in the context of Massive Open Online Courses (MOOCs) by analyzing 1825 peer-reviewed assignments collected from a Coursera course. The authors assess the reliability and validity of peer grading by comparing aggregated peer scores with instructor evaluations and by examining students’ perceptions of the assessment process. Their empirical results show that student graders’ collective judgments closely match instructor scores, and that most participants perceive peer grading as fair and beneficial, supporting the practical usefulness of peer assessment in large-scale learning environments. Nevertheless, the study focuses on a single MOOC setting. It relies on aggregate agreement measures, leaving open questions about individual grader bias, strategic behavior, and the robustness of peer-grading mechanisms across diverse courses and learner populations. Capuano et al. [20] address the problem of peer assessment in MOOCs by using Fuzzy Ordinal Peer Assessment (FOPA), a group decision-making model designed to handle unreliable and incomplete ordinal evaluations provided by students. The two-step FOPA procedure first completes partial rankings by transforming them into fuzzy preference relations and estimating missing values. Then, it aggregates the enriched preferences using ordered weighted averaging (OWA) operators. Experimental results show that FOPA improves both ranking reconstruction and grade estimation when compared with existing ordinal and cardinal peer assessment methods, thereby demonstrating the practical value of fuzzy aggregation in large-scale educational settings. However, the approach requires eliciting fuzzy intensity judgments and entails additional computational overhead, which may limit its scalability and ease of adoption in very large or time-constrained MOOC environments, and motivates further research on simpler yet robust ordinal aggregation mechanisms. Langan et al. [21] examine evaluation and self-evaluation within the student–teacher relationship by analyzing peer, self, and tutor assessments of oral presentations in higher education. The authors investigate how student characteristics, such as gender and academic attainment, influence assessment outcomes and compare the resulting marks across the three evaluation modes. Their findings show that peer assessments align more closely with tutor evaluations than self-assessments, while self-evaluation exhibits systematic biases, notably gender-related underestimation of performance by female students. The study also reveals contextual effects, including grading compression among peers and variability linked to session structure and student conditions, such as fatigue. Although the empirical analysis provides valuable insights into sources of bias in educational assessment, the work focuses on a specific assessment format and institutional setting, limiting the generalizability of the results and motivating further research on bias-aware, context-adaptive aggregation mechanisms for self- and peer evaluations.

In the above-mentioned works, even when differences in experts’ priorities appear, they result either from external intervention (moderation) [18], from automatic rules aimed at achieving decision consensus [9,12], or from assessments of an expert’s quality based on their past performance [22]. To the best of our knowledge, however, no study has addressed a situation in which an expert’s priority (i.e., an assessment of their quality) is determined simultaneously with their participation in the decision-making process. This gap in the literature is precisely what the present work aims to address.

To address this gap, we introduce two alternative opinion aggregation methods for situations in which evaluators and evaluated individuals coincide. Both methods (additive and multiplicative) make the strength of a given expert’s voice (the weight of their opinion) dependent on the evaluation result they have received. Thus, the more respected a given expert is (and, therefore, the more they deserve the award), the greater the weight assigned to their opinions. Conversely, the less respect a person enjoys within the group, the less influence their opinion has on the final result. A defined additive method always admits a solution that is computable via an iterative procedure. In our work, we indicate the convergence speed of iterations. The additive method also always has a solution. However, it is not always possible to calculate it using the iterative procedure. In this case, the use of optimization methods is more important. Formal analysis proves the existence of a global solution and identifies the condition for its uniqueness. In addition, Monte Carlo experiments demonstrate the effectiveness of the calculations in the multiplicative case. The paper also analyzes the properties of both aggregation functions. One of the motivations for undertaking this topic in the paper was the problem of fairness, often colloquially expressed in the statement “equal does not mean fair.” For this reason, in the conclusion of the paper, we discuss the relationship between the proposed aggregation methods and the equity theory. The presented research will help to create more fair models for aggregating the opinions of experts and decision-makers.

The article consists of eight parts. The Introduction (Section 1) and Preliminaries (Section 2) are intended to familiarize the reader with the subject’s literature and recall the necessary concepts and definitions related to the pairwise comparison method and group decision-making. The third section, “Peer Ranking Aggregation,” describes the problem of a group of experts’ self-assessment, introduces the proposed two methods of aggregating the results, additive and multiplicative, and indicates how to calculate the results. This section also contains theorems concerning the existence of a solution and its uniqueness in the multiplicative problem. Section 4 analyzes the properties of the defined methods. The next section indicates the place of the proposed aggregation methods in the decision-making process. The sixth section contains an analysis based on Montecarlo’s experiments, showing, in practice, the possibility of calculating the result in the multiplicative case. The last two sections of the paper discuss the relationship between the proposed aggregation methods and the issue of equity, and summarize the proposed aggregation functions and the obtained results.

2. Preliminaries

2.1. Pairwise Comparisons

Many methods, including AHP, HRE, MACBETH, and BWM, as well as outranking methods like PROMETHEE [6,23], and various hybrid solutions, use pairwise comparisons (PC) as a source of information about decision-makers’ preferences. The approach has been extended to cover different data representations, including intervals [24], fuzzy sets [14], or gray numbers [25]. The best-known method based on pairwise comparisons of alternatives is AHP. This is also the most criticized, although the criticism is often constructive and has led to many improvements and extensions of the original proposal. Among the more critical areas of improvement are the methods for calculating the vector of weights [26], including ranking incomplete matrices [27,28,29], methods for measuring inconsistencies [30,31], and others. One can find an interesting critical analysis of the AHP method in the work of Munier and Hontoria [32].

In the PC method, experts, also known as decision-makers, compare alternatives in pairs. Let $A=\{a_1,\dots ,a_n\}$ be a finite set of alternatives and $E=\{e_1,\dots ,e_k\}$ a set of experts. Each expert compares the alternatives with each other to form a set of comparisons represented in the form of a square matrix $C_q=\{c_{ijq}\in {\mathbb{R}}_{+}:i,j=1,\dots ,n\}$ where $1\le q\le k$, and q indicates the number of the expert. A single $c_{ijq}$ denotes the relative importance of the i-th alternative compared to the j-th alternative according to the expert’s opinion $e_q$. For the sake of legibility of notation, whenever possible we will omit the expert index q by writing the PC matrix as $C=\left[c_{ij}\right]$. The PC matrix is used to synthesize a vector of weights. Let us define the function resulting from this calculation.

Definition 1.

Let A be a set of alternatives. The priority function for A is the mapping $w:A\to {\mathbb{R}}_{+}$ assigning a real and positive number to each alternative.

The priority vector for A resulting from $C_q$ takes the following form:

$$w_q={\left[w_q\left(a_1\right),\dots ,w_q\left(a_n\right)\right]}^T.$$

(1)

Many methods have been described in the literature to calculate the value of priority vector w. The first one, which is still quite popular, is the eigenvalue method (EVM). Saaty proposed this approach in his seminal paper [1], and it consists of calculating a principal eigenvector and then normalizing it so the sum of entries is one. Another method is based on calculating the geometric means (referred to as the geometric mean method (GMM).) of the matrix rows and taking the result of the i-th row as the priority of the i-th alternative [33]. The result thus obtained is also subject to normalization at the end of the calculation. In addition to the two mentioned methods, several others for calculating the priority vector exist. For example, Srdjevic et al. compare the performance of the six popular priority-deriving methods [34]. Other examples of prioritization methods can be found in [35,36].

The matrices used in the pairwise comparison method have several specific properties. Below, we recall the Frobenius–Perron theorem as it applies to matrices used in opinion aggregation.

Definition 2.

A square matrix A is stochastic if all of its entries are non-negative, and the entries of each column sum to $1$. A matrix is positive if all of its entries are positive numbers [37] (p. 349).

Theorem 1.

Let A be a positive stochastic matrix. Then, A admits a unique normalized steady state vector x, which spans the $1$-eigenspace. Moreover, for any vector $v_0$ with entries summing to some number c, the iterates

$$v_1=A{v}_0,v_2=A{v}_1,\dots ,v_t=A{v}_{t-1},\dots$$

approach $c·x$ as t becomes large [37] (p. 351).

Claim 1.

From the above assertion, the following statements are true [37] (p. 351):

1. 

The $1$-eigenspace of a positive stochastic matrix is a line.

2. 

The $1$-eigenspace contains a vector with positive entries.

3. 

All vectors approach the $1$-eigenspace upon repeated multiplication by A.

In pairwise comparison matrix analysis, an important element is the analysis of matrix inconsistency.

Definition 3.

The pairwise comparisons matrix $C=\left[c_{ij}\right]$ with dimensions $n\times n$ is consistent if for every $1\le i,j,k\le n$ holds $c_{ik}{c}_{kj}=c_{ij}$.

The basic methods for determining the degree of inconsistency among pairwise comparison matrices are inconsistency indices. Many different inconsistency indicators can be found in the literature [30,31]. The most commonly used and earliest introduced is that defined by Saaty [1], as follows:

$$\mathrm{CI}\left(C\right)\stackrel{\mathrm{df}}{=}{\displaystyle \frac{{\lambda }_{\max }-n}{n-1}},$$

(2)

where ${\lambda }_{\max }$ is the spectral radius of a square matrix C with dimensions $n\times n$. For a valid multiplicative pairwise comparison matrix $C=\left[c_{ij}\right]$ with reciprocity, i.e., for every $1\le i,j\le n$ holds $c_{ij}=1/c_{ji}$, it is true that ${\lambda }_{\max }\ge n$. Whereby ${\lambda }_{\max }=n$ when C is consistent, i.e., $\mathrm{CI}\left(C\right)=0$. For the purposes of AHP, Saaty also defined the consistency ratio $\mathrm{CR}\left(C\right)$, understood as the quotient of the inconsistency index of matrix C and the average value of the inconsistency index for a random matrix of pairwise comparisons with the given dimensions and comparison scale. He suggested that matrices for which the value of CR is no greater than $0.1$ (a matrix at least 10 times more consistent than a comparable random matrix) should be treated as sufficiently coherent and the weight vector calculated on this basis is considered credible.

The measurement of inconsistency applies to a single matrix in pairs. In the case of multiple experts and multiple matrices, a different method of determining their concordance is required. One such method is Saaty’s compatibility index [6,38], given as follows:

$$\mathrm{cmp}(C_1,C_2)={\displaystyle \frac{1}{n^2}}{e}^T{C}_1\odot C_2^{T}e={\displaystyle \frac{1}{n^2}}\sum _{i,j=1}^n{c}_{ij}^{\left(1\right)}{c}_{ji}^{\left(2\right)},$$

(3)

where ⊙ is a Hadamard product of $C_1$ and $C_2$. If both matrices are fully compatible with each other, their compatibility index (3) is equal to 1. Other ways of determining and improving the concordance (consensus) of weight vectors from different experts can be found in the literature [39]. These could include, for example, checking the distance of a given expert’s opinion from the average recommendation, combined with the elimination of outliers [16] or calculating the GCCI (Geometric Cardinal Consensus Index) and GOCI (Geometric Ordinal Consensus Index) group incompatibility measures [9], among others.

2.2. Group Decision-Making

2.2.1. Aggregation Procedures

When more than one expert participates in decision-making, their opinions must be aggregated. As with a single PC matrix, the final result should be a vector of weights assigning real numbers to each alternative. The aggregation of expert opinions can be achieved both at the level of individual comparisons and at the level of individual weight vectors. An example of the first approach is the AIJ (aggregating of individual judgments) [4] procedure, according to which each comparison of the i-th and j-th alternatives by each of k experts is averaged, and then a vector of weights is calculated for the matrix composed of the averaged values. In addition to AIJ, several optimization methods directly use expert comparison results. The previously mentioned model, based on minimization of group Euclidean distance (GED) values [8], is an example. A different approach is the AIP procedure, in which priority vectors for individual experts are calculated first, and only then are the results aggregated. Compared to AIJ, the AIP approach has several significant advantages. One undoubted advantage of the AIP approach is that both geometric and arithmetic averages can be used to aggregate preferences [4]. Moreover, the second method does not impose the need for a group of experts to act as a unit, leaving them more freedom to make individual judgments [4]. It can also be easily implemented in the case of incomplete data from different experts. In such cases, it is sufficient to use appropriate prioritization methods for incomplete PC matrices [27,29] to calculate individual weight vectors. The weight vectors obtained in this way may then be subjected to aggregation. In addition, AIP satisfies the Pareto principle with an arithmetic or a geometric mean.

Aggregation procedures allow each expert’s opinion to be assigned a different strength (priorities). Thus, experts considered less competent can have less influence on the final opinion, while more recognized experts can significantly impact the final outcome. In the following two sections, we will look at two variants of the AIP method that allow for different priorities to be assigned to individual experts.

2.2.2. Additive Aggregation of Individual Priorities

Let the vector of weights calculated from the pairwise comparison matrix $C_q$ provided by the q-th expert be denoted as follows:

$$w_q=\left(\begin{array}{c}{w}_q\left(a_1\right)\\ w_q\left(a_2\right)\\ ⋮\\ w_q\left(a_n\right)\end{array}\right),\mathrm{for}q=1,\dots ,k.$$

(4)

Then, let W be a matrix of weight vectors so that

$$W=\left(w_1,\dots ,w_k\right),$$

i.e.,

$$W=\left(\begin{array}{ccccc}{w}_1\left(a_1\right)& w_2\left(a_1\right)& \dots & \dots & w_k\left(a_1\right)\\ w_1\left(a_2\right)& w_2\left(a_2\right)& \dots & \dots & w_k\left(a_2\right)\\ w_1\left(a_3\right)& ⋮& \ddots & ⋮& ⋮\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ w_1\left(a_n\right)& w_2\left(a_n\right)& \dots & \dots & w_k\left(a_n\right)\end{array}\right),$$

(5)

and let p be the vector of priorities for individual experts, i.e.,

$$p=\left(\begin{array}{c}{p}_1\\ p_2\\ ⋮\\ p_k\end{array}\right),$$

(6)

so that ${\sum }_{i=1}^k{p}_i=1$. Then,

$$\mathrm{AAIP}(W,p)\stackrel{\mathit{df}}{=}\left(\begin{array}{c}\sum _{i=1}^k{p}_i{w}_i\left(a_1\right)\\ \sum _{i=1}^k{p}_i{w}_i\left(a_2\right)\\ ⋮\\ ⋮\\ \sum _{i=1}^k{p}_i{w}_i\left(a_n\right)\end{array}\right)$$

(7)

denotes the AIP procedure using a weighted arithmetic mean. To simplify the notation, we will omit the vector p when the priorities of the experts are equal, i.e.,

$$\mathrm{AAIP}\left(W\right)\stackrel{\mathit{df}}{=}\mathrm{AAIP}\left(W,{\left({\displaystyle \frac{1}{n}},\dots ,{\displaystyle \frac{1}{n}}\right)}^T\right).$$

2.2.3. Multiplicative Aggregation of Individual Priorities

Let the priority vector provided by the q-th expert be $w_q$ (4), the matrix of priority vectors be W (5), and the vector of expert priorities be p (6), so that ${\sum }_{i=1}^k{p}_i=1$. Then,

$$\mathrm{MAIP}(W,p)\stackrel{\mathit{df}}{=}\left(\begin{array}{c}\prod _{i=1}^k{w}_i^{p_i}\left(a_1\right)\\ \prod _{i=1}^k{w}_i^{p_i}\left(a_2\right)\\ ⋮\\ ⋮\\ \prod _{i=1}^k{w}_i^{p_i}\left(a_n\right)\end{array}\right)$$

(8)

denotes the AIP procedure using a weighted geometric mean. As before, we will drop the vector p in a situation of equal priorities, i.e.,

$$\mathrm{MAIP}\left(W\right)\stackrel{\mathit{df}}{=}\mathrm{MAIP}\left(W,{\left({\displaystyle \frac{1}{n}},\dots ,{\displaystyle \frac{1}{n}}\right)}^T\right).$$

3. Peer Ranking Aggregation

3.1. Problem Description

Sometimes, the expert team members have to perform a peer self-assessment. This may be related, for example, to selecting the best expert, distributing a reward for achieving an intended goal, or creating a ranking of experts for any other purpose. The evaluation criterion can be professionalism, commitment to the team, or simply appreciation of others. In such a situation, team members, on the one hand, become the subjects of evaluation themselves; on the other hand, they have the knowledge and appropriate competence to participate as experts (decision-makers) in the review. However, while their participation as experts in the decision-making process seems necessary, the weight of their opinions should be tied to the assessment they received. For example, an expert who is recognized by their colleagues should have a greater influence on the team’s work outcomes than those rated lower. We may encounter such a situation when the team is small, and there is no possibility of selecting or naming an impartial person to evaluate team members. Such a team may, for example, be a small group of collaborating scientists preparing a joint publication, a team of peer programmers developing a joint application, an autonomous task force in an organization that needs to distribute bonuses, etc. To handle such situations, we propose two methods of aggregating expert evaluations. In our proposal, the weight of a given expert’s voice depends on the ranking score they receive.

3.2. Assessment-Driven Aggregation Model (ADAM)

The above considerations led us to propose a modified peer-review model based on pairwise comparisons of alternatives. In the solution presented, group members have a dual role. They are simultaneously the object of comparison, i.e., the alternatives $A=\{a_1,\dots ,a_n\}$ in the PC method, and the experts providing their judgments, i.e., $E=\{e_1,\dots ,e_n\}$. Each of the experts, i.e., team members, provides comparisons in the form of a PC matrix $C=\left[c_{ij}\right]$, where $c_{ij}\in {\mathbb{R}}_{+}$, in which they assess the relative level of recognition of the other members of the group. Based on the collected ratings $C_1$,…,$C_n$, vectors of weights $w_1,\dots ,w_n$ are created for individual experts. The calculated vectors are then subjected to aggregation, resulting in the formation of a final priority vector:

$$w=\left(\begin{array}{c}w\left(a_1\right)\\ ⋮\\ ⋮\\ w\left(a_n\right)\end{array}\right)$$

reflecting the share of each person evaluated for the award. However, in line with the observation that people who are more respected are often more competent judges of others, the priorities given to expert opinions during aggregation may differ to reflect this regularity. Therefore, let $p:A\to \mathbb{R}$ be an expert priority function such that, for two experts $e_i$ and $e_j$, if their final evaluation share meets $w\left(a_i\right)\ge w\left(a_j\right)$, then their priorities during the aggregation process also meet $p\left(a_i\right)\ge p\left(a_j\right)$. This regularity follows from the observation that individuals held in higher esteem by their peers have a correspondingly stronger claim to influence the distribution of the reward. Accordingly, we adopt the identity mapping $p\left(a_i\right)=w\left(a_i\right)$, which assigns decision-making power is directly in proportion to the level of recognition expressed in the assessment process. This choice preserves the proportional relationship between recognition and influence without introducing additional normative assumptions or distortions, ensures that aggregation remains strictly assessment-driven, and offers a transparent and easily interpretable mechanism. For these reasons, the identity function constitutes not only the simplest but also, in the considered context, the most equitable specification of p. Below, following [4], we propose two assessment-driven aggregation methods to support the proposed ADAM model: multiplicative and additive.

3.3. Additive Assessment-Driven Aggregation Method (AADAM)

When proceeding to aggregate the results, we assume that each team member $e_q\in E$ provided a PC matrix $C_q$, based on which the ranking vector $w_q={\left(w_q\left(a_1\right),\dots ,w_q\left(a_n\right)\right)}^T$ was calculated. These vectors form a matrix W (see (5)). Denoting the vector of experts’ priorities as $p={\left(p\left(a_1\right),\dots ,p\left(a_n\right)\right)}^T$ we obtain an extra condition that the final solution must satisfy:

$$\mathrm{AAIP}(W,{\displaystyle \frac{p}{{∥p∥}_1}})=\left(\begin{array}{c}p\left(a_1\right)\\ p\left(a_2\right)\\ ⋮\\ ⋮\\ p\left(a_n\right)\end{array}\right),$$

(9)

where $p\left(a_i\right)>0$ for $i=1,\dots ,n$ and for any $u\in {\mathbb{R}}^n$ it holds that ${∥u∥}_1\stackrel{\mathit{df}}{=}{\sum }_i\left|u_i\right|$. The above condition (9) states that the ranking values of the individual alternatives/experts (i.e., the right side of the equation) are equal to the priorities (after normalization) with which the opinions of the individual experts were considered (the left side of the equation). The Equation (9) is

$$W{\displaystyle \frac{p}{{∥p∥}_1}}=p.$$

(10)

Since W is a stochastic matrix and irreducible and aperiodic, i.e., $W>0$, ${\sum }_{j=1}^n{w}_i\left(a_j\right)=1$ and $w_i\left(a_i\right)>0$ for $i=1,\dots ,n$; thus, due to the Perron–Frobenus theorem (see Theorem 1) there is a unique and positive eigenvector $p>0$ with the eigenvalue 1 whose entries add up to one. Thus, (10) takes the form

$$Wp=p.$$

Moreover, the vector p can be calculated iteratively as the limit of the iteration procedure $W(\dots W\left(W{p}_0\right)\dots )\to p$ where entries of the initial vector $p_0$ have to sum up to 1. Using properties of W, we can easily estimate the convergence speed of an iterative method.

Property 1.

Since $W>0$, the matrix is primitive, so its principal eigenvalue is ${\lambda }_1=1$ and all other eigenvalues satisfy $|{\lambda }_i|<1$. Let $\rho =|{\lambda }_2|<1$ be the second largest eigenvalue (SLEM). Then, $\parallel p^{\left(t\right)}-p\parallel \le O\left({\left|\rho \right|}^t\right)$ [40] (p. 331). Hence, the number of iterations needed to achieve accuracy ε (e.g., $\parallel p^{\left(t\right)}-p\parallel \le \epsilon$) is in the order of $t=O\left({\displaystyle \frac{log(1/\epsilon )}{-log\rho }}\right)$, and for $\rho \approx 1$ the number is $t\approx O\left({\displaystyle \frac{log(1/\epsilon )}{1-\rho }}\right)$.

It is worth noting that eigenvector calculation algorithms W, as well as global optimization methods, can also be used to calculate the solution. For example, condition (9) induces the following optimization problem:

$$\begin{array}{c}\hfill minh(W,r)\mathrm{s}.\mathrm{t}.\\ \hfill h(W,r)=\sum _{i=1}^n{\left(\sum _{j=1}^{n}r\left(a_j\right)w_j\left(a_i\right)-p\left(a_i\right)\right)}^2,\\ \hfill r\left(a_i\right)={\displaystyle \frac{p\left(a_i\right)}{{\sum }_{j=1}^{n}p\left(a_j\right)}}\mathrm{and}r\left(a_i\right)>0\mathrm{for}i=1,\dots ,n.\end{array}$$

(11)

whose solution is the sought-after vector p (10).

Regardless of the calculation method adopted, we will denote the aggregation result using this model as $\mathrm{AADAM}\left(W\right)$, where W is a matrix in which the columns are vectors of weights derived from individual experts.

3.4. Multiplicative Assessment-Driven Aggregation Method (MADAM)

In the case of the geometric mean, the condition (9) takes the following form:

$$\mathrm{MAIP}(W,{\displaystyle \frac{p}{{∥p∥}_1}})=\left(\begin{array}{c}p\left(a_1\right)\\ p\left(a_2\right)\\ ⋮\\ ⋮\\ p\left(a_n\right)\end{array}\right),$$

(12)

where $p\left(a_i\right)>0$ for $i=1,\dots ,n$.

The equivalent non-linear optimization problem is as follows:

$$\begin{array}{c}\hfill ming\left(r\right)\mathrm{s}.\mathrm{t}.\\ \hfill g\left(r\right)=\sum _{i=1}^n{\left(\prod _{j=1}^n{w}_j^{r\left(a_j\right)}\left(a_i\right)-p\left(a_i\right)\right)}^2,\\ \hfill r\left(a_i\right)={\displaystyle \frac{p\left(a_i\right)}{{\sum }_{j=1}^{n}p\left(a_j\right)}}\mathrm{and}r\left(a_i\right)>0\mathrm{for}i=1,\dots ,n.\end{array}$$

(13)

The accepted constraints correspond to the conditions of the MAIP problem, i.e., the value of priorities for each expert is positive and their sum is 1. The above (13) can be solved using constrained global optimization methods. One can also attempt to find the final vector of weights using a direct iterative algorithm (DIA); however, is should be noted that it may not always converge. The iterative algorithm works according to the following scheme.

If the Algorithm 1 returns the correct value of the vector of weights, this result can be taken as the correct solution. If the calculated result is too inaccurate, we can increase the number of iterations and try to run the algorithm again. However, if the calculation does not converge despite increasing parameters $\gamma$ and $ϵ$, it is necessary to use dedicated software with optimization algorithms. Regardless of the calculation method adopted, we denote the aggregation result using this model as $\mathrm{MADAM}\left(W\right)$, where W is a matrix in which the columns are vectors of weights derived from individual experts.

Algorithm 1 Direct Iterative Algorithm (DIA)

1: procedure DIA(W, $\gamma$, $ϵ$) 2:       calculate the first approximation of $w^{\prime }$ using $\mathrm{MAIP}\left(W\right)$ 3:       normalize it so that ${∥w^{\prime }∥}_1=1$ 4:       then in at most $\gamma$ iterations repeat the lines (2–3), verifying each time if a sufficiently accurate solution has been achieved, i.e., $g(W,w^{\prime })\le ϵ$ holds. If this is the case, we stop the iteration and return the result $w^{\prime }$. Otherwise we continue iterations. If after $\gamma$ iterations the condition is not satisfied, we consider that the algorithm has not found a solution. 5: end procedure

The computational complexity of the DIA algorithm, assuming that the standard 64-bit representation of a real number, is the maximum of the computational complexity of $MAIP\left(W\right)$, $g(W,w^{\prime })$ and vector normalization $w^{\prime }$ multiplied by the maximum number of iterations $\gamma$. Since $\mathrm{MAIP}\left(W\right)$, $g(W,w^{\prime })$ have a complexity of $n^2$ and normalization has a linear complexity, the final execution time of the DIA algorithm is $O\left(\gamma n^2\right)$ = $O\left(n^2\right)$ ($\gamma$ is a constant).

3.5. The Existence of a Solution in the Multiplicative Approach

In both approaches, i.e., additive AADAM and multiplicative MADAM, the solution exists. However, while in AADAM its existence was almost immediate and resulted from a well-known property of the stochastic matrix, in the multiplicative case this fact requires separate justification. Let us, therefore, show that the MADAM optimization problem (13) always has a solution, i.e., function g always reaches the global minimum. To present this statement in the form of a theorem, for convenience, let us shorten the notation by assuming $r_i\stackrel{\mathit{df}}{=}r\left(a_i\right)$, $p_i\stackrel{\mathit{df}}{=}p\left(a_i\right)$, $w_{ij}\stackrel{\mathit{df}}{=}{w}_j\left(a_i\right)$, $r=\left(r_1,\dots ,r_n\right)$ and $p=\left(p_1,\dots ,p_n\right)$.

Theorem 2.

For $g:{\mathbb{R}}_{+}^n\to \mathbb{R}$ such that $g\left(r\right)={\sum }_{i=1}^n{\left({\prod }_{j=1}^n{w}_{ij}^{r_j}-p_i\right)}^2$ where $r>0$ and $r_i=p_i/{\sum }_{j=1}^n{p}_j$ for $i=1,\dots ,n$ there is always a global minimizer $r^{*}$, such that $g\left(r^{*}\right)=0$.

Proof. 

Note that the condition $r_i=p_i/{\sum }_{j=1}^n{p}_j$ is equivalent to $p_i=S·r_i$, where $S={\sum }_{j=1}^n{p}_j$. Thus, $p=S·r$. It is the case that $r\in \Delta \stackrel{\mathit{df}}{=}\left\{r\in {\mathbb{R}}_{\ge 0}^n:{\sum }_{j=1}^n{r}_j=1\right\}$, where $\Delta$ is a closed unit sphere in ${\mathbb{R}}^n$ [41] (p. 11). Let us define

$$b_i\left(r\right)\stackrel{\mathit{df}}{=}\prod _{j=1}^n{w}_{ij}^{r_j},$$

(14)

and $b\left(r\right)=\left(b_1\left(r\right),\dots ,b_n\left(r\right)\right)$. Since $w_{ij}>0$ for every $1\le i,j\le n$, then for every $r\in \Delta$, $b_i\left(r\right)>0$ holds. It is also true that the function $b\left(r\right)$ is a continuous function on $\Delta$. Let $F:\Delta \to \Delta$ have the following mapping:

$$F\left(r\right)\stackrel{\mathit{df}}{=}{\displaystyle \frac{b\left(r\right)}{{\sum }_{k=1}^n{b}_k\left(r\right)}}.$$

(15)

Function F is continuous and well-defined. The denominator is positive because $b_i\left(r\right)>0$ and, for the same reason, all its components are strictly positive. Since $\Delta$ is a closed unit sphere (compact and convex), and F is continuous, then according to Brouwer’s theorem [41] (p. 220) there exists $r^{*}\in \Delta$ such that $F\left(r^{*}\right)=r^{*}$.

Let us adopt the following as the minimum required:

$$p_i^{*}\stackrel{\mathit{df}}{=}{b}_i\left(r^{*}\right),$$

(16)

where, of course, it holds that

$$b_i\left(r^{*}\right)=\prod _{j=1}^n{w}_{ij}^{r_j^{*}}>0.$$

(17)

The value $p_i$ is well-chosen, i.e., it satisfies the constraints of the function g (13). Let us check the following:

$${\displaystyle \frac{p^{*}}{{\sum }_k{p}_k^{*}}}\underset{\left(1\right)}{=}{\displaystyle \frac{b\left(r^{*}\right)}{{\sum }_k{b}_k\left(r^{*}\right)}}\underset{\left(2\right)}{=}F\left(r^{*}\right)\underset{\left(3\right)}{=}{r}^{*},$$

where the first equality comes from (16), the second results from (15), and the third follows directly from Brouwer’s theorem. Thus,

$${\displaystyle \frac{p^{*}}{{\sum }_k{p}_k^{*}}}=r^{*}.$$

Furthermore, as (16) and (17), it holds that

$$\prod _{j=1}^n{w}_{ij}^{r_j^{*}}-p_i^{*}=b_i\left(r^{*}\right)-b_i\left(r^{*}\right)=0,$$

for every $i=1,\dots ,n$. Therefore, due to the form of g, we have the following

$$g\left(r^{*}\right)=\sum _{i=1}^n{\left(\prod _{j=1}^n{w}_{ij}^{r_j^{*}}-p_i^{*}\right)}^2=\sum _{i=1}^n{0}^2=0.$$

Since for every $r\in \Delta$, $g\left(r\right)\ge 0$, then $r^{*}$ is the global minimum of the function g and $g\left(r^{*}\right)=0$. □

Unfortunately, the above statement does not guarantee that the minimum found is unique. However, we can attempt to formulate a sufficient condition for the existence of a unique solution to problem (13). To achieve this end, let us define a measure of agreement between the expert opinions contained in W (5).

Definition 4.

For a given matrix $W=\left[w_{ij}\right]=\left[w_j\left(a_i\right)\right]$, where the rows are weight vectors derived from experts, let us take the matrix of logarithms $L=[ln{w}_{ij}]=\left[l_{ij}\right]$. Let $D_{ij}$ be given as

$$D_{ik}\stackrel{\mathit{df}}{=}\sum _{j=1}^n|l_{ij}-l_{kj}|=\sum _{j=1}^n\left|ln{w}_{ij}-ln{w}_{kj}\right|=\sum _{j=1}^n\left|ln{\displaystyle \frac{w_{ij}}{w_{kj}}}\right|.$$

We will refer to $DI\left(W\right)$ as the index of disagreement among the experts’ opinions presented in matrix W, where

$$DI\left(W\right)\stackrel{\mathit{df}}{=}\underset{i,k}{max}{D}_{ik}.$$

In deed, if the i-th and k-th rows in matrix W are identical, then $D_{ik}=0$. If they are similar, i.e., $w_{ij}\approx w_{kj}$, then $D_{ik}\approx 0$. The introduction of the disagreement index allows us to set an easily numerically verifiable condition for the existence of a unique global solution using the MADAM method.

Theorem 3.

The solution to problem (13) is unique if the preferences of the experts expressed in the matrix W are sufficiently similar, i.e., if $DI\left(W\right)<2$.

Proof. 

Using the notation from Theorem 2, let us observe that $w_{ij}^{r_j}=exp\left(r_{j}ln{w}_{ij}\right).$ Hence,

$$\prod _{j=1}^n{w}_{ij}^{r_j}=\prod _{j=1}^{n}exp\left(r_{j}log{w}_{ij}\right)=exp\left(\sum _{j=1}^n{r}_{j}log{w}_{ij}\right).$$

(18)

Therefore, for the matrix $L=\left[l_{ij}\right]=[ln{w}_{ij}]$, we have

$${\left(Lr\right)}_i=\sum _{j=1}^n{l}_{ij}{r}_j=\sum _{j=1}^n{r}_{j}log{w}_{ij}.$$

(19)

Since $b_i\left(r\right)={\prod }_{j=1}^n{w}_{ij}^{r_j}$ (14) due to (18) i (19), we obtain

$$b_i\left(r\right)=exp{\left(Lr\right)}_i.$$

Thus, taking the definition (15), we can see that $F\left(r\right)=\sigma \left(Lr\right)$, where $\sigma :{\mathbb{R}}^n\to {(0,1)}^n$ is the softmax function, given as

$$\sigma {\left(r\right)}_i\stackrel{\mathit{df}}{=}{\displaystyle \frac{e^{r_i}}{{\sum }_{j=1}^n{e}^{r_j}}}.$$

Let us consider $x=r-s$, where $r,s\in \Delta$, i.e., ${∥x∥}_1=0$; we obtain

$${\left(Lx\right)}_i-{\left(Lx\right)}_k=\sum _{j=1}^n(l_{ij}-l_{kj})x_j.$$

Thus,

$${|\left(Lx\right)}_i-{\left(Lx\right)}_k|\le \sum _{j=1}^n|l_{ij}-l_{kj}\left|\right|x_j|\le D_{ik}{\parallel x\parallel }_1\le \mathrm{DI}\left(W\right){\parallel x\parallel }_1\mathrm{for}\mathrm{every}1\le i,k\le n.$$

(20)

Let us define the variance of the vector $Lx$ elements as $\mathrm{var}\left(Lx\right)\stackrel{\mathit{df}}{=}{max}_i{\left(Lx\right)}_i-{min}_i{\left(Lx\right)}_i$. Thus, due to (20), it holds that

$$\mathrm{var}\left(Lx\right)\le {\mathrm{DI}\left(W\right)\parallel x\parallel }_1.$$

From Lemma 1 (proven below), it holds that ${\parallel \sigma \left(u\right)-\sigma \left(v\right)\parallel }_1\le \frac{1}{2}\mathrm{var}(u-v)$. Bearing in mind that $x=r-s$, we obtain the following:

$${∥F\left(r\right)-F\left(s\right)∥}_1={∥\sigma \left(Lr\right)-\sigma \left(Ls\right)∥}_1\le {\displaystyle \frac{1}{2}}\mathrm{var}\left(Lr-Ls\right)=$$

$$={\displaystyle \frac{1}{2}}\mathrm{var}\left(L\left(r-s\right)\right)\le \underset{\alpha }{\underbrace{{\displaystyle \frac{1}{2}}\mathrm{DI}\left(W\right)}}{\parallel r-s\parallel }_1.$$

From Banach’s fixed point principle [41,42] we know that there is exactly one $r^{*}$, such that $F\left(r^{*}\right)=r^{*}$ if F is a contraction, i.e., there exists a Lipszyc constant $0<\alpha <1$, such that ${∥F\left(r\right)-F\left(s\right)∥}_1\le \alpha {\parallel r-s\parallel }_1$. Therefore, for F to be a contraction, the following is necessary:

$$\alpha ={\displaystyle \frac{1}{2}}\mathrm{DI}\left(W\right)<1,$$

which is true when

$$\mathrm{DI}\left(W\right)<2.$$

□

Additionally, let us prove the property of the softmax function that we used in the above theorem.

Lemma 1.

For the softmax function $\sigma :{\mathbb{R}}^n\to {(0,1)}^n$, defined as $\sigma {\left(r\right)}_i\stackrel{\mathit{df}}{=}{e}^{r_i}/{\sum }_{j=1}^n{e}^{r_j}$, and the variance of variables for vector r, given as $var\left(r\right)\stackrel{\mathit{df}}{=}{max}_i{r}_i-{min}_i{r}_i$, it holds that

$${\parallel \sigma \left(u\right)-\sigma \left(v\right)\parallel }_1\le \frac{1}{2}var(u-v).$$

Proof. 

Let $J_{\sigma }$ denote the Jacobian matrix of the softmax function. We know [43] that $J_{\sigma }\left(z\right)=\mathrm{Diag}\left(s\right)-s·s^T$, where $\sigma \left(z\right)=s$. Furthermore, for $\sigma$, the following inequality holds:

$$\sigma \left(u\right)-\sigma \left(v\right)={\int }_0^{1}J\left(v+t(u-v)\right)(u-v)dt.$$

Hence, due to Minkowski’s integral inequality

$${\parallel \sigma \left(u\right)-\sigma \left(v\right)\parallel }_1\le {\int }_0^1{∥J\left(v+t(u-v)\right)(u-v)∥}_{1}dt.$$

(21)

For any $x\in {\mathbb{R}}^n$, denote $\mu \stackrel{\mathit{df}}{=}{s}^{\top }x$(weighted average x as $s\in \Delta$). Since ${\left(\mathrm{Diag}\left(s\right)x\right)}_i=s_i{x}_i$ and ${\left(s{s}^{\top }x\right)}_i=s_i\left(s^{\top }x\right)=s_i\mu$, then

$${\left(J_{\sigma }\left(z\right)x\right)}_i={\left(\mathrm{Diag}\left(s\right)-s·s^T\right)}_i=s_i(x_i-\mu ).$$

Therefore,

$$\parallel J_{\sigma }{\left(z\right)x\parallel }_1=\sum _{i=1}^n|s_i(x_i-\mu )|=\sum _{i=1}^n{s}_i|x_i-\mu |.$$

For any numbers $x_1,\dots ,x_n$ and any weight distribution $s\in \Delta$, the following holds:

$$\sum _{i=1}^n{s}_i|x_i-\mu |\le \frac{1}{2}(\underset{i}{max}{x}_i-\underset{i}{min}{x}_i),$$

where $\mu ={\sum }_i{s}_i{x}_i$. Hence, in particular, it follows that

$$\parallel J_{\sigma }{\left(z\right)x\parallel }_1\le \frac{1}{2}(\underset{i}{max}{x}_i-\underset{i}{min}{x}_i)={\displaystyle \frac{1}{2}}\mathrm{var}\left(x\right).$$

(22)

Hence, taking (21) and (22), we obtain

$${\parallel \sigma \left(u\right)-\sigma \left(v\right)\parallel }_1\le {\int }_0^1\frac{1}{2}\mathrm{var}(u-v)dt=\frac{1}{2}\mathrm{var}(u-v).$$

□

Theorem 3 defines a simple, numerically verifiable, sufficient condition for a global solution to be a unique solution of an optimization problem (13). However, this is not a necessary condition. This means that even if the conditions of this theorem are not met, the global minimum $t^{*}$ of the function g may still be unique.

3.6. Illustrative Example

The opinion aggregation models proposed above can be applied in many cases. To illustrate how and in what context the above models can be used, let us consider the following example.

Example 1.

Four scientists—John, Samantha, Michael, and Frank—of equal seniority and professional experience worked on a joint project that resulted in a scientific article. John proposed the research idea and coordinated the team’s activities. However, during the project, Samantha’s ideas substantially reshaped the research plan, and she ultimately made the largest contribution. She also collaborated most intensively with the other team members. Michael wrote the software and ran the experiments. After completing the initial programming tasks, he focused mainly on testing various data sets and collaborated only occasionally with other team members. Frank collected and prepared the experimental data but did not participate in most team meetings. The team published the results in a respected journal. The university awarded the team a financial prize and left its distribution to the project participants. John, Samantha, Michael, and Frank agreed that each researcher’s share of the award should reflect their contribution and involvement in the project. At the same time, they noted that the ability to evaluate contributions accurately increases with the level of involvement. As a result, they proposed that each participant evaluate all team members using pairwise comparisons and aggregate the resulting opinions with the AADAM method, which appropriately differentiates the priorities of individual experts.

John observed that Samantha invested substantially more effort and commitment than the others. However, he believed that the original idea and initiative carried equal importance. Therefore, he rated Samantha’s contribution as equal to his own. He rated Michel’s and Frank’s contributions as two and three times smaller than his own, respectively. Consequently, his pairwise comparison matrix $C_J$ takes the following form:

$$C_J=\left(\begin{array}{cccc}1& 1& 2& 3\\ 1& 1& 2& 3\\ {\displaystyle \frac{1}{2}}& {\displaystyle \frac{1}{2}}& 1& {\displaystyle \frac{3}{2}}\\ {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{3}}& {\displaystyle \frac{2}{3}}& 1\end{array}\right).$$

Samantha believes that the actual amount of work performed should guide the distribution of the award. Moreover, she considers her ideas to be at least as relevant as, and possibly more relevant than, John’s. Accordingly, she rated her contribution slightly higher than John’s. She rated Michael’s and Frank’s contributions as three and four times lower than her own, respectively. Consequently, her set of judgments, represented by the matrix $C_S$ looks as follows:

$$C_S=\left(\begin{array}{cccc}1& {\displaystyle \frac{4}{5}}& 2& 3\\ {\displaystyle \frac{5}{4}}& 1& 3& 4\\ {\displaystyle \frac{1}{2}}& {\displaystyle \frac{1}{3}}& 1& {\displaystyle \frac{4}{3}}\\ {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{4}}& {\displaystyle \frac{3}{4}}& 1\end{array}\right).$$

Michael recognizes that his contribution to the project was smaller than that of the others. However, he communicated more frequently with Samantha than with the other team members. Consequently, he perceives her contribution as more important than John’s. In his view, the amount of work and time Frank devoted exceeded half of his own contribution. Accordingly, his judgments, represented by the matrix $C_M$ are as follows:

$$C_M=\left(\begin{array}{cccc}1& {\displaystyle \frac{3}{5}}& {\displaystyle \frac{3}{2}}& 2\\ {\displaystyle \frac{5}{3}}& 1& 2& {\displaystyle \frac{5}{2}}\\ {\displaystyle \frac{2}{3}}& {\displaystyle \frac{1}{2}}& 1& {\displaystyle \frac{5}{3}}\\ {\displaystyle \frac{1}{2}}& {\displaystyle \frac{2}{5}}& {\displaystyle \frac{3}{5}}& 1\end{array}\right).$$

Frank realistically assesses his role in the project. He considers his participation in the project to be non-critical. He assigns similar importance to the contributions of the two key researchers, John and Samantha. Accordingly, he does not differentiate between their levels of participation. He views his own involvement in the project as ancillary. He holds a similar view of Michael’s role. Therefore, he evaluates Michael’s contribution as comparable to his own. As a result, his set of evaluations, given as matrix $C_F$, is as follows:

$$C_F=\left(\begin{array}{cccc}1& 1& 3& 3\\ 1& 1& 3& 3\\ {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{3}}& 1& 1\\ {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{3}}& 1& 1\end{array}\right).$$

After calculating the priority vectors using GMM for each participant, we obtain the following:

$$w_J={\left(0.352941,0.352941,0.176471,0.117647\right)}^T,$$

$$w_S={\left(0.319363,0.424617,0.14814,0.107881\right)}^T,$$

$$w_M={\left(0.268793,0.394279,0.200346,0.136582\right)}^T,$$

$$w_F={\left(0.375,0.375,0.125,0.125\right)}^T.$$

After aggregation using the AADAM the final priority vector is as follows:

$$w_{\mathrm{AADAM}}={\left(0.328698,0.39026,0.163249,0.117793\right)}^T,$$

whereas the result using MADAM is

$$w_{\mathrm{MADAM}}={\left(0.328706,0.391014,0.162337,0.117943\right)}^T.$$

All project participants agreed that the obtained result represented a reasonable compromise between their expectations regarding the distribution of the prize.

4. Properties of the Aggregation Methods

4.1. Formal Properties

When considering various aggregation methods, one of the most important questions is whether they satisfy the Pareto principle [4,12], according to which if all experts prefer one alternative to another, this preference is reflected in the final results. The MAIP and AAIP aggregation methods satisfy the Pareto postulate [4]. This property carries over to the MADAM and AADAM aggregation methods. Similarly, both methods satisfy the homogeneity condition.

Proposition 1.

Both modified aggregation methods, MAIP and AAIP, satisfy the Pareto principle [44], i.e., if for any two alternatives $i,j$, all experts prefer the i-th alternative over the j-th alternative, i.e., $w_k\left(a_i\right)>w_k\left(a_j\right)$ for $k=1,\dots ,n$, then this relationship holds in the aggregated vector of weights, i.e., $w\left(a_i\right)>w\left(a_j\right)$.

Proof. 

Let MAIP and AAIP be as defined in (8) and (7) correspondingly. We fix $i,j\in \{1,\dots ,n\}$. For each $p\in {\mathbb{R}}^n$ and for each $k\in \{1,\dots ,n\}$

$$w_k\left(a_i\right)>w_k\left(a_j\right)\Rightarrow \left(w_k^{p_k}\left(a_i\right)>w_k^{p_k}\left(a_j\right) \mathrm{and} p_k{w}_k\left(a_i\right)>p_k{w}_k\left(a_j\right)\right),$$

which implies that

$$w_1^{p_1}\left(a_i\right)·w_2^{p_2}\left(a_i\right)\cdots w_n^{p_n}\left(a_i\right)>w_1^{p_1}\left(a_j\right)·w_2^{p_2}\left(a_j\right)\cdots w_n^{p_n}\left(a_j\right)$$

and

$$p_1{w}_1\left(a_i\right)+p_2{w}_2\left(a_i\right)+\cdots +p_n{w}_n\left(a_i\right)>p_1{w}_1\left(a_j\right)+p_2{w}_2\left(a_j\right)+\cdots +p_n{w}_n\left(a_j\right).$$

The above inequalities complete the proof. □

In addition, it is possible to estimate the norm value of both resultant vectors.

Proposition 2.

MAIP and AAIP maps are bounded by $\sqrt{n}$, which is also a Lipschitz constant for $\mathit{AAIP}(W,·)$, for any initial matrix W.

Proof. 

Since $p,w\left(a_i\right)\in {\mathbb{R}}^n$ for $i=1,\dots ,n$ with a scalar product $⟨p,w\left(a_i\right)⟩={\sum }_{j=1}^n{p}_j{w}_j\left(a_i\right)$, by virtue of the Cauchy–Schwarz inequality, the following holds: $|⟨p,w\left(a_i\right)⟩|\le \parallel p\parallel \parallel w\left(a_i\right)\parallel$. Thus, using the inequality between the geometric mean and the arithmetic mean, we obtain

$$\begin{array}{ccc}& & \left|\right|\mathrm{MAIP}(W,p)\left|\right|\hfill \\ & & =\sqrt{{(w_1^{p_1}\left(a_1\right)\cdots w_n^{p_n}\left(a_1\right))}^2+\dots +{(w_1^{p_1}\left(a_n\right)\cdots w_n^{p_n}\left(a_n\right))}^2}\hfill \\ & & \le \sqrt{{(p_1{w}_1\left(a_1\right)+\dots +p_n{w}_n\left(a_1\right))}^2+\dots +{(p_1{w}_1\left(a_n\right)+\dots +p_n{w}_n\left(a_n\right))}^2}\hfill \\ & & =\left|\right|\mathrm{AAIP}(W,p)\left|\right|\hfill \\ & & =\sqrt{{⟨p,w_1⟩}^2+\dots +{⟨p,w_n⟩}^2}\le \sqrt{{\left|\right|p\left|\right|}^2\left|\right|w_1{\left|\right|}^2+\dots +{\left|\right|p\left|\right|}^2\left|\right|w_n{\left|\right|}^2}\hfill \\ & & =\left|\right|p\left|\right|\sqrt{\left|\right|w_1{\left|\right|}^2+\dots +\left|\right|w_n{\left|\right|}^2}\le \left|\right|p\left|\right|\sqrt{n}\le \sqrt{n}.\hfill \end{array}$$

Obviously, the $\mathrm{AAIP}(W,·)$ map is linear, so for each $p,\widehat{p}\in {\mathbb{R}}^n$

$$\left|\right|\mathrm{AAIP}(W,p)-\mathrm{AAIP}(W,\widehat{p})\left|\right|=\left|\right|\mathrm{AAIP}(W,p-\widehat{p})\left|\right|\le \sqrt{n}\left|\right|p-\widehat{p}\left|\right|.$$

□

Notice that for $p={({\displaystyle \frac{1}{n}},\dots ,{\displaystyle \frac{1}{n}})}^T$, the norm

$$\left|\right|p\left|\right|={\displaystyle \frac{1}{\sqrt{n}}},$$

so

$$\left|\right|\mathrm{MAIP}(W,p)\left|\right|\le \left|\right|\mathrm{AAIP}(W,p)\left|\right|\le 1.$$

On the other hand, if for each $i\in \{1,\dots ,n\}$ we have

$$w_1\left(a_i\right)=w_2\left(a_i\right)=\dots =w_n\left(a_i\right),$$

then, for each $j\in \{1,\dots ,n\}$ and each p, it follows that

$$\mathrm{MAIP}(W,p)=\mathrm{AAIP}(W,p)={(w_j\left(a_1\right),\dots ,w_j\left(a_n\right))}^T.$$

This means that if all the experts scored each other the same way, then neither of the two algorithms will influence the resulting ranking.

One of the arguments for using the geometric mean to aggregate results, in addition to satisfying the Pareto principle, is that it satisfies the homogeneity postulate. According to [2,4], if all experts judge a pair of alternatives t times as large as another pair of alternatives, then the resulting vector also preserves this ratio. For this reason, Aczel and Saaty [2] argue that for AIJ (see Section 2.2.1), the geometric mean must be used. As the weighted geometric and arithmetic means also support the homogeneity postulate [2], then this condition also holds for MADAM and AADAM.

Definition 5.

The homogeneity condition [2] states the following:

$$f(s{x}_1,s{x}_2,\dots ,s{x}_n)=sf(x_1,x_2,\dots ,x_n),$$

and $s>0$ and $x_i,s{x}_i\in R$, where R is a positive set of numbers also containing their reciprocity values, i.e., if $x\in R$, then $1/x\in R$.

The homogeneity condition means that the aggregating transformation f is invariant due to a change in scale or unit (e.g., converting kilograms to pounds).

Proposition 3.

The MADAM and AADAM aggregation methods satisfy the homogeneity postulate.

Proof. 

For the additive case, we have

$$f(w\left(a_1\right),\dots ,w\left(a_n\right))\stackrel{\mathit{df}}{=}\sum _{i=1}^k{\displaystyle \frac{p_i}{{\sum }_j{p}_j}}{w}_i\left(a_k\right).$$

As ${\sum }_j{p}_j=1$, then $f(w\left(a_1\right),\dots ,w\left(a_n\right))={\sum }_{i=1}^k{p}_i{w}_i\left(a_k\right)$. Hence, for $k=1,\dots ,n$, there is

$$f(sw\left(a_1\right),\dots ,sw\left(a_n\right))=\sum _{i=1}^{k}s{p}_i{w}_i\left(a_k\right)=\dots$$

$$\dots =s\sum _{i=1}^k{p}_i{w}_i\left(a_k\right)=sf(w\left(a_1\right),\dots ,w\left(a_n\right))$$

Similarly, for the multiplicative aggregation,

$$f(w\left(a_1\right),\dots ,w\left(a_n\right))\stackrel{\mathit{df}}{=}\prod _{i=1}^k{w}_i^{{\displaystyle \frac{p_i}{\sum p_j}}}\left(a_k\right).$$

Thus, for $k=1,\dots ,n$

$$f(s{w}_1\left(a_k\right),\dots ,s{w}_n\left(a_k\right))=\prod _{i=1}^{k}s{w}_i^{{\displaystyle \frac{p_i}{\sum p_j}}}\left(a_k\right)=\dots$$

$$\dots =\prod _{i=1}^k{s}^{{\displaystyle \frac{p_i}{\sum p_j}}}{w}_i^{{\displaystyle \frac{p_i}{\sum p_j}}}\left(a_k\right)=\prod _{i=1}^k{s}^{{\displaystyle \frac{p_i}{\sum p_j}}}\prod _{i=1}^k{w}_i^{{\displaystyle \frac{p_i}{\sum p_j}}}\left(a_k\right)=\dots$$

$$\dots =\left(s^{{\displaystyle \frac{p_1}{\sum p_j}}+\dots +{\displaystyle \frac{p_n}{\sum p_j}}}\right)\prod _{i=1}^k{w}_i^{{\displaystyle \frac{p_i}{\sum p_j}}}\left(a_k\right)=s\prod _{i=1}^k{w}_i^{{\displaystyle \frac{p_i}{\sum p_j}}}\left(a_k\right)=sf(w\left(a_1\right),\dots ,w\left(a_n\right)).$$

□

The proof of homogeneity of MADAM and AADAM shows that if all the experts evaluate the ratio between the two chosen alternatives $a_x$ and $a_y$ in the same way, i.e.,

$${\displaystyle \frac{w_i\left(a_s\right)}{w_i\left(a_t\right)}}={\alpha }_{st},\mathrm{for}i=1,\dots ,n.$$

then the same ratio will also be preserved in the aggregate result, i.e.,

$${\displaystyle \frac{p\left(a_s\right)}{p\left(a_t\right)}}={\alpha }_{st}.$$

In particular, it follows from the above that if all experts agree on the quantitative relationship between two alternatives, then this relationship holds in the resulting vector. This observation also means that the Pareto condition is satisfied.

4.2. Numerical Examples

When discussing a new method for computing rankings, it is helpful to examine its behavior on concrete and simple examples. Below, we present four examples that illustrate how the proposed methods operate in practice.

Example 2 models a situation of unanimity and shows that, as expected, the aggregation functions reproduce each expert’s opinion in the output. Example 3 models a coalition of experts and demonstrates that averaging-based aggregation functions favor coalitions. Example 4 represents a tie situation and confirms the intuition that if each expert receives the same level of support, then all experts obtain the same evaluation. The final example reflects on the role of expert priority, showing that an expert who enjoys high recognition from others may, when the proposed aggregation methods are applied, exert a real and significant influence on the final outcome.

Example 2.

Following the observation at the end of Section 4.1, let us consider case of unanimity, i.e., a situation in which each of the five experts expressed exactly the same opinions, which translated into five identical vectors of weights. In this case, one can expect that the aggregation function will duplicate the selection of each expert. Thus, the considered matrix will look as follows:

$$W=\left(\begin{array}{ccccc}{\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{3}}\\ {\displaystyle \frac{4}{15}}& {\displaystyle \frac{4}{15}}& {\displaystyle \frac{4}{15}}& {\displaystyle \frac{4}{15}}& {\displaystyle \frac{4}{15}}\\ {\displaystyle \frac{1}{5}}& {\displaystyle \frac{1}{5}}& {\displaystyle \frac{1}{5}}& {\displaystyle \frac{1}{5}}& {\displaystyle \frac{1}{5}}\\ {\displaystyle \frac{2}{15}}& {\displaystyle \frac{2}{15}}& {\displaystyle \frac{2}{15}}& {\displaystyle \frac{2}{15}}& {\displaystyle \frac{2}{15}}\\ {\displaystyle \frac{1}{15}}& {\displaystyle \frac{1}{15}}& {\displaystyle \frac{1}{15}}& {\displaystyle \frac{1}{15}}& {\displaystyle \frac{1}{15}}\end{array}\right).$$

Indeed, all the experts expressed exactly the same opinion. It is easy to check that the result of the aggregation of vectors in W using MADAM and AADAM is identical to each of the columns in W:

$$\mathrm{AADAM}\left(W\right)=\mathrm{MADAM}\left(W\right)={\left({\displaystyle \frac{1}{3}},{\displaystyle \frac{4}{15}},{\displaystyle \frac{1}{5}},{\displaystyle \frac{2}{15}},{\displaystyle \frac{1}{15}}\right)}^T.$$

Example 3.

It is easy to imagine a situation in which two participants support each other. This is a case of a coalition of two experts who support each other. Let us consider a situation in which expert $e_1$ will rate $e_5$ highly and, vice versa, $e_5$ will rate $e_1$ highly. The other experts will rate everyone the same. The matrix W corresponding to such a case will look as follows:

$$W=\left(\begin{array}{ccccc}{\displaystyle \frac{1}{13}}& 1& 1& 1& {\displaystyle \frac{9}{13}}\\ {\displaystyle \frac{1}{13}}& 1& 1& 1& {\displaystyle \frac{1}{13}}\\ {\displaystyle \frac{1}{13}}& 1& 1& 1& {\displaystyle \frac{1}{13}}\\ {\displaystyle \frac{1}{13}}& 1& 1& 1& {\displaystyle \frac{1}{13}}\\ {\displaystyle \frac{9}{13}}& 1& 1& 1& {\displaystyle \frac{1}{13}}\end{array}\right).$$

Using the arithmetic mean with equal experts’ priorities to aggregate the results (8) leads to a weight vector of the following form:

$$\mathit{AAIP}\left(W\right)={\left(0.222,0.186,0.186,0.186,0.222\right)}^T,$$

while AADAM leads to the following:

$$\mathit{AADAM}\left(W\right)={\left(0.226,0.183,0.183,0.183,0.226\right)}^T.$$

Both of these means of aggregation lead to higher expert priorities of $e_1$ and $e_5$ than the others; however, in AADAM, the priorities are slightly higher. Indeed, someone considered these experts more important, which further strengthened their opinion. A similar, but slightly stronger, effect can be observed with the geometric approach, i.e.,

$$\mathit{MAIP}\left(W\right)={\left(0.254,0.164,0.164,0.164,0.254\right)}^T,$$

$$\mathit{MADAM}\left(W\right)={\left(0.275,0.15,0.15,0.15,0.275\right)}^T.$$

The stronger influence of the opinions of the first and last experts translates into a larger difference between the weight vectors in the multiplicative case. Thus, while the euclidean distance for the additive method is very small $∥\mathit{AAIP}\left(W\right)-\mathit{AADAM}\left(W\right)∥=0.008$, for the multiplicative one it is an order of magnitude larger: $∥\mathit{MAIP}\left(W\right)-\mathit{MADAM}\left(W\right)∥=0.037$.

Example 4.

In Example 3, experts $e_1$ and $e_5$ mutually assigned the same weight to each other. Thus, in a situation where the indications form a cycle, i.e., expert $e_1$ prefers $e_2$ over all others, $e_2$ will similarly indicate $e_3$, etc., and the last expert, again, will prefer $e_1$, a tie can be expected. Indeed for

$$W=\left(\begin{array}{ccccc}{\displaystyle \frac{1}{13}}& {\displaystyle \frac{9}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}\\ {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{9}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}\\ {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{9}{13}}& {\displaystyle \frac{1}{13}}\\ {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{9}{13}}\\ {\displaystyle \frac{9}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}& {\displaystyle \frac{1}{13}}\end{array}\right)$$

the following result is obtained:

$$\mathit{AAIP}\left(W\right)=\mathit{AADAM}\left(W\right)=\mathit{MAIP}\left(W\right)=\mathit{MADAM}\left(W\right)={\left(0.2,\dots ,0.2\right)}^T.$$

Example 5.

One may be concerned about bestowing too much influence on one expert, who will end up dominating the ranking. Let us try to consider an example where four experts, back-to-back, will decisively identify a fifth person as the ranking winner. The fifth expert, with a separate opinion, will not give himself (i.e., the fifth alternative) as much weight. Let us consider an example matrix W corresponding to such a situation:

$$W=\left(\begin{array}{ccccc}{\displaystyle \frac{1}{10}}& {\displaystyle \frac{1}{15}}& {\displaystyle \frac{1}{20}}& {\displaystyle \frac{1}{20}}& {\displaystyle \frac{1}{10}}\\ {\displaystyle \frac{1}{20}}& {\displaystyle \frac{1}{10}}& {\displaystyle \frac{1}{20}}& {\displaystyle \frac{1}{15}}& {\displaystyle \frac{2}{3}}\\ {\displaystyle \frac{1}{10}}& {\displaystyle \frac{1}{10}}& {\displaystyle \frac{1}{5}}& {\displaystyle \frac{1}{10}}& {\displaystyle \frac{1}{20}}\\ {\displaystyle \frac{1}{20}}& {\displaystyle \frac{1}{15}}& {\displaystyle \frac{1}{20}}& {\displaystyle \frac{1}{5}}& {\displaystyle \frac{1}{20}}\\ {\displaystyle \frac{7}{10}}& {\displaystyle \frac{2}{3}}& {\displaystyle \frac{13}{20}}& {\displaystyle \frac{7}{12}}& {\displaystyle \frac{2}{15}}\end{array}\right).$$

Aggregation using the simple arithmetic mean leads to the following weight vector:

$$\mathit{AAIP}\left(W\right)={\left(0.073,0.186,0.11,0.083,0.546\right)}^T,$$

while the AADAM calculation leads to the following result:

$$\mathit{AADAM}\left(W\right)={\left(0.081,0.334,0.087,0.065,0.432\right)}^T.$$

We can note that the fifth expert did not manage to “avoid” the fate of the ranking winner. The weight assigned to them by the rest of the team is still the highest, i.e., $0.432$, although less than that achieved with the standard aggregation method (value: $0.546$). This reduction occurred at their own request. The strong voice of the fifth expert had a noticeable effect on increasing the weight of the second expert. However, it is worth noting that this expert, using the standard way of counting, has a significant advantage over the others (excluding expert no. $5$). The influence of the last expert is noticeable, but their influence is effectively limited by the will of the majority.

In the multiplicative case, we obtain the following:

$$\mathit{MAIP}\left(W\right)={\left(0.0858,0.125,0.123,0.0857,0.58\right)}^T,$$

and

$$\mathit{MADAM}\left(W\right)={\left(0.107,0.261,0.108,0.082,0.44\right)}^T.$$

As before, the weight of the fifth expert decreases and the value of the second expert increases. The weights of the remaining experts behave similarly. Interestingly, the overall change in the vector of weights in the second case is slightly larger, i.e., $∥\mathit{MAIP}\left(W\right)-\mathit{MADAM}\left(W\right)∥=0.196$, where $∥·∥$ is an Euclidean distance, while for the additive case we have $∥\mathit{AAIP}\left(W\right)-\mathit{AADAM}\left(W\right)∥=0.189$.

5. Self-Assessment Process

The AADAM and MADAM aggregation methods can effectively support group self-assessment processes. Such a process can consist of four phases (Figure 1). In the first stage, each expert constructs an individual preference vector. To this end, they may use the pairwise comparison method. When experts use this method, they should check the consistency of their judgments, for example, by applying Saaty’s consistency index CI (2) and the consistency ratio $\mathrm{CR}.$ If the consistency check fails, the process may incorporate additional measures, such as improving consistency through negotiation, applying automatic consistency-improvement procedures [45] or excluding the expert’s opinion and treating it as an outlier. The next step involves computing the individual preference vector for each expert. In many decision-making methods (e.g., TOPSIS, PROMETHEE [23]), this step constitutes the first stage of the process. In these methods, experts specify the weight vector directly rather than deriving it through calculations. After computing the individual preference vectors, the process should assess the compatibility of experts’ judgments. To this end, the procedure may apply a compatibility index (3) or the disagreement index introduced in this study (Definition 4). When experts’ judgments differ significantly, the process may initiate steps aimed at reaching consensus. A good overview of methods used for reaching consensus in group decision-making methods can be found in the works of Tran et al. and Dong [39,46]. In the third phase, the process aggregates the experts’ preferences. In this case, the preferred methods are MADAM and AADAM (Section 3.3 and Section 3.4). The aggregated result can further support the consensus-building process. In particular, it enables the comparison of individual experts’ opinions with the aggregated outcome. Excessive differences may indicate the need for additional consensus-building steps. In the final phase, the process formulates a recommendation and presents it to all interested parties.

Figure 1. Place of the AADAM and MADAM aggregation methods in the group peer assessment process using the pairwise comparisons method.

The four-stage peer review process described above (Figure 1) is consistent with the general consensus framework [39] (pp. 7–9). However, depending on the method applied, reaching consensus often requires an iterative process. After computing the aggregated result, the procedure may ask one or more experts to revise their positions, which, in turn, triggers a recalculation of the model.

6. Montecarlo Experiments

6.1. Test Preparation

From a practical perspective, the key issue concerns how easily one can compute the desired result. The additive APDAM method allows for straightforward computation of the solution and poses no practical difficulties (Property 1 reports the asymptotic convergence speed of the iterative algorithm). In contrast, the multiplicative variant of PDAM may require substantially greater computational effort. The DIA procedure presented in Section 3.4) runs quickly and is easy to implement, but it does not guarantee success. In such cases, global optimization methods can provide a viable alternative. However, this effectiveness comes at the cost of longer computation times. For this reason, we propose a hybrid approach, in which we first try to use DIA, and in the event of failure, we explore optimization methods (one or more in turn). Following this hybrid strategy, we first execute DIA in the tests (with a maximum of $\gamma \le {10}^4$ iterations and success defined as $ϵ<0.1^4$); if DIA fails, we then apply library algorithms such as simulated annealing, Nelder–Mead, and differential evolution. We used implementations of these three algorithms from the standard Wolfram Mathematica library (https://reference.wolfram.com/language/tutorial/ConstrainedOptimizationGlobalNumerical.html, accessed 27 December, 2025).

To evaluate the effectiveness of MPDAM using this hybrid computational scheme, we generated $900,000$ W matrices (5), where each column represents the weight vector of an individual expert.

We created each vector by randomly selecting successive elements from the range $1/15$ to 15 and then normalizing them. As a result, the vectors provided by different experts may differ substantially. For the case of two experts, we generated ${10}^5$ matrices of 2 by 2, for three experts ${10}^5$ matrices of 3 by 3, similar matrices for four experts, and so on up to the case of 10 experts, i.e., ${10}^5$ matrices of 10 by 10. We assumed that each expert provides a normalized weight vector, meaning that the sum of the weights in each matrix column equals 1.

6.2. Numerical Results

In the multiplicative method, DIA does not always find a solution. The mere existence of a correct solution does not guarantee the convergence of an iterative algorithm. However, as the number of experts increases, i.e., the size of the W matrix containing their weights, the DIA’s success frequency increases significantly. For two experts, the DIA procedure returns the correct solution in $92,515$ times out of ${10}^5$ cases, for five $97,576$ times out of ${10}^5$ cases, and for 10 experts, $99,998$ times out of ${10}^5$ tested cases. This means that in the case of ten experts, only in two cases, i.e., $0.002\%$ of all decision scenarios, was it necessary to use optimization algorithms (Figure 2).

Figure 2. The increase in DIA effectiveness alongside the increase in the number of experts.

The computational experiments were performed on a server equipped with an Apple M2 Max processor (Apple Inc., Cupertino, CA, USA). The experimental environment for the conducted tests was Wolfram Mathematica^TM software version 13.3. Although the tests were conducted in parallel, the algorithms that were executed were run sequentially. The DIA procedure used the following parameter values: $\gamma =10,000$ and $ϵ={10}^4$. The other algorithms were run with standard parameterization in the Wolfram Mathematica^TM environment, except the accuracy goal, which was set to 5. This parameter specifies how many effective digits of accuracy should be sought in the final result. The average times of sequential execution of the calculations are shown in Figure 3. As expected, execution times increase as the number of experts increases. For most cases, the DIA procedure was able to return the correct solution very quickly. In the case that after 10,000 iterations ($\gamma ={10}^4$) it was unable to provide an acceptable solution, i.e., one for which ($ϵ<0.1^4$), the execution time DIA was significantly higher but still shorter than the other global optimization algorithms. The running time of the professional-grade implementations of optimization algorithms used for the considered decision models also increases along with the size of the decision model. The average solution running times for the best optimization algorithm in each case are presented in Figure 3. For larger models (number of experts greater than 3), optimization calculations take longer for the given case than calculations using DIA in the event of failure. However, what is essential from a practical point of view is that as the size of the model increases, the efficiency of DIA rapidly increases. So, while for a model with two experts (W matrix 2 by 2), DIA was unable to return the correct result in $7\%$ of cases (Figure 2), for a W of size 6 by 6, the percentage is just slightly over $1\%$, and for a W of size 10 by 10, it drops to an imperceptible $0.002\%$.

Figure 3. Average, minimum, and maximum solution computation times for the DIA algorithm and optimization algorithms.

Adopting a hybrid strategy according to which we first try to calculate the solution using DIA, and if the algorithm fails then we use optimization methods, allows us to calculate the expected computation time for a given size of decision model. Let $\mathrm{ET}\left(n\right)$ denote the expected time required to compute the solution for the i-th model of size n, i.e.,

$$\mathrm{ET}\left(n\right)=\sum _{i\in I_{\mathit{DS}}}{10}^{-5}·\mathrm{DIA}(i,n)+\sum _{i\in I_{\mathit{DU}}}{10}^{-5}\left(\mathrm{DIA}(i,n)+\mathit{AVGOPT}(i,n)\right),$$

where $I_{\mathit{DS}}$ is the set of indices of models for which DIA is able to calculate the solution, $I_{\mathit{DU}}$ is the set of indices of models for which DIA fails (i.e., it performs the maximum set number of iterations but does not achieve satisfactory accuracy), $\mathrm{DIA}(i,n)$ is the running time of DIA for the i-th model of size n, and $\mathit{AVGOPT}(i,n)$ is the average running time of the optimization algorithms (Nelder–Mead, simulated annealing and differential evolution) for the i-th model of size n. It turns out that the average expected running time for the hybrid strategy $\mathrm{ET}\left(n\right)$ decreases along with the model’s size (Figure 4). The decline slows down around the model size 9, caused by a significant reduction in the share of global optimization algorithms, and begins a slow, natural growth based on the increasing computation time of the DIA algorithm.

Figure 4. The average expected running time using the hybrid strategy combining the DIA and optimization algorithms.

7. Discussion

7.1. The Problem of Equity

Although the ADAM model originates from judgment aggregation methods for group pairwise comparisons, its underlying idea extends beyond the domain of decision-making techniques. At its core, ADAM aims to strengthen the perceived sense of justice among peer team members. More specifically, this discussion concerns distributive justice—the problem of fair reward allocation. This problem is not new, and has been explored by researchers, philosophers, and politicians since the time of Aristotle. Ulpian (Gnaeus Domitius Annius Ulpianus), a Roman jurist and politician in the second century AD, defines justice as “the firm and perpetual will to give to each his due.” (lat. “Iustitia est firma et perpetua voluntas suum cuique tribuendi” in Gnaeus Domitius Annius Ulpianus, Iustiniani Augusti Digesta seu Pandectae.). Ulpian’s definition indicates that justice remains in the quantitative relationship between the reward and merit of the individual. We can also find similar ways of defining justice today. For example, Homans [47,48] and Adams [49,50] argue that distributive justice (Adams prefers to use the term “equity” when referring to distributive justice; thus, a lack of “justice” is referred to as “inequity”) between people who are in an exchange relationship with each other takes place when the profits of each correspond to their investment. Schematically, we can represent this relationship for two individuals, A and B, by equating two ratios:

$${\displaystyle \frac{\mathrm{A}’\mathrm{s}\mathrm{profits}}{\mathrm{A}’\mathrm{s}\mathrm{investments}}}={\displaystyle \frac{\mathrm{B}’\mathrm{s}\mathrm{profits}}{\mathrm{B}’\mathrm{s}\mathrm{investments}}}.$$

If such correspondence exists, distributive justice between A and B holds. If not, participants in the exchange experience a sense of injustice. The exchange relationship can take different forms. It is often an exchange of services (labor) for money (salary) between an employee and employer. In other words, as a rule, profit should be proportional to the work performed. Of course, the researchers point out that there may be differences in education, experience, length of service between employees, and the cost they incurred in performing the work. All these elements are considered by appropriately expanding the concepts of profit (outcome) and investment (input). Sayles [51] suggests that the group of workers under consideration “calculate” the fairness of their wages using the following equation:

$${\displaystyle \frac{\mathrm{our}\mathrm{importance}\mathrm{in}\mathrm{the}\mathrm{factory}}{\mathrm{any}\mathrm{other}\mathrm{group}’\mathrm{s}\mathrm{importance}}}={\displaystyle \frac{\mathrm{our}\mathrm{earnings}}{\mathrm{their}\mathrm{earnings}}}.$$

As before, as long as the above equality is maintained, workers perceive their pay as fair. “Importance in the factory” can be considered equivalent to the perceived investments of group members, including skills and type of work performed, seniority, etc. Patchen [52] defines the condition for distributive justice very similarly. This proportional view of justice becomes particularly relevant when interpreted through the lens of the ADAM model defined above (Section 3.2). In ADAM, an expert’s priority directly corresponds to “importance in the factory,” while the ranking score plays the role of earnings. Thus, by proposing that the “priority of expert X” should be equal to their ranking score, the ADAM model implements the distributive justice relation for which:

$${\displaystyle \frac{\mathrm{my}\mathrm{priority},\mathrm{i}.\mathrm{e}.,\mathrm{my}\mathrm{importance}\mathrm{in}\mathrm{the}\mathrm{plant}}{\mathrm{priority}\mathrm{of}\mathrm{someone}\mathrm{else},\mathrm{importance}\mathrm{of}\mathrm{someone}\mathrm{else}}}={\displaystyle \frac{\mathrm{my}\mathrm{earning}}{\mathrm{earnings}\mathrm{of}\mathrm{someone}\mathrm{else}}}.$$

These considerations show that ADAM works well in settings where an expert’s priority reliably represents inputs such as effort, experience, or commitment to the project. The ranking result, which translates into a share of the final prize, corresponds to earnings. In the context of the company’s peer team, investments in terms of equity theory, such as education, experience, and seniority, are similar. Hence, the time spent on a task (often equated with commitment) will play a primary role. Similarly, at a university, a group of researchers at a similar career level working together on a scientific article can be considered a team of peers. In this case, the amount of work may be the main or only input represented by the team member’s ranking result.

Therefore, the ADAM model is a good solution if an individual’s priority effectively represents its inputs. Specifically, this means that all members of a peer-reviewing group should have the ability to evaluate the investments of others, including both workload and education, seniority, position, responsibility held, etc. When team members cannot access or reliably assess such information, the rationale for using the ADAM model becomes questionable.

The proposed solution seeks a compromise between two extremes: equal distribution of priorities among experts and the dominance of a single individual. We deal with the first case when we use a simple arithmetic or geometric mean for aggregation. In a situation where there are significant differences in the inputs of individuals, this method of aggregation can create an understandable sense of injustice among participants. Even if those who made substantial contributions received more in the end than those with small inputs, it remains an open question as to why these people had the same influence on the decision as those whose input was small, perhaps almost none. The second case is when everyone unanimously offers a large share of the award, and thus the final decision, to one distinguished person. In such a situation, in the ADAM model, the final vector of weights begins to look more and more like the vector of weights of a recognized individual. Of course, total agreement would occur if everyone unanimously gave a rating of 1 to the chosen one and all the others 0, which is rather unlikely. However, giving a relatively significant influence on the final decision to one recognized person may raise additional concerns. The primary concern is whether such a person can remain fair and objective when they gain significant influence over the final weight vector. This observation leads us to believe that the ADAM model is worth special consideration when other team members trust those likely to earn a high ranking. Otherwise, there may be a tactical “downgrading of priority” of the giving individuals due to concerns about their honesty. The assumption of mutual trust in a team of peers is also a limitation of the scope of the ADAM model, underscoring the crucial role of trust in decision-making processes.

Despite this concern, however, it is worth noting that there is often a dictatorial approach to reward distribution in practice. This happens when a group leader, a manager, is authorized by, for example, the company’s board of directors to distribute the reward according to his best judgment.

Consequently, a situation in which one individual’s priority and share of the reward dominates the rest of the team closely resembles a model in which a single leader assumes full responsibility for reward distribution.

7.2. Limitations and Extensions

The presented solutions primarily provide procedures for the quantitative aggregation of weight vectors. Their unusual (unique) nature lies in the fact that the voting power of a given expert is directly dependent on the rating that they receive in the decision-making process. Other self-assessment models [18,19,20] usually assume that each participant in the process has constant, equal voting power. Similarly, in most studies focusing on weight aggregation functions, the aggregated values do not depend on the ranking results [53,54,55]. This distinctive feature has important implications for the applicability of the proposed approach in existing group decision-making frameworks.

On the one hand, the fundamental nature of the proposed solution may facilitate its integration with various other solutions; on the other hand, there may be particular difficulties in its direct application. Many decision-making methods use functions aggregating assessments from different experts. Often, these functions are various forms of weighted arithmetic or geometric means. Mohd and Abdullach [53] cite at least 25 studies that use these types of means in group decision-making methods. Of course, the mere fact that such averages are used to aggregate opinions does not mean that the AADAM or MADAM can be directly used in these solutions, i.e., that those methods could automatically become self-assessment techniques. However, this may be a good starting point for further research. These challenges become particularly evident in more realistic, technically complex decision-making settings.

In practice, many solutions are more complex and have various additional mechanisms. For example, if the source of the weight vectors is a pairwise comparison matrix, it undergoes inconsistency verification. If this check fails, the expert who prepared the set of comparisons is asked to revise their opinions to be more consistent, or their views are deemed unreliable and may be excluded from aggregation. In the literature, there are many papers describing various methods for determining the inconsistency of pairwise comparisons [30,31,56,57]. Similarly, it may occur that while experts are consistent enough, their weight vectors differ significantly. These differences may indeed result from discrepancies in how the problem is assessed. Still, they may also be attempts to manipulate the decision-making process or manifestations of uncooperative behavior. Many studies address the issue of weight vector aggregation in the case of manipulation risk [16,58,59]. Similarly, several studies aim to eliminate or reduce differences of opinion among experts by proposing a compromise (consensus) solution [12,60,61]. Although these works do not essentially concern mutual evaluation by experts, they contain many elements that could be used in the self-assessment process. One example is a simple procedure that eliminates extreme opinions, or an iterative approach in which the degree of consensus is assessed and, if it is too low, the assessment is repeated in whole or in part [60]. We believe that integrating the proposed aggregation procedures with frameworks that enable consensus to be reached or manipulative behavior to be detected is an interesting direction for further research.

8. Conclusions

In the paper, we propose ADAM, a new judgment aggregation model for group decision-making based on the pairwise comparison method, in which experts’ priorities directly influence the aggregation result. The model reflects situations where experts both evaluate others and are themselves evaluated. As an illustrative example, we consider the problem of distributing rewards for team achievements within a peer group, such as preparing and publishing a scientific article or reaching a milestone in a software engineering team.

Beyond its formal formulation, ADAM also admits a natural normative interpretation within the theory of injustice [49,50,51], according to which an individual’s contribution should be proportional to earnings. In the case of ADAM, this contribution (understood as a substitute for components such as work effort, commitment, position, education, etc.) is reflected in the priority assigned to each team member’s opinion. Under the above assumptions, the solution calculated by our model satisfies the condition of fair employee remuneration.

To operationalize these assumptions, we introduce two new priority aggregation methods within the ADAM framework: additive (AADAM) and multiplicative (MADAM). For AADAM, we demonstrated the existence of an admissible solution and a simple iterative method for computing it. While AADAM admits a straightforward computational procedure, MADAM requires a different approach. For MADAM, we propose a method for calculating the result based on an optimization model and an iterative algorithm. Although the latter cannot always find the right solution, Monte Carlo tests indicate that the chance of success increases with problem size. In the tests conducted for the MADAM method, we obtained the correct solution with the assumed numerical accuracy in each case. Both aggregation methods, AADAM and MADAM, satisfy the Pareto principle and the homogeneity condition.

Taken together, these results suggest that ADAM is not limited to the specific setting studied in this paper. Although our research focuses on group decision-making using the pairwise comparison method, the presented approach can be extended to many other decision-making frameworks. It would be sufficient if the input to the aggregation procedure were vectors of weights representing the positions of individual experts. The ADAM model can also have applications other than counterpart reviews of peer team members. For these reasons, we plan to focus on future research on extending the applicability of the proposed model to other decision-making methods and application areas.