- Article
In national research assessment, peer reviewers must collapse heterogeneous criteria into a single judgment, the source of what the literature calls commensuration bias. This paper integrates the absolute measurement variant of the Analytic Hierarchy Process (AHP) into Italy’s Research Quality Evaluation (VQR): pairwise comparisons derive panel-specific weights for three criteria (Originality, Methodology, Impact) and ten sub-criteria, while publications are scored directly on the sub-criteria and aggregated additively. The framework does not eliminate commensuration bias; it constrains and documents it at the aggregation stage, leaving a residual component in sub-criterion scoring. The framework also includes a ranking stability audit, a Monte Carlo procedure assessing how sensitive results are to weighting and scoring uncertainty. In an illustrative simulation of 20 publications, explicit weighting shifts 15 publications relative to an unweighted mean (1.25 positions on average across the 20 publications). Under the specified perturbations, rankings are stable when consistent pairwise judgments are perturbed (mean Spearman ρ ≥ 0.97) and more sensitive to scoring noise (mean ρ down to 0.81), but weight uncertainty gains importance as sub-criterion profiles become less correlated. The study is a proof of concept; empirical validation with real VQR data is left to future research.
11 November 2026





