Innovative Indicator-Based Support Tools for High-Quality Participation in Disaster Risk Management and Urban Resilience Building

Abstract

Despite broad consensus on the importance of participatory processes in disaster risk management and urban resilience building, substantial gaps persist, including scarce research on monitoring and evaluating participation, lack of comparative studies, underexplored policy and institutional roles. The paper provides methodological and empirical insights by developing and validating two indicator-based tools: one for ex ante assessment of institutional capacity and the other for supporting monitoring and ex post evaluation of participatory processes. The paper also tests them through a comparative study employing a standardizable and reproducible methodology and synthesizes findings from a systematic review of case studies and a semi-systematic review of grey literature to compile a comprehensive pool of criteria and indicators. These are screened, assigned a weight (either by Equal Weight or Best Worst Method) and are aggregated in the two innovative tools mentioned above. These are tested on four case studies: recent local-scale participatory processes aimed at reducing disaster risk and promoting urban resilience addressing multi-hazard scenarios. The research quali-quantitatively demonstrates how, in the four case studies, greater institutional capacity turns into a higher-quality participatory process. Furthermore, the paper improves practical knowledge on participatory processes in disaster risk management and urban resilience building and lays the foundation for evidence-based innovative guidelines for their planning a priori.

1. Introduction

Contemporary urban planning and design are shaped by a multitude of challenges including natural, anthropogenic, and climate-related risks; constrained resources; crises of political representation and declining trust in institutions; and persistent social and material inequalities. In this context, the adoption of adaptable and iterative participatory governance models in master planning, disaster risk management (DRM) and urban resilience building (URB) shall be encouraged. DRM is a comprehensive process aimed at preventing new hazards, reducing existing vulnerabilities, and managing residual risks to minimize disaster losses and ultimately strengthen resilience [1]. URB enhances the capacity of urban systems to anticipate, absorb, adapt to, and recover from multiple, interconnected threats while sustaining and reinforcing core urban functions [2]. Whereas DRM primarily focuses on risk-specific interventions and policy measures, URB emphasizes the transformative potential of urban systems in response to complex, multi-scalar challenges. The paper examines DRM and URB in tandem under the umbrella of participatory governance due to their overlapping institutional arrangements, stakeholder networks, and policy processes. Both domains operate within polycentric and multilayered governance structures, comprising public institutions with overlapping jurisdictions and vertically integrated systems that connect community-level actors to regional, national, and international scales [3]. They both demand flexible, multi-level platforms that foster collaboration among diverse stakeholders, including policymakers, public agencies at different level, scientific organizations, practitioners, risk insurance companies, non-governmental organizations, local communities, vulnerable social groups, etc. [4]. Furthermore, recent studies indicate a convergence between previously distinct DRM and URB policy regimes, as contemporary urban resilience frameworks integrate DRM considerations into land-use planning, hazard mapping, and the regulation of critical infrastructure [5].

The call for participatory governance models in DRM and URB is firmly embedded within international policy frameworks. Beginning with the Rio Declaration [6] and subsequently reaffirmed in key global instruments such as the Paris Agreement [7], the 2030 Agenda for Sustainable Development [8], the Sendai Framework for Disaster Risk Reduction [9], and the New Urban Agenda [10], the principle of inclusive participation is recognized as a cornerstone for advancing equitable, context-sensitive, and resilient development trajectories. This approach is also substantiated by a growing body of research across the disciplines of urban planning, sociology, political science, and economics. Empirical evidence demonstrates that traditional top-down and techno-centric approaches to DRM and URB often fail to achieve sustainable outcomes, as they tend to marginalize stakeholder needs, knowledge, and agency [11,12,13,14].

Compared to traditional top-down approaches, participation involves the activation of processes that include individuals, groups and organizations who choose to take an active role in making decisions that affect them [15]. There is, as yet, no single universal definition. The aforementioned multidisciplinary proliferation has generated a wide array of synonyms and conceptualizations, varying across academic, disciplinary, professional, and geographic contexts. Rather, public participation is generally understood as an umbrella term encompassing a broad spectrum of participatory planning approaches documented in both academic and grey literature [16]. At one end are highly institutionalized forms of engagement—often superficial or formalistic—primarily intended to secure public consensus and preserve the status quo. At the other end are autonomous, frequently contentious, bottom-up initiatives that seek to redistribute resources and transform the use of space, services, and public assets [17]. Between these two poles lies a continuum of intermediate practices characterized by dialogue between institutions and local actors, displaying varying degrees of formality and experimentation with more inclusive and participatory governance approaches. This paper focuses primarily on the latter, specifically as it manifests in participatory processes. Participation is grounded in the promise of enhanced democratic legitimacy and more effective decision-making [18]. It offers numerous benefits, including the integration of local knowledge, the promotion of social learning and the cultivation of shared responsibility in context of scarce services and resources [19]. Particularly in the field of DRM and URB, research highlights the importance of participatory approaches in overcoming fragmented, silo-based decision-making models. Such approaches foster trust and collaboration among stakeholders [20], which are essential for effective policy implementation and the efficient allocation of resources. Clearly, participation depends on the quality of the process, and, for this reason, some scholars have proposed different frameworks: the ladder [21], the wheel [22], the spectrum [23], the tree of participation [24], etc. However, numerous aspects remain insufficiently explored, preventing the full realization of the promises of participation [25]. In this regard, research should move beyond viewing participation as a panacea [26] and instead concentrate on addressing the existing gaps in knowledge and practice.

Research Gaps and Rationale for This Paper

As evidenced in the most prominent and recent (since 2019 onwards) literature reviews on participatory planning in DRM and URB (see Supplementary S1 in Supplementary Materials to view the meta-analysis and, therefore, the pool of reviews mentioned above), significant gaps persist in both understanding and implementing participation. These, which are examined in more depth later in this section, can be categorized as empirical (e.g., limited experimentation with innovative frameworks and a lack of comparative studies), methodological (e.g., absence of monitoring and evaluation frameworks and lack of standardized approaches to documenting and disseminating participation), and political/institutional (e.g., insufficient investigation into the role of policy frameworks and institutional capacities).

Several authors have underscored the need to broaden the scientific debate concerning the application of new models and concepts of collaboration and partnership [27]. Empirical evidence indicates that experimentation with participatory mechanisms in DRM and URB remains relatively underdeveloped, particularly in relation to adaptation rather than mitigation strategies [28]. Most contemporary studies focus primarily on informing or consulting stakeholders [29]. However, more interventionist approaches—such as power delegation and citizen control—are required to foster knowledge co-production and enable transformative changes in socioecological systems [30]. Furthermore, only a limited number of participatory frameworks in DRM and URB currently embrace a multidisciplinary perspective, wherein diverse knowledge domains contribute collaboratively to the same planning objective [31], aligning efforts to achieve social, environmental, and distributive justice [32]. Yet, communication across disciplines, and especially between science and policy, remains largely ineffective. For instance, there is still no consensus on the interpretation and operationalization of core concepts such as risk, vulnerability, and adaptive capacity [33]. Similarly, participatory frameworks seldom incorporate systemic and multi-hazard perspectives [34]. Greater efforts are also needed to design intergenerational and inclusive platforms for participation that explicitly account for cultural sensitivities (e.g., non-discrimination) [35], power asymmetries [30], and skill disparities (e.g., the digital divide) [36]. From this standpoint, it becomes crucial to recognize the multidimensional and layered nature of local communities [37].

Few studies have sought to monitor and evaluate participatory processes in practice. Most papers addressing participation in DRM and URB rely on narrative accounts that describe experiences and lessons learned [38]. When monitoring or evaluation does occur, it typically involves qualitative discussions and interviews, often without the support of specific or validated evaluation frameworks [29]. Even existing frameworks do not generally provide clear sets of criteria and indicators [39,40]. By contrast, they tend to offer checklists addressing discrete aspects of participation in isolation. This lack of systematic monitoring and evaluation hampers the ability to determine whether and how participatory planning fosters transformative and ambitious actions [41], at the levels of public policy, community practice [33], and social outcomes [42].

Current research also offers limited guidance on the systematic documentation and large-scale dissemination of participatory experiences [37]. Many studies selectively report data, thereby constraining future reviews and comparative analyses (e.g., participant representativeness is often unspecified) [32]. A major obstacle lies in the absence of a standardized, flexible, and well-defined set of categories for data collection and reporting [43].

Closely related to the lack of monitoring and evaluation frameworks, and the absence of standardized documentation practices, is the scarcity of systematic or comparative empirical studies. This deficiency undermines the ability to comprehensively identify the enablers of participation management, the barriers encountered, and the strategies to overcome them [35,44]. Consequently, it also hinders the identification of effective methods, techniques, and tools for analyzing local contexts and fostering a culture of participation [34].

Finally, only a limited number of studies examine the policy-related determinants of participation in depth [45]. Existing research consistently call for the mainstreaming of participation, emphasizing two key priorities: first, the need for a more structured understanding of how stakeholder participation is incorporated into grey literature and policy documents [33]; and second, the establishment of legislative arrangements and participation standards that place participation at the core of planning processes, rather than treating it as a subsidiary dimension [35]. Moreover, research concerning governance models and institutional capacities (including political, administrative, and technical dimensions) that enable actor engagement [28] and underpin the realization of win–win participatory models remains fragmented [46].

Overall, research remains limited in prioritizing the creation of indicator-based frameworks and tools that can support decision-makers in the effective implementation of participatory processes, contributing to more resilient and socially equitable urban transformations.

In light of this background, the paper intends to contribute to the scholarly debate on stakeholder participation in DRM and URB by addressing the following research questions:

• RQ1: Which criteria and indicators are most appropriate for the ex ante assessment of institutional capacity to design and manage participatory processes in DRM and URB?
• RQ2: What criteria and indicators should be evaluated in progress and ex post to determine the quality or otherwise of participatory processes in DRM and URB?
• RQ3: What is the relationship between institutional capacity and the quality of participation?
• RQ4: How and why participation can or cannot lead the decision-making process toward adopting transformative strategies in DRM and URB?

Through the analysis of multiple case studies based on a standardizable and reproducible methodology, the paper aims to achieve the following research objectives:

• Developing and validating two novel, indicator-based tools: one designed for the ex ante assessment of institutional capacity, and the other intended to support monitoring and ex post evaluation of participation in DRM and URB.
• Testing the relationship between institutional capacity and participatory processes quality.
• Defining future research trajectories.

The rest of the paper is structured as follows: Section 2 contextualizes and describes the methodology used in this research. Section 3 applies the methodology to four case studies: recent (since 2019 onwards) participatory processes at a local scale aimed at managing disaster risk and building urban resilience, dealing with multi-hazard scenarios (natural, anthropogenic and climatological hazards). Section 4 discusses the main results of the research. In conclusion, Section 5 draws the main conclusions of the research, highlights key limitations and opportunities for further improvement.

2. Materials and Methods

The methodology is fundamentally organized into five principal phases (Figure 1): 1_Performing a systematic literature review (LR) of case studies to identify the main barriers and enablers of high-quality participation, and the variables relevant to monitoring and evaluating participatory processes; 2_Integrating the above findings with the main grey literature on the topic (since 2019 onwards to ensure that the findings are relevant and up-to-date); 3a_Establishing an indicator-based framework for assessing the institutional capacity for managing participatory processes in DRM and URB; 3b_Establishing an indicator-based framework for monitoring and evaluating participatory processes in DRM and URB; 4a_Developing an ex ante assessment tool for the institutional capacity to manage participatory processes in DRM and URB; 4b_Developing a support tool for monitoring and evaluating participatory processes in DRM and URB; 5_Testing the two novel tools on multiple case studies.

The following subsections provide a detailed explanation of each phase.

2.1. Conducting the Systematic Case Study LR and the Grey Literature LR (PHASES 1, 2)

The systematic case study LR is guided by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology [47], which ensures a rigorous and transparent procedure despite the large volume of data to be collected. The protocol consists of four steps-identification, screening, eligibility, and inclusion-through which relevant literature is selected in accordance with the research’s objectives. Two major citation databases, Scopus and Web of Science, are employed for the search process, while papers indexed exclusively in other databases are excluded. The following search string is applied: (“participat*” OR “engag*” OR “involv*” OR “co-design*” OR “collaborat*” OR “co-product*”) AND (“stakeholder” OR “public” OR “communit*” OR “citiz*” OR “expert”) AND (“urban resilience” OR “resilient urban syste*” OR “resilient cit*” OR “resilience in cities” OR “city resilience”) in title, abstract and keywords.

Table 1. Eligibility and exclusion criteria underlying the review protocol.

Criterion	Eligibility	Elimination
Document type	Article	Review article | Proceeding paper | Book chapter | Book | Abstract | Short survey | Note | Editorial | Letter
Language	English	non-English
Coverage	Whole world	Dealing with informal settlements
Topic	Participation in DRM and URB	No consideration about participation in DRM and URB
Methodology	Case study	non-Case study
Publication stage	Published	In press
Distribution	Open access	non-Open access
Journal ranking	Q1	non-Q1

presents the main eligibility and exclusion criteria that guide the selection of literature included in this review.

The LR intends to analyze scientific papers that clearly report case studies of participation in the field of DRM and URB in structured urban areas (informal settlements are not the object of the study). Eligible papers discuss at least the objectives set, the study area and scale involved, the methods and techniques used, the actors engaged, and the results achieved. Such papers are accessible (open access) and of high technical-methodological quality (a criterion virtually guaranteed by including in the research only papers published in Q1 journals).

Grey literature LR, on the contrary, occurs in a semi-systematic way [48]. Its purpose is to map which scientific findings have been incorporated into national and international guidelines, reports, white papers, regulatory documents, toolkits, publications by NGOs, etc. Therefore, it aims at gathering what is now considered good practice and/or benchmarks by national and international bodies. The collection of such literature is done through various sources: the knowledge of experts on the topic; consultation of national and international organization websites; access to mini sites of publicly funded governance or research projects; snowballing [49]. Eligible documents are published since 2019 onwards (guaranteeing the inclusion of recent and up-to-date documents relative to the post-2015 literature, following the publication of the Sendai Framework for Disaster Risk Reduction [9]). They address the topic of public participation in DRM and URB and demonstrate a certain level of methodological rigour (qualitative aspects). They are originated from recognized single institutions or networks of national or international significance (see Supplementary S2 in Supplementary Materials to view the lists of the most significant grey literature documents from 2019 onwards, catalogued by document type: guidelines or toolkit).

At this stage of the research, data from both the systematic case study LR and the grey literature LR, covering barriers, enablers and variables relevant to participatory process monitoring and evaluation, are extracted and compiled into dedicated databases (spreadsheets) in an unstructured way. The entries are sorted solely by the main author and publication date.

2.2. Establishing a Framework of Indicators for the Ex Ante Assessment of Institutional Capacity to Manage the Participatory Process in DRM and URB (PHASE 3a)

The term capacity refers to the strengths, attributes and resources available within an organization, community or society to manage and reduce disaster risks and strengthen resilience [50]. Institutional capacity is defined as “the ability of administrative and government organizations and agencies to respond to and manage current social and environmental challenges through decision-making, planning and implementation processes” [51]. The literature argues that providing and improving capacity and resources in local public bodies is associated with achieving results in public policy development, strengthening support (political, social and economic) for risk management measures and URB [52]. Certainly, local institutional capacities are only one of the contributing factors, among others. However, they can drive toward deep sustainable and resilient transformations, based on innovative and more radical norms and visions of the future [53].

From both the systematic case studies LR and the grey literature LR, all identified barriers and enablers that may fall within the broader notion of institutional capacity for managing participatory processes aimed at DRM and URB are extracted. All extracted data are stored in a database, organized according to the author of each source. Synonymous and homonymous barriers and enablers are subsequently merged, with linguistic variations (e.g., suffixes, verb tenses) standardized for consistency. Redundancies are then addressed through a categorization process, whereby each barrier or enabler is assigned to a set of superordinate criteria. These criteria are informed by established literature and, where necessary, by the research team’s expert judgement. Although no studies specifically examine institutional capacity for managing participatory processes in the context of DRM and URB, several existing frameworks address institutional capacities for DRM and URB more broadly, incorporating participation and the promotion of collaborative or participatory governance as one of several key dimensions. Some authors have defined institutional capacity using the following criteria: knowledge resources; relational resources; mobilization capacity [54]. Subsequently, the adaptive capacity wheel [55], one of the most widespread models in the field of DRM and URB, identifies the following dimensions: variety; learning capacity; room for autonomous change; leadership; resources; fair governance. A more recent study tests a framework for assessing institutional capacity for promoting sustainable development and mitigating/adapting to climate change based on strategic or leadership capacity; analytical and data management capacity; organizational management capacity; collaborative or network management capacity. The authors suggest future research should also consider variables such as leadership, political will, technical expertise or social support for the definition of urban policies [56]. Categorizing barriers and enablers facilitates both the refinement of existing conceptual understandings and the generation of new labels. Then, barriers and enablers with vague or ambiguous wording are excluded. barriers and enablers characterized by vague or ambiguous wording are excluded. At this stage, the remaining barriers and enablers are translated into preliminary indicators (to be further refined, as discussed later) through the completion of a specific matrix comprising the following fields: indicator name, description, corresponding superordinate criterion, and proposed measurement method. For instance, the indicator “place attachment”—defined as the emotional bond between an individual and a specific place—falls under the criterion “socio-environmental variables” and can be measured through proxy such as the willingness to pay to support the management of that place itself [57]. This process aims to ensure both the internal coherence of the indicator set and its long-term sustainability [58].

A screening phase of indicators is then initiated in alignment with the overall research objectives. The full set of indicators undergoes a rapid assessment designed to evaluate each indicator’s relevance to the research goal (i.e., managing participation in DRM and URB), its applicability at the local scale, and the clarity and comprehensibility of its formulation. Each individual indicator is assigned a score from 0 (the indicator is not relevant, not applicable at the local scale, not understandable and unclearly formulated) to 4 (the indicator is relevant to the research objective, applicable at the local scale, understandable and clearly formulated). This procedure is inspired by established methodologies in the literature [59]. To enhance consistency and reduce subjectivity in the scoring process, multiple reviewers independently assess the indicators, and any discrepancies are resolved through discussion and consensus. Through a recursive process involving iterative revisions of the indicator set, a final shortlist is developed. As recommended by some authors [60], this final shortlist of indicators is then analyzed as a whole to ensure the absence of overlap, verifying that each indicator captures a distinct aspect without duplicating information.

2.3. Establishing a Framework of Indicators for Monitoring and Ex Post Evaluation Participatory Processes in DRM and URB (PHASE 3b)

Public participation is inherently difficult to measure with precision. First, it encompasses a very broad and multifaceted set of practices, serving as an umbrella term that includes a wide variety of experience. Moreover, participatory processes are highly susceptible to the influence of environmental, psychosocial, cultural, demographic, and economic confounding variables [61]. The challenge is further compounded by the general absence of a monitoring, evaluation, and learning culture within public institutions. Nevertheless, systematically monitoring and evaluating participation could help render it a truly “public matter,” thereby reducing potential manipulation by administrations. Such practices would require public authorities to clarify their expectations and be fully transparent in their intentions, while encouraging civil society to engage collaboratively and make reasonable demands [62]. Traditionally, research efforts aimed at developing models for monitoring and evaluating participation have essentially focused on:

• Characteristics of the participatory process [63], such as the context of reference, the objectives, the roles of the actors, the selection of participants, the size of the group of participants, their knowledge and expectations, the duration, the degree of turnover, the ways of use, the degree of participation and the methods and techniques used.
• Participatory procedures, such as fairness, legitimacy, effectiveness, efficiency, satisfaction [64], as well as inclusiveness, information quality and learning, influence on political decisions [65,66].
• Outputs, such as policy outputs, perceived expected/unexpected outcomes in social, economic and inter-institutional fields [67], interpersonal trust, common understanding and attitude towards the process [68].

To develop a support tool that integrates, in an organized yet streamlined manner, the characteristics, procedural features, and outputs of participatory processes, the barriers, enablers, and variables relevant to monitoring and evaluating participation are initially collected from the case study systematic LR and the grey literature LR, respectively. These data are compiled into a single database and, following the methodology described in Section 2.1, are: categorized according to superordinate criteria identified in the literature or defined by the research team; and independently screened by multiple reviewers based on their relevance to the research objectives, applicability at the local scale, and clarity and comprehensibility, using a scoring procedure ranging from 0 to 4. Once a set of indicators is established, it undergoes iterative cyclical review and is analyzed holistically to ensure that all indicators are conceptually distinct. Finally, each indicator in the finalized shortlist is entered into a matrix similar to that described in Section 2.2, which includes the following fields: indicator name, description, corresponding superordinate criterion, and measurement method.

2.4. Developing Innovative Indicator-Based Tool (PHASE 4a, 4b)

The research aims to develop and validate two innovative indicator-based tools for use by researchers and practitioners in the fields of DRM and URB. The ex ante institutional capacity assessment tool, to be employed before a participatory process commences, is designed to equip institutions—acting as initiators of participatory processes for urban planning purposes—with a comprehensive analysis of the current state of affairs. This enables institutions to calibrate realistic participation ambitions, define clear objectives, and outline the essential procedural aspects of the process. Indeed, evidence indicates that weak institutional capacity can compromise the quality of participation. The support tool for monitoring and ex post evaluating (designed for deployment during and after the execution of a participatory process) facilitates the measurement and interpretation of ongoing participatory processes, enabling the identification of factors contributing to failure and the implementation of mitigation strategies in areas such as capacity building, knowledge capitalization, and innovation. It can also be used to understand the reasons behind and the scope of participatory process outcomes. When employed for comparative analyses, the tool enables the identification of good practices. In addition, it serves as an effective means of communicating territorial performance to key stakeholders, including potential initiators of participatory processes, national and international agencies, external evaluators, and the scientific community.

Following the screening process described in Section 2.2 and Section 2.3 which produces a final shortlist of indicators, these are subjected to a weighting procedure. The weights assigned to institutional capacity indicators reflect the relative importance of different dimensions in contributing to the performance of an institutional body responsible for instigating and managing participatory process in DRM and URB. Similarly, the weights assigned to indicators for monitoring and evaluating participatory processes indicate the relative significance of various dimensions in determining the quality of a participatory process in DRM and URB. No or Equal weights, Budget Allocation Process, Analytical Hierarchy Process (AHP), Conjoint Analysis, Multiple Linear Regression Analysis, Principal Component Analysis, Factor Analysis, Data Envelopment Analysis are just some of the weighting approaches in existence. There is no “one-size-fits-all” solution and no weighting system is free from criticism. The choice of methodology is left to researchers and practitioners, depending on the specific objectives and context of the study. In this paper, different and specific strands of reasoning are outlined for each innovative indicator-based tool. For the ex ante assessment tool of institutional capacity, all indicators are assigned equal weight. Given that the tool is meant to be used a priori of participatory processes in DRM and URB, and that it cannot be assumed that institutions have specific expertise in on the topic, Equal Weight is the optimal system [69]. This method is the most transparent (allowing any stakeholder to understand how indicators are developed) and the simplest to calculate and communicate. In contrast, for the support tool for monitoring and evaluating participatory processes, quantitative weight for each indicator are determined using the Best-Worst Method (BWM), one of the most recent multi-criteria decision making (MCDM) approaches based on two vectors of pairwise comparisons [70]. The choice of BWM is justified by several factors: it effectively manages the complexity inherent in the measuring participation, which involves a large number of variables; it reduces inconsistencies and guarantees reliability in the results; it requires less cognitive effort from experts and stakeholders by minimizing the number of pairwise comparisons (less than AHP); and it has based on a robust, simple and computationally efficient algorithm [71,72]. BWM has been successfully applied in various DRM and URB contexts, including supporting the selection of optimal sites for recovery shelters [73], identifying suitable locations for new emergency facilities [74], mapping landslide susceptibility [75], assessing district-scale resilience and prioritizing urban regeneration in neighbourhoods [76] and so on. Despite these promising applications, BWM remains relatively understudied. To the best of the authors’ knowledge, no prior studies have applied it to develop a tool for monitoring and evaluating participatory processes [77].

The BWM procedure involves the following steps [78]. First, it is necessary to determine a set of criteria (C_n) and sub-criteria (C_nm). In this paper, they are the criteria and indicators for monitoring and evaluating participatory processes, respectively. n experts and stakeholders are involved via an online questionnaire in determining the most (B) and the least (W) important and desirable criterion based on their own knowledge and experience, amongst the m criteria. The questionnaire happens to be the most suitable participatory method to conduct the survey. Within this research, the questionnaire is administrated remotely via LimeSurvey GmbH 6.12.0+250310. Once this step has been completed, experts and stakeholders are then asked to evaluate the preference of the B criterion over the other criteria using a numerical scale where 1 = equal preference and 9 = extreme preference of B over the other criteria. By doing so, the result is a Best-to-Others vector (1):

$$A_{Bi}^k=(a_{B1}^k, a_{B2}^k, \dots , a_{Bm}^k)$$

(1)

$a_{Bi}^k$ is the preference of the expert/stakeholder k for the B criterion over i criterion, i = 1, 2, … n. Therefore, the same n experts and stakeholders are engaged in giving their judgement on the preference of the criteria over the W based on numerical scale where 1 = equal preference and 9 = extreme preference of the C_n criterion over W. This procedure produces the Others-to-Worst vector (2):

$$A_{Wi}^k=(a_{W1}^k, a_{W2}^k, \dots , a_{Wm}^k)$$

(2)

$a_{iW}^k$ is the preference of the expert/stakeholder k for the i criterion over the W criterion, i = 1, 2, … n.

Table 2. The evaluation scale for pairwise comparison in BWM.

Best-to-Others Evaluation		Others-to-Worst Evaluation
Scores	Linguistic Terms	Scores	Linguistic Terms
1	B and Cn are equally important	1	Cn and W are equally important
2	Intermediate evaluation	2	Intermediate evaluation
3	B is moderately more important than Cn	3	Cn is moderately more important than W
4	Intermediate evaluation	4	Intermediate evaluation
5	B is strongly more important than Cn	5	Cn is strongly more important than W
6	Intermediate evaluation	6	Intermediate evaluation
7	B is very strongly more important than Cn	7	Cn is very strongly more important than W
8	Intermediate evaluation	8	Intermediate evaluation
9	B is extremely more important than Cn	9	Cn is extremely more important than W

conveys the evaluation scale for pairwise comparison of B criterion over criteria and the latter over W criterion.

At this point of the research, it is possible to calculate the optimal weights for each criterion (w₁ * w₂ * … * w_i) through the linear model proposed by Rezaei [79]. Such optimal criterion weights must satisfy the following requirements: for each pair of w_B/w_i and w_i/w_W, the ideal situation is that w_B/w_i = a_Bi and w_i/w_W = a_iW (w_B is the weight of the B criterion; w_i is the weight of the i criterion; w_W is the weight of the W criterion). Aiming to get as close as possible to the ideal situation, we should minimize the maximum in the set of {|w_B − a_Biw_i|,|w_i − a_iWw_W|}. Below is a representation of this problem (3):

$${min max}_i\left\{\left|w_B-a_{Bi}{w}_{Wi}\right|, \left|w_i-a_{iW}{w}_W\right|\right\}, s.t. {\sum }_i{w}_i=1\left(1\right), w_i\ge 0 for all i$$

(3)

which can be solved by the following linear programming problem (4):

$$\min {\xi }^L s.t. \left|w_B-a_{Bi}{w}_{Wi}\right|\le {\xi }^L for all i$$

(4)

$$\left|w_i-a_{iW}{w}_W\right|\le {\xi }^L for all i$$

$${\sum }_i{w}_i=1\left(2\right), w_i\ge 0 for all i$$

The results of solving (4) are the optimal weights and ξ^L, i.e., the maximum acceptable deviation between the observed and ideal weight ratios (w_B/w_i = a_Bi and w_i/w_W = a_iW), thereby measuring the degree of inconsistency in the pairwise comparisons among the criteria. ξ^L is essential to compute a consistency check according to the following Formula (5):

$$CR={\displaystyle \frac{{\xi }^L}{CI}}$$

(5)

CR is the consistency ratio; it has a range of (0,1). Values approaching 0 indicate higher levels of consistency and reliability in the pairwise comparisons (inconsistent judgments result in the exclusion of the expert or stakeholder from the sample). CI is the consistency index, i.e., the maximum possible ξ as derived from predefined tables (

Table 3. Consistency Index table.

aBW	2	3	4	5	6	7	8	9
CI	0.44	1.00	1.63	2.30	3.00	3.73	4.47	5.23

where a_BW denotes the preference of the B criterion over the W criterion.

).

In line with the studies available in the literature [80], the final weight value of each criterion reflects the arithmetic mean of the weights calculated based on the inputs received from the n experts and stakeholders separately.

The same procedure described so far is also applied to the C_nm sub-criteria pertaining to each criterion. In this case, the local final weight (i.e., that depends on the inputs of the k expert/stakeholder) is the average value of the weights of all the experts/stakeholders; the global final weight of each C_nm is the result of the multiplication of the local weight and the final weight of the pertaining criterion [81].

The weight assigned to each indicator is meant to be multiplied to the value of each indicator itself. But be careful: indicators might be both quantitative and qualitative; normalization is essential to get everything consistent. Normalization is processed as the following Formula (6):

$$i\prime ={\displaystyle \frac{i-\mathrm{m}\mathrm{i}\mathrm{n}\left(i\right)}{\max \left(i\right)-\mathrm{m}\mathrm{i}\mathrm{n}\left(i\right)}}$$

(6)

where i′ represents the normalized value of the indicator i.

There are also many aggregation methods. Some authors defined three categories thereof: compensatory (additive and geometric methods), non-compensatory (ELECTRE and PROMETHEE methods) and mixed methods (MPI, Penalty of the Bottleneck, Mean-Min Function, ZD model, Directional Benefit-of-the-Doubt methods) [82]. In the context of this paper, the weighted geometric mean is the method used (7):

$$I={\prod }_{i=1}^m{I}_i^{\omega i}$$

(7)

The term I is the ultimate composite index, ωi the weight of the i indicator, and I_i the normalized score of the i indicator. Although it is considered a compensatory method, the weighted geometric mean limits the compensability within certain limits (indicators with very low scores cannot be completely compensated by indicators with high scores) [83]. Once it is calculated, the index score is assigned to an appropriate category, which are defined below for both the index of institutional capacity in managing participatory processes and the index of high-quality participation in DRM and URB (

Table 4. Score categories of the defined indices.

Institutional Capacity Index		High-Quality Participation Index
Scores	Categories	Scores	Categories
0 < I ≤ 0.2	Struggling	0 < I ≤ 0.2	Very Low
0.2 < I ≤ 0.4	Surviving	0.2 < I ≤ 0.4	Low
0.4 < I ≤ 0.6	Stabilizing	0.4 < I ≤ 0.6	Medium
0.6 < I ≤ 0.8	Heading	0.6 < I ≤ 0.8	High
0.8 < I ≤ 1	Flowing	0.8 < I ≤ 1	Very High

).

2.5. Testing on Multiple Case Studies (PHASE 5)

The research aims to apply the ex ante assessment tool of institutional capacity for the management of participatory processes in DRM and URB, as well as the support tool for monitoring and ex post evaluating the participatory process already completed, to multiple case studies. Case study selection is based on the following criteria. First, the operational setting: the focus is on participatory processes at the local scale aimed at disaster risk management and urban resilience, addressing multi-hazard scenarios, including natural, anthropogenic, and climatological hazards. Second, the processes are relatively recent, since 2019 onwards. This time threshold is chosen to exclude processes based on outdated models, frameworks, methods and techniques, and to include processes that necessarily addressed the COVID-19 health emergency and its consequences across social, cultural, methodological, scientific and economic dimensions. Additionally, this period ensures the availability of high-quality data, particularly when involving participating stakeholders, reducing cognitive biases related to memory. Third, since triangulation is essential when performing a summative evaluation [84], combining multiple data collection methods—such as field notes, ethnographic observation, primary and secondary document analysis, open-ended interviews, and closed-ended survey questions—the selected processes include those in which the evaluators (i.e., the authors of this paper) have been directly or indirectly involved. Finally, as a highly innovative aspect of this research, at least three case studies are considered, including at least two processes initiated by the same public institution at different times. The design allows for a comparative analysis of institutional capacity in managing participation in DRM and URB through repeated measures—a research direction that, to the best of the authors’ knowledge, has not been previously explored. Consequently, this approach enables both the comparison of how the capacities of different institutions influence high-quality participation and the monitoring of the same institution’s capacity across different periods, including the objectives it sets and the effectiveness of implemented participatory processes.

In this paper, the relationship between institutional capacity and high-quality participation is analyzed using a qualitative approach. Figure 2 presents a graphical framework, including a template and an example, to describe and represent the results emerging from the assessment of institutional capacity, the evaluation of participatory process effectiveness, and their interrelationship.

3. Results

In this section, the application of the methodology is described in detail. The discussion and the conclusions of the research are addressed in Section 4 and Section 5, respectively.

3.1. The Systematic Case Study LR and the Grey Literature LR (PHASES 1, 2)

Applying the research protocol for the systematic case studies LR (Section 2), an initial set of 1232 papers was gathered from Web of Science and Scopus. 28 were excluded because not being written in English, 304 were removed due to document type, 334 were identified as duplicates, and one was withdrawn from publication. The titles, keywords, and abstracts of all remaining papers were reviewed, and those unrelated to participation in DRM and URB were excluded. Additionally, 40 papers could not be retrieved, as they were not openly accessible and were therefore eliminated. A full-text analysis was conducted on the remaining 120 papers, leading to further exclusions: 19 focused on case studies related to informal settlements, which fell outside the scope of this study; 9 were not actually case studies; and 19 were not published in Q1 journals, ensuring a quality standard for inclusion. After this selection process, 73 papers are identified as relevant for the case study systematic LR. The selection process is detailed in Figure 3.

According to the methodology (Section 2.1), from the 73 papers identified and included in the systematic case study LR (see Supplementary S3 in Supplementary Materials to view the lists of the case study papers included in the review), the barriers and enablers to high-quality participation, as well as the variables relevant for the monitoring and evaluation of participatory processes, were extracted and listed in a spreadsheet. These data were integrated with what emerges from the grey literature (Supplementary S2 in Supplementary Materials).

3.2. An Indicator-Based Tool for the Ex Ante Assessment of Institutional Capacity (PHASES 3a, 4a)

Barriers and enablers extracted from both the systematic case study literature review and the grey literature review—on the basis that they may fall within the concept of institutional capacity for managing participatory processes in DRM and URB—were subjected to the following procedure: (1) merging in cases of redundancy, (2) classification according to superordinate criteria, and (3) exclusion if the wording was vague or ambiguous. With regard to the superordinate criteria, these were identified through two main approaches: first, by coining labels based on the knowledge and expertise of the researchers involved in the study, following a bottom-up approach; and second, by drawing on concepts established in the relevant scientific literature. The studies that informed this phase of the work are listed in

Table 5. Recent literature (last 5 years) on institutional and governance capacity in DRM and URB.

Authors	Year	Title
Brockhoff et al.	2019	Pluvial Flooding in Utrecht: On Its Way to a Flood-Proof City	[85]
Holscher et al.	2019	Steering transformations under climate change: capacities for transformative climate governance and the case of Rotterdam, the Netherlands	[86]
Hernantes et al.	2019	Towards resilient cities: A maturity model for operationalizing resilience	[87]
Labaka et al.	2019	Defining the roadmap towards city resilience	[88]
Marana et al.	2019	Towards a resilience management guideline—Cities as a starting point for societal resilience	[89]
Romano et al.	2019	Water Governance in Cities: Current Trends and Future Challenges	[90]
Shah et al.	2019	Current capacities, preparedness and needs of local institutions in dealing with disaster risk reduction in Khyber Pakhtunkhwa, Pakistan	[91]
Chang et al.	2020	Evaluation on the integrated water resources management in China’s major cities—Based on City Blueprint Approach	[92]
Marana et al.	2020	We need them all: development of a public private people partnership to support a city resilience building process	[93]
Fastiggi et al.	2021	Governing urban resilience: Organisational structures and coordination strategies in 20 North American city governments	[94]
Poland et al.	2021	A Connected Community Approach: Citizens and Formal Institutions Working Together to Build Community-Centred Resilience	[95]
Algaba et al.	2022	Assessment and Actions to Support Integrated Water Resources Management of Seville (Spain)	[96]
Aguilar et al.	2022	Governance challenges and opportunities for implementing resource recovery from organic waste streams in urban areas of Latin America: insights from Chía, Colombia	[97]
Glaas et al.	2022	Disentangling municipal capacities for citizen participation in transformative climate adaptation	[98]
Jukic et al.	2022	Organizational maturity for co-creation: Towards a multi-attribute decision support model for public organizations	[99]
Ejigu et al.	2023	Integrating resource oriented sanitation technologies with urban agriculture in developing countries: measuring the governance capacity of Arba Minch City, Ethiopia	[100]
Sharma	2023	Civil society organizations’ institutional climate capacity for community-based conservation projects: Characteristics, factors and issues	[101]
Edwards et al.	2024	Achieving net zero neighborhoods: A case study review of circular economy initiatives for South Wales	[102]

.

This procedure produced a set of 49 barriers and 54 facilitators, which were subsequently transformed into indicators using a structured matrix comprising the following fields: name, description, superordinate criterion of affiliation, and measurement method. The resulting pool of indicators and associated criteria was then subjected to the screening procedure described in Section 2.2, yielding a final shortlist of 20 conceptually discrete indicators and four superordinate criteria. The shortlist of indicators for assessing institutional capacity in the management of participatory processes in DRM and URB is presented in Figure 4.

The ex ante assessment tool for institutional capacity is designed as a self-assessment checklist for several reasons. First, the checklist—structured as a series of simple, qualitative controls—facilitates use through its immediacy and concreteness, reducing the risk of interpretation errors and enhancing the reliability of the results obtained. Second, it emphasizes practical and observable aspects without requiring complex calculations or the interpretation of numerical data, promoting a more accessible, inclusive, and replicable self-assessment process. This approach supports continuous improvement based on qualitative evidence that is easily understood and managed by non-specialist personnel, while also fostering greater compliance and motivation within the organization.

3.3. A Support Tool for Monitoring and Ex Post Evaluation of Participatory Process (PHASES 3b, 4b)

To develop a weighted indicator-based support tool for monitoring and ex post evaluation of participatory processes, barriers and enablers deemed analytically relevant, as well as variables pertinent to monitoring and evaluation, were extracted from the systematic case study literature review and the grey literature review. As described in Section 2.3, these results were first screened and merged to remove redundancies, then classified according to superordinate criteria, and finally assessed for clarity, with vague or ambiguous items excluded. The remaining barriers, enablers, and variables were subsequently transformed into indicators, compiled into a structured matrix comprising the following fields: indicator name, description, superordinate criterion, and measurement method. This process produced a total of 93 indicators, each of which was individually evaluated for relevance to the research objectives, local applicability, clarity, and comprehensibility using an iterative approach. The final shortlist was then assessed holistically, resulting in the identification of 21 conceptually discrete indicators.

Application of the BWM for the Monitoring and Ex Post Evaluation Tool

The shortlist of indicators for monitoring and ex post evaluation participatory processes in DRM and URB is presented in

Table 6. Hierarchical structure of superordinate criteria and indicators.

Criteria	Sub-Criteria	Code
Characteristics	Heterogeneity of the Stakeholders Involved in Each Risk Considered	CHR01
	$\mathrm{H} = {\mathit{C}}_{\mathit{k}}{\mathit{H}}_{\mathit{k}}{\mathit{H}}_{\mathit{r}}$ with H = Heterogeneity Index of Stakeholders Involved ∈ [0,1] [a] ${\mathit{C}}_{\mathit{k}} = {\displaystyle \frac{{\mathit{N}}_{\mathit{k}}}{\mathit{N}}}$ with $C_s$ = Index of Category of Stakeholder Covered ∈ [0,1], $N_k$ = number of stakeholder categories actually involved and $N$ = number of stakeholder categories that can theoretically be involved (6 = government, industries, academia, third sector, citizenry, visitors) [b] ${\mathit{H}}_{\mathit{k}} = -{\displaystyle \frac{\mathbf{1}}{\mathbf{log}{\mathit{N}}_{\mathit{k}}}}{\sum }_{\mathit{n}=\mathbf{1}}^{{\mathit{N}}_{\mathit{k}}}{\mathit{w}}_{\mathit{k}}\mathbf{log}\left({\mathit{w}}_{\mathit{k}}\right)$ with $H_k$ = Intra-Category Heterogeneity Index ∈ [0,1] and $w_k$ = weight assigned to each stakeholder category k ∈ [0,1] [c] ${\mathit{p}}_{\mathit{r}} = {\sum }_{\mathit{n}=\mathbf{1}}^{{\mathit{N}}_{\mathit{k}}}{\mathit{w}}_{\mathit{k}}{\mathit{M}}_{\mathit{k}\mathit{r}}$ with $p_k$ = share of involvement in risk r and $M_{kr}$ = involvement matrix [d] ${\mathit{H}}_{\mathit{r}} = -{\displaystyle \frac{\mathbf{1}}{\mathbf{log}\mathit{R}}}{\sum }_{\mathit{r}=\mathbf{1}}^{\mathit{R}}{\mathit{p}}_{\mathit{r}}\mathbf{log}\left({\mathit{p}}_{\mathit{p}}\right)$ with $H_k$ = Intra-Risk Heterogeneity Index ∈ [0,1]
	Implementation of Participatory Activities in a Blended Setting	CHR02
	${\mathit{B}\mathit{L}}_{\mathit{t}\mathit{o}\mathit{t}} = {\sum }_{\mathbf{s}=\mathbf{1}}^{\mathit{S}}{\mathit{w}}_{\mathit{r}}{\displaystyle \frac{{\mathit{B}}_{\mathit{r}}}{{\mathit{A}}_{\mathit{r}}}}$ with BL = Index of Blended Methods Share ∈ [0,1], $B_r$ = number of methods that are blended (in-person and remote) and $A_r$ = total number of methods that address the risk r and $w_r$ = weight given to each risk (r) considered ∈ [0,1]
	Triangulation of Quantitative and Qualitative Data	CHR03
	$\mathit{T}\mathit{Q}\mathbf{2}={\sum }_{\mathit{r}=\mathbf{1}}^{\mathit{R}}{\mathit{w}}_{\mathit{o}}{\displaystyle \frac{{\mathit{Q}}_{\mathit{n}\mathit{r}}+{\mathit{Q}}_{\mathit{l}\mathit{r}}}{\mathbf{2}}}{\mathit{I}}_{\mathit{r}}$$\mathrm{with} \mathit{TQ}2 =$ Index of Triangulation ∈ [0,1]$, w_r$ = weight assigned to each risk r ∈ [0,1]$, Q_{nr}$$= \mathrm{presence} \left(1\right) \mathrm{or} \mathrm{absence} \left(0\right) \mathrm{of} \mathrm{quantitative} \mathrm{data} \mathrm{for} \mathrm{risk} \mathrm{r}, Q_{lr}$$= \mathrm{presence} \left(1\right) \mathrm{or} \mathrm{absence} \left(0\right) \mathrm{of} \mathrm{qualitative} \mathrm{data} \mathrm{for} \mathrm{risk} \mathrm{r} \mathrm{and} I_r$ = degree of integration between quantitative and qualitative data for risk r (0 = data collected but analyzed separately, without comparison; 0.5 = data compared and discussed together, but not integrated into a single model; 1 = quantitative and qualitative data fully integrated into a single model or synthesis)
	Degree to which Stakeholders Withdraw from the Participatory Process	CHR04
	$\mathbf{W} = {\sum }_{\mathbf{k}=\mathbf{1}}^{\mathit{K}}{\mathit{w}}_{\mathit{k}}{\displaystyle \frac{{\mathit{N}}_{\mathit{s}\mathit{t}\mathit{a}\mathit{r}\mathit{t},\mathit{o}}-{\mathit{N}}_{\mathit{e}\mathit{n}\mathit{d},\mathit{o}}}{{\mathit{N}}_{\mathit{s}\mathit{t}\mathit{a}\mathit{r}\mathit{t},\mathit{o}}}}$ with W = Stakeholder Withdrawal Index ∈ [0,1]$, N_{start,o}$$= \mathrm{initial} \mathrm{number} \mathrm{of} \mathrm{stakeholders} \mathrm{in} \mathrm{the} \mathrm{category} \mathrm{k}, N_{start,o}$$= \mathrm{final} \mathrm{number} \mathrm{of} \mathrm{stakeholders} \mathrm{in} \mathrm{the} \mathrm{category} \mathrm{k} \mathrm{and} w_k$= weight assigned to each stakeholder category k ∈ [0,1]
	Representativeness of the Sample of Stakeholders Involved	CHR05
	$\mathit{R}= {\displaystyle \frac{{\sum }_{\mathit{k}=\mathbf{1}}^{\mathit{K}}{\mathit{w}}_{\mathit{k}}{\mathit{s}}_{\mathit{k}}}{{\sum }_{\mathit{k}=\mathbf{1}}^{\mathit{K}}{\mathit{w}}_{\mathit{k}}}}$$\mathrm{with} R$ = Representativeness index ∈ [0,1]$, s_k$ = score based on the coverage of the subgroup of the stakeholder category k (0 = not involved subgroup, 1 = involved subgroup)$\mathrm{and} w_k$ = weight given to each subgroup of category of stakeholders k ∈ [0,1]
	Coordination between Administrative Bodies across Multiple Levels	CHR06
	$\mathbf{CML} = {\displaystyle \frac{\mathbf{1}}{\mathit{L}}}{\sum }_{\mathbf{l}=\mathbf{1}}^{\mathit{L}}{\mathit{I}}_{\mathit{l}}$ with CML = Multi-Level Coordination Index ∈ [0,1]$, L = \mathrm{administrative} \mathrm{levels} \mathrm{theoretically} \mathrm{involved} (\mathrm{e}.\mathrm{g}., \mathrm{micro-local}, \mathrm{municipal}, \mathrm{provincial}/\mathrm{metropolitan}, \mathrm{regional}, \mathrm{national} \mathrm{and} \mathrm{international}) \mathrm{and} I_l$ = Index of the Quality of Coordination per level l (0 = levels not involved; 0.25 = levels are only informed; 0.5 = levels are only consulted; 0.75 = levels collaborate in the specific decision-making; 1 = shared decision-making and planning between levels)
	Adaptability of the Structure of the Participatory Process to Ever-Changing External Factors	CHR07
	$\mathit{A}\mathit{D}\mathit{A}\mathit{P}\mathit{T}={\sum }_{\mathit{p}=\mathbf{1}}^{\mathit{P}}{\mathit{w}}_{\mathit{p}}({{\mathit{w}}_{\mathit{o}\mathit{f}}\mathit{s}}_{\mathit{o}\mathit{f}}+{\mathit{w}}_{\mathit{m}\mathit{v}}{\mathit{s}}_{\mathit{m}\mathit{v}}+{\mathit{w}}_{\mathit{i}}{\mathit{s}}_{\mathit{i}})$$\mathrm{with} \mathit{ADAPT} =$ Index of Adaptability ∈ [0,1]$, w_p$ = weight assigned to each phase p ∈ [0,1]$, s_{of}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{organization} \mathrm{flexibility} (0 = \mathrm{rigidity} \mathrm{e}.\mathrm{g}., \mathrm{fixed} \mathrm{dates}, \mathrm{non-modifiable} \mathrm{tools}, \mathrm{etc}.; 0.5 = \mathrm{limited} \mathrm{room} \mathrm{for} \mathrm{maneuver}; 1 = \mathrm{high} \mathrm{flexibility} \mathrm{and} \mathrm{ease} \mathrm{in} \mathrm{adjusting} \mathrm{the} \mathrm{process}), s_{mv}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{methodological} \mathrm{variety} (0 = \mathrm{few} \mathrm{methods} \mathrm{are} \mathrm{used} \mathrm{in} \mathrm{a} \mathrm{rigid} \mathrm{way}; 0.5 = \mathrm{some} \mathrm{methodological} \mathrm{variation}, \mathrm{but} \mathrm{only} \mathrm{marginal}; 1 = \mathrm{wide} \mathrm{range} \mathrm{of} \mathrm{methods}, \mathrm{selected} \mathrm{and} \mathrm{combined} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{situation}), s_i$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{ongoing} \mathrm{inclusion} (0 = \mathrm{no} \mathrm{new} \mathrm{stakeholders} \mathrm{can} \mathrm{join} \mathrm{the} \mathrm{ongoing} \mathrm{process}; 0.5 = \mathrm{partial} \mathrm{openness}, \mathrm{but} \mathrm{with} \mathrm{significant} \mathrm{constraints}; 1 = \mathrm{open} \mathrm{and} \mathrm{inclusive} \mathrm{process}, \mathrm{new} \mathrm{stakeholders} \mathrm{can} \mathrm{be} \mathrm{seamlessly} \mathrm{integrated}) \mathrm{and} w_x$ = weight assigned to each component s ∈ [0,1]
	Frequency of Participant Engagement Activities	CHR08
	$\mathit{F}= {\displaystyle \frac{{\sum }_{\mathit{n}=\mathbf{1}}^{\mathit{N}}{\mathit{w}}_{\mathit{n}}{\mathit{D}}_{\mathit{n}}}{\mathit{D}}}$$\mathrm{with} F$ = Index of Frequency ∈ [0,1]$, D_n$$= \mathrm{days} \mathrm{occupied} \mathrm{by} \mathrm{engagement} \mathrm{activity} \mathrm{n} \mathrm{and} w_n$ = weight assigned to each engagement activity n ∈ [0,1]
	Regular monitoring of the progress of participation	CHR09
	$\mathit{M}\mathit{O}\mathit{N}={\sum }_{\mathit{p}=\mathbf{1}}^{\mathit{P}}{{\mathit{w}}_{\mathit{p}}\mathit{s}}_{\mathit{p}}$ with MON = Monitoring Index ∈ [0,1]$, s_p$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{degree} \mathrm{of} \mathrm{monitoring} (0 = \mathrm{no} \mathrm{monitoring}; 0.25 = \mathrm{sporadic} \mathrm{and} \mathrm{intermittent}; 0.5 = \mathrm{partial} \mathrm{but} \mathrm{not} \mathrm{systematic}; 0.75 = \mathrm{fairly} \mathrm{systematic} \mathrm{monitoring}; 1 = \mathrm{high} \mathrm{constant} \mathrm{monitoring}) \mathrm{and} w_p$ = weight assigned to each phase p of the process ∈ [0,1]
Procedures	Prevailing Level of Intensity of Participation	PRC01
	$\mathit{I}\mathit{N}\mathit{P}= {\sum }_{\mathit{f}=\mathbf{1}}^{\mathit{P}}{{\sum }_{\mathit{k}=\mathbf{1}}^{\mathit{K}}\mathit{w}}_{\mathit{p}}{\mathit{w}}_{\mathit{k}}\mathit{s}\left({\mathit{L}}_{\mathit{p}\mathit{k}}\right)$ with INP = Index of Participation Intensity ∈ [0,1], $s\left(L_{fk}\right)$ = score based on the level of participation observed at each phase p for each stakeholder category k (0 = category k is not involved; 0.20 = category k is only informed; 0.40 = category k is only involved; 0.60 = category k is consulted; 0.80 = category k collaborates; 1 = category k is empowered), $w_p$ = weight assigned to each phase p ∈ [0,1] and $w_k$ = weight assigned to each category of stakeholder k ∈ [0,1]
	Share of Participatory Activities that Implement a Multi-Risk Approach	PRC02
	$\mathit{M}\mathit{U}\mathit{L}\mathit{T}\mathit{I}\mathbf{01}={\displaystyle \frac{{\sum }_{\mathit{m}=\mathbf{1}}^{\mathit{M}}{\mathit{w}}_{\mathit{m}}{\mathit{s}}_{\mathit{m}}}{{\sum }_{\mathit{m}=\mathbf{1}}^{\mathit{M}}{\mathit{w}}_{\mathit{m}}}}\mathit{M}\mathit{R}$ with MULTI01 $=$ Index of Multi-Risk Share ∈ [0,1], $w_m$ = weight assigned to each method m ∈ [0,1] and MR multi-risk coefficient (0 = single-hazard; 0.20 = single risk; 0.40 = multilayer single-hazard; 0.60 = multihazard; 0.80 = multihazard risk; 1 = multirisk) [a] ${\mathit{s}}_{\mathit{m}}={\displaystyle \frac{{\sum }_{\mathit{r}=\mathbf{1}}^{\mathit{R}}{\mathit{w}}_{\mathit{r}}{\mathit{s}}_{\mathit{m}\mathit{r}}-\mathit{m}\mathit{i}\mathit{n}\left(\mathit{w}\right)}{{\sum }_{\mathit{r}=\mathbf{1}}^{\mathit{R}}{\mathit{w}}_{\mathit{r}}-\mathit{m}\mathit{i}\mathit{n}\left(\mathit{w}\right)}}$ with $w_r$ = weight assigned to each risk r ∈ [0,1] and $s_{mr}$ = score based on whether the method m considers (1) or does not consider (0) the risk r
	Intergenerationality in Decision-Making	PRC03
	$\mathit{I}\mathit{N}\mathit{T}\mathit{G}= {\sum }_{\mathit{p}=\mathbf{1}}^{\mathit{P}}{\mathit{w}}_{\mathit{p}}{\displaystyle \frac{{\sum }_{\mathit{g}=\mathbf{1}}^{\mathit{G}}{\mathit{s}}_{\mathit{o}\mathit{g}}}{\left|\mathit{G}\right|}}$$\mathrm{with} INTG$ = Index of Intergenerationality ∈ [0,1]$, w_p$ = weight assigned to each phase p ∈ [0,1]$\mathrm{and} s_{og}$ = score based on whether generation g is involved in the p phase (0 = not involved; 1 = involved)
	Interculturalism in Decision-Making	PRC04
	$\mathit{I}\mathit{N}\mathit{T}\mathit{C}= {\sum }_{\mathit{p}=\mathbf{1}}^{\mathit{P}}{\mathit{w}}_{\mathit{p}}{\displaystyle \frac{{\sum }_{\mathit{g}=\mathbf{1}}^{\mathit{G}}{\mathit{s}}_{\mathit{o}\mathit{g}}}{\left|\mathit{G}\right|}}$$\mathrm{with} INTG$ = Index of Interculturalism ∈ [0,1]$, w_p$ = weight assigned to each phase p ∈ [0,1]$\mathrm{and} s_{og}$= score based on whether social group g is involved in the phase p (0 = not involved; 1 = involved)
	Gender Equality in Decision-Making	PRC05
	$\mathit{G}\mathit{E}=\mathbf{1}-{\displaystyle \frac{\mathbf{1}}{\mathbf{P}}}{\sum }_{\mathit{p}=\mathbf{1}}^{\mathit{P}}{\displaystyle \frac{|{\mathit{P}}_{\mathit{f}\mathit{p}}-{\mathit{P}}_{\mathit{f}\mathit{m}}|}{{\mathit{P}}_{\mathit{f}\mathit{p}}+{\mathit{P}}_{\mathit{f}\mathit{m}}}}$$\mathrm{with} \mathit{GE} =$ Index of Gender Equality ∈ [0,1]$, P_{fo}$$= \mathrm{number} \mathrm{of} \mathrm{female} \mathrm{participants} \mathrm{in} \mathrm{the} \mathrm{phase} \mathrm{o} \mathrm{and} P_{mo}$ = number of male participants in the phase o
	Degree of Cross-Contamination and Integration between Disciplines	PRC06
	$\mathrm{T}={\sum }_{\mathit{p}=\mathbf{1}}^{\mathit{P}}{\mathit{w}}_{\mathit{p}}\left({\displaystyle \frac{{\mathit{I}}_{\mathit{p}}-\mathbf{1}}{\mathit{m}-\mathbf{1}}}{\displaystyle \frac{{\mathit{d}}_{\mathit{p}}}{\mathit{D}}}\right)$ with T = Index of Transdisciplinarity ∈ [0,1]$, w_p$ = weight assigned to each phase p ∈ [0,1]$, I_p$$= \mathrm{degree} \mathrm{of} \mathrm{integration} \mathrm{observed} \mathrm{in} \mathrm{the} \mathrm{phase} \mathrm{p} (1 = \sin \mathrm{gle} \mathrm{discipline}; 2 = \mathrm{multi-disciplinarity}; 3 = \mathrm{interdisciplinarity}; 4 = \mathrm{transdisciplinarity}), m$$= \mathrm{maximum} \mathrm{value} \mathrm{of} \mathrm{the} \mathrm{degree} \mathrm{of} \mathrm{integration}, d_p$ = number of disciplines involved in the phase p
	Level of Transparency in Returning Data and Disseminating of Results	PRC07
	$\mathit{T}\mathit{R}\mathit{A}={\sum }_{\mathit{p}=\mathbf{1}}^{\mathit{P}}{\mathit{w}}_{\mathit{p}}({{\mathit{w}}_{\mathit{a}}\mathit{s}}_{\mathit{a}}+{\mathit{w}}_{\mathit{r}\mathit{d}}{\mathit{s}}_{\mathit{r}\mathit{d}}+{\mathit{w}}_{\mathit{c}}{\mathit{s}}_{\mathit{c}})$$\mathrm{with} \mathit{TRA} =$ Index of Transparency ∈ [0,1]$, w_p$ = weight assigned to each phase p ∈ [0,1]$, s_a$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{accessibility} \mathrm{of} \mathrm{results} (0 = \mathrm{privilege} \mathrm{information} \mathrm{for} \mathrm{a} \mathrm{few}; 0.5 = \mathrm{partial} \mathrm{or} \mathrm{delayed} \mathrm{access} \mathrm{to} \mathrm{information}; 1 = \mathrm{full} \mathrm{equality} \mathrm{of} \mathrm{access} \mathrm{to} \mathrm{information}), s_{rd}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{reporting} \mathrm{and} \mathrm{accessible} \mathrm{dissemination} \mathrm{of} \mathrm{results} (0 = \mathrm{no} \mathrm{reports} \mathrm{available}; 0.5 = \mathrm{sporadic} \mathrm{and} \mathrm{unclear} \mathrm{reports}; 1 = \mathrm{complete} \mathrm{and} \mathrm{frequent} \mathrm{reports}), s_c$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{clarity} \mathrm{of} \mathrm{the} \mathrm{results} (0 = \mathrm{incomprehensible} \mathrm{technical} \mathrm{documents}; 0.5 = \mathrm{partially} \mathrm{comprehensible}; 1 = \mathrm{clear}, \mathrm{concise}, \mathrm{and} \mathrm{understandable} \mathrm{by} \mathrm{all}) \mathrm{and} w_x$ = weight assigned to each component s ∈ [0,1]
	Level of Integration of Grassroots Practices in the Decision-Making	PRC08
	$\mathit{G}\mathit{R}\mathit{A}\mathit{S}\mathit{S}={\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}{\mathit{s}}_{\mathit{o}}$ with GRASS = Index of Integration of Grassroots Practices ∈ [0,1]$, w_O$ = weight assigned to each output o ∈ [0,1], $s_o$ = score based on the level of integration of grassroots practices (0 = no integration; 0.5 = partial integration; 1 = full integration)
	Accessibility and User-Friendliness of Methods, Techniques and Tools Used	PRC09
	$\mathit{A}\mathit{U}\mathit{F}={\sum }_{\mathit{m}=\mathbf{1}}^{\mathit{M}}{\mathit{w}}_{\mathit{m}}({{\mathit{w}}_{\mathit{l}\mathit{a}}\mathit{s}}_{\mathit{l}\mathit{a}}+{\mathit{w}}_{\mathit{s}\mathit{e}\mathit{i}}{\mathit{s}}_{\mathit{s}\mathit{e}\mathit{i}}+{\mathit{w}}_{\mathit{c}\mathit{s}}{\mathit{s}}_{\mathit{c}\mathit{s}}+{\mathit{w}}_{\mathit{u}}{\mathit{s}}_{\mathit{u}}+{\mathit{w}}_{\mathit{f}}{\mathit{s}}_{\mathit{f}})$$\mathrm{with} \mathit{AUF} =$ Index of Accessibility and User-friendliness ∈ [0,1]$, w_p$ = weight assigned to each phase p ∈ [0,1]$, s_{la}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{language} \mathrm{accessibility} (0 = \mathrm{use} \mathrm{of} \mathrm{language} \mathrm{incompatible} \mathrm{with} \mathrm{the} \mathrm{target} \mathrm{involved}; 1 = \mathrm{use} \mathrm{of} \mathrm{language} \mathrm{compatible} \mathrm{with} \mathrm{the} \mathrm{target} \mathrm{involved}), s_{sei}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{socio-economic} \mathrm{inclusiveness} (0 = \mathrm{times} \mathrm{and} \mathrm{locations} \mathrm{not} \mathrm{compatible} \mathrm{with} \mathrm{the} \mathrm{target} \mathrm{involved}; 1 = \mathrm{times} \mathrm{and} \mathrm{locations} \mathrm{compatible} \mathrm{with} \mathrm{the} \mathrm{target} \mathrm{involved}), s_c$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{clarity} \mathrm{and} \mathrm{simplicity} (0 = \mathrm{rules} \mathrm{and} \mathrm{objectives} \mathrm{not} \mathrm{easily} \mathrm{understandable} \mathrm{by} \mathrm{the} \mathrm{target} \mathrm{involved}; 1 = \mathrm{rules} \mathrm{and} \mathrm{objectives} \mathrm{easily} \mathrm{understandable} \mathrm{by} \mathrm{the} \mathrm{target} \mathrm{involved}), s_u$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{usability} (0 = \mathrm{materials} \mathrm{not} \mathrm{intuitive} \mathrm{and} \mathrm{difficult} \mathrm{to} \mathrm{use}; 1 = \mathrm{materials} \mathrm{intuitive} \mathrm{and} \mathrm{easy} \mathrm{to} \mathrm{use}), s_f$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{support} \mathrm{and} \mathrm{facilitation} (0 = \mathrm{no} \mathrm{facilitators} \mathrm{or} \mathrm{guides} \mathrm{available}; 1 = \mathrm{presence} \mathrm{of} \mathrm{facilitators} \mathrm{or} \mathrm{guides} \mathrm{available}) \mathrm{and} w_x$ = weight assigned to each component s ∈ [0,1]
Outputs	Degree of Consistency between Objectives Set and Results Achieved	OTP01
	$\mathrm{COR}={\displaystyle \frac{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}{\mathit{s}}_{\mathit{o}}}{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}}}$ with COR = Index of Objectives-Results Consistency ∈ [0,1], $s_o$ = score based on the degree of consistency (0 = total inconsistency; 0.5 = partial consistency; 1 = total consistency) and $w_o$ = weight assigned to each output o ∈ [0,1]
	Integration of Participation Inputs in Decision-Making	OTP02
	$\mathit{I}\mathit{P}\mathit{I}={\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{N}}{\mathit{w}}_{\mathit{o}}({{\mathit{w}}_{\mathit{e}\mathit{d}}\mathit{s}}_{\mathit{e}\mathit{d}}+{\mathit{w}}_{\mathit{i}}{\mathit{s}}_{\mathit{i}}+{{\mathit{w}}_{\mathit{e}\mathit{x}}\mathit{s}}_{\mathit{e}\mathit{x}})$ with IPI $=$ Index of Participation Index ∈ [0,1], $w_o$ = weight assigned to each output o ∈ [0,1], $s_{ed}$ = score based on documentary evidence of the integration of participation inputs (0 = no documentary evidence; 0.5 = output o refers to contributions indirectly or generically; 1 = output o explicitly cites participants contributions), $s_i$ = score based on the relevance of participation input (0 = no influence of participation input on output o; 0.5 = only a partial influence on output o; 1 = participation inputs directly influences output o), $s_{ex}$ = score based on the extent of participation input used (0 = none were used; 0.5 = only a portion were used; 1 = all or most relevant inputs were used) and $w_x$ = weight assigned to each component s ∈ [0,1]
	Share of Urban Solutions Addressing Multiple-Interacting Risks	OTP03
	$\mathit{M}\mathit{U}\mathit{L}\mathit{T}\mathit{I}\mathbf{02}={\displaystyle \frac{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}{\mathit{s}}_{\mathit{o}}}{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}}}\mathit{M}\mathit{R}$$\mathrm{with} \mathit{MULTI02} =$ Index of Multi-Risk Share ∈ [0,1], $w_o$ = weight assigned to each output o ∈ [0,1] and MR multi-risk coefficient (0 = single-hazard; 0.20 = single risk; 0.40 = multilayer single-hazard; 0.60 = multihazard; 0.80 = multihazard risk; 1 = multirisk) $[\mathrm{a}] {\mathit{s}}_{\mathit{o}}={\displaystyle \frac{{\sum }_{\mathit{r}=\mathbf{1}}^{\mathit{R}}{\mathit{w}}_{\mathit{r}}{\mathit{s}}_{\mathit{o}\mathit{r}}-\mathit{m}\mathit{i}\mathit{n}\left(\mathit{w}\right)}{{\sum }_{\mathit{r}=\mathbf{1}}^{\mathit{R}}{\mathit{w}}_{\mathit{r}}-\mathit{m}\mathit{i}\mathit{n}\left(\mathit{w}\right)}}$$\mathrm{with} w_r$ = weight assigned to each risk r ∈ [0,1]$\mathrm{and} s_{or}$ = score based on whether the output o considers (1) or does not consider (0) the risk r
	Level of Disclosure of Tradeoffs	OTP04
	$\mathit{D}={\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}{\mathit{s}}_{\mathit{o}}$ with D = Index of Disclosure ∈ [0,1]$, w_O$ = weight assigned to each output o ∈ [0,1]$, s_o$ = score based on tradeoffs disclosure (0 = no disclosure; 1 = disclosure)
	Complementary between Outputs and Existing Urban Planning Tools	OTP05
	$\mathit{C}\mathit{O}\mathit{T}={\displaystyle \frac{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}{\mathit{s}}_{\mathit{o}}}{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}}}$$\mathrm{with} \mathit{COT} =$ Index of Consistency Outputs-Urban Planning Tools ∈ [0,1], $s_o$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{degree} \mathrm{of} \mathrm{consistency} (0 = \mathrm{total} \mathrm{inconsistency}; 0.5 = \mathrm{partial} \mathrm{consistency}; 1 = \mathrm{total} \mathrm{consistency}) \mathrm{and} w_o$ = weight assigned to each output o ∈ [0,1]
	Establishment of New Formal and Informal Networks	OTP06
	$\mathit{N}\mathit{E}\mathit{T}={\sum }_{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\mathit{w}}_{\mathit{i}}({{\mathit{w}}_{\mathit{h}}\mathit{s}}_{\mathit{h}}+{\mathit{w}}_{\mathit{s}}{\mathit{s}}_{\mathit{s}}+{{\mathit{w}}_{\mathit{f}}\mathit{s}}_{\mathit{f}}+{{\mathit{w}}_{\mathit{o}}\mathit{s}}_{\mathit{o}})$$\mathrm{with} \mathit{NET} =$ Index of Network Establishment ∈ [0,1]$, w_i$ = weight assigned to each network i ∈ [0,1]$, s_h$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{involvement} \mathrm{from} 1 \mathrm{category} \mathrm{of} \mathrm{stakeholders} \left(0\right) \mathrm{to} 6 \left(1\right) \mathrm{i}.\mathrm{e}., \mathrm{government}, \mathrm{industries}, \mathrm{academia}, \mathrm{third} \sec \mathrm{tors}, \mathrm{citizenry} \mathrm{and} \mathrm{visitors}, s_s$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{stability} \mathrm{of} \mathrm{the} \mathrm{network} \mathrm{I} \mathrm{at} \mathrm{the} \mathrm{end} \mathrm{of} \mathrm{the} \mathrm{participatory} \mathrm{process} (0 = \mathrm{it} \mathrm{dissolved}; 0.5 = \mathrm{it} \mathrm{continues} \mathrm{but} \mathrm{in} \mathrm{a} \mathrm{very} \mathrm{limited} \mathrm{or} \mathrm{sporadic} \mathrm{form}; 1 = \mathrm{it} \mathrm{actively} \mathrm{continues}), s_f$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{degree} \mathrm{of} \mathrm{formalization} \mathrm{of} \mathrm{the} \mathrm{network} \mathrm{i} (0 = \mathrm{informal} \mathrm{network} \mathrm{with} \mathrm{no} \mathrm{written} \mathrm{agreement}; 0.5 = \mathrm{network} \mathrm{with} \mathrm{informal} \mathrm{agreement}; 1 = \mathrm{formalized} \mathrm{network}), s_o$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{origin} \mathrm{of} \mathrm{the} \mathrm{network} \mathrm{i} (0 = \mathrm{pre}-\mathrm{existing} \mathrm{network}; 0.5 = \mathrm{existing} \mathrm{network} \mathrm{but} \mathrm{significantly} \mathrm{strengthened} \mathrm{by} \mathrm{the} \mathrm{process}; 1 = \mathrm{new} \mathrm{network}) \mathrm{and} w_x$ = weight assigned to each component s ∈ [0,1]
	Testing Collaborative Management Models	OTP07
	$\mathit{C}\mathit{O}\mathit{M}={\displaystyle \frac{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}{\mathit{s}}_{\mathit{o}}}{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}}}$$\mathrm{with} \mathit{COM} =$ Index of Co-Management ∈ [0,1]$, s_o$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{degree} \mathrm{of} \mathrm{activation} \mathrm{of} \mathrm{co-management} \mathrm{models} (0 = \mathrm{no} \mathrm{co-management} \mathrm{model} \mathrm{planned}; 0.25 = \mathrm{mention} \mathrm{only} \mathrm{or} \mathrm{theoretical} \mathrm{possibility}; 0.5 = \mathrm{planned} \mathrm{and} \mathrm{piloted} \mathrm{but} \mathrm{not} \mathrm{formalized}; 0.75 = \mathrm{partial} \mathrm{agreements} \mathrm{or} \mathrm{resources} \mathrm{planned}; 1 = \mathrm{co-management} \mathrm{activated} \mathrm{and} \mathrm{formalized}, \mathrm{resources} \mathrm{allocated}) \mathrm{and} w_o$ = weight assigned to each output o ∈ [0,1]
	Governance Experimentation and Transformation	OTP08
	$\mathit{G}\mathit{O}\mathit{V}={\displaystyle \frac{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}{\mathit{s}}_{\mathit{o}}}{{\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{O}}{\mathit{w}}_{\mathit{o}}}}$$\mathrm{with} GOV$ = Index of Governance Change ∈ [0,1]$, s_o$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{type} \mathrm{of} \mathrm{governance} \mathrm{change} (0 = \mathrm{business-as-usual}; 0.5 = \mathrm{temporary} \mathrm{model} \mathrm{test}; 1 = \mathrm{structural} \mathrm{governance} \mathrm{transformation}) \mathrm{and} w_o$ = weight assigned to each output o ∈ [0,1]
	Level of Clarity in the Workplan	OTP09
	$\mathit{C}\mathit{W}\mathit{P}={\sum }_{\mathit{o}=\mathbf{1}}^{\mathit{N}}{\mathit{w}}_{\mathit{o}}({{\mathit{w}}_{\mathit{o}\mathit{c}}\mathit{s}}_{\mathit{o}\mathit{c}}+{\mathit{w}}_{\mathit{r}\mathit{c}}{\mathit{s}}_{\mathit{r}\mathit{c}}+{{\mathit{w}}_{\mathit{p}\mathit{c}}\mathit{s}}_{\mathit{p}\mathit{c}}+{{\mathit{w}}_{\mathit{f}\mathit{c}}\mathit{s}}_{\mathit{f}\mathit{c}})$$\mathrm{with} \mathit{CWP} =$ Index of Clarity of the Workplan ∈ [0,1]$, w_i$ = weight assigned to each output o ∈ [0,1]$, s_{oc}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{level} \mathrm{of} \mathrm{clarity} \mathrm{in} \mathrm{delineating} \mathrm{objectives} (0 = \mathrm{vague} \mathrm{or} \mathrm{absent} \mathrm{objectives}; 0.5 = \mathrm{main} \mathrm{objectives} \mathrm{are} \mathrm{indicated} \mathrm{but} \mathrm{are} \mathrm{generic} \mathrm{or} \mathrm{poorly} \mathrm{measurable}; 1 = \mathrm{well-formulated} \mathrm{and} \mathrm{measurable} \mathrm{objectives}), s_{rc}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{level} \mathrm{of} \mathrm{clarity} \mathrm{in} \mathrm{defining} \mathrm{roles} (0 = \mathrm{roles} \mathrm{are} \mathrm{unclear}, \mathrm{it} \mathrm{is} \mathrm{not} \mathrm{indicated} \mathrm{who} \mathrm{does} \mathrm{what}; 0.5 = \mathrm{some} \mathrm{main} \mathrm{roles} \mathrm{are} \mathrm{indicated}, \mathrm{but} \mathrm{precise} \mathrm{references} \mathrm{are} \mathrm{missing}; 1 = \mathrm{all} \mathrm{roles} \mathrm{are} \mathrm{clearly} \mathrm{defined}, \mathrm{with} \mathrm{responsibilities} \mathrm{and} \mathrm{contacts}), s_{pc}$ = score based on the level of clarity in procedures (0 = procedures not defined; 0.5 = some are described, but details on times or operating methods are missing; 1 = clear explanation of procedures and times), $s_{fc}$$= \mathrm{score} \mathrm{based} \mathrm{on} \mathrm{the} \mathrm{origin} \mathrm{of} \mathrm{the} \mathrm{network} \mathrm{i} (0 = \mathrm{There} \mathrm{are} \mathrm{no} \mathrm{details} \mathrm{on} \cos \mathrm{ts}, \mathrm{funding}, \mathrm{or} \mathrm{funding} \mathrm{sources}; 0.5 = \mathrm{Some} \mathrm{financial} \mathrm{information} \mathrm{is} \mathrm{provided}, \mathrm{but} \mathrm{details} \mathrm{are} \mathrm{missing}; 1 = \mathrm{All} \mathrm{funding}, \mathrm{estimated} \cos \mathrm{ts}, \mathrm{sources}, \mathrm{and} \mathrm{allocation} \mathrm{methods} \mathrm{are} \mathrm{clearly} \mathrm{and} \mathrm{transparently} \mathrm{described}) \mathrm{and} w_x$ = weight assigned to each component s ∈ [0,1]

.

To implement the BWM as a weighting system, an ad hoc online questionnaire was developed to guide participants through the different stages of the evaluation process. The questionnaire was structured into four main sections: (1) an introduction, designed to familiarize experts with the objectives of the research project and their role; (2) a methodological overview, aimed at explaining the rationale underlying the questionnaire, which differs from more conventional surveys; (3) the core BWM section; and (4) a closing section, in which participants’ status, area of expertise, years of professional experience, country of residence. The closing section also included three questions aimed at investigating the following aspects and validating the study results: experts’ perceived knowledge of participatory processes and urban governance; perceived knowledge of multi-hazard urban resilience planning; and perceived confidence in the potential of participatory processes to contribute to building multi-hazard urban resilience. The questionnaire was administered to a panel of experts with demonstrated professional and academic experience in either participatory processes and urban governance or multi-risk urban resilience planning. Experts were recruited through the researchers’ professional networks and via snowball sampling to ensure both heterogeneity of backgrounds and complementarity of expertise. The final panel comprised twenty respondents (see Figure 5 for details). Only fully completed questionnaires were considered for analysis; partial or inconsistent responses were excluded.

The resulting weights for each criterion and sub-criterion are presented in

Table 7. The resulting weights for each criterion and sub-criterion.

Criteria	W	Criteria	W	Criteria	W
Characteristics	0.33	Procedures	0.44 (BEST)	Outputs	0.22 (WORST)
Sub-Criteria	W	Sub-Criteria	W	Sub-Criteria	W
CHR01	0.15	PRC01	0.11	OTP01	0.12
CHR02	0.04 (WORST)	PRC02	0.13 (BEST)	OTP02	0.14 (BEST)
CHR03	0.12	PRC03	0.12	OTP03	0.11
CHR04	0.07	PRC04	0.10 (WORST)	OTP04	0.08 (WORST)
CHR05	0.16 (BEST)	PRC05	0.10 (WORST)	OTP05	0.11
CHR06	0.09	PRC06	0.10 (WORST)	OTP06	0.09
CHR07	0.15	PRC07	0.11	OTP07	0.09
CHR08	0.08	PRC08	0.13 (BEST)	OTP08	0.14 (BEST)
CHR09	0.14	PRC09	0.11	OTP09	0.11

.

3.4. Testing the Innovative Indicator-Based Tools on Multiple Case Studies

In this paper, the following case studies have been identified for testing the proposed methodology (

Table 8. Comparative overview of selected case studies.

Case Studies	TFC	URCA!	RETURN	HERIT ADAPT
Funding source	URBACT	PRIN	NextGenerationEU	Interreg Euro-MED
Time frame	2019–2022	2023–2025	2022–2025	2024–2025
Scale of intervention	Municipal	Neighbourhood	Municipal	Neighbourhood
Hazards addressed	Environmental Biological Technological	Hydrometeorological Technological	Hydrometeorological Geological	Hydrometeorological Environmental Technological
Actors involved	Government Industries Academia NGOs Citizenry Visitors	Government Academia NGOs Citizenry Visitors	Government Industries Academia	Government Industries Academia NGOs Citizenry
Main output	Municipal Strategy Urban Project	Decision support tool	Municipal Strategy	Cross-border Strategy Urban Project

).

Table 8 presents, for each participatory process selected as a case study, the following information: the identifier corresponding to the name of the research or governance project that received public funding and within which the participatory process was implemented; the time frame of the participatory process (start and end year); the scale of intervention; the hazards addressed, categorized according to UNDRR [103]; the types of actors involved; and the main outputs. These case studies satisfy the selection criteria described in Section 2.5: they focus on managing disaster risk and building urban resilience in multi-hazard scenarios at the local scale, all fall within the defined time frame (since 2019 onwards), and the research team was directly or indirectly involved. Two of the case studies were initiated by the same public institution (the same municipal office) in 2019 and 2024, while the third and fourth processes, initiated by a different public institution, enable comparative analysis.

• Tourism-friendly Cities (TFC) [104] aimed to collaboratively define strategies and pilot actions at the municipal level for planning sustainable and resilient tourist destinations, addressing primarily overtourism and challenges arising from the COVID-19 pandemic. The process initially involved a stakeholder working group composed mainly of institutional actors already engaged in other projects, before expanding to include local communities and both domestic and international tourists. The outputs are twofold: (1) the co-development of the Integrated Action Plan for Sustainable Tourism in the City of Genoa, and (2) the co-implementation of a small-scale action, specifically the restoration of a hiking trail in the peri-urban green belt (instigator: Municipality of Genoa) [105].
• URCA! [106] implemented a community-based participatory research process to co-design a decision-support tool for planning sustainable urban drainage systems at the sub-basin scale. This initiative, established through a collaboration between the municipal authority and a university, formed an interdisciplinary working group that engaged intergenerational local communities (ages 6+) in Genoa’s Sampierdarena neighbourhood. Participants co-produced flood risk maps and planned both structural and non-structural flood adaptation measures (instigator: University of Genoa, Department of Civil, Chemical, and Environmental Engineering) [107].
• RETURN [108] was an expert-based participatory planning process for Civil Protection in Sanremo. The initiative focused on defining the Emergency Limit Condition to address simultaneous or sequential seismic and flood hazards (instigator: Municipality of Sanremo) [109].
• HERIT ADAPT [110] intended to collaboratively develop mitigation and adaptation strategies to address overtourism and the impacts of climate change on UNESCO sites. In this case, the Steering Committee for the “Le Strade Nuove and the System of the Palazzi dei Rolli” UNESCO site in Genoa was expanded to incorporate new technical and non-technical expertise. The outputs of the process include the development of a cross-border strategy for protecting natural and cultural assets and enhancing the resilience of tourist destinations, as well as the improvement of accessibility at a museum in Genoa through the installation of dedicated signage (instigator: Municipality of Genoa).

As described in Section 2.5, at least two of the selected case studies—TFC and HERIT ADAPT—were initiated by the same public administration. Figure 6 presents the results of the application of the institutional capacity self-assessment checklist, completed on behalf of the Municipality of Genoa.

The overall institutional capacity index of the Municipality of Genoa prior to the TFC project planning was 0.6, corresponding to the “stabilizing” category, whereas the a priori index for HERIT ADAPT was 0.55, corresponding to the “surviving” category.

Table 9. The high-quality participation indicators for evaluating participatory processes in DRM and URB.

Indicators	TFC		URCA!		RETURN		HERIT ADAPT
	Municipality of Genoa (2019)		Municipality of Genoa (2022)		Municipality of Sanremo (2023)		Municipality of Genoa (2024)
	s	i	s	i	s	i	s	i
CHR01	0.99	0.15	0.67	0.10	0.67	0.10	0.82	0.12
CHR02	0.10	0	0.43	0.02	0	0	0.33	0.01
CHR03	0.5	0.06	1	0.12	1	0.12	0.5	0.06
CHR04	1	0.07	1	0.07	1	0.07	1	0.07
CHR05	0.68	0.11	0.45	0.07	0.26	0.04	0.39	0.06
CHR06	0.875	0.04	0.33	0.02	0.42	0.02	0.875	0.04
CHR07	0.80	0.12	1	0.15	0.30	0.05	1	0.15
CHR08	0.18	0.01	0.33	0.03	0	0	0.19	0.02
CHR09	0.45	0.06	0.45	0.06	0	0	0.45	0.06
PRC01	0.62	0.07	0.66	0.07	0.43	0.05	0.59	0.06
PRC02	0.34	0.04	0.60	0.08	0.60	0.08	0.47	0.06
PRC03	0.63	0.08	0.80	0.10	0.47	0.06	0.57	0.07
PRC04	0.17	0.02	0.17	0.02	0.17	0.02	0.17	0.02
PRC05	0.72	0.07	0.40	0.04	0.36	0.04	0.38	0.04
PRC06	0.43	0.04	0.60	0.06	0.53	0.05	0.40	0.04
PRC07	0.97	0.11	1	0.11	0.80	0.09	0.93	0.10
PRC08	0.68	0.09	0	0	0	0	0.50	0.07
PRC09	0.93	0.11	0.97	0.11	0.8	0.09	0.93	0.11
OTP01	1	0.12	1	0.12	1	0.12	0.38	0.05
OTP02	1	0.14	1	0.14	1	0.14	0.38	0.05
OTP03	0.40	0.04	0.60	0.07	0.60	0.07	0.19	0.02
OTP04	0	0	0	0	0	0	0	0
OTP05	1	0.11	1	0.11	1	0.11	1	0.11
OTP06	0.875	0.08	0.71	0.06	0.71	0.06	0.875	0.08
OTP07	0.51	0.05	0	0	0	0	0	0
OTP08	0.5	0.07	0.5	0.07	0.5	0.07	0.5	0.07
OTP09	0.76	0.08	0	0	0.75	0.08	1	0.11

presents the results of applying the support tool for monitoring and ex post evaluation of participatory processes in DRM and URB across all four case studies.

Figure 7 presents: (i) a graphical representation, employing a traffic light colour scale, of the results from the application of the support tool for monitoring and ex post evaluation of participatory processes in DRM and URB, allowing for a comparative investigation of the four selected case studies; and (ii) the completed graphical framework correlating the assessment of the Municipality of Genoa’s institutional capacity with the ex post evaluation of the TFC and HERIT ADAPT participatory processes.

4. Discussion

The objectives of the study were as follows: to develop and validate two innovative indicator-based tools—one designed for the ex ante assessment of institutional capacity, and the other intended to support monitoring and ex post evaluation of participation in DRM and URB; and to examine the relationship between institutional capacity and the quality of participatory processes.

A first finding emerges from the assessment of the institutional capacity of the Municipality of Genoa prior to the initiation of the participatory processes TFC and HERIT-ADAPT. The analysis reveals a decline in institutional capacity to manage participatory processes in DRM and URB, with the overall score decreasing from 0.6 (stabilizing) in 2019 to 0.55 (surviving) in 2024. This finding highlights that institutional capacity is not an inherent characteristic but rather a quality that can be actively developed and strengthened [111]. Moreover, the fact that HERIT ADAPT was implemented after TFC yet continued to encounter similar constraints underscores the importance of continuous monitoring. In this regard, the self-assessment checklist can serve as an effective tool for tracking institutional capacity over time, thereby supporting more adaptive, effective, and high-quality participatory processes in DRM and URB.

In contrast to the ex ante institutional capacity assessment tool, which was chiefly informed by scientific and grey literature, the development of the monitoring and ex post evaluation tool engaged a panel of domain experts in the weighting phase through the BWM. Consequently, the weights assigned to the indicators also reflect the perspectives and priorities of these experts. Among the main criteria, procedures emerged as the most important (weight: 0.44), followed by characteristics (0.33) and outputs (0.22). This distribution suggests that, in participatory processes, how things are done is perceived as more important than what is done or what outcomes are achieved. Experts thus tend to view participation as an opportunity to democratize decision-making, rather than as a mere methodology for producing predefined outputs. Participation as both process and goal, rather than merely a tool [112] or a slogan [113]. The analysis of sub-criteria further clarifies which elements best characterize a high-quality participatory process. Within the procedures dimension, the most relevant aspects include the extent to which participation incorporates a multi-hazard approach and the degree of integration of existing grassroots practices into formal decision-making processes. Under characteristics, the representativeness of the local stakeholder sample involved emerges as a key factor. Finally, within outputs, the quality of decision-making is linked to the extent to which participatory inputs are effectively integrated and to the degree of transformation in the governance model resulting from the participatory process. The application of the support tool for the ex post evaluation to the four participatory processes reveals a clear hierarchy among the case studies. TFC ranks as the most effective participatory process, with an overall score of 0.63. It is followed by URCA! (score: 0.59), whose performance is primarily driven by a high score in the characteristics criterion; HERIT ADAPT (score: 0.55), which shows relatively balanced results across criteria; and finally RETURN (score: 0.49), which is significantly constrained by its low performance in the characteristics dimension. The findings presented here underscore several noteworthy aspects. All participatory processes demonstrated the capacity to engage a relatively heterogeneous audience of participants through a process structure that proved adaptable to external factors, accompanied by transparent dissemination strategies and accessible methods, and without recording significant dropouts. According to the literature [34] stakeholder heterogeneity is not always adequately ensured, as vulnerable and marginalized groups are often underrepresented. Nevertheless, in some cases, such as the RETURN project, the sample of stakeholders involved was not highly representative of the territorial context. Specifically, only public administrations, industries, and academia were engaged. Moreover, each of these stakeholder categories proved to be rather homogeneous internally. For instance, the industries involved belonged exclusively to the civil engineering sector, thereby excluding other key sectors relevant to post-earthquake or post-flood recovery, as well as to cascading or concurrent events. Participation appears to be relatively intergenerational, particularly in the URCA! initiative, which notably involved participants aged six and above. Conversely, lower scores were observed with respect to interculturalism and gender equality in decision-making. Specifically, none of the processes succeeded in effectively engaging many of the minority groups residing within the intervention areas (challenges included limited access to local community networks, the absence of multilingual communication strategies, and the lack of intercultural mediators, among other constraints). Moreover, most processes exhibited a gender imbalance: men were fewer in number but often occupied leadership positions, while women were more numerous yet predominantly held subordinate roles—with the exception of the TFC case. This finding raises important questions about the need to integrate intersectional and feminist perspectives into participatory and highly inclusive urban planning processes [114,115,116]. In terms of the outputs achieved, all processes produced results consistent with the main planning tools in force at the time of project design. However, in most cases, it appears that each individual output was unable to address multiple hazards within a multi-risk framework, instead adhering to more traditional single-hazard or multilayered single-risk approaches. This outcome is consistent with the limited extent to which the multi-risk perspective was integrated into the participatory processes themselves (only a few participatory activities within each process invited participants to reflect simultaneously on multiple hazards and their interrelations). Moreover, this result is not surprising, given that the processes also exhibited a generally low to moderate degree of cross-disciplinary exchange. Furthermore, it seems that these processes, upon completion, did not succeed in fostering a transformation of the governance model, remaining largely within a business-as-usual paradigm rather than experimenting with more collaborative approaches to public space management. Notably, in the URCA! and RETURN cases, the decision-making processes failed to integrate grassroots practices. This stands in contrast to a strand of the literature [38] arguing that participation should aspire to a culture of full decision-making power, with institutions acting as facilitators and supporters rather than decision-makers. Lastly, none of the processes adopted systematic strategies for monitoring participatory progress. This omission constitutes a missed opportunity, as previous scholars [33] have highlighted that risk reduction and the advancement of urban resilience are inherently long-term, uncertain, and dynamic processes that demand sustained and adaptive monitoring efforts.

By applying a comparative quali–quantitative framework (see Figure 2 and Figure 7), the relationship between the institutional capacity of the Municipality of Genoa in 2019 and 2024 and the quality of the participatory processes TFC and HERIT-ADAPT was examined. The analysis suggests that greater institutional capacity is associated with higher-quality participatory processes. This finding aligns with previous studies indicating that, when formal participation takes place, limited technical, institutional, and economic capacity can substantially undermine the real impact of planning, risking that participatory efforts remain largely rhetorical or merely formalized on paper [117]. Examination of Figure 6 reveals that, among the indicators included in the ex ante institutional capacity assessment tool, only a small subset of critical factors accounted for the variation in the quality scores of the TFC and HERIT ADAPT participatory processes, with political stability identified as the most decisive. The two participatory processes were planned and implemented at different stages of the city council’s legislative term. When TFC was designed, the council was already established and fully operational. In contrast, for HERIT ADAPT it was known that the early phases of the process would coincide with the electoral campaign (political stability score: 0.25). The effects of this electoral context soon became evident: stakeholder meetings became less frequent, several planned activities were postponed, and systematic participation monitoring was interrupted. During this period of stagnation, many of the participants’ inputs were lost and were not reintegrated once the process resumed. This void was swiftly filled by the administrative leadership’s own priorities -largely independent of the participatory process’s intended goals-thereby producing a misalignment between the original objectives and the outputs eventually attained. Limited political stability also had a substantial impact on the time available for conducting the participatory process. Initially, several activities were postponed; subsequently, following the elections, the political composition of the local government changed, resulting in additional delays. On one hand, the new council’s vision diverged from that of the previous administration; on the other, the newly appointed officials were required to engage in an already ongoing participatory process and needed time to familiarize themselves with its structure and objectives. Time constraints are a well-documented challenge for participatory initiatives implemented within call for tender projects, where engagement activities must often conform to the project lifecycle rather than to stakeholders’ availability. In the case of HERIT ADAPT, however, this limitation was further exacerbated by the combined effects of the electoral campaign and the subsequent legislative transition. The compressed schedule prevented the organization of in-person meetings and hindered the triangulation of qualitative and quantitative data. For instance, the mapping of territorial vulnerability and exposure to flood, heat-related, and wind-related risks was conducted solely through the collection of participants’ perceptions and by cross-referencing them with data already available in existing planning tools. It was therefore not possible to complement these outputs with a comprehensive multi-risk assessment or integrated multi-hazard maps; as a result, the three hazards were largely analyzed in isolation. The HERIT ADAPT experience therefore highlights the importance of incorporating buffer periods during the design phase of participatory processes, in order to accommodate potential disruptions such as electoral campaigns or governmental transitions. Another factor that significantly affected the quality of both participatory processes was time availability.

5. Conclusions

Through a comparative research design, the paper develops and validates two innovative indicator-based tools: one for ex ante assessment of institutional capacity and the other for supporting the monitoring and ex post evaluation of participatory processes in DRM and URB. The research presents some limitations related to the use of both qualitative and quantitative indicators and the BMW as a weighting method. Regarding the use of qualitative and quantitative indicators, the main limitation is related to the quality of data collection and coding used to calculate the indicators (e.g., regarding data derived from the recollection of the pre-participatory process status quo) as well as the theoretical completeness of the indicators that have been chosen. Furthermore, bias may also arise from the scoring thresholds adopted, with the risk of introducing a certain degree of interpretive subjectivity. Moreover, contrary to what was envisaged in the methodological approach described in Section 2.2 and Section 2.3, it was not possible for the scoring to be conducted independently by multiple reviewers. The use of the BWM, while offering a systematic and structured approach to weighting criteria, requires experts involved to clearly identify the “best” and “worst” options, a process that can be influenced by cognitive biases, incomplete information, or divergent interpretations among experts. Furthermore, the assumption of preference consistency does not always reflect the complexity of real-world decisions, while the sensitivity of the results to the initial choices (best/worst) can reduce the stability of the conclusions. These aspects suggest caution in interpreting the results and reinforcing the need to integrate the methods used with complementary approaches (e.g., sensitivity analysis or replication on larger samples) to validate and consolidate the evidence produced. An additional limitation of the study relates to the composition of the sample used for the BWM implementation: only one respondent reported working or residing outside Italy (in France), and over half of the participants were female, aged 35–50 years.

Considering the limitations identified, future research could pursue several directions. Theoretically, the range of quali-quantitative indicators integrated into the support tool could be expanded to enable both monitoring and ex post evaluation of participatory processes in DRM and URB contexts. Further studies might also examine how conflict management was approached, whether long-term follow-ups were implemented, and which technical or procedural factors potentially constrained the effective execution of participatory processes. Methodologically, future research could benefit from broadening the empirical base by incorporating a larger number of cases and testing the robustness of the identified patterns through sensitivity analyses or mixed-method designs. Integrating QCA with conventional quantitative approaches could further enhance the external validity of the findings, achieving a more effective balance between qualitative depth and generalizability. With regard to BWM, subsequent studies could engage a more heterogeneous panel of experts or experiment with methodological variants (e.g., fuzzy BWM or hybrid approaches combining BWM with AHP or DEMATEL) to better capture the uncertainty and complexity inherent in decision-making preferences. It would also be valuable to investigate how BWM outcomes vary across different application contexts, decision-making structures, and informational settings. Future research could seek to replicate the study across diverse contexts (spanning sectors, countries, and organizational cultures) in order to evaluate the transferability of the findings and discern recurring patterns as well as context-dependent factors. In doing so, the results would gain greater scientific rigour and applied significance.

In conclusion, the testing and validation of the support tool designed to monitor and evaluate ex post participatory processes in DRM and URB also revealed its potential as a highly effective planning tool during the ex ante phase. In this capacity, the tool can facilitate the planning and design of participatory governance models by refining the definition of objectives and targets, anticipating vulnerabilities and opportunities, and optimizing resource allocation prior to implementation. Moreover, this planning support tool could provide a robust empirical foundation for the development of innovative, evidence-based guidelines for participation in DRM and URB. Such guidelines would not only inform the design of participatory methodologies but also strengthen adaptive and learning-oriented governance systems within disaster risk management and urban resilience contexts.