Next Article in Journal
Improving Energy Efficiency in Buildings Using an Interactive Mathematical Programming Approach
Next Article in Special Issue
The Use of Cool Pavements for the Regeneration of Industrial Districts
Previous Article in Journal
A Managerial and Behavioral Approach in Aligning Stakeholder Goals in Sustainable Last Mile Logistics: A Case Study in the Netherlands
Previous Article in Special Issue
Evaluating the Transition Towards Post-Carbon Cities: A Literature Review
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Framing Sustainability. Evidence from Participatory Forums to Taylor the Regional 2030 Agenda to Local Contexts

by
Maria Stella Righettini
Department of Political Science, Law and International Studies, University of Padova, 35122 Padova, Italy
Sustainability 2021, 13(8), 4435; https://doi.org/10.3390/su13084435
Submission received: 1 April 2021 / Revised: 12 April 2021 / Accepted: 14 April 2021 / Published: 15 April 2021

Abstract

:
The present article introduces an original theoretical framework to investigate how public participation reframes governmental sustainability topics along four dimensions: exploration, prioritization, embedding, and integration. The literature highlights public participation as a strategic sustainability governance tool that can help governments extract local communities’ knowledge to better design policy. In light of this, the article proposes the participatory assessment grid (PAG) to assessing participatory contribution to framing sustainability. The present study adopts a topic detection method to gauging and comparing qualitatively and quantitatively the seven Veneto region participatory forums’ output. In so doing, the article tests the PAG and highlights the bottom-up contribution to the sustainability strategic plan formulation within the regional 2030 Agenda design. As the analysis reveals, participatory forums’ communicative content gives several potential contributions to government policy formulation that are drivers both to enhance public engagement and improve sustainability policy design.

1. Introduction

Participation is considered a pillar of 2030 Agenda governance to adapt global goals, targets, and indicators at the regional and local levels [1]. In light of this effort, it has become commonplace for sustainability policymakers and managers to engage the public in collaborative practices [2], collectively known as public participation. To governments, participation is a strategic tool in policy design [3], particularly when address multidimensional and cross-sectoral issues. Actors involved focusing their attention on different and sometimes conflicting aspects of the policy seeking to build support for their interests or positions [4,5,6]. Participation can be adopted to prepare starting conditions and address the institutional design of complex and potentially conflicting policies and programs. As the 2030 Agenda brings multi-issue aspects competing for attention, framing is crucial in formulating a multidimensional action plan. Several interlinkages between sustainable development goals (SDGs) and an even more significant number of targets are glocal [7] and compete for local public attention needing to be framed and incorporated both in state and local governments’ programs and citizens’ real life [8]. Recent contributions confirm urban sustainable development planning addresses citizens’ concern about the quality of life and environmental protection by increasing their participation in policymaking processes and their involvement in designing nature-based and socially inclusive solutions [9,10,11]. What is commonly agreed upon is that participation can influence SDGs framing, meant as deliberate crafting. One issue can be perceived and shared by actors from various perspectives and can be constructed as having implications for multiple values and actions [12]. As there can be simultaneously many ways of understanding a problem and finding solutions, according to the government and actors involved, participation is considered a suitable tool to elicit and mediate between distinct preferences and facilitate different and more adaptive solutions. Engaging citizens at the local level needs to rethink collaboration settings and partnerships [9] to better establish tools for integrating traditional approaches [10] to empower local communities in practice, allowing them to portray the context of sustainability better and build public acceptance in policymaking. Despite widespread support for public participation, there is no explicit consensus on the purpose or evidence of this practice’s actual outputs and its effects on policy framing. In the vast literature on sustainability framing, there are two fundamental approaches: the one considering framing an independent variable of sustainability policies, and the other one considering framing a dependent variable of local government organization and culture [8,13] scientific expertise [14] and civic engagement [13,15,16,17,18]. This article is part of the second field of studies and considers sustainability framing a dependent variable of public engagement and participation.
Regarding participatory practices policy, the literature highlights methodological challenges in assessing how individual actor engagement can determine how issues are defined collectively [12] and how to extract valuable knowledge from the participatory output. Building specific taxonomies may help examine sustainability social construction and discourses by citizens and stakeholders in planning processes within the 2030 Agenda. The present article addresses such questions from a methodological perspective by developing an analytical tool for overcoming such difficulties by categorizing and assessing participatory output features [19]. Extending a previous analysis of the same case study [20]. In doing so, it relies both on policy framing and policy design literature analyzing participatory tools in plan formulation [19]. The article argues sustainability reframing results from the asymmetric dialog between government and stakeholders [3] having diverse problem perceptions and salience distribution. The participatory assessment grid (PAG) we present does not consider organizational variables [20] nor procedural aspects [21]. It is based on discursive participation output, written texts, and content features and highlights several framing dimensions that participation can contribute. The PAG is tested on the outputs of seven participatory forums activated in the Italian region within the Venetian regional government’s provincial forums during 2019 when drafting the regional strategic program planning for the 2030 Agenda [22]. More precisely, this research’s main endeavor is to explore how bottom-up public reframing matters in updating government discourse and agenda in the formulation phase. In doing so, the article uses topic detection [23], a lexical-textual approach focusing on relationships between qualitative aspects (e.g., innovative word clusters and their interrelations) and quantitative factors (e.g., word frequencies and territorial distribution) of lexical variables characterizing public participation outputs.
The paper is structured as follows: the second section illustrates the theoretical framework and the PAG to categorize framing features in collaborative processes in the 2030 Agenda formulation phase; the third section is devoted to illustrating the method, specifically the case study and the content analysis approach (topic detection) to validate the PAG and be applied to the public participatory output; the fourth section presents the topic detection results related to the seven public forums activated by the Veneto region; the last section, discusses the research results and draw some conclusions.

2. The Theoretical Framework

While the framing process refers to the varieties of modes by which people develop a particular conceptualization of an issue or reorient their thinking about an issue, the reframing process is defined as “a process seeking to change how other actors initially framed issues” [24] (p. 2). Such a process that goes from framing towards reframing may result in a conflict or mutual learning process [25] whereby knowledge production about a problem results from diverse actor knowledge interacting with each other. Framing activity has been frequently analyzed as a top-down strategy adopted by political leaders to manipulate and deceive individuals through communication; reframing activity has been addressed as a bottom-up strategy by which people acquire or modify common beliefs through participation self-coordination [26]. From the top-down perspective, framing is an élite activity intending to mobilize attention on policies that are likely to be adopted or implemented [27], or it is “an attitude towards an object, in this view, is the weighted sum of a series of evaluative beliefs about that object” [28] (p. 113). In complex policymaking, people are exposed not just to one frame of a problem, rather to several competing frames [27]. Multiple potentially conflicting goals characterize the 2030 Agenda to meet global decarbonization, as in the competition between promoting environmental protection and economic growth, economic growth and overtourism, agricultural exploitation, and consumer health protection [10]. The mainstream global agenda has generated a complex “top-down” framing of goals, targets, and measures that lead to the following question: to what extent local communities recognize and embrace the top-down sustainability conceptualization of local needs, policies, and measures? Top-down sustainability, generally defined and supported by a technocratic élite working for governments, tends to exclude citizens, stakeholders, and local communities in policy formulation and knowledge production [29,30]. Scholars have argued that besides top-down framing, tools [31], bottom-up strategies must be fostered and put in place by national and local governments to enhance the 2030 Agenda impact. Beyond the technicalities, a bottom-up sustainability strategy claims a more inclusive definition via policy takers and stakeholders’ direct participation in collaboratively understanding, designing specific tools [22]. The “collaborative governance” (CG) is a method allowing for novel insights on participatory policy formulation while holding different perspectives together [2]. CG is a strategy to mobilize and organize public attention and to extract information from potential policy beneficiaries and stakeholders. It is broadly defined as a dynamic interaction between public and private actors to secure the provision of common goods [21]. More concretely, it is a place-based process in which either the public authority “invites” social and business actors to participate in different policymaking phases or when the latter actors mobilize the public authority to engage in joint solutions of sustainability problems. The joint involvement of diverse public and private actors generates new challenges and potential benefits regarding the greater compatibility between short-term interests with a long-term perspective necessary to achieve sustainability objectives [20,32]. CG’s ideas and practices are grounded in the assumption that policymakers are not the problem but could be part of the solution [4]. Generally, CG practice is grounded in participatory and disciplined initiatives [33,34] and on methods for shared knowledge production, mobilizing policy takers, and eliciting latent social knowledge within multilevel and multi-actor contributions in the program formulation and creation [35]. Therefore, users, citizens, and stakeholders contribute on the assumption that they are experts for their problems [19]. This approach marks a paradigm shift from the dominant consideration of citizens as passive targets or users of public programs and services towards proactive citizens, users, and stakeholders, who are program co-designers [36,37,38,39] and could actively contribute increased shared knowledge in the policymaking processes. The rise of collaborative sustainability policymaking has led to developing specific participatory tools that bring diverse people together at a specific location to facilitate the dialog on specific problems and to activate mutual knowledge processes. In so doing, individual actors use each other’s competencies to face increasingly challenging sustainability problems and balance potentially conflicting shared goals and strategies [21,31].
In facilitating collaborative tools, the literature points out that leaders often selectively include individual stakeholders with specific skills or perspectives expected to contribute significantly but exclude others who might increase transaction costs without making significant or unique contributions [40]. Therefore, procedural aspects of participatory tools have been examined as crucial aspects of participatory methods in collaborative governance, providing involvement without discrimination. By design, they include all those affected by a problem into a collaborative process and discuss solutions. Within a CG framework, participatory tools open up the structuring phase of sustainability problem definition to collect as much information as possible on the issues at stake. This aspect is particularly relevant as the 2030 Agenda is not representative per se of sustainability topic mixes that characterize different local policy perceptions and design. For example, in the case [41], the same overtourism issue could be framed differently by people living in economically underdeveloped areas or people residing in areas characterized by highly developed tourism exploitation. Therefore, an inclusive and bottom-up formulation of sustainability policies is achievable through mechanisms taming and embedding global problems in local contexts and fostering policy takers’ ability to define achievable ends by suitable means [42]. The theoretical framework illustrated in this section (Table 1) modifies and expands the model used to assess participation contribution in the policy process by focusing on the policy formulation step [19]. In all collaboration, governments are set up in the policy formulation phase by directly involving individual citizens, individuals through associations they belong to, individuals and associations, or involving other public and private organizations. The rationale of framing and reframing is to activate a communicative process. The empirical analysis of communicative processes and gauge of conceptualization entanglements are the methodological challenges [41]. The article relies on framing theory and participatory assessment methodology in policy formulation [14,19,43,44] to elaborate an original theoretical framework. It aims to grasp better the local participatory content and contribution to sustainability agenda formulation [45]. Given the potential asymmetry between top-down government framework and bottom-up local stakeholder perception and experience (reframing), we ask what happens during the participatory process? What types of contributions about sustainability conceptualization emerge from the public debate? What are the reframing aspects from which governments could learn to tailor sustainability policies? The participatory assessment grid (PAG) [19] identifies four types of participatory contribution (Table 1) sustainability framing in planning processes to be and gauged by appropriate indicators.
The communicative output of a collaborative experience unfolds and articulates as follows:
(1)
Exploration refers to the extent to which participation identifies (or does not) a higher number of sustainability topics than those formulated by other actors at the start of the participatory process. Through facilitating the emergence of new concepts and issues, exploration reveals to what extent government framing and participant reframing differ in the number and content of sustainability topics and measure unexplored issue-framework;
(2)
Prioritization: refers to the degree of importance/relevance participants attribute to sustainability issues. The prioritization reveals what the people urge to address before in accordance or contrast with the government. The most frequently a topic is addressed, and the most it is salient;
(3)
Embedding: refers to participants’ attention according to local needs and real-life experiences. The embedding dimension reveals whether and to what extent local communities differentiate from each other or converge in conceptualizing sustainability problems and solutions. This is the case when the same problem is perceived as negative or positive according to different local contexts;
(4)
Integration: It refers to relationships participants establish within items in the same topic (clusters) and across sustainability topics (overlapping). The integration contribution allows understanding how sustainability issues and targets are understood and connected.
The PGA allows us to empirically observe, in a dynamic way, how the complexity of participatory contributions unfold and articulates in the formulation phase of government planning.
The following section illustrates the case study, the Veneto region participatory tool characteristics, and the methodology to assess the participatory contributions under scrutiny.

3. Methodology

3.1. The Case Study

This section analyses the case of the seven regional forums in the Veneto region (Veneto region, Italy) around the initiative to adapt the 2030 Agenda to the local context’s needs [46]. Fundamental to the novel methodological approach is that this paper’s author was directly involved in the initiatives understudy. (At the time of the study, the author was part of the scientific board supporting the steering group to adopt the regional sustainability strategic plan.) The great variety of goals, targets and indicators included in the 2030 Agenda have raised the core problem of defining the 17 Sustainable Development Goals (SDGs) sustainability using a common language to better adapt SDGs to local contexts. This common language includes several targets explicitly addressed in a cross-sectoral mode, which boost interrelation among economic, societal, environmental, and governance sustainability targets. The common language of intertwined goals and targets should be addressed and build through inclusive governance at the national and sub-national levels by drawing on partnerships between public organizations, the private sector, and civil society.
The present section illustrates such a process started within the Italian Environmental Ministry’s initiative involving all the twenty Italian regions. In Spring 1999, the National Strategy for Sustainable Development (NSSD) required Italian regions to approve their strategies consistent with the National objectives and indications. The National Strategy contains strategic choices and objectives divided into five thematic areas: people, planet, peace, prosperity, and partnership and five governance vectors of sustainability: (1) common knowledge, (2) monitoring and evaluation of policies, (3) participation and partnerships, (4) education-awareness-communication, (5) efficiency of public administration-management and public resources. According to the national framework, each regional government had to decide how and to what extent to design a participatory process to share the sustainability agenda’s formulation phase with society and the cities. We specifically analyze the Veneto region’s case and the process of participatory sustainability agenda building that involved all the seven Veneto Provinces in Autumn 2019. The decentralized participatory tool and process adopted were inspired by the idea that a less formal procedure can facilitate people’s engagement and a constructive contribution to the ongoing planning process. The Veneto region has five million inhabitants, is located in Northeast Italy, the capital city is Venice, and the region includes six other medium-sized provinces. Agriculture, manufacturing, and tourism are among the primary sources of economic wealth and development in the region. The rapid economic growth in the 1970s has increased regional GDP and the employment rate. It has also caused severe stress on the territory, increasing land consumption, exploitation of water resources, and pollution growth. This region belongs partially to the Po river valley. It is one of the hotspots in Europe for air quality, where efforts to meet the European standard for PM emissions have been generally unsuccessful [47].
The decision to undertake an open and decentralized participatory process was favored by the political situation: the upcoming administrative elections. The outgoing government would reapply with an excellent probability of victory. The regional government prepared the participatory process by analyzing statistics that highlighted the regional overcoming of critical national thresholds in some activity areas. Veneto region is second in Italy (after Lombardy) for land consumption rate and nonrenewable raw material consumption rate; furthermore, most of the region’s cities are affected by overtourism impacts. Sustainable lifestyle issues were conversely framed in a more abstract and general way [48] and referred to social and sanitary issues without using statistics. In each forum, a regional government representative illustrated the Veneto region’s positioning concerning the National Sustainability Agenda and its 17 SDGs, targets, and indicators. The positioning report specifically identified regional strengths and critical points compared with the other nineteen Italian regions. In the light of this, the regional government decided to put under people’s attention statistics regarding the most critical issues, land use, circular economy, sustainable lifestyle and sustainable tourism, to star the public debate.
The Veneto region officially invited all its stakeholders throughout the region to participate in seven regional forums (72% of the formally invited attended the forums). The invited stakeholders included various subjects: municipalities and their investee companies, nonprofit organizations, environmental associations, interest groups, regional and local service companies, companies in which the region participates, and schools and regional educational authorities. In addition to the formal invitations, the Veneto region made public the Forums’ Agenda on its website, inviting everyone to participate freely and informally. Following the same format, the forums took place in all the region’s provincial capital cities in October and November 2019. Local stakeholders could formulate, contrast, and discuss their policy preferences on the four earlier selected sustainability issues. The forum involved representatives across different territorial government levels (municipalities, provinces, public companies, state agencies, and decentralized authorities) and traditional policy sectors. Before the session starts, participants were divided into four groups, according to the four themes of interest of the region (i.e., land use, circular economy, sustainable lifestyle and sustainable tourism), and distributed, ensuring the same variety of organizations present and preventing colleagues from the same organization from participating in the same group. Then, an interactive, multistep process started. In the first step, each participant individually and freely formulated their proposals to face the four policy topics initially identified by the region. In a second step, the interaction took place on an interpersonal level, as each participant contributed through a new proposal that modifying or improving the other ones. Finally, in a third step, each group was collectively asked to reorganize all the proposals on a single topic and identify commonalities and differences emerging from the joint interactive participation. To conclude, each group was asked to vote on the most innovative proposal and the most easily achievable target in short on the topic at stake and present their conclusions to the other groups.
At the same time, a complementary online platform was made available for citizens, who, for various reasons, could not reach the places where the forums took place. The online platform served both to ensure the forum’s transparency and participation of most fragile categories of subjects. Overall, around 380 entities, associations, public and private enterprises, and citizens took part in the regional forums and contributed to animate the public debate on the future regional sustainability agenda [22]. All texts produced during the three phases build the corpus for topic detection analysis. During the regional forums, the work carried out produced 2688 Post-it notes, each of which contained a proposal merging written contribution at individual and group level. According to the national framework and methodology adopted, the next step would be to increase the awareness and communication of participatory outputs. Finally, the Veneto region invited (December 2019) all those who participated in regional forums to discuss the participatory tool and process results. The high participation in this last phase proved that people’s awareness of being onboard the regional strategic planning formulation improves the communicative process. It happens when participants repeatedly interact in a public space and, most importantly, when they can evaluate and discuss their outputs [22]. We are now presenting the variables considered in analyzing the participatory output content.

3.2. Methods

Even though many techniques are available [43,49], evaluating public participatory outputs and contribution to sustainability conceptualization is rare. This article adopts a method that relies on automated content analysis procedures of the participatory forums’ output. The content analysis allows assessing the participatory contributions to regional sustainability planning formulation using lexical variables and transforming PAG analytical dimensions (participation purposes) into variables (Table 2). The central motivation in using content analysis is to extract topics (lexical clusters) by classifying portions of texts and elaborating them quantitatively and quantitatively according to the PAG dimensions (Table 2).
The gathered data were processed using the R-based version of an open-source software IRaMuTeQ for topic detection, a lexical approach to written texts according to the Reinert method [50,51] using correspondence analysis (CA). The algorithm was applied to the written texts produced during participatory forums’ sessions [52]. Topic detection requires verification of the corpus tractability (The corpus tractability is determined by the number of hapaxes, words used only once, on the total of words, which must be minor of 50%. In our case, the hapax index is 45%) and prior work of cleaning the textual corpus by eliminating errors, checking grammar, and labeling written texts [23]. Then, topic detection systematically explores participatory output by analyzing co-occurrences of words as they appear in the portions of the text (phrases of at least 40 words) and assigning them to a limited number of clusters (topics or semantic areas) that reflect their primary content. The topic detection extracts inductively latent topics meant as the most frequent issues addressed by the participants. It served to translate individual and group outputs into collective output by analyzing the participatory production as a whole [12,50]. Topic modeling serves for validating the PAG by rendering operational the idea of “framing”, “reframing” and its variance along with three of the four specific dimensions of interest: the exploration of new sustainability topics (new cluster of words), their prioritization meant as the general and territorial order of relevance (frequencies); finally, it allows assessing lexical relationships (closeness/distance and overlaps) between sustainability topics (Table 2). The PAG embedding dimension across the regional forums is assessed using CA through the chi-squared correlation measure between topics according to the territorial variable.

4. Research Results

The first research result regards the frequency of participatory outputs according to the four initial governmental topics (Table 3). Regional forums generated 2688 written proposals for intervention that constitute the corpus of 35,800 occurrences/words to be processed by the software. Table 3 shows participatory written outputs distribution between the initial four critical issues proposed for developing the regional 2030 Agenda. It shows that sustainable lifestyles attract more attention among participants to the regional forums among the four topics: 754 proposals to sustainable lifestyles, 666 proposals to sustainable development and the circular economy, and 602 proposals to land use and oil consumption.
According to the PAG illustrated above (Table 2), a second elaboration of the participatory outputs analyses more deeply how the initial government framework has been re-elaborated by participants. As the ambiguity of multifaceted policy problems stems from the fact that their interpretation is not given but rather a matter of definition [35,53], we explored the diverse participation contributions throughout the four dimensions evidenced by PAG. With this premise, participation is the independent variable, a local sense-making device for extracting helpful knowledge, shaping, and adapting sustainability in the local context. Participants’ written proposals are the natural source to study the reframing output (our dependent variable) in its various dimensions.
Starting with the PAG explorative dimension, topic detection reveals new hidden topics emerging from the overall participatory processes. The software identifies somatic groups or classes with similar content (semantic words) through a clustering procedure based on the co-occurrence of words. In Table 4, we can observe seven clusters of items (instead of four) that emerged from the analysis that corresponds to a more significant number of SDGs and relative targets addressed in the participatory forums.
Table 4 also shows most of the SDGs are addressed by the regional government and regional forums, except for SDGs 3, 4, and 9, which are addressed exclusively by the regional forums. These latter focus attention on the educational aspects of circular economy (SDG 4) and on the interrelationships between health protection and reduction of polluting transport systems (SDGs 3 and 9). These themes emerge as the most innovative as regards the exploratory contribution of participatory forums. Contrariwise, SDGs n. 1, 2, 5, 10, 14, 16, and 17 are absent over the participatory process.
Among the seven topics emerging from participatory forums [8], some topics attract more attention than others in the overall participatory production. Table 5 highlights attention distribution between the seven topics, among which the d) projects for green area repurposing and e) circular economy are the most relevant. Latent topics divide into two prominent families: topics (d) and (e) related to industrial production sustainability, circular economy, reuse (29.7% total of occurrences), and all the other topics related to urban sustainability (71.3% total of occurrences).
The topic prioritization (the most frequent references in the written texts) shows stakeholders and citizens are interested in urban sustainability. They highlight crucial aspects of urban systems, including alternative transportation systems, green space scarcity, public services availability, quality of life in urban neighborhoods, and innovative and sustainable local tourism. Participants’ concern for industrial production impact and circular economy development are prominent items related to waste management and recycling systems and sustainability education.
The contribution to embedding sustainability in local contexts unfolds from the same corpus of texts using CA. Figure 1 shows differences and convergence in topic reframing by province. When provinces stay near the zero lines, their topic frame is similar in intensity and content to that of the other provinces. In contrast, provinces far from the zero line are negatively (under framing) or positively (over framing) correlated with the topics and differentiate significantly from the others.
As evidenced by Figure 1, local participants differentiate sustainability reframing across the territories according to topic mixes characterized by different degrees of attention and salience. The seven Venetian provinces prioritize the macro topics differently. As theory argued [41], local sustainability framing variance reflects differences, negative or positive, in salience and attention local actors give to the same problem. The provincial characterizations of the sustainability prioritization were as follows: in Belluno, a small town located in a mountain area and characterized by limited economic development and depopulation, participants overemphasized topics concerning tourism innovation, digital tourism to enhance tourism economy-based and local tourism valorization while in Venice, structurally affected by overtourism phenomena, participants underemphasized these aspects; in Padua and Verona, two intensely industrialized and polluted areas, participants were the most concerned with redevelopment and green space regeneration added to developing new meeting places for local communities; in Venice participants highlighted reuse issue, reflecting concern and salience for the waste problem caused by overtourism. Interestingly, along with emerging new shared sustainability topics (smart transportation) and business-related issues (circular economy), local communities’ expectations for future policies and interventions were more converging. Transportation and waste reduction are problems perceived throughout all territories. Together, the regional forums diverge further on policy issues, such as urban green spaces and especially the reuse, their concern is related to local context characteristics.
To investigate participatory integration contribution, we used topic analysis again. The Cartesian plane in Figure 2 highlights the closeness between semantic clusters, lexical profiles, and within each cluster between keywords.
Each colored cluster indicates homogeneous solutions to sustainability problems; some clusters are far from the center, while others are close to the center and close to each other. The spatial position indicates lexical correspondence within and across clusters or words (topics). In Figure 2, we may observe semantic correspondence within each of the seven clusters whose words are characterized by different dimensions, spatial positions, and linkages. In each cluster, the most frequent linkages between words deploy as follows: in the topic (a) digital platform (piattaforma) linked to the operator (operator) and tourist offer (offer); in topic (b), the most frequent linkages are tourism (Turismo), valorization (valorizzazione), local (territorio) and traditions (tradizioni); in topic (c) the most frequent linkages are historical center (Centro Storico), commerce (commercial) and aggregation (aggregation) linked to family (famiglia), space (Spazio) and nursery schools (nidi); in Topic (d) the most frequent linkages are green (Verde), buildings (Edificio), area (area) and repurposing (riqualificazione), warehouse (capannone). In topic (e), the most frequent linkages are as follows: circular and economy (circolare and economia), raise awareness (sensibilizzare) and school (Scuola), and in the most intertwined topic (f), the most frequent linkages are product (Prodotto), production (produzione), material (materiale), recycling (riciclare) and packaging (imballaggio). Finally, in topic (g), the most frequent linkages are transport (trasporto), mobility (mobilità), public (pubblico), cycle path (ciclabile e pista), parking (parcheggio), traffic and car-sharing, and economic incentives provision and education are among the most frequent items linked to transport system transformation.
The automatic elaboration allows observing also overlap and integrations across different topics emerging from participatory forums. Looking at the Cartesian axes (Figure 2), we observe some interconnected clusters. Clusters of words closer to the center tend to be most intertwined or to overlap. In contrast, the farther clusters are more peculiar and specific and tend to differentiate more from the others. It is primarily the smart mobility cluster, and, to a lesser extent, the circular economy cluster tends to be more specific (distant from the center) than others.
The most interconnected are “local tourism” and “reuse”, close to each other. Reuse is specifically linked to waste reduction by tourists and green energy, energy-saving, and incentives devoted to fostering sustainable packaging. Additionally, local tourism exploitation throughout cultural heritage valorization (environmental, cultural, and architectural) and local tourism innovation are remarkably close to each other and local tourism and tourism innovation, which call for new digital platform development. Repurposing public spaces and old buildings and green regeneration is at the crossroad between green spaces and local tourism valorization. The “smart mobility” cluster, on the contrary, is far from the others and, therefore, indicates an autonomous and independent significance from other aspects participants dealt with. Within this cluster, the city transport system” s inefficiency, services for reducing intercity mobility inconvenience attract much attention to correcting. The mobility issue salience stems from a historical structural deficiency of the region.
The regional government has planning and financing competencies—the regional transport action plan was approved in September 2019 after twenty years—while municipalities and private transport service companies share local transport services implementation and evaluation competencies. Regional forums insisted on intelligent mobility, namely on cycle routes increasing, electric mobility, intermodal car parks, car-sharing and pooling, skateboard mobility, and growth of low carbon public transportation services. Local communities indicate interconnections between urban mobility features, pollution reduction, and energy efficiency. In conclusion, we may affirm two main original aspects that emerged from the sustainability reframing process stemming from the local participation: smart transportation—not yet included in the initial government framing, and the educational approach to sustainability and circularity.
The sustainable development strategic plan approved by the Venetian regional government in June 2020 contains some reflections and objectives identified by regional forums, for example, smart mobility for persons and sustainable logistic transport for goods, which were adapted and integrated into the new pandemic context of COVID-19.

5. Discussion and Conclusions

The present article analyzed local sustainability reframing emerging from participatory forums within the regional sustainability planning formulation. The study did this by adopting an original analytical framework to investigate participation dimensions along, which sustainability reframing can alter the top-down framework. It highlights how local communities can integrate technocratic knowledge based on data, thresholds, and statistical indicators [54]. More informal and decentralized procedures serve policymaking appropriately, developing a dynamic voluntary learning process where public and private actors are encouraged to learn how to use each other’s competencies [10] and can develop new perspectives and frameworks to better cope with local sustainability challenges [31,55]. Through the PAG and the assessment of participatory outputs using topic detection, the article aimed to provide an alternative approach to assess participatory contributions on a large scale based on the contents of participatory outputs. The twofold contribution to the existing literature is as follows: First, the article confirms the PAG’s validity in highlighting the policy framing process’s multidimensionality. Participatory tools create conditions for constructive discussions, expand opportunities to acquire local factual knowledge, and explore the potential of increasing open and generative planning processes [21]. The research results show the participative process modify and partially integrate how the regional government initially defined the sustainable development plan’s priorities.
Exploration. We observed local actors’ capacity to conceptually unfold and open out the government policy topics to translate them into new operational and concrete local dimensions and sustainability practices. Seven latent sustainability topics, instead of the initial four, emerged from multi-actor interactions. Intelligent transportation systems and services drew regional policymakers’ attention to neglected issues and related specific sustainability items to be addressed by the strategic regional sustainability plan.
Prioritization. Stakeholders prioritized latent sustainability items, attributing them to a different level of attention and importance. Higher attention (number of the written proposals) is devoted to waste collection, recycling, and reuse education, on one side, and to urban local service availability and quality, on the other side. Social/urban and environmental sustainability are the most relevant issues to local communities.
Embedding. The inductive discovery of latent policy frames reveals topic covariance across different local communities characterized by diverse economic and environmental conditions. The topic mix represents the local specificity of sustainability discourse and how priorities may vary and combine differently.
Integration. Participants contribute to focus on sustainability issue overlapping, particularly in urban sustainability, on one side, and circular economy and reuse, on the other side. In the formulation phase, sustainability issue integration and interaction seem related to participants’ ability to collectively come to a grip with interconnected challenges associated with more sustainable urban policies and services.
In the light of this, the PAG allowed observing the enrichment government could draw from the bottom-up reframing process.
Second, the article contributes to further methodological advancement in analyzing participatory output in a qualitative/quantitative fashion. Topic detection helped operationalize the reframing process, explore the local sustainability hidden agenda, and gauge the several aspects of sustainability reframing contributions.
From the methodological point of view, limitations in this exercise of extracting and analyzing participation output could be in the self-selection bias, which is how citizens and stakeholders decide to participate or exclude themselves from intervening in the process. Changing the forums’ participants would probably change the reframing process output and potentially impact each of the four identified dimensions: exploration prioritization, embedding, and integration. The research method adopted (topic detection) [23], connecting individual discourse to latent broader collective knowledge and communication, reveals a spectrum of participation contribution. Due to the narrow case study presented, PAG validation by topic detection needs to be extended to a more significant number of cases and a larger body of texts.
The approach used and illustrated in this article could also apply to participation and policy change studies across other policymaking phases and government levels. On a larger scale, it might also interest specific sustainability policy fields and SDG. The PAG may be adapted and extended beyond the formulation phase as the framing process continues through implementation. The approach used has more general implications for policymakers to increasing the legitimacy of decisions at the regional and local levels. Since it is difficult for governments to have an overall and evidence-based vision of local communities’ real needs, attention to the output of the participatory process and its enhancement could become an ineliminable requirement within the 2030 Agenda governance. In stimulating and creating more inclusive sustainability policies, a systematic investigation and follow-up of participation results can enhance the local community’s communicative transformation capacity within the regional and local policy cycle. They can probably also contribute to spreading and consolidating this type of tool and practice.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Florini, A.; Pauli, M. Collaborative Governance for the Sustainable Development Goals. Asia Pac. Policy Stud. 2018, 5, 583–598. [Google Scholar] [CrossRef] [Green Version]
  2. Ansell, C. Collaborative governance. In The Oxford Handbook of Governance; Levi-Faure, D., Ed.; Oxford University Press: Oxford, UK, 2012; pp. 498–511. [Google Scholar]
  3. Ansell, C.; Gash, A. Collaborative Governance in Theory and Practice. J. Public Adm. Res. Theory 2008, 18, 543–571. [Google Scholar] [CrossRef] [Green Version]
  4. Schmitter, P.C. Participation in Governance Arrangements: Is there any reason to expect it will achieve “Sustainable and Innovative Policies in a Multi-Level Context”. In Participatory Governance; Grote, J.R., Gbikpi, B., Eds.; VS Verlag für Sozialwissenschaften: Wiesbaden, Germany, 2002; pp. 51–69. ISBN 978-3-8100-3237-9. [Google Scholar]
  5. Heinelt, H. Participatory Governance. Debating Democr. Legitimacy Eur. Union 2007, 217–232. [Google Scholar]
  6. Heinelt, H.; Getimis, P.; Kafkalas, G.; Smith, R.; Swyngedouw, E. Participatory Governance in Multi-Level Context; VS Verlag für Sozialwissenschaften Springer Fachmedien: Wiesbaden, Germany, 2002. [Google Scholar]
  7. Gupta, J.; van der Leeuw, K.; de Moel, H. Climate Change: A ‘Glocal’ Problem Requiring ‘Glocal’ Action. Environ. Sci. 2007, 4, 139–148. [Google Scholar] [CrossRef] [Green Version]
  8. Laurian, L.; Walker, M.; Crawford, J. Implementing Environmental Sustainability in Local Government: The Impacts of Framing, Agency Culture, and Structure in US Cities and Counties. Int. J. Public Adm. 2017, 40, 270–283. [Google Scholar] [CrossRef]
  9. Constantinescu, M.; Orîndaru, A.; Căescu, Ș.-C.; Pachițanu, A. Sustainable Development of Urban Green Areas for Quality of Life Improvement—Argument for Increased Citizen Participation. Sustainability 2019, 11, 4868. [Google Scholar] [CrossRef] [Green Version]
  10. LopezDeAsiain, M.; Díaz-García, V. The Importance of the Participatory Dimension in Urban Resilience Improvement Processes. Sustainability 2020, 12, 7305. [Google Scholar] [CrossRef]
  11. Ferreira, V.; Barreira, A.P.; Loures, L.; Antunes, D.; Panagopoulos, T. Stakeholders’ Engagement on Nature-Based Solutions: A Systematic Literature Review. Sustainability 2020, 12, 640. [Google Scholar] [CrossRef] [Green Version]
  12. Baumgartner, F.R.; Mahoney, C. Forum Section: The Two Faces of Framing: Individual-Level Framing and Collective Issue Definition in the European Union. Eur. Union Polit. 2008, 9, 435–449. [Google Scholar] [CrossRef]
  13. Portney, K. Civic Engagement and Sustainable Cities in the United States. Public Adm. Rev. 2005, 65, 579–591. [Google Scholar] [CrossRef]
  14. Bäckstrand, K. Civic Science for Sustainability: Reframing the Role of Experts, Policy-Makers and Citizens in Environmental Governance. Glob. Environ. Polit. 2003, 3, 24–41. [Google Scholar] [CrossRef]
  15. Craig, D.; Porter, D. Framing Participation. Dev. Pract. 1997, 7, 229–236. [Google Scholar] [CrossRef]
  16. Healey, P.; Booher, D.E.; Torfing, J.; Sørensen, E.; Ng, M.K.; Peterson, P.; Albrechts, L. Civic Engagement, Spatial Planning and Democracy as a Way of Life Civic Engagement and the Quality of Urban Places Enhancing Effective and Democratic Governance through Empowered Participation: Some Critical Reflections One Humble Journey towards Planning for a More Sustainable Hong Kong: A Need to Institutionalise Civic Engagement Civic Engagement and Urban Reform in Brazil Setting the Scene. Plan. Theory Pract. 2008, 9, 379–414. [Google Scholar]
  17. Sarzynski, A. Public Participation, Civic Capacity, and Climate Change Adaptation in Cities. Urban Clim. 2015, 14, 52–67. [Google Scholar] [CrossRef]
  18. Wahlström, M.; Wennerhag, M.; Rootes, C. Framing “the Climate Issue”: Patterns of Participation and Prognostic Frames among Climate Summit Protesters. Glob. Environ. Polit. 2013, 13, 101–122. [Google Scholar] [CrossRef]
  19. Hisschemöller, M.; Cuppen, E. Participatory assessment: Tools for empowering, learning and legitimating. In The Tools of Policy Formulation; Edward Elgar Publishing: Cheltenam, UK, 2015. [Google Scholar]
  20. Garmendia, E.; Stagl, S. Public Participation for Sustainability and Social Learning: Concepts and Lessons from Three Case Studies in Europe. Ecol. Econ. 2010, 69, 1712–1722. [Google Scholar] [CrossRef]
  21. Tosun, J.; Koos, S.; Shore, J. Co-Governing Common Goods: Interaction Patterns of Private and Public Actors. Policy Soc. 2016, 35, 1–12. [Google Scholar] [CrossRef] [Green Version]
  22. Righettini, M.S. Multilevel Co-Governance Within the 2030 Agenda: The Impact of Participatory Processes in the Veneto Region Sustainable Development Strategic Planning. In Proceedings of the New Metropolitan Perspectives; Bevilacqua, C., Calabrò, F., Della Spina, L., Eds.; Springer International Publishing: Cham, Switzerland, 2021; pp. 536–544. [Google Scholar]
  23. Sbalchiero, S. Topic Detection: A Statistical Model and a Quali-Quantitative Method. Tracing Life Cycle Ideas Humanit. Soc. Sci. 2018, 189–210. [Google Scholar] [CrossRef]
  24. Kaufman, S.; Smith, J. Framing and Reframing in Land Use Change Conflicts. J. Archit. Plan. Res. 1999, 16, 164–180. [Google Scholar]
  25. Fischer, F. Reframing Public Policy: Discursive Politics and Deliberative Practices; Oxford University Press: Oxford, UK, 2003. [Google Scholar]
  26. Chong, D.; Druckman, J.N. Framing Theory. Annu. Rev. Polit. Sci. 2007, 10, 103–126. [Google Scholar] [CrossRef]
  27. Gamson, W.A. Talking Politics; Cambridge University Press: Cambridge, UK, 1992. [Google Scholar]
  28. Chong, D.; Druckman, J.N. A Theory of Framing and Opinion Formation in Competitive Elite Environments. J. Commun. 2007, 57, 99–118. [Google Scholar] [CrossRef]
  29. Turnpenny, J.R.; Jordan, A.J.; Benson, D.; Rayner, T. The tools of policy formulation: An introduction. In The Tools of Policy Formulation; Edward Elgar Publishing: Cheltenam, UK, 2015. [Google Scholar]
  30. Jordan, A.J.; Turnpenny, J.R.; Rayner, T. The tools of policy formulation: New perspectives and new challenges. In The Tools of Policy Formulation; Edward Elgar Publishing: Cheltenam, UK, 2015. [Google Scholar]
  31. Zeitlin, J. EU Experimentalist Governance in Times of Crisis. West Eur. Polit. 2016, 39, 1073–1094. [Google Scholar] [CrossRef] [Green Version]
  32. Koppenjan, J.F.; Enserink, B. Public-Private Partnerships in Urban Infrastructures: Reconciling Private Sector Participation and Sustainability. Public Adm. Rev. 2009, 69, 284–296. [Google Scholar] [CrossRef]
  33. Radnor, Z.; Osborne, S.P.; Kinder, T.; Mutton, J. Operationalizing Co-Production in Public Services Delivery: The Contribution of Service Blueprinting. Public Manag. Rev. 2014, 16, 402–423. [Google Scholar] [CrossRef] [Green Version]
  34. Osborne, S.P.; Radnor, Z.; Strokosch, K. Co-Production and the Co-Creation of Value in Public Services: A Suitable Case for Treatment? Public Manag. Rev. 2016, 18, 639–653. [Google Scholar] [CrossRef] [Green Version]
  35. Somerville, P.; Haines, N. Prospects for Local Co-Governance. Local Gov. Stud. 2008, 34, 61–79. [Google Scholar] [CrossRef] [Green Version]
  36. Bovaird, T.; Stoker, G.; Jones, T.; Loeffler, E.; Pinilla Roncancio, M. Activating Collective Co-Production of Public Services: Influencing Citizens to Participate in Complex Governance Mechanisms in the UK. Int. Rev. Adm. Sci. 2016, 82, 47–68. [Google Scholar] [CrossRef] [Green Version]
  37. Loeffler, E.; Bovaird, T. Assessing the Impact of Co-Production on Pathways to Outcomes in Public Services: The Case of Policing and Criminal Justice. Int. Public Manag. J. 2020, 23, 205–223. [Google Scholar] [CrossRef]
  38. Loeffler, E.; Bovaird, T. From participation to co-production: Widening and deepening the contributions of citizens to public services and outcomes. In The Palgrave Handbook of Public Administration and Management in Europe; Springer: Berlin/Heidelberg, Germany, 2018; pp. 403–423. [Google Scholar]
  39. Bovaird, T.; Loeffler, E. What has co-production ever done for interactive governance. In Critical Reflections on Interactive Governance; Edward Elgar Publishing: Cheltenam, UK, 2016. [Google Scholar]
  40. Miller, C.A.; Wyborn, C. Co-Production in Global Sustainability: Histories and Theories. Environ. Sci. Policy 2018. [Google Scholar] [CrossRef]
  41. Brugnach, M.; Ingram, H. Ambiguity: The Challenge of Knowing and Deciding Together. Environ. Sci. Policy 2012, 15, 60–71. [Google Scholar] [CrossRef] [Green Version]
  42. Wu, X.; Ramesh, M.; Howlett, M. Policy Capacity: A Conceptual Framework for Understanding Policy Competencies and Capabilities. Policy Soc. 2015, 34, 165–171. [Google Scholar] [CrossRef]
  43. Arroyo, P. A New Taxonomy for Examining the Multi-Role of Campus Sustainability Assessments in Organizational Change. J. Clean. Prod. 2017, 140, 1763–1774. [Google Scholar] [CrossRef]
  44. Breukers, S.; Hisschemöller, M.; Cuppen, E.; Suurs, R. Analysing the Past and Exploring the Future of Sustainable Biomass. Participatory Stakeholder Dialogue and Technological Innovation Systems Research. Technol. Forecast. Soc. Chang. 2014, 81, 227–235. [Google Scholar] [CrossRef]
  45. Dola, K.; Mijan, D. Public Participation in Planning for Sustainable Development: Operational Questions and Issues. Int. J. Sustain. Trop. Des. Res. Pract. 2006, 1, 1–8. [Google Scholar]
  46. Brundtland, G.H.; Khalid, M.; Agnelli, S.; Al-Athel, S.; Chidzero, B. Our Common Future. In Earth and Us; Tolba, M.K., Biswas, A.K., Eds.; Butterworth-Heinemann: New York, NY, USA, 1987; Volume 8. [Google Scholar]
  47. Masiol, M.; Benetello, F.; Harrison, R.M.; Formenton, G.; De Gaspari, F.; Pavoni, B. Spatial, Seasonal Trends and Transboundary Transport of PM2. 5 Inorganic Ions in the Veneto Region (Northeastern Italy). Atmos. Environ. 2015, 117, 19–31. [Google Scholar] [CrossRef]
  48. Regione Veneto—Servizio Statistico Regionale Veneto Sostenibile. Gli Obiettivi Di Sviluppo Sostenibile: Il Posizionamento Del Veneto; Regione Veneto: Venezia, Italy, 2019. [Google Scholar]
  49. Guzman, C.; Bhatia, R.; Grande, O. Participatory Impact Assessment: A Tool for Framing Community Voices into Policy Actions. In Proceedings of the The 132nd Annual Meeting, Washington, DC, USA, 8–11 January 2004. [Google Scholar]
  50. Benoit, K. Text as Data: An Overview. In E SAGE Handbook of Research Methods in Political Science and International Relatations; Curini, L., Franzese, R., Eds.; SAGE Publishing: London, UK, 2019. [Google Scholar]
  51. Laver, M.; Benoit, K.; Garry, J. Extracting Policy Positions from Political Texts Using Words as Data. Am. Polit. Sci. Rev. 2003, 97, 311–331. [Google Scholar] [CrossRef] [Green Version]
  52. Sbalchiero, S.; Righettini, M.S. Rhetorical Manifestation of Institutional Transformation. Qual. Quant. 2017, 51, 1279–1296. [Google Scholar] [CrossRef]
  53. Stenvoll, D.; Svensson, P. Contestable Contexts: The Transparent Anchoring of Contextualization in Text-as-Data. Qual. Res. 2011, 11, 570–586. [Google Scholar] [CrossRef]
  54. Portney, K.E. Sustainability; MIT Press: Cambridge, MA, USA, 2015. [Google Scholar]
  55. van Hulst, M.; Yanow, D. From Policy “Frames” to “Framing”: Theorizing a More Dynamic, Political Approach. Am. Rev. Public Adm. 2016, 46, 92–112. [Google Scholar] [CrossRef]
Figure 1. Sustainability framing variance and prioritization across provinces (2019). Source: authors’ elaboration of Veneto region data.
Figure 1. Sustainability framing variance and prioritization across provinces (2019). Source: authors’ elaboration of Veneto region data.
Sustainability 13 04435 g001
Figure 2. Sustainability topic integration and overlaps emerging from provincial forum participation (2019). Source: authors’ elaboration of Veneto region data.
Figure 2. Sustainability topic integration and overlaps emerging from provincial forum participation (2019). Source: authors’ elaboration of Veneto region data.
Sustainability 13 04435 g002
Table 1. Participatory assessment grid (PAG). Analytical dimensions of reframing process.
Table 1. Participatory assessment grid (PAG). Analytical dimensions of reframing process.
Type of ContributionPurpose
1. ExplorationInvestigation of new topics
2. PrioritizationPut topics in a relevant order
3. EmbeddingAttribute salience according to local context and perception
4. IntegrationEstablish item relationships within a topic and between topics (overlapping)
Table 2. Participatory assessment grid (PAG). Reframing variables and measures of participatory outputs constituted by written texts.
Table 2. Participatory assessment grid (PAG). Reframing variables and measures of participatory outputs constituted by written texts.
Type of ContributionPurpose of Sustainability Framing ActivityVariables
1. ExplorationParticipants investigate new sustainability topics.New words and cluster of words in the participatory output
2. PrioritizationParticipants put topics in order of relevance/salienceMost relevant topics: most frequent words and the cluster of words in forum output
3. EmbeddingParticipants assign sustainability topics salience by local areasMost frequent words and the cluster of words by each regional forum
4. IntegrationParticipants establish relationships between words within a topic and between topics (overlapping).Distance or closeness between semantic areas or between words within the same semantic area.
Source: Author’s elaboration.
Table 3. Number of written texts generated by regional forums on governmental topics (2019).
Table 3. Number of written texts generated by regional forums on governmental topics (2019).
602 Land use666 Circular economy
666 Sustainable development754 Sustainable lifestyles between urban and rural contexts.
Source: author’s elaboration on the data available from Righettini (2020) [20] (p. 543).
Table 4. Framing and reframing topics: sustainable development goals (SDGs) involved (2019).
Table 4. Framing and reframing topics: sustainable development goals (SDGs) involved (2019).
Reframing. Topics Addressed in Participation Local ForumsSDGsFraming. Topics Proposed by Regional GovernmentSDGs
a. Tourism innovation through digital management.6, 9, 8, 11,1. Land use and soil sealing12, 13, 15
b. Local tourism development through local heritage promotion.8, 9, 11, 152. Circular economy and waste management7, 13, 12, 15
c. Public spaces, new opportunities for public meetings.11, 12,3. Sustainable development7, 11, 12, 13, 15
d. Projects for green area repurposing.11, 12, 13, 154. Sustainable lifestyle8, 6, 11, 12, 15
e. Circular economy better awareness.4, 9, 11, 12, 15
f. Incentives for product and resource reusing.9, 11, 12, 13, 15
g. Development of alternative transport systems.3, 7, 11, 12, 15,
Source: author’s elaboration.
Table 5. Sustainability topic salience—prioritization of sustainability topics (2019).
Table 5. Sustainability topic salience—prioritization of sustainability topics (2019).
Topics from Participatory Forums% of Output
a. Tourism innovation through digital management10.5
b. Local tourism development through local heritage promotion14.3
c. Public spaces, new opportunities for public meetings7.4
d. Projects for green area repurposing22.2
e. Circular economy better awareness22.1
f. Incentives for reusing products and resources10.1
g. Development of alternative smart mobility systems13.5
Source: author’s elaboration on the data available from Righettini (2021) [22] (p. 543).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Righettini, M.S. Framing Sustainability. Evidence from Participatory Forums to Taylor the Regional 2030 Agenda to Local Contexts. Sustainability 2021, 13, 4435. https://doi.org/10.3390/su13084435

AMA Style

Righettini MS. Framing Sustainability. Evidence from Participatory Forums to Taylor the Regional 2030 Agenda to Local Contexts. Sustainability. 2021; 13(8):4435. https://doi.org/10.3390/su13084435

Chicago/Turabian Style

Righettini, Maria Stella. 2021. "Framing Sustainability. Evidence from Participatory Forums to Taylor the Regional 2030 Agenda to Local Contexts" Sustainability 13, no. 8: 4435. https://doi.org/10.3390/su13084435

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop