A Framework Proposal for the Ex Post Evaluation of a Solution-Driven PES Scheme: The Case of Medvednica Nature Park

Abstract

Payments for ecosystem services are a voluntary market-based instrument to remunerate provider(s) of ecosystem services by those who benefit from them. Our research aimed to create an ex post evaluation framework to identify bottlenecks and elements hindering the success of a solution-driven PES scheme. The framework was applied to a case study to assess its feasibility and concerns the provision of health and recreational services in the Medvednica Nature Park (Zagreb, Croatia). The framework was set up through three main sources: the study of PES implementation project documentation, semi-structured interviews with visitors and key stakeholders, and web-scraping of TripAdvisor reviews of the park. The main findings confirm society’s interest in the park, but the lack of mapping, quantification, and accounting of the services analysed, the little or no demand from society to pay for their provision, and confused knowledge of the property rights of some ecosystem service providers in the area limit the success of the PES scheme. The framework was useful to describe the chosen PES scheme and to identify bottlenecks and fragilities of the system in place, allowing it to correct its application flaws and, on the other hand, to demonstrate its replicability in other contexts.

1. Introduction

In recent decades, there has been ample evidence that using market-based instruments (MBIs), as alternative policy instruments to command-and-control, can help improve the provision of ecosystem services and ensure sustainable local development [1,2,3,4]. These include payments for ecosystem services (PES), or payments for environmental services, schemes defined by Wunder [5] as: “voluntary transactions between service users and service providers that are conditional on agreed rules of natural resource management for generating offsite services”. The main benefits of ecosystem services internalised by PES include regulating services, such as biodiversity protection [6], hydrogeological protection [7], and carbon storage [8]. In recent times, cultural services, such as tourism–recreational activities and health are also gaining recognition [9,10].

However in order to be defined as PES schemes in the strict sense, they must meet certain preconditions, which Wunder [11] identifies as a transaction voluntarily; a clear definition of the ecosystem service provided or the resource that provides it; the presence of at least one buyer/user of the service and one seller/provider of the service; and conditionality, namely, if the provision of the service ceases or diminishes, payment for it is stopped or reduced. Other preconditions are added to the previous ones, according to Smith et al. [12]: additionality, namely, payment is made for improvement or maintenance interventions that the provider makes on the resource and that would not otherwise be there; prevention of leakage, in other words, an undesirable negative impact on the provision of other ecosystem services or another natural resource to the detriment of the guarantee of the service or the resource protected by the PES scheme; the permanence of the service, in the sense that the interventions made by the provider should not have reduced effectiveness and rapid reversibility.

A PES scheme to be defined as such must fulfil all the previous preconditions. If only some of them are met it is defined as PES-like and in a broad sense it becomes an economic incentive or MBI [13]. Several definitions of PES schemes exist in the literature [11,14,15,16,17,18,19], which allow for the inclusion of a large part of the world’s PES programmes that do not meet the restrictive criteria proposed by Wunder. However, at the same time, they lead to a lack of harmonisation of results and replication and transfer of experiences from one site to another, due to a non-unique definition [20,21]. For the development of this study, the narrowest meaning of the term was considered. PES schemes originated mainly in countries with developing economies to reduce poverty, environmental degradation (e.g., REDD+ programme), and the economic divide, but today they are also widely applied in developed economies, as witnessed by programmes in Costa Rica [22], Brazil [23], Mexico [24], Vietnam [25], the United States [26], Poland [27], and Finland [28]. Nevertheless, the geographical areas most targeted are China and the American continent [29,30]. It is also important to point out that many PES programmes only exist in the grey literature and that some organisations have tried to collect data from them but have produced reports that are difficult to understand [20].

At the same time, it should be noted that for the adequate protection and enhancement of natural resources, it is not enough just to use MBIs, such as PES, but there is a need to adopt other instruments, methods, and approaches beyond the market alone. This is because the limitations associated with such instruments are widely discussed in the literature and range from biophysical, institutional, ethical issues to rights-based aspects [31].

Review of the Literature

There are several reports in the grey literature that provided guidelines or best practices on the design and implementation of PES schemes [12,32,33], while others collected the different PES currently in place [34], while others demonstrated both [35,36,37,38,39,40]. The problem with these reports is that—apart from not having received a peer-review process—they are often long and complex to read or focus exclusively on PES in countries with developing economies. At the same time, in the scientific literature, several authors hypothesised methodological frameworks related to PES schemes with more emphasis on some aspects than others; indeed some focused on the development of early PES schemes for poor and developing economies [41,42]; on the institutional and political economy framework [18]; on the environmental component with life-cycle assessment (LCA) [43]; on the creation of a PES case study dataset [44]; on socio-economic outcomes [45,46]; on behavioural economics and social psychology for the motivation to conserve and manage a natural resource behind a reward [47]; on choosing between different alternatives of PES schemes [48]; on the improvement of already existing methodological frameworks [29]; and on the integration of a framework containing biodiversity and carbon sequestration [49].

Most of the frameworks listed above, however, serve for implementing PES schemes and not for their ex post evaluation. Impact evaluation studies do exist in the literature, but they have been carried out for PES schemes implemented in threatened and degraded areas, for conservation and protection or poverty alleviation [50,51,52,53]. For PES schemes that are instead implemented at the suggestions of external organisations, for example by scientific projects (i.e., solution-driven PES scheme; see Section 2.1.1), and that do not originate to address a threat, but rather to increase the welfare of the local communities, some have identified the successful elements for their implementation [54] or have performed an ex ante evaluation [28], but an ex post evaluation framework is lacking.

This is the context for our research, the aim of which is to investigate the bottlenecks and elements that might hinder the success of a solution-driven PES scheme. In this study, a methodological framework for the ex post evaluation of the design, implementation, and impacts of a PES scheme was created to respond to our research question. Document analysis, web scraping, and semi-structured interviews were the methods used to answer the different blocks of the framework.

2. Materials and Methods

2.1. A Solution-Driven PES Framework Proposal: Design, Implementation, and Impacts

Figure 1 shows the methodological framework for the ex post evaluation of the design, implementation, and impacts of a solution-driven PES scheme. Specifically, the proposed methodological framework is the contribution of the authors, while some of the elements that make up the various boxes refer to the literature, as extensively documented in the following Section 2.1.1, Section 2.1.2, Section 2.1.3, Section 2.1.4, Section 2.1.5, Section 2.1.6 and Section 2.1.7. In this section, an attempt is made to answer the following questions: How is the area that the PES scheme is to be implemented in identified (Section 2.1.1)? What are the governance elements required to initiate a PES scheme (Section 2.1.2)? How is ecosystem service provision assessed (Section 2.1.3)? What are the conditions that create the market (Section 2.1.4)? Who are the actors involved (Section 2.1.5)? What elements constitute a PES scheme (Section 2.1.6)? How are the impacts of its implementation evaluated (Section 2.1.7)?

2.1.1. Goal and Identification Area

The levers that create the initial interest in setting up a PES scheme are various. An International Union for Conservation of Nature (IUCN) report [55] divides them into three main categories:

• Demand-driven: this is the case when service users encounter a problem in its provision and are willing to pay for its maintenance or improved provision—for example, improving the quality and quantity of drinking water (off-site). In this case, service providers are incentivised to produce it;
• Supply-driven: this is found when there is a problem on-site, related to the conservation and management of the ecosystem. In this case, the economic contribution by the service users helps the provider to maintain or improve the management of the natural resource for the benefit of both the ecosystem and the users;
• Solution-driven: this occurs when a third-party organisation identifies cases where the creation of a PES scheme would be feasible and beneficial.

In the case of a solution-driven PES, the area can be identified by analysing its social interest, usage, and frequentation. In this respect, the use of sociometric methods such as site and social content analysis or social network analysis [56] can help identify the potential area [57,58].

2.1.2. Governance

For a PES scheme to be successful, it must have effective governance upstream, capable of considering the legal, institutional, and property rights framework in the territory in which it operates. The legal framework helps to define the role of institutions, and the characteristics of the PES mechanism, with its limitations and massive diffusion as legally recognised. Legal references can refer to different spatial scales, from the international scale, for example with agreements, to the local scale with regulations, to the supranational, national, and regional scales with guidelines, regulations, constitutions, and laws.

The institutional framework is useful to define the role and respective “arenas of power” of the social, economic, and political actors involved in a PES scheme. The institutional framework can be purely public, private, or mixed. Institutions remove different barriers; they can, for example, help reduce transaction costs and risks, manage possible conservation or ecosystem use conflicts, provide economic support, and coordinate different mechanisms and policies [59].

Property rights regulate land tenure, namely, the relationship between a good or resource and the individual or group of people concerned. Starting with ownership as the most exclusive right, FAO [60] identifies three different sub-categories of rights:

• Access and use: the possibility of choosing who can have access to and whom to exclude from the land for use;
• Control: the possibility of choosing what land is to be used for;
• Transfer: the possibility of transferring the right of access, use, and control to other persons, by selling, mortgaging, or bequeathing the land tenure.

To identify the elements of governance, diagnostic analytics of documents can be useful.

2.1.3. Ecosystem Services

The spatial identification of ecosystem services and their quantification, evaluation, and accounting (necessary to understand their trade-offs) is crucial for the successful design and implementation of a PES scheme. Regarding spatial targeting, Wunder et al. [44] emphasise the importance of identifying both areas where the density of ecosystem services (i.e., supply) is highest, and areas threatened by unsustainable management practices where the adoption of a PES scheme could make a difference and bring added value. For the quantification and evaluation of ecosystem services in the literature, there are several valuation methods, including biophysical, economic, and socio-cultural methods [61], and decision support systems and models, such as ARIES, InVEST, ORVal, and MESH that allow mapping, quantifying, and valuing ecosystem services in one go [62].

For accounting purposes, a framework called system of environmental economic accounting (SEEA) [63] was developed at the international level that is capable of monitoring and integrating the biophysical component of ecosystem stocks and ecosystem service flows with the related economic component in monetary terms. This framework allows for harmonised and comparable statistics across countries.

2.1.4. Market

The analysis of supply and demand is essential in the development of a PES scheme. In demand, there is a distinction to be made between payments made to obtain the provision of a service and funding for the creation of the PES scheme. Payments can be made by a public or a private component. To assess the demand, there are various monetary economic valuation methods [62], such as those based on stated or revealed preferences, which can estimate the willingness-to-pay (WTP) on the part of users/buyers for the provision of the service. For there to be demand, it is important that there is the precondition of additionality and, thus, that the service provider produces benefits for the users/buyers of the service. Such benefits, according to Porras et al. [19], may relate to the restoration of destroyed or degraded land; the reforestation or afforestation of an area; the conservation and protection of the managed resource; and the management and improvement of land practices.

Similar to demand, the same preference-based methods can be used for supply to estimate the willingness-to-accept (WTA) compensation from providers. This WTA is the leverage that moves the provider to maximise the provision of other ecosystem services at the expense of productive activity alone (e.g., agricultural harvest or forest cutting) and, thus, the private return. Indeed, if the WTA does not cover the private return that would have been obtained from the productive activity alone, the PES scheme fails. Conversely, if the provision of the service(s) increases the private return beyond that derived from mere production, the provider has a greater incentive to participate in the scheme.

The service provider/seller and the service user/buyer are stakeholders in a PES scheme and are discussed in more detail in the following subsection.

2.1.5. Stakeholders

Knowing who the stakeholders are involved in a PES scheme allows one to understand their roles and relationships and the respective powers with which they can influence its design and implementation. The broader the participation in the scheme, the more transparent and socially accepted the process is [59]. To date, several classifications of stakeholder groups exist [12,19,64]; here, we propose that from Paletto et al. [7], modified as follows:

• Service producers/sellers: the public or private land and forest owners who conserve and manage the natural resource;
• Service users/buyers: who are willing to pay for the provision of the service and may also be public or private;
• Intermediaries: who connect producers and users and who support the creation of the PES scheme, for example, trade associations, public institutions, and NGOs. This category also includes donors, regulators who influence, control, and facilitate the start-up and effectiveness of PES and funding agencies that support the start-up and operation of the scheme with for example feasibility studies;
• Knowledge providers: those who provide advice, knowledge, and assistance for the development of the PES scheme, such as experts, planners, universities, and research institutes and consultants.

The use of participatory methods, such as focus groups, the Delphi method, semi-structured interviews, deliberative workshops, and meetings can support the decision-making process related to the design and implementation of the PES scheme by analysing the different interests, values, opinions, and behaviours of the stakeholders. The identification and selection of stakeholders at the beginning of the design of a PES scheme are of paramount importance for its success. In this context, the use of stakeholder analysis can help to identify who needs to be involved, kept informed, and satisfied and the role they play in the scheme.

2.1.6. PES Scheme

Several key design aspects need to be considered when developing a PES scheme. The funding source of the PES scheme can be public, private, or mixed depending on whether the payer is a governmental body, a private party or both. The role of stakeholders in the scheme may be different (see Section 2.1.5) and be configured in different ways depending on the number of service users and service providers [12]. It can be:

• A single service provider and service user (one-to-one): such as the case of the government or a company that comes into direct contact with an individual forest or landowner [12];
• Many service providers and a single service user (many-to-one): such as the case of Vittel with farmers in France [65];
• A single service provider and many service users (one-to-many): such as the case of the UK Ministry of Defence and retail companies and the North Pennines AONB with the UK Woodland Carbon Code in Cumbria [66];
• Many service providers and service users (many-to-many): such as the case of water certificates issued by the Bonneville Environmental Foundation between private sector businesses and landowners in the USA [67].

The spatial scale of application of the scheme can vary from local/neighbourhood, through catchment areas and valleys, to national and international depending on market needs and ecological processes. At the same time, the time scale may also vary from short- to long-term depending on the typology of the ecosystem service provided, the conservation status of the ecosystem; the duration of the project, programme, and environmental policy; and the ongoing provision of financing by service users [37].

Ecosystem services can be sold and packaged in different ways [12] depending on whether a service user or a consortium of service users buys all the ecosystem services produced in that PES-scheme area in one solution (bundling); several service users buy several ecosystem services provided by the area (layering or stacking); and whether a service user buys a single or a few ecosystem services, but also receives the provision of other services from the area for free (piggy-backing, “free-riding” phenomenon).

There are also several payment characteristics to consider [35]:

• Arrangement: whether it is based on improved management practices (action-based), on the actual provision of the ecosystem service (result-based), or mixed (hybrid-based);
• Type: whether it is cash, in-kind, or mixed;
• Time: whether it occurs as a one-off, or whether it is periodic, or mixed;
• Frequency: whether it occurs upfront, after practice improvement, or after service delivery.

2.1.7. Impact Evaluation

Once the PES scheme has been implemented, it is important to verify the results both to trigger the feedback process, should it be necessary, to modify or implement parts of the scheme or integrate its prerequisites (learning), and to make stakeholders more aware of and accountable for the PES scheme they have implemented (accountability) [68]. The impact evaluation is based on the cause–effect theory and the adoption of a logic model or theory of change makes it possible to explain the causal link between the resources/inputs and treatment employed and the outputs, results, and impacts achieved [69]. Outputs are the short-term products obtained from the resources used, outcomes are the medium-term effects of the problem to be solved, and impacts are the long-term outcomes. The impact evaluation can consider several criteria such as the efficiency of the scheme (output/input), its effectiveness in solving the problem for which it was implemented (output/outcome), its cost-effectiveness and availability of the inputs used, and equity, namely, the right balance between benefits and costs distributed among the various stakeholders. Monitoring and enforcement can be used in the evaluation of outputs, whereas the evaluation of outcomes and impacts requires the counterfactual approach and, thus, measuring the difference between a scenario with the treatment adopted and one without it.

2.2. Case Study

The Medvednica Nature Park (IUCN category V) is one of 12 Croatian nature parks and is located near the capital Zagreb [70]. The total area of the park (17,938 ha) is a Natura 2000 site, of which approximately 81% is covered by forests. It has an extension ranging from 120 to 1035 m, culminating in a peak named Sljeme. More than a quarter of the Croatian population lives in its surroundings. The park provides various forest ecosystem services depending on the season; for example, in summer it mitigates high temperatures and in winter it provides recreational activities such as skiing. In this respect, the park has various accommodations, sports, and cultural facilities as well as more than 70 hiking and biking trails. Ownership of the forests is split between the state (50%, Ministry of Agriculture of the Republic of Croatia), private individuals (47%), and the Faculty of Forestry and Wood Technology, University of Zagreb (3%). Almost all private individuals, however, own less than 5 ha of land, and more than 80% of them have only 1 ha [71].

The park was selected by the Spurring INnovations for forest eCosystem sERvices in Europe (SINCERE) project of the Horizon2020 programme (https://sincereforests.eu/) to raise the awareness of civil society on forest ecosystem services, and design and implement a PES scheme on the health and recreational services offered by the park. The payment mechanisms selected for the PES scheme are donation boxes placed at the beginning of two trails and available to any visitor, and one-time concession permits, i.e., a fee for sports organisations and other entities carrying out activities in the park.

2.3. Data Collection

To identify bottlenecks and elements hindering the success of a solution-driven PES scheme, a diagnostic analytics was conducted on the Medvednica Nature Park and PES scheme developed in the SINCERE project. Data collection ranged from data mining to document review and semi-structured interviews.

2.3.1. Data Mining

A data-mining technique called web scraping was used to collect society’s interest in the case study. This technique enables the extraction of data from websites using ad hoc software programmes [72]. Data were collected in December 2022 and January 2023 from the TripAdvisor review platform for the “Medvednica mountain” page (https://www.tripadvisor.com/Attraction_Review-g294454-d549693-Reviews-Medvednica_mountain_Sljeme-Zagreb_Central_Croatia.html, accessed on 15 January 2023).

2.3.2. Document Review

To identify the institutional and legal framework of PES in Croatia, as well as to identify the stakeholders involved in the PES of the Medvednica Nature Park, its characteristics and the results obtained from the project, several documents were collected and subsequently analysed, which are shown in

Table 1. Review documents.

Document Category	Document Name
Institutional and legal framework	Constitution of the Republic of Croatia [73] Forest Act and its regulations (OG 68/18) [74] Environment Protection Act (OG 80/13, 153/13, 78/15, 12/18, 118/18) [75]
Stakeholders	Three multi-actor group (MAG) meeting [76]
Characteristics of PES and project results	Report D4.1—Assessing the upscaling potential of SINCERE IAs using a theory of change structure [77] Report D4.2—Synthesis report of the experiences and lessons learnt, situating them in the global experiences and knowledge [78]

.

2.3.3. Semi-Structured Interviews

To analyse demand, namely, the willingness of park visitors to pay for the forest ecosystem services, and to analyse stakeholders’ opinions on the success factors and bottlenecks encountered during the implementation of the PES scheme, interviews were conducted. The participants of the survey were selected among the visitors of the Nature Park. Employees of the Medvednica nature park approached the visitors of the park and if they were willing to take a part in the research, they were interviewed. The research for demand was conducted in two rounds—the first part in the second half of 2018 with 155 participants, and the second part in the second half of 2020 with 148 participants of different ages and origins. The interviews consisted of 11 questions with a first part related to socio-demographics and a second part related to FES and the WTP stated by visitors (Appendix A).

The stakeholder opinion interviews were conducted at the end of the SINCERE project between May and June 2022 on a sample of 10 respondents. The interviews consisted of six questions with a first warm-up part and a second part aimed at identifying the stakeholders’ opinions on the elements they consider successful for a PES scheme, the main difficulties encountered in the development of the scheme, and the added value of being involved in the project (Appendix B).

2.4. Data Analysis

The visual software WebHarvy was used for web scraping [79]. This paid software allows the automatic identification of data patterns on web pages without the need to write scripts or code for data extraction. The items collected from the TripAdvisor page on “Medvednica Mountain” concern the reviewer (name, origin, and the number of reviews made on the site) and the reviews (month and year, travel objective, and title and text of the review). We identified 247 reviewers, of whom 197 left a review in addition to the evaluation alone between 2008 and 2022. To collect and analyse the data, we simply entered the URL of the page of interest into the software and visually selected the categories of items. The methodology used by Sergiacomi et al. [57] was adopted for the analyses.

MAXQDA Analytics Pro 2022 software was then used to analyse the transcripts of the semi-structured interviews on stakeholders’ opinions. Stakeholders were grouped into three broad categories (users, managers, and consultants) according to their role in the PES scheme. For coding, the code–category analytical approach was used for qualitative data analysis [80] on textual data that ranged from single words to whole sentences. The naming of codes and categories (merged codes) were identified through a review of the literature, which allowed the interview questions to be structured. Coding was performed using a bottom-up approach; therefore, codes and categories were identified from the text without having previously identified them. Various coding methods were used for coding [80], both first-cycle methods such as elemental methods (structural coding, in vivo coding, and initial coding) and affective methods (emotion coding), and second-cycle methods such as focused coding. For the first review of the transcripts, initial coding was used, an open-ended method without specific guidelines incorporating in vivo coding, which allowed parts of the transcripts to be coded using the same words and language as the respondents. Subsequently, content-based structural coding was used to further segment the transcripts and emotion coding to intercept emotions expressed by the respondents or inferred by the researcher. Ultimately, focused coding was used to categorise similar codes. The frequencies of the codes are not shown in the results because we wanted to focus exclusively on their variety and not on their quantity. The analysis performed allowed for an ex post understanding of what worked, what did not, and what could be improved in projects related to the implementation of PES schemes.

3. Results

Figure 2 shows the results from the application of the ex post evaluation framework. A general description of the individual boxes follows.

The initial interest in the development of a PES scheme in the Medvednica Nature Park was solution-driven because it was chosen as part of a research project (the SINCERE project). The choice of the area as a case study can be supported by the analysis conducted on TripAdvisor. Figure 3 shows the time trend of the collected reviews both as frequency (Figure 3a) and growth rate (Figure 3b). The results suggest a bimodal distribution of reviews with peaks in 2013 and 2015, possibly due to specific sports and/or recreational events in the mountains. There is also a sharp decline in reviews in 2020 (a drop of −80%), probably due to the COVID-19 health emergency and a slow recovery from the following year.

Figure 4 shows the reviews by seasonal (Figure 4a) and monthly (Figure 4b) trends. A constant interest emerges in the different seasons, with a strong peak in the summer period, specifically in August and June, and in the winter season, probably due to both recreational events (e.g., fairs and events) and sports competitions (such as skiing and orienteering).

The continents and European countries of the provenance of the reviewers/tourists are shown in Figure 5. Europe ranks first as the continent of origin (n = 125), followed by America (n = 25). In Europe, Croatia is the country with the most reviewers (n = 71) due to its proximity to the study area, followed by the United Kingdom (n = 22) and Germany (n = 5).

In general, it can be inferred that the study area is frequented by a good number of tourists and society living in neighbouring areas continuously throughout the year. The decision to develop a PES scheme for tourism and health aspects is, therefore, cohesive.

In terms of governance, Croatia has a well-defined legal and institutional framework. It recognises the importance of forests and natural resources in its constitution and, since 1990, has developed ad hoc legislation (Amendments to the Forest Act OG 41/90) for the first Croatian PES forestry, which was a green tax that has now been merged into Articles 65 and 68 of the latest Croatian Forest Act OG 68/18. Regarding institutions, forests are a state competence, as the Ministry of Agriculture with the company Croatian Forests Ltd. oversees forest management, while the Ministry of Environment Protection and Energy deals with environmental protection, which includes forests and natural resources in general (see the various Environmental Protection Acts).

Property rights, however, are only clear for the state park manager, but for the PES scheme to work, the entire Medvednica area must be studied for the provision of its forest ecosystem services. In this area, there are several conflicts of ownership between private individuals and little knowledge of their rights, as also revealed in the stakeholder interviews (

Table 2. Stakeholder opinions are broken down by attribute, category, and type of stakeholder.

Category	Attribute	Role
		User	Manager	Advisor
Emotions	Positive	x	x	x
	Negative	x	x
	Neutral	x
Elements for PES success	“Paying a tax for transit”	x		x
	“Mindset”		x	x
	“Showing the difference between Business-As-Usual and PES scenario”			x
	“Charge for the facilities/things that are used in the PES area”	x
	“Raising the awareness and education of users”		x	x
	“Good cooperation with all stakeholders”	x		x
	“Understanding the willingness to pay for forest ecosystem services”
	“Transparency”		x
	“Legislation”	x
	“Better communication to the public”			x
Difficulties for PES implementation	“Different point of views”	x
	“Misunderstanding”	x	x
	“Ownership conflicts”		x
	“Nature is free”	x	x
	“Do not consider different scenarios”			x
Project added value	“Cooperation with different stakeholders”		x	x
	“Implications only in theory and not in practice”	x	x
	“Good foundation for the future”		x
	“Raising the awareness about the benefits of forest ecosystem services to society”	x	x
	“Implementation and replication of a PES mechanism”		x
	“Nothing is changed”	x	x
	“Fostering new innovative approaches to the monetary valuation of ecosystem service”		x

).

The stakeholders who participated in the semi-structured interviews used various attributes to express their opinions on PES schemes and the SINCERE project (Table 2). These attributes are subsequently grouped into four main categories (“emotions”, “elements for the success of a PES”, “difficulties for the implementation of a PES”, and “value added by the project”). Managers were the stakeholders who most contributed to the identification of the attributes (n = 15), followed by users (n = 13) and consultants (n = 9). Concerning the categories, all roles perceived positive emotions in being involved in such a project.

These emotions, as reported during the interviews, changed negatively during the development of the project because of various conflicts and the general perception that nothing would change. Regarding the elements considered most important for the success of a PES scheme, the social component was prioritised, namely, good cooperation and mentality of those involved in the implementation of such schemes and increased awareness and education of users and visitors about the role and importance of forest ecosystem services provided by ecosystems. The main difficulties perceived during the development of the PES scheme are related to the misunderstanding among the stakeholders involved and the users’ and visitors’ conception that nature is free, and, thus, the lack of demand to trigger the PES mechanism. Finally, according to stakeholders, the SINCERE project has brought added value in terms of its ability to create collaborations and potential new opportunities between stakeholders and in raising stakeholders’ awareness of the importance of forest ecosystem services. However, negative aspects also emerged from the interviews, such as the perception that the project remained only at a theoretical level and that nothing had changed.

No mapping, biophysical quantification, or accounting of forest ecosystem services was found, probably because it was not the aim of the project and because biophysical quantification and accounting of cultural ecosystem services is difficult.

Regarding the market, participants were asked if they knew what forest ecosystem services were, and most of them answered that they did not know—106 (68.8%) of them in 2018 and 104 (80%) in 2020. In 2018, 48 visitors stated that they were aware of forest ecosystem services, and the most frequently mentioned services were oxygen production (14 visitors), provision of natural recreational space (10 visitors), protection from soil erosion and fresh air (7 visitors), water treatment, impact on psychophysical health and animal habitat (5 visitors), provision of a natural resting place, health services, and air purification (4 visitors). Other forest ecosystem services mentioned were CO2 sequestration, biodiversity, climate change mitigation, tourism, balance management, and many others.

In the survey conducted in 2020, 26 visitors responded to knowing what forest ecosystem services are, and the most frequently mentioned were providing a natural recreational place (11 visitors), oxygen production (7 visitors), water purification (6 visitors), health services and air purification (5 visitors each), and food and timber (4 visitors each). Visitors also mentioned tourism, soil protection from erosion, animal habitat, offering a natural resting place, offering a natural educational site, biodiversity, CO2 sequestration, and many others.

Participants were also asked to compare the impact of the experience of visiting the Medvednica Nature Park on health and well-being with commercial products and services (Figure 6). In a survey conducted in 2018, most respondents (33.3%) compared this impact to a visit to the gym (HRK 35—about 5 EUR-per hour), 20.1% to that of a day at a wellness centre (HRK 150—about 20 EUR-per day), and 15.7% to that of going to the cinema. The lowest number of visitors, 4.4%, compared a visit to the park to a concert worth HRK 200 (about EUR 25).

Similar results were shown in the research conducted in 2020—33.8% of visitors compared the effect of a visit to Medvednica to a visit to the gym (HRK 35 per hour), followed by 14.4% who compared it to a day at the spa (HRK 150 per day), and the same percentage who compared it to local trips (HRK 350—about EUR 50). The smallest number of visitors, 2.9 per cent, compared a visit to Medvednica with an international trip worth HRK 2100 (about EUR 280).

Regarding the question of the WTP stated to visit the Medvednica Nature Park (Figure 7), of the 102 visitors who answered the question in the survey conducted in 2018, 54 respondents (35.3%) answered no, while 99 (64.7%) answered yes. Out of 99 respondents who said yes, 48.5% are willing to pay HRK 10 (about EUR 1.50), 26.3% HRK 15 (about EUR 2), and 25.3% of respondents are willing to pay HRK 5 (about EUR 0.60).

The research conducted in 2020 showed slightly different results than that conducted in 2018. Although most of the visitors stated that they are willing to pay to visit Medvednica (54.7%), their share is lower than in the research conducted two years earlier. On the other hand, the share is lower, but the amount they are willing to pay is higher—44.4% of visitors who are willing to pay for staying in Medvednica Nature Park are willing to pay HRK 15, 38.3% HRK 10, and 17.3% HRK 5.

The WTA has not been estimated to compensate landowners and foresters for continuing with their forest management practices in the area for the benefit of the community and at the expense of their private return from productive activity alone (i.e., forest logging). This is probably because, for the publicly managed portion of forests, it would not make sense to estimate the WTA, while for the privately managed portion, little or no knowledge of forest owners or conflicting property rights undermine the success of its estimation.

Intermediaries and knowledge providers were also involved in the development of the PES scheme. The providers were the private owners and the public administration of the Medvednica Nature Park. The service users were visitors to the donation boxes and organisations and entities organising events in the park for one-time concession permits (e.g., Croatian Mountain rescue service and triathlon club). The intermediaries were civil society and business activities (e.g., web service providers). The knowledge providers were research institutions (e.g., Faculty of Forestry, University of Zagreb). During the project, three meetings were held between 2018 and 2019 to involve stakeholders from the area. The first meeting involved only five stakeholders, but in the other meetings, their participation increased.

The PES scheme designed with the two mechanisms has private funding, with a many-to-many configuration due to the different service providers and users, is applied locally, and has a validity period equal to the duration of the research project (short-term). Recreational and health services are sold and packaged in the form of piggy-backing as users who buy these services also benefit from others free of charge. In terms of payment characteristics, the arrangement is result-based, namely, based on the actual delivery of the services, with cash, one-off payment for the one-time concession permits, and periodically for the donation boxes, and takes place after the delivery of services.

Regarding the evaluation of impacts, it was not possible to assess the mechanism of the donation boxes, because they were vandalised and destroyed. About the one-time concession permits mechanism, reports from the SINCERE project show that the involvement of different stakeholders in the meetings creates a greater awareness of forest ecosystem services and the importance to pay for such services. Another result of the project shows that the mechanism allows for a re-direction of conflicts between the users of the area, because it assigned each group a different area and the funds collected from the permits could be used for investments in recreational infrastructures. Furthermore, there was a monitoring of fauna, flora, and soil damage. However, there was no impact assessment because there was no comparison with the counterfactual scenario, but it is likely that there will be an improvement in the recreational experience in the park with a consequent increase in demand for recreational services. However, the experience gained from this mechanism did not trigger the feedback or verification process to improve or implement parts of the PES scheme and its duration.

4. Discussion

PES schemes can contribute to the conservation of threatened areas and ecosystems and improve the well-being of local communities, as reported by several authors [50,81,82]. For PES schemes to be successfully implemented, however, their effectiveness and efficiency, as well as the results obtained, must be analysed. Otherwise, they remain a mere theoretical exercise and lose credibility, as argued by Wunder et al. [44]. The framework proposed here aimed to evaluate ex post a solution-driven PES scheme, independently of the ecosystem service analysed. This was performed by attempting to consider both the political/institutional and social, economic, and environmental components. Our attempt differs from those found in the literature, such as the one proposed by Page et al. [43] about the evaluation of local PES proposals by monitoring their environmental impact with LCA, or the one proposed by Schomers et al. [83] about assessing the potential of intermediaries to improve the effectiveness of PES schemes, or the one proposed by Tikkanen et al. [28] for the ex ante evaluation of schemes applied to nature-based tourism.

The application of the framework allowed us to answer our research question and identify bottlenecks that potentially limited the success of the PES scheme implemented in the Medvednica Nature Park. For a successful implementation of a PES scheme, it is important to have a suitable background, as argued by Brand [84]. In this regard, the results reveal a strong societal interest in the study area, as also reported by Bakarić et al. [71] and Tisma et al. [70], and, thus, the potential for implementing a PES scheme. At the same time, as reported in the study conducted by Vuletić et al. [85] on water-related PES schemes in south-eastern European countries, Croatia has more than 30 years of experience with the legal and institutional and, thus, theoretical framework of PES schemes. The absence of such a factor would likely have led to the failure of the scheme, as argued by the review conducted by Yan et al. [86] on PES as an essential instrument for improving ecosystem services. However, although a large part of the forests in the park are in public ownership, the remaining private part faces a lack of clear ownership rights among the landowners, as argued by some managers involved in stakeholder interviews, which not only limits the success of up-scaling the scheme, and of the SINCERE project, but also risks undermining its success and generating leakage situations. In a review conducted by Adhikari and Agrawal [87] on the ecological and social outcomes of PES projects, the importance of property rights and tenure security for a project’s success emerged, as also reported by Yan et al. [86].

The methodological framework also highlights the lack of spatial identification and quantification of forest ecosystem services and the areas with their greatest hotspot within the study area. According to Wunder et al. [29] and Ezzine-de-Blas et al. [50], who analysed past experiences of PES schemes, these factors are one of the main factors for environmental additionality along with payment differentiation and conditionality. However, the case study analysed cultural ecosystem services that, due to their intrinsic characteristics, are difficult to biophysically quantify and account for. The results indicate a lack of knowledge of forest ecosystem services, which probably led to reduced or no willingness on the part of society to pay for them. During the course of the project, however, as reported by Tisma et al. [70], society’s perception and knowledge of such services increased, and this will certainly influence their stated WTP in future research projects. In this regard, Poudyal et al. [88] highlight society’s knowledge of ecosystem services and PES as one of the socio-economic factors affecting the implementation of PES schemes.

In general, the methodological framework reveals potential bottlenecks, especially in the preparatory context for a PES scheme, but the design and implementation are successful in the study area. A final element to be reported is the evaluation of the impacts that was partially carried out, but the conditionality for the continuous provision of recreational and health-related services by private landowners was not verified. This may be due both to the fact that monitoring the effects of the scheme would increase transaction costs by not making the instrument as cost-effective compared to other economic/political instruments, and to the short duration of this scheme, which had the same validity as the SINCERE research project. There is, however, a need to emphasise the importance of monitoring, verifying conditionality, and sanctioning non-compliance, as reported by several authors [44,50,89] so that PES schemes are stable over time and do not become mere economic additions to “business as usual” conditions where the service is either not provided or poorly provided.

5. Conclusions

Recently in the field of environmental economics, there has been a shift from a “polluter pays” approach typical of political and command-and-control-based instruments to a “provider gets” approach, such as that of market-based instruments, capable of creating virtuous and cooperative behaviour among the actors involved. Among MBIs, PES schemes have attracted increasing interest over the past three decades, first as a tool for protecting threatened ecosystems in developing and emerging economies, and then as a tool for enhancing ecosystems in developed economies. In this context, there is a need for evaluation frameworks, guidelines, and supports that can monitor and evaluate the effectiveness and success of such schemes.

This study attempted to establish an ex post evaluation framework to identify bottlenecks and obstacles to the success of solution-driven PES schemes by integrating the four main spheres of sustainability, namely, environmental/ecological, social, economic, and political, into one proposal.

The results express a clear societal interest in the area subjected to the PES scheme—given its year-round attendance—but at the same time, reveal potential obstructive elements for the successful implementation of such a scheme. The mechanism based on donation boxes for visitors to the park fail due to vandalism. In contrast, the mechanism based on one-time concession permits, which mainly involve organisations and entities organising events and recreative activities in the park, although moderately successful and interesting, encounter obstacles in the unclear property rights of landowners, the absence of mapping, quantification, and accounting of the forest ecosystem services under study, little or no WTP on the part of the demand side for such services, and the absence of application of the feedback process of the functioning of the scheme for its improvement or integration. Regarding the WTP from the demand side, this is probably one of the main weaknesses of solution-driven schemes, where a third party dreams up a scheme without a particular need from the market/society. There is also a need to add that both mechanisms proposed in the project fail in the proposed PES schemes defined by Wunder, as no spatial externalities are produced, but ecosystem services are consumed locally. Therefore, it would be more correct to talk about innovative mechanisms, yes, but related to MBIs and not to PES schemes.

In a general sense, the proposed assessment framework lends itself to replication in other geographical contexts with an even greater spatial extent than the local context, depending on the application of the PES scheme. The framework is versatile and can be modified and integrated with other boxes depending on the context, such as the need to integrate local communities’ knowledge of ecosystem services and their uses. The main limitations of the framework relate to technical capacity, data availability, and a clear view of the political–institutional complexity of the case study in which it is applied. With reference to technical capacity, there is a need to have trained figures with some level of knowledge of the topic or several figures that may be involved, each specializing in a particular part of the framework. Regarding data availability, this is a problem that plagues all research, although the digitization of information, the presence of digital data collection platforms, and the era of big and smart data are improving this situation. Regarding the institutional complexity and the possible different stakeholders in it, at the European level there is the presence of agricultural knowledge and information systems (AKIS) diagrams that map the relational and institutional framework of a country’s agricultural and forestry sector. However, there is a present need to investigate for potential similar diagrams internationally. Our future developments in the research may concern both the creation of guidelines for the proper use of the framework and its application to other case studies, as well as to begin to create a kind of portfolio of analysed case studies and validate the goodness and adaptability of the methodological framework in other contexts.

Regarding the possible implications for decision-makers and those involved in the development of PES schemes, it can be a valuable support in initially identifying the elements that hinder their success, however, there is a need for different technical skills to be able to analyse the different boxes of the framework and there is a need to identify strategies to cope with the possible phenomenon of individual free-riding of scheme participants. In such a case, solutions such as those proposed by Naime et al. [90], namely, monitoring by public detection of individual actions or monetary sanctions by an external central government agency or internally by the community of stakeholders related to the scheme, can be proposed. For researchers, such a framework can be a first step for the ex post evaluation of PES schemes at the academic level and an element on which further research can be developed to overcome the limitation of its application only for solution-driven cases. Such cases are in the minority compared to PES schemes created to protect an ecosystem or to support landowners and foresters in their management practices.