EASIER: An Evaluation Model for Public–Private Partnerships Contributing to the Sustainable Development Goals

Abstract

Recently, public–private partnerships (PPPs) have attracted renewed attention as a valuable tool to close the gap between public services and social needs. In fact, the United Nations (UN) proposed collaboration across multiple stakeholders as one of the key goals for securing global sustainable development. Yet, PPPs remain a controversial proposition for many due to, among other factors, the complexity and limitation of current systems to assess their impact beyond the notion of value for money. This study offers a conceptual model (EASIER) that accounts for six dimensions that are relevant for social, environmental, and economic progress. We also propose a questionnaire to assess the impact of PPPs on UN Sustainable Development Goals (SDGs) and apply it to a PPP contract as an illustration. We advocate the use of EASIER as an initial evaluation model due to its simplicity and its holistic perspective.

1. Introduction

The societal and environmental challenges that the world is facing are materializing at an unprecedented rate and degree. Climate change is causing the devastation of natural ecosystems around the planet, and is becoming a more urgent issue than ever. The human environmental footprint is severely affecting human health, with a significant portion of the population breathing polluted air, which was responsible for over four million premature deaths in 2016 [1]. Child labor and poor work conditions are still present in many countries [2], wrecking human dignity. Cities are struggling to accommodate the rapid growth of the urban population, creating deep pockets of poverty and fueling inequality [3]. As a response to these pervasive societal problems, all the member countries of the United Nations (UN) adopted 17 Sustainable Development Goals (SDGs) in 2015 to end poverty, protect the planet, and ensure prosperity by 2030. While these 17 goals have been welcomed equally by governments, firms, and academia alike—as they represent a new opportunity to build more prosperous, greener, and more equal societies—they also impose significant challenges and tensions that are difficult to resolve. The magnitude and the ambition of the SDGs indicate that they cannot be tackled by a single entity, but instead they require the joint effort of several societal actors [4]. Acknowledging the importance of collaboration, SDG 17 calls for “partnership for the goals”, encouraging multi-stakeholder partnership in general and effective public–private partnerships (PPPs) in particular (target 17.17).

PPPs are defined as “any long-term collaborative relationships between one or more private actors and public bodies that combine public sector management or oversight with a private partner’s resources and competencies for a direct provision of a public good or service” [5] (p. 273). PPPs are strategically important for SDGs for at least two reasons. First, they are a valid vehicle to overcome the massive infrastructure gap, which is globally estimated at around US$800 billion to US$1 trillion per year, depending on the source [6,7]. For developing countries alone, the investment gap is even higher; it is estimated at $US2.5 trillion per year [8]. The lack of the needed infrastructure not only prohibits the access to clean water, electricity, and food, damaging the quality of life of billions of people around the world, but also hinders mobility and the ability to connect markets and create jobs. Through the involvement of the private sector, which is expected to have a higher willingness to take risks, greater managerial capabilities, and the capability of assuming the upfront costs, governments can provide cost-effective services and infrastructure through recurrent smaller payments over extended periods of time, easing the impact of these investments on public budgets [9].

Second, in addition to the instrumental economic value they can offer, PPPs also have the potential to create social value beyond that created by the firm working alone [10,11]. That is, the public and the private sector can adopt hybrid arrangements to satisfy collective societal needs, increasing efficiency in basic services provision, including, but not limited to, affordable housing, access to quality education, and a robust healthcare system [12]. This blended value approach [13,14] allows the integration of economic, social, and environmental performances. This is why governments and practitioners around the world are increasingly promoting PPPs as key strategic tools to implement sustainable development projects. Recognizably, this mixed approach is inherently aligned with the essence of SDGs, which attempt to secure progress in the environmental, social, and economic domains.

However, despite the potentially beneficial aspects of the use of PPPs for public services provision—such as access to capital for the public sector, leveraging on the efficiency of the private sector and proper risk transfer—prior research has also identified several disadvantages that need to be considered. These disadvantages include, among others, high capital costs, stifled innovation, high monitoring costs, and management difficulties over extended periods of time [12]. Prior studies have explored potential mechanisms to align goals and minimize conflict in long-term PPP arrangements, such as contractual and relational governance mechanisms [15]. Still, significant gaps remain in scholarly and practitioner understanding of PPPs. In this regard, Roehrich et al. [16] found in their bibliometric and content analyses that despite the importance and scale of the PPP, there is limited conceptualization and empirical research.

Moreover, although there is a recognized need to include sustainability considerations in projects delivered through PPPs or in public procurement [17,18], there is limited research focused on how governments can advance economic, social, and environmental policy goals in parallel. A notable exception in studying how governmental contracts—i.e., public procurement—can encourage sustainability is the work by Brammer and Walker [19], which explores the variety of social and environmental ends through procurement across different countries. The authors showed that some governments are using sustainable procurement (SP) to promote social and environmental policy goals, especially in regard to local business, sustainable labor, and safety practices [19]. Yet, some areas of sustainability are based on a “fragile theoretical base” [20] (p. 37), and there is a need to engage in more conceptual building in order to advance the sustainability agenda.

Adding to the limitations mentioned above of the current literature, the most popular methods to evaluate PPP are overly focused on the economic dimension. For instance, widespread models such as the economic impact assessment (EIA) or the computable general equilibrium (CGE) model provide a proper assessment of the effect of a given event on the economy, but they offer limited analysis of the social and environmental consequences [21]. The disproportionate attention to the economic rents and costs is problematic, as it offers an incomplete view of the overall impact of PPP, ignoring the social and environmental aspects, and thus biasing both policy and private decision making. Moreover, since these assessment tools rely heavily on complex economic models, specific skills are needed not only to calculate them but also to interpret them, limiting their reach and impact.

In short, despite the potentialities of PPPs in combining the strengths of private and public actors to provide high-quality infrastructure and services while delivering social well-being, there is a clear need to conduct additional research to explore the extent to which this is truly performed [16]. The purpose of this article is to overcome the aforementioned gaps in the literature by offering a conceptual model that provides an initial evaluation of PPPs’ contribution to the SDGs, highlighting the ability of PPPs to contribute to sustainable development. Our model accounts for six different dimensions and we call it EASIER, which is an acronym for:

• Engagement of Stakeholders, in reference to the scope that the PPP has in the involvement of multiple constituencies;
• Access, which refers to the extent to which the PPP is oriented toward increasing access to social interest services to the population;
• Scalability and Replicability, regarding the degree to which a PPP can achieve profitable growth and be copied to other geographies;
• Inclusiveness, which refers to the level of coverage that a PPP offers on a non-discriminatory ground;
• Economic Impact, which refers to the impact of the PPP on the economy and its contribution to economic growth;
• Resilience and Environment, which refers to the ability of the PPP to build resilient and ecological communities.

We justify the dimensions of our EASIER model, which is derived from the cross-disciplinary literature that has addressed various aspects of PPP and sustainability. Further, in Section 3, we develop a questionnaire comprising 53 items—tapping into the six dimensions of the EASIER model—and offering an evaluation tool that helps holistically analyze the implications and impacts of PPPs on sustainable development. Then, we apply the EASIER evaluation tool to a specific PPP project in Section 4 in order to provide initial validation. The article closes with a discussion and the conclusion in Section 5.

This paper makes three general contributions. First, we add to the emerging literature studying the SDGs [22,23,24,25,26] by setting up the conditions that PPPs need to comply with in order to become meaningful to the achievement of a sustainable future. This is important given the growing reliance that governments are having on various PPP models. Second, we complement existing literature on PPP evaluation by elaborating a holistic assessment approach that accounts for the three general domains of sustainable development, adding to current models that are more centered on PPPs’ economic impacts. Thus, our work complements those focused only on value for money, by including also the notions of value for people and value for the planet. Third, our work also has practical value, as we offer a simple evaluation model that provides a first initial assessment of how a PPP is properly aligned with the outcomes contemplated by the SDGs. This can be of practical value for both public administration and private organizations that are interested in contributing to the achievement of the SDGs.

2. Conceptual Model

The 2030 Agenda for Sustainable Development establishes a global plan of action “for people, planet, and prosperity” [27]. At the core of the agenda are the 17 SDGs and 169 targets, covering all the crucial areas of sustainable development, including health, education, economic development, climate change, and environmental protection, among others. Thus, it constitutes an ambitious, multifaceted, and broad-reaching project. Besides, the agenda specifically calls for the action of all countries and all stakeholders to mobilize global efforts—in collaborative partnerships—to reach the goals and targets by 2030. Due to their multi-sectoral, multidimensional, and multi-actor essence, achieving the SDGs presents itself as a challenging endeavor [26].

In order to make the SDGs more accessible and easier to implement, there have been recent attempts to simplify them into a more manageable number. For instance, the UN Secretary General suggested a simpler structure around six essential elements—i.e., people, dignity, prosperity, justice, partnership, and the planet—to help frame, communicate, and deliver the SDGs [28]. Similarly, the Organization for Economic Co-operation and Development (OECD) attempted to organize the SDGs according to its well-being framework, which covers 11 dimensions of current well-being in OECD countries, recognizing overlaps but also important differences [29]. In addition, and taking into account the interlinked nature of the SDGs and their targets, several authors have proposed to reduce the SGDs into fewer categories or areas, with the aim of easing their implementation and allowing the maximization of synergies and identification of trade-offs. For example, Griggs et al. [30] proposed a more straightforward framework based on six areas around basic human development and environmental matters, namely: thriving lives and livelihoods, sustainable food security, sustainable water security, universal clean energy, healthy and productive ecosystems, and governance for sustainable societies. Likewise, Elder, Bengtsson, and Akenji [22] proposed an integrated framework that grouped the SDGs into six main categories or functions: social objectives, resources, economy, environment, education, and governance. Similarly, we argue for the need of a simpler classification around the three traditional pillars of sustainable development, the so-called triple bottom line approach [31]; that is, the environmental, social, and economic domains. This would provide an easier way to tackle the multidimensional and cross-linked nature of the SDGs.

Moreover, the SDGs are not only challenging in terms of magnitude, complexity, and ambition, but they also require significant budget, knowledge, and resources that governments or other actors might not have. With the adoption of the 2030 agenda, PPPs have become strategic for at least two reasons. First, as previously mentioned, PPPs have the potential to become key agents to overcome the current, massive infrastructure gap by providing financing capabilities, cost-effective services, and infrastructures, and because they tend to have higher willingness to take risks. Second, besides their existing value for money, PPPs also have the potential to create social value or value for the people by incorporating social objectives beyond common market objectives [10]. However, the complexity in the design and management of these long-term public–private arrangements also presents some limitations [12,15]. These issues and limitations of PPPs are important to bear in mind when analyzing the potentialities of PPPs driving sustainable development.

In order to realize their social value beyond their economic value, PPPs need to be fit for purpose. That means that PPPs need to move from being a mere financing tool to becoming an instrument that also provides value for people and value for the environment. In this context, The United Nations Economic Commission for Europe (UNECE) has developed the People-first PPP (PfPPP) approach to provide a model that will “foster access to essential public services for all where sustainable development as [sic] its objective and putting people first at the core” [32]. People-first PPP is defined as “a long-term contractual relationship between the public and private sector, where delivering value for people is the core objective, there is a commitment to serving and protecting the community, and the project is developed with the real interests of people in mind” [33]. That is, PPPs need to be measured according to their impacts to be in line with the SDGs, including having strong economic effectiveness and impact, being replicable, reducing CO₂ emissions, fostering resilience, and/or engaging stakeholders [32]. As a result, PPPs can become powerful tools to achieve the targets and goals of the 2030 Agenda for Sustainable Development.

2.1. EASIER Conceptual Model

Building on the existing literature, and recognizing the strong and strategic value that PPPs have due to their ability to positively affect sustainable development beyond their value for money, this article offers a conceptual model and an evaluation tool for an initial assessment of PPPs’ contribution to the SDGs. Our conceptual model aims to overcome the above-mentioned limitations of the current methodologies that evaluate the impact of PPPs—which tend to cover just one dimension or area—by offering a more holistic approach that accounts for six dimensions that are relevant for sustainable development from economic, social, and ecological perspectives. We have called it EASIER, which is an acronym of the six dimensions of our model, namely: Engagement of Stakeholders; Access; Scalability and Replicability; Inclusiveness; Economic Impact; Resilience and Environment (see Figure 1).

Hereinafter, each of the dimensions is explained in more detail, defining the concepts based on existing literature, as well as describing the link between each of them and their contribution to the SDGs.

Table 1. Contributions of the EASIER dimensions to the SDGs.

SDGs 1	Engagement of Stakeholders	Access	Scalability and Replicability	Inclusiveness	Economic Impact	Resilience and Environment
SDG 1		X	X	X	X	X
SDG 2		X	X	X	X	X
SDG 3		X	X	X		X
SDG 4		X	X	X
SDG 5		X	X	X
SDG 6		X		X		X
SDG 7		X		X		X
SDG 8		X		X	X	X
SDG 9		X	X	X	X	X
SDG 10		X	X	X	X
SDG 11		X	X	X		X
SDG 12				X	X	X
SDG 13						X
SDG 14			X			X
SDG 15			X			X
SDG 16		X	X	X
SDG 17	X	X	X

¹ SDG 1: No poverty; SDG 2: Zero hunger; SDG 3: Good health and well-being; SDG 4: Quality education; SDG 5: Gender equality; SDG 6: Clean water and sanitation; SDG 7: Affordable and clean energy; SDG 8: Decent work and economic growth; SDG 9: Industry, innovation, and infrastructure; SDG 10: Reduced inequalities; SDG 11: Sustainable cities and communities; SDG 12: Responsible consumption and production; SDG 13: Climate action; SDG 14: Life below water; SDG 15: Life on land; SDG 16: Peace, justice, and strong institutions; SDG 17: Partnerships for the goals.

illustrates this relationship between each of the six EASIER dimensions and the SDGs.

2.1.1. Engagement of Stakeholders

A sustainable development that considers value for the people, and therefore includes the social dimension of sustainability, requires the participation of all relevant stakeholders. Traditionally, a stakeholder is defined as “any group or individual who can affect or is affected by the achievement of the organization’s objectives” [34]. Over the years, different approaches and definitions of the initial stakeholder theory have evolved, depending on the context and the issue. For this research, and in the framework of PPP projects, a “project stakeholder” is defined as “a person or group of people who have a vested interest in the success of a project and the environment within which the project operates” [35]. That includes, in a broad sense, civil society organizations, businesses, and public administrations at all levels.

PPP projects are, in essence, cooperation agreements between the private and public sectors.

However, in order to have an inclusive social sustainable development, the engagement and participation of all relevant stakeholders (e.g., supplier, operator, local community’s organizations, unions, ecologists) must be ensured. In fact, multiple cases exhibit that a positive involvement with stakeholders can be a decisive factor for the success of a project [36]. The engagement of stakeholders refers not only to making information available to the public—which is indeed necessary to achieve public accountability [37]—but also to actively involve the society in the PPP process to incorporate public values and preferences in decision making [38]. An active public involvement refers to the gathering of citizens’ opinions and perceptions through several activities such as public hearings, open houses, citizen surveys, or web-based forums, among others [39]. The participation and engagement of stakeholders can be enabled at the different stages of the PPP life cycle, including in the design phase, the building or construction phase, and during the operation of the PPP project (Figure 2).

Of particular significance is the participation of stakeholders in the initial stages of the project, as it is when the parties’ requirements are being negotiated [41], which helps to increase public support of the project. Moreover, it also allows the possibility of including stakeholders’ suggestions or demands in the final project at a reasonable cost. For instance, Boyer, Van Slyke, and Rogers [39] analyzed public involvement on transportation PPPs and found that the involvement of citizens can increase support and improve the customization of projects’ designs to local conditions. However, the engagement and participation of stakeholders cannot be limited to the initial stage of a PPP, and it should be balanced through the whole life cycle of the project. Overall, the benefits of public involvement in the context of PPPs are widely supported in the literature, with positive effects on enhanced perceptions of political fairness and trust [42], improved tailoring of projects to particular local needs [43,44], and leading to more innovative projects [45], among others.

In the context of the 2030 Agenda for Sustainable Development, the utmost importance of implementing multi-actor partnerships and stakeholders’ engagement and participation is highlighted by having a stand-alone goal, SDG 17: “partnerships for the goals”. More precisely, target 17.17 encourages and promotes effective public, public–private, and civil society partnerships. In our model, this dimension includes the involvement and participation of citizenship, civil society, business, and governments at the different stages of the project life. For instance, the model evaluates if the public authorities have consulted citizenship or civil society before the final approval of the project, which could be done through consultative mechanisms such as voting (in person or online), public meetings, or by opening a project desk. In addition, the model also considers whether the project’s design phase has included the involvement of civil society and/or other public administrations. All of these different actors can provide a diverse field of experience, knowledge, technology, or financial resources to the PPP project, and as a result, contribute to the above-mentioned pillars of sustainable development (economic growth, social justice, and ecological protection) and have a direct impact on the achievement of the SDGs.

2.1.2. Access

Socially sustainable development requires access to social interest services and infrastructures to all populations, without any discrimination concerning social diversity or on the grounds of gender, race, disabilities, and minorities. For instance, women still face more limited access to education, healthcare, and other essential services in comparison to men, both in developing and developed countries [46]. Raha, Osbahr, and Garforth [47] found existing gaps between policy objectives and reality, in particular in the context of women’s accessibilities and rights. If a society wants to develop people’s potentialities fully, it is a necessary condition that all populations have access to essential services. Therefore, providing access to public interest services such as education, water, health, energy, transport, or telecommunications, among others, is fundamental to developing the full capabilities of people and societies, reducing poverty and inequalities, and fostering economic development.

The more people that can benefit from having access to public interest services, the higher the positive impact the project will have on improving human well-being, increasing social justice and equity, and boosting economic development. For instance, electrification works are a starting point to promote sustainable development, particularly in rural areas [48]. Improving access to housing is also of utmost importance in order to reduce poverty and inequality, especially in slum areas and developing countries [49]. Access to water resources is an integral part of social and economic development [50]. Additionally, there is extensive evidence in the literature of the positive effects that education has on economic growth and economic development [51,52,53], as well as the positive impact that good health has on economic progress and growth [54,55,56]. Overall, access to all of these public interest services is of paramount importance to achieve all the crucial areas of sustainable development.

In this context, PPP projects can positively contribute to providing access to these services and, as a result, help advance toward the achievement of the 2030 Agenda on Sustainable Development. This dimension evaluates the direct and indirect impact of PPP projects to different public interest services, including education (SDG 4), energy (SDG 7), health (SDG 3), housing (SDGs 1 and 11), security (SDG 16), social services (SDGs 1 and 2), transport (SDGs 8, 9, and 11), telecommunications (SDGs 8 and 9), water (SDG 6), and public space (SDG 11). For instance, a PPP project that builds a new water treatment facility or that extends the existing electricity network in an area directly increases access to water and energy, respectively. The dimension also assesses if this access is offered in a non-restrictive way on the grounds of religion, race, gender, or disabilities, thus reducing inequalities (SDGs 5 and 10). Finally, not only the increase in access to the above-mentioned services is evaluated, but also if the project improves access to the specific service and even to other services. An example would be a public transport project that not only increases access to transportation as a service, but also helps people reach other basic services, such as schools or healthcare centers while increasing access to jobs and opportunities (SDG 8) (see Table 1 for a complete list of SDGs in each dimension).

2.1.3. Scalability and Replicability

The concepts of scalability and replicability can refer to different issues, depending on the context and the topic. Here, we understand scalability as the capacity to expand in the geographical area where it was successfully implemented [57]. Therefore, a PPP project will be scalable if it can be enlarged or expanded to meet growing demand; consequently, it has the potential to provide service to a larger segment of the population than that for which it was initially conceived. In general, scaling up takes place in the same area or periphery of the original project. On the other hand, replicability refers to the possibility of transporting or copying the results of a project to other geographies or localities, even if different boundary conditions prevail [57]. Hence, a PPP project, both in its construction and operation, has the potential to be replicable if it can be exported or copied to other regions or communities.

In our model, and consistent with the UNECE’s PfPPP approach [32], both scalability and replicability are important features to increase the PPP’s transformational impact in the context of the 2030 Agenda for Sustainable Development. If a particular project or approach has the capability of being scaled up and increasing the percentage of the population covered by its service, or if it can be repeated in other geographies, and thus allowing the development of further projects, this will result in a higher attraction of investments and a higher impact of the project [33]. This scaling up of impact is important for at least two reasons. First, taking into account the current massive infrastructure gap and the amount of investment needed to achieve the SDGs, the ability of a project to be able to duplicate and/or enlarge will increase both the value for the people and the value for the economy. In this sense, PPPs that are designed with scalability aspirations from the very beginning will more likely succeed when reaching their scaling stage [58]. Second, replication is not only limited to copying infrastructures or services; it may also include the reproduction of the management processes and/or the transmission of training and knowledge. There is extensive evidence on the importance of continuous capacity building for the successful implementation of PPPs [59,60]. For instance, capacity building can take place through training and education programs for both public sector staff and private actors, or through the communication of lessons learned to both public and civic actors [61]. As a result, if the PPP project considers the training of the different employees and stakeholders involved, and/or ensures know-how transmission, its potential impact further escalates.

Therefore, both scalability and replicability relate to the ability of a PPP project to contribute to the SDGs. For instance, in view of the expected increase in urbanization rates—with the world’s urban population expected to increase to 68% of the total population by 2050 [62]—and in line with SDG 11 (sustainable cities and communities), the ability of a project to be enlarged to cope with the increasing demand will be of highest importance. However, in order to further contribute to the 2030 Agenda on Sustainable Development, this expansion must not only be financially sustainable, it must also be socially and environmentally sustainable. On the one hand, this implies the achievement of objectives such as equity, gender rights, education, or health [63], which are in line with many SDGs (mainly related to SDGs 1 to 5 and SDG 10); on the other hand, this also implies accomplishing sustainability-related objectives (including SDGs 9, 14, and 15) (see Table 1). Additionally, a PPP project can facilitate or foster its replicability by, for instance, making the project’s information freely available online (contributing to SDG 16: peace, justice, and strong institutions), or by, as mentioned before, conducting the training of local staff and other related stakeholders, and ensuring know-how transmission (SDG 4 and SDG 17: partnerships for the goals).

2.1.4. Inclusiveness

With the adoption of the 2030 agenda, UN member states pledged that “no one will be left behind” [27]. This means taking action to combat poverty, inequalities, and discrimination, and accelerating the progress of those furthest behind [64]. In other words, it requires ensuring that growth and development are equitable and inclusive. Inclusiveness is a broad concept that can refer to different aspects in different contexts, but generally, it refers to the inclusion or integration of all individuals and groups regardless of their gender, age, ethnicity, sexual orientation, culture, or religion. Therefore, inclusiveness is the ability of a system or community to avoid exclusion [65], and to ensure that opportunities are available for everyone to fully participate in society and for all stakeholders to contribute and influence decision making [66]. In this sense, inclusiveness is the result of the inclusion of diverse populations in society by ensuring universal access to basic services and the opportunity to participate without any discrimination.

Inclusiveness is an essential dimension to reach community well-being and reduce inequalities among individuals, regions, and countries. The 2030 Agenda for Sustainable Development presents a unique opportunity to achieve a more inclusive and sustainable development that takes into account the value for the people and the social dimension of sustainability. In fact, the SDG text exhibits a rhetorical commitment to inclusiveness with over 40 references to inclusion [67]—including aspects related to jobs, infrastructures, industrialization, education, gender, health, etc.—and a stand-alone goal to reduce inequality (SDG 10). PPPs have the potential to be powerful tools to provide essential services that enhance inclusiveness. However, in order to be inclusive, PPPs need to aim for the fulfillment of needs for all, especially for the most vulnerable groups, providing equal access to services and opportunities that meet the above-mentioned goal of “leaving no one behind” [64]. This means that new or existing PPP projects aiming to contribute to the SDGs need to take into consideration inclusiveness aspects when designing, building, and operating a PPP project. That includes, among other elements, making sure that the service or infrastructure provided by the PPP can be accessed by all individuals and groups—especially by the most vulnerable populations and the poorest—and promoting women’s empowerment in the different stages of the project.

Additionally, PPPs aiming to promote inclusiveness, and therefore contribute to the SDGs, need to take into account the importance of context-specific aspects, such as the geographical context and the governance context [68]. In this sense, the geographical place is likely to influence the levels of social and economic inequality. Low-income economies face a chronic lack of public infrastructure that needs to be solved in order to provide their populations with the necessary essential services [69]. Yet, and at the same time, middle and high-income economies are starting to feature increasingly concentrated pockets of poverty and exclusion, such as those of urban slums [64]. Therefore, PPP projects that promote inclusiveness in the most vulnerable areas, both in developing and developed countries, become particularly important in order to achieve the SDGs. Likewise, and in regard to the governance context, PPPs can play a significant role. For instance, PPPs have the potential to provide collective goods in cases of limited statehood [70], or they can help ensure good governance by promoting participation, transparency, accountability, fairness, or sustainable development, in particular in contexts of corruption [71].

As a result, by ensuring and promoting inclusiveness, this dimension contributes first to the reduction of poverty (SDG 1) and inequality—both income inequality and gender inequality (SDGs 10 and 5). Second, it increases access to proper nutrition, health, and education (SDGs 2, 3, and 4), as well as to other essential and public interest services, such as water or energy (SDGs 6 and 7). Third, it increases the opportunities to access jobs (SDG 8) and, if the PPP project takes place in developing countries, it has the potential to trigger innovation (SDG 9), promote responsible consumption and production (SDG 12), and foster sustainable cities and communities (SDG 11). Lastly, by reducing social exclusion, this dimension also contributes to reducing social conflict, and thus increasing peace and justice (SDG 16) (see Table 1).

2.1.5. Economic Impact

Investments in infrastructures and other public interest services, such as education or health, are crucial to help economies grow. For instance, Garcia-Milà and McGuire [72] analyzed the contribution to the output of two publicly provided inputs—highways and education—across states and over time, and found publicly provided infrastructure to be an important component of economic growth. Similarly, Roller and Waverman [73] also found positive effects of telecommunications infrastructure investments on economic growth. This dimension assesses how and to what extent PPP projects can contribute to economic growth and economic development in the area where they are implemented.

In the current context in which most governments face fiscal and public resources constraints, PPPs offer a powerful tool that can help finance the current investment gap. PPPs can be of particular importance in contexts of rapidly growing economies where governments alone cannot finance all the needed infrastructure investments [74]. Investments in infrastructure and basic services have the potential to unleash economic progress, create jobs, increase productivity, and boost trade. For example, Cohen and Morrison [75] found significant effects of public infrastructure investment on the cost savings and productivity increases of private enterprises, and thus on economic growth. Likewise, Fageda and Gonzalez-Aregall [76] found the density of motorways and the amount of port traffic to be significant determinants of industrial employment in a region, and therefore on the ability to stimulate job creation.

Achieving sustainable economic growth and lasting prosperity is at the core of the 2030 Agenda for Sustainable Development. With this stand-alone goal in mind, SDG 8—the promotion of “sustained, inclusive and sustainable economic growth”—is therefore essential in order to achieve the development goals. This dimension covers, first, the capacity of a PPP project to generate local employment and decent jobs (SDG 8). Although being employed does not guarantee escaping poverty [77], unemployment is positively related to inequality and poverty within a society [78]. Therefore, PPP projects with the capacity to generate decent jobs and increase employment—both short-term in infrastructure projects and long-term in operation and maintenance PPP projects—have the potential to contribute to economic growth and reduce inequalities within and between societies (SDG 10).

Second, PPP projects can have a positive economic impact on the area by promoting technological upgrading and technological transfer, as well as fostering research and innovation (SDG 8 and SDG 9). For instance, PPP projects that allow for the transfer and adoption of new technologies to developing economies—resulting in increases in economic productivity and potentially reducing poverty and inequalities (SDGs 1, 2, and 10)—can allow developing economies or the most vulnerable areas to catch up with those who have previously experienced greater progress [64]. Additionally, technological and innovative capacities can also help the advance toward more sustainable patterns of consumption and production (SDG 12), influencing not only the economic dimension of sustainability, but also the ecological one. Lastly, this dimension also covers PPP projects that promote industrialization as a source of employment generation and economic growth, especially in the least-developed economies (SDG9) (see Table 1 for a complete list of the SDGs in each of the dimensions).

2.1.6. Resilience and Environment

Sustainable development requires a long-term view [79]. The traditional definition of sustainable development as “the development that meets the needs of the present without compromising the ability of future generations to meet their own needs” [80] exhibits this core idea of long-term sustainability. In this sense, preserving the Earth’s natural resources, building effective resilience to natural hazards, tackling climate change, and protecting the environment are crucial to achieving ecological sustainable development in the long run.

First, resilience is defined as “the ability of a system, community or society exposed to hazards to resist, absorb, accommodate, adapt to, transform and recover from the effects of a hazard in a timely and efficient manner, including through the preservation and restoration of its essential basic structures and functions through risk management” [81]. In other words, it refers to the capacity of a system or ecosystem to respond to a perturbation by resisting damage and recovering quickly. This includes building resilience to future disasters and reducing disaster risks. Second, the capacity of a project to preserve the natural environment and combat climate change and its impacts is integral to sustainable development. Therefore, this dimension refers to the ability of a PPP project to build resilient infrastructures and ecological communities, as well as its capacity to preserve the natural environment and its ecosystems.

Developing new infrastructures can create significant benefits for economic and social development, but it can also result in substantial environmental costs. For instance, the construction and operation of infrastructures may result in the excessive consumption of natural resources, pollution of the surrounding environment, changes in land use, or in adverse effects on ecosystems [82,83]. As a result, although the role of sustainability in PPP projects is still limited [18], there is a necessity to include environmental considerations in infrastructure and other PPP projects in order to reduce their negative environmental impacts and achieve sustainable development. In this regard, legislation, for instance, has been found to be an important facilitator to implement sustainable procurement [19]. Additionally, in the context of developing countries, several studies have found that through the use of PPPs, governments can incentivize and promote sustainability, including environmental protection [84,85]. In this context, this dimension aims to assess to what extent the PPP project under evaluation contributes to ecological sustainability.

In fact, ecological sustainability is at the core of the 2030 agenda, as it is intrinsic to many of the SDGs: from eliminating hunger and poverty (SDGs 1 and 2), to achieving good health and well-being (SDG 3), to preserving life below water and on land (SDGs 14 and 15). This dimension of our model includes, for instance, whether a PPP project establishes disaster risk reduction strategies and climate change adaptation plans, such as contingency plans, as an integral part of building resilient infrastructures (SDGs 9, 11, and 13). It also assesses the contribution of the PPP project to reducing CO₂ emissions and air pollution (PM2.5 and PM10) (SDG 13), or if the PPP makes use of environmentally friendly materials during the building of the infrastructure (SDGs 8, 9, and 12). Additionally, this dimension also assesses the adoption of renewable energy solutions and clean technologies, as well as increases in energy efficiency, as one of the main strategies in PPP projects to mitigate climate change and reduce global greenhouse gas (GHG) emissions (SDGs 7 and 13). Finally, this last dimension also assesses whether the project contributes to the sustainable management of the resulting waste generated from its construction and/or operation, and whether it ensures responsible consumption and production (SDGs 6 and 12), among others (see Table 1 for a complete list of the SDGs in each of the dimensions). This last dimension chiefly adds the value for the planet to our model, which is crucial and a cornerstone to achieving sustainable development that can support the needs of present and future generations.

3. Methodology

The EASIER conceptual model described in Section 2 is complemented by the EASIER evaluation model, which is a methodological tool for an initial evaluation of the impact of PPPs on the SDGs, and is described below.

EASIER Evaluation Model

This research proposes an evaluation method based on a questionnaire, tapping into the six dimensions of the EASIER conceptual model. For each of the dimensions, several questions have been developed, based on existing literature, covering the different phases of the PPP projects, as well as the various impacts on the people, the economy, and the planet. The research methodology followed in this research is depicted in Figure 3.

First, we examined the questionnaire with a panel of experts on PPPs and sustainability, and pre-tested it in a specific PPP project. The use of a panel of experts is often recommended to maximize the likelihood of content-valid, well-constructed data collection instruments [86]. In our case, expert reviews were instrumental in developing and validating the conceptual model and the items of the questionnaire. The group of experts was selected following specific criteria. First, the group had to be composed of a balanced mix of academics and professionals who were knowledgeable about PPPs and sustainability. Second, in addition to knowledge, they had to be currently active in those two fields, in order to assure relevant and up-to-date insights. Finally, because the intended goal was to develop an assessment model, at least one member had to have experience and knowledge about the evaluation of public policies. With these criteria in mind, eight individuals were invited to be part of the panel, five of them holding PhDs in economics, management, and/or quantitative methods. They had strong research backgrounds in fields such as PPP, sustainability, strategic management, urban development, and governance. Importantly, they were contributing experts to the UNECE International PPP Centre of Excellence, which assured that they were familiar with the United Nations activities, including the SDGs. Two additional members were professional consultants, with more than 15 years of experience combined in the implementation, management, and advisory in PPPs. Finally, the group of experts was completed with an experienced PhD candidate in economics whose topic of interest was the evaluation of public policies. During several meetings with the panel members and through an iterative process, we were able to define the conceptual framework and the questionnaire.

For the selection of cases, the Specialist Centre on PPPs in Smart and Sustainable Cities (PPP for Cities) has provided an archive of practical case studies to test the methodology. In order to select the cases for pre-testing and test our questionnaire, we adopted a purposeful sampling approach, which is a widely used method in qualitative research for the identification and selection of information-rich cases [87], such as those explored in this article. Among the numerous purposeful sampling strategies that exist, we followed the Criterion-i model. Criterion-i’s objective is to identify and select all the cases that meet some predetermined criterion of importance [87]. From the perspective of qualitative analysis, this strategy will allow the selection of cases regarding their ability to meet or exceed specific criteria, and thus ensuring that they will be information-rich cases that will enable the EASIER evaluation to be carried out. However, this type of sampling strategy requires some prior information about the case study to be available in order to decide whether it meets the criteria.

We applied the following criteria for the selection of case studies:

• It must be a PPP project, where both the public and private sectors are clearly identified.
• The construction or building phase of the PPP project must be finished. The PPP project could be either in its Operation and Maintenance (O&M) phase or be fully completed.
• The case study must afford enough data to address the questions in the EASIER questionnaire.
• The sources of information must be independent and reliable. If information sources are scarce, biased, or inaccessible for any other reason, the case is of little value, since the evaluators will not be able to complete the questionnaire.

The initial questionnaire was firstly validated through the expert reviews and pre-tested in the case of the PPP project of VLT Carioca, which is a light train in the municipality of Rio de Janeiro (Brazil). The VLT Carioca PPP project fulfilled all of the above-mentioned criteria for the selection of case studies. Then, after this initial pre-test, the questionnaire was readjusted and improved. The number of questions was substantially reduced (the initial questionnaire had more than 80 questions), while some items were modified, and new questions were added. Finally, the questionnaire exhibited in Appendix A

Table A1. EASIER evaluation questionnaire.

“E” Engagement of Stakeholders

Did the public authority consult the citizenship before the final approval of the project? Did the public authority consult the civil society before the final approval of the project? Did the public authority consult the business sector before the final approval of the project? Was the project designed with the involvement of civil society? Was the project designed with the involvement of other public administration (different from the contracting authority)? Does the operation involve civil society? Does the operation involve public administrations?

“A” Access

8. Does the project increase access to the following essential or public interest services? Education Energy Health Housing Security Social services Transport Telecommunications Water Public space 9. Does the project improve access to any other service? 10. Is the access to the project service open to all the population, without restrictions on the grounds of gender, race, disabilities, minorities, etc.? 11. Is the infrastructure integrated into a system?

“S” Scalability and Replicability

Scalability 12. Is it possible to enlarge the project without any significant social cost? 13. Is it possible to enlarge the project without any significant environmental cost? Replicability 14. Is the project’s information freely available online? (i.e., tender documents, bidders, etc.) 15. Does the PPP contract management involve training of public administration employees or know-how transmission? 16. Does the PPP contract management involve training of the concessionaire employees or know-how transmission? 17. Does the infrastructure involve training of concessionaire employees or related industries’ employees or know-how transmission? 18. Does the operation involve training of the concessionaire employees or know-how transmission? 19. Does the operation involve training of public administration employees or know-how transmission?

“I” Inclusiveness

20. Does the access to service offer more significant benefit to the more vulnerable population? 21. Are subsidies for the more vulnerable people considered? 22. Is the PPP located in a developing economy? 23. Does the project promote women’s empowerment? 24. Does the project include a significant proportion of women in the design, construction, and/or operation of the project? 25. Were the local communities able to stay in their settlements despite the development of the PPP project? 26. Does the service provided benefit the rural areas? 27. Does the service provided particularly benefit people living in slums? 28. Is the project conceived from, or is it part of, a poverty reduction or social inclusiveness program? 29. Does the service help to tackle hunger, food insecurity, and malnutrition? 30. Is the project located in an area subject to violence and/or social conflict? 31. Does the project help to reduce maternal mortality, neonatal mortality, and mortality in children under five years old?

“E” Economic Impact

32. Does the infrastructure generate local employment? 33. Does the project operation generate local employment? 34. Does the project enhance technological transfer? 35. Does the project promote industrialization? 36. Does the project promote scientific research and/or innovation?

“R” Resilience and Environment

Resilience 37. Are there contingency plans and procedures in case of natural disasters? 38. Is the infrastructure insured against natural disasters? Environment 39. Does the project help reduce CO2 emissions? 40. Does the project help reduce PM2.5 and PM10? 41. Are the infrastructure building materials environmentally friendly? 42. Does the project make use of clean fuels/clean technology/renewable energy? 43. Does the project contribute to reducing the levels of energy consumption/increasing energy efficiency? 44. Does the project consider any waste management plan? 45. Does the project promote the conservation of oceans? 46. Does the project help combat land degradation and desertification? 47. Does the project halt biodiversity loss? 48. Does the project compensate for biodiversity losses resulting from construction or operation? 49. Has the project not been subject to any accident that has produced water, land, or air poisoning? 50. Has the project not resulted in a change of laws affecting protected areas? 51. Does the project ensure sustainable production? 52. Does the project stimulate responsible consumption? 53. Does the project stimulate recycling?

was completed with 53 questions divided into the six dimensions of the EASIER model. A pre-questionnaire regarding the PPPs’ economic efficiency was also developed. The pre-questionnaire presented in Appendix A

Table A2. EASIER evaluation pre-questionnaire: PPP’s economic efficiency.

Economic Efficiency

Are the procurement methods and bidding process well designed? (a) Does the procurement design stimulate competition without excluding relevant firms? (b) Do the tender documents define the service standard to be achieved during operation? (c) Is the contract awarded to the most advantageous economic bid? (d) Does the procurement process maximize mechanisms to guarantee full transparency? (e) Is a cost–benefit analysis (CBA) performed ex ante? Does the contract allocate the risks in an adequate form? Does the payment method incentivize an efficient use of economic resources? Is there an adequate institutional structure along the project lifetime? (a) Does the project have a planning agency? (b) Is the planning agency in charge of designing, evaluating, and selecting projects? (c) Does the project have an external board? (d) Does the external board review the CBA? (e) Does the external board revise the PPP contract? (f) Does the project have a PPP superintendence? (g) Does the PPP superintendence insure compliance of the contract? (h) Is the PPP superintendence in charge of contract monitoring and quality of service? (i) Does the project have a panel of experts? (j) Is the panel of experts in charge of reviewing contract renegotiations and adjudicating conflicts? Was the performance of the project as initially expected? (a) Was the infrastructure delivered on time? (b) Was the infrastructure built with the original budget? (c) Is the infrastructure/operation functioning according to the original planning?

accounts for five questions—with three of them having several sub-questions, creating a total of 20 questions. The pre-questionnaire covers aspects in regard to the correct design and structure of the PPP, in terms of a transparent and competitive tender process, a correct assignment of the risk, or payment methods linked to the risk assignment, among other areas. The final questionnaire with 53 questions was subsequently validated with the experts’ panel in an iterative process, and finally tested in the case of La Chira, which is a wastewater treatment plant in Lima (Peru). The La Chira PPP project, similar to VLT Carioca, was also made available to us by the PPP for Cities center, and it fulfilled all of the aforementioned criteria for sampling selection. The La Chira case study, which is described in Section 4, offers a concrete example of how the EASIER model could be implemented. The described process to develop the methodology (i.e., sampling logic, pre-test, and final test) has been carried out in order to ensure a preliminary validation of both the conceptual model and the questionnaire.

Scoring

This section offers a proposal for how scoring and rating could be conducted in the EASIER evaluation model. This has been the evaluation process followed in the case study described in the following Section 4. However, it should be noted that this proposal is an initial approximation for the EASIER evaluation. Therefore, the approach described in this section is mainly descriptive rather than normative. As more PPP projects will be analyzed using the present evaluation tool, the current model, including the questionnaire and its scoring system, will be further validated and improved.

The methodology proposed here is based on the observation and collection of data and information from different sources by the evaluators, who answered the questionnaire. In order to ensure validity, two different evaluators participated in the evaluation process and applied the EASIER questionnaire independently. As described above, the EASIER evaluation proposed in this research was conducted in a two-step process and consisted of a binary score, which is defined for each of the questions. The first step was the pre-questionnaire on PPP’s economic efficiency (Appendix A Table A2). If a PPP is not able to reach a minimum score in the pre-questionnaire on the PPP’s financial and economic sustainability, we recommend that the evaluation does not continue. We understand that the first condition of a PPP to positively contribute to the SDGs is that it is correctly designed as a PPP. In this regard, by making this a pre-condition for the EASIER evaluation, we want to make sure that the disadvantages and limitations of the PPPs mentioned in the introduction—i.e., high capital costs, stifled innovation, high monitoring costs, and difficulties in management over extended periods of time [12]—are taken into consideration when analyzing the potentialities of PPPs driving sustainable development. It is clear that depending on the type of project and the external factors where it is developed, the PPP can hardly accomplish a perfect score in practice. Therefore, it is under the criterion of the evaluators to determine the minimum score that an efficient PPP can obtain, depending on these factors.

The second step was the proper EASIER evaluation. Firstly, evaluators had to determine whether the questions applied or did not apply to the PPP analyzed in terms of area and sector. Secondly, for the questions that applied, the evaluator had to answer the questions and give them a score, which had a binary value. If a question did not apply, it was not considered in the final score. Therefore, the final score of the EASIER evaluation was equal to the average relative to the number of questions, n, that applied. Similarly, each dimension obtained a score, D, which was equal to the average relative to the number of questions, n, that applied in each dimension. The final EASIER score was a number that allowed the determination of the level of impact of the PPP, depending on which rank of the score was assigned.

Table 2. Level of the impact of the PPP on the SDGs, according to the EASIER evaluation.

Rank of Scores	Level of Impact
80–100%	Very high impact
60–80%	High impact
40–60%	Medium impact
20–40%	Low impact
0–20%	Negligible impact

illustrates the level of impact defined in the EASIER model, depending on the final score of the EASIER evaluation.

4. Case Study

In order to assess the EASIER evaluation tool, the questionnaire has been applied to the PPP for Cities’ case study of La Chira, which is a wastewater treatment plant (WWTP) located in Lima (Peru). The La Chira WWTP started operations in June 2016 to treat wastewater before its disposal into the sea. Before the plant was constructed, the wastewater was directly disposed into the sea, resulting in significant adverse impacts on public health, marine resources, and the environment in general. The plant had four different goals: (1) to improve public health by treating wastewater before its disposal into the sea; (2) to reduce the pollution of seawater, therefore improving marine resources; (3) the environmental recovery of beach areas to gain new leisure public spaces and thus foster economic activity; and (4) to dispose of wastewater into the sea, according to the environmental law existing in the country. The project was developed between the Peruvian government and a consortium of one national and one international private company, using a PPP framework.

To test the EASIER evaluation tool, two evaluators have carried out the questionnaire independently following the methodology explained in Section 3. First, they conducted the pre-questionnaire regarding the PPP’s economic efficiency (Table A2), in which both evaluators obtained an 80% score. Therefore, the first condition was considered fulfilled, and the proper EASIER questionnaire (Table A1) was carried out. The two evaluators predominantly looked for information that was freely and publicly available, including the PPP contract, the environmental impact study, the PPP for Cities’ case study, academic reports, and international media news, as well as official statistics from the national government and public agencies.

The summary for each evaluator’s results is presented in

Table 3. Summary of the EASIER evaluation results for the La Chira case study.

Evaluator	EASIER	Max Points Case Study	Points La Chira	Score (%)
Evaluator 1	“E” Economic Efficiency	5	4	80.00
	“E” Engagement of Stakeholders	6	6	100.00
	“A” Access	3	3	100.00
	“S” Scalability and Replicability	8	6	75.00
	“I” Inclusiveness	11	8	72.73
	“E” Economic Impact	5	3	60.00
	“R” Resilience and Environment	16	10	62.50
	TOTAL SCORE	49	36	73.47
Evaluator 2	“E” Economic Efficiency	5	4	80.00
	“E” Engagement of Stakeholders	7	5	71.43
	“A” Access	4	4	100.00
	“S” Scalability and Replicability	8	5	62.50
	“I” Inclusiveness	12	10	83.33
	“E” Economic Impact	5	3	60.00
	“R” Resilience and Environment	16	11	68.75
	TOTAL SCORE	52	38	73.08

. After conducting the questionnaire independently, the two evaluators obtained an almost identical score for the EASIER evaluation of the PPP project: 73.47% in the case of the first evaluator, and 73.08% in the case of the second. Therefore, the La Chira WWTP, with a score of 73%, is considered a high-impact PPP project contributing to the SDGs, according to the EASIER evaluation tool. However, there were disagreements between the evaluators in regard to the number of questions that applied. In the case of Evaluator 1, 49 questions were considered applicable; while 52 questions were counted as applicable by Evaluator 2. For those questions where there was a disagreement or variance between the evaluators, the responses were discussed among them, and an agreement was reached in all the dimensions. The most significant disagreement between the evaluators was found in the Engagement of Stakeholders (almost 30 percentage points of variance), followed by Scalability and Replicability and Inclusiveness (around 10 percentage points of variance each). Lastly, in regards to the analysis per dimension, the highest scores were obtained by the following dimensions: Access, Inclusiveness, and Engagement of Stakeholders, obtaining very high impact in these three dimensions.

Therefore, the La Chira WWTP is considered a high-impact PPP project contributing to the SDGs, predominantly due to the high level of involvement of different stakeholders in the different phases of the project and its contribution to increasing access and inclusiveness. This is partly due to the nature of the project to improve public health, reduce pollution, improve marine resources, and achieve environmental objectives while having an economic impact for local communities.

Raters’ Reliability

The results presented in Table 3 summarize the scores obtained by each evaluator in the independent evaluation for a given PPP, and represent an example of the possible implementation of the EASIER evaluation proposed in this work. Still, there is a need for validation of the results, considering their categorical nature. The literature offers different methods for analyzing categorical data [88]. In this line, inter-rater reliability or inter-rater agreement has excelled in the applied literature to compute the extent of agreement between two raters—in our context, the evaluators—on nominally scaled data [89].

The basic approach to find consensus between two or more raters would be to compute the observed proportion of questions in which they agreed, and focus the analysis on these measurements. Some initial work on this was done by Goodman and Kruskal [90]. However, this approach has argued to be inadequate, given that it does not adjust for the possibility of a certain amount of agreement coming from chance alone [91]. In this context, the most influential chance-corrected coefficient was proposed by Cohen [92] and its weighted extension [93] in what is come to be known as the Cohen’s kappa. Nonetheless, this coefficient occasionally yields unexpected results in situations known as the paradoxes of kappa (see Gwet [89]), which is why is common in the applied literature to present it along with other inter-rater reliability tests.

In

Table 4. Summary of inter-rater weighted agreement coefficients.

	Economic Efficiency		EASIER
	Coef.	Std. Err.	Coef.	Std. Err.
Brennan and Prediger	0.9 ***	0.1	0.809 ***	0.0726
Cohen/Conger’s Kappa	0.8864 ***	0.1129	0.6694 ***	0.1127
Scott/Fleiss’ Pi	0.886 ***	0.1139	0.6679 ***	0.1144
Gwet’s AC	0.9109 ***	0.0908	0.865 ***	0.0551
Krippendorff’s Alpha	0.8889 ***	0.1139	0.6711 ***	0.6711

* p < 0.1, ** p < 0.05, *** p < 0.01.

, we present Cohen’s kappa [92,93] and its generalization by Conger [94], along with the Brennan and Prediger coefficient [95], Scott/Fleiss’s kappa/pi [96], Gwet’s AC [89,97], and Krippendorff’s alpha coefficient [98] for the case study of the La Chira WWTP. The inter-rater agreement coefficients are estimated taking into account the disagreement weight for full chance-correction. The coefficients can be interpreted as the level of agreement of consensus between the two evaluators. These coefficients were calculated after the independent evaluation of the case study by the two evaluators, but before they met and reached an agreement on the different dimensions. Their statistical significance rejects the null hypothesis that the evaluators are making their determinations randomly, i.e., that their agreement is not due to chance. Based on the Landis and Koch [99] benchmark scale, the coefficients of the economic efficiency can be categorized as an almost perfect agreement (between 0.81–1.00), while the EASIER evaluation lands on the substantial (between 0.61–0.80) as well as the almost perfect category.

The results shown above seem to indicate a correct validation both for the EASIER evaluation pre-questionnaire on PPP’s economic efficiency (Table A2) and for the EASIER evaluation questionnaire (Table A1), as they reach a significant inter-rater agreement. However, these results cannot be interpreted as final. As described in the following last section, the EASIER methodology proposed in this article requires further validation. As more PPP projects and case studies are analyzed using the present evaluation tool, the current model will be further validated and improved.

5. Conclusions and Discussion

PPPs are potentially important instruments to achieve sustainable development. Private actors and businesses can contribute significantly to the SDGs by providing financing, sector-specific expertise and knowledge, managerial and enforcement capacity, and a higher willingness to take risks. In fact, the 2030 Agenda for Sustainable Development calls explicitly for action to “all countries and all stakeholders, acting in collaborative partnership, to implement the plan” [27]. In this context, the effectiveness of these PPPs will be crucial to achieving the SDGs [100], regarding its ability to create sustainable futures in terms of economic, social, and environmental considerations. Yet, prior studies have shown that PPPs also offer a series of disadvantages or limitations, such as high transaction costs, limited innovation, or difficulties integrating service models in the case of multiple stakeholders with conflicting interests [12,101]. This needs to be compounded with there being important limits in the existing conceptual frameworks when it comes to sustainability issues [20]. This paper provides a novel conceptual framework and an initial examination of the phenomenon of PPPs achieving sustainability. In doing so, we contribute to the existing literature in different ways.

First, most of the current theoretical approaches and evaluation tools to measure the impact of PPPs are primarily focused on the economic dimensions or economic implications of PPP projects. This article expands this narrow focus and proposes a model that accounts not only for PPPs’ economic impact or value for money, but also their impact on society (value for people) and the environment (value for the planet), and therefore, their contribution to the achievement of the SDGs. Unlike other models, the EASIER framework gives equal weight to the economic, social, and environmental aspects, which reinforces the notion that addressing sustainability requires the absence of any a priori predominance between the three dimensions [102].

Second, we bridge the literature of PPPs [5,12,13,15,16] and sustainability [17,18,19,20]. While prior studies have indicated that collaboration between the public sector and private firms can yield positive results in terms of social and environmental progress [10,19], the evidence is rather limited. Our study offers the possibility to scholars who are interested in this field to explore the extent to which, and under which conditions, PPPs can truly contribute to the achievement of societal goals.

Third, we add to the growing literature addressing SDGs [22,23,24,25,26]. Despite the increasing importance of SDGs in the academic arena, there is a dearth of studies that understand the role of PPPs in achieving them. This is striking given that one of the SDGs explicitly refers to the importance of collaboration (SDG 17). We bring the collaboration between the public and the private sectors to the forefront of sustainability agenda by assessing the extent to which a given PPP can meaningfully contribute to the social, environmental, and economic progress.

Our work also has implications for policymakers. Given the multidimensionality of the SDGs—accounting for multiple goals, which are often cross-linked among them—and the complex nature of PPPs—with different types of contracts, which tend to be intricate and long-term, and involve multiple stakeholders—governments often struggle to evaluate the inherent and broad value of PPP contracts. Our model offers the possibility of making a preliminary appraisal of this value.

In this regard, it is important to highlight that the EASIER model aims to complement, not to substitute, existing evaluation tools that are more focused on PPPs’ economic efficiency or value for money. Well-known assessment tools such as the EIA or the CGE rely heavily on complex economic models, which provide a more precise calibration of the economic impact, but overlook the societal and environmental aspects. The EASIER evaluation aims at offering a qualitative and non-exhaustive evaluation tool that offers an initial evaluation of PPPs’ impact on the SDGs from a holistic perspective, which is relatively more accessible and easier to use than more complex models. However, the EASIER model should be understood as a complementary tool to improve both the quality and impact of current methods of evaluating PPPs, in particular in regard to its relation to the SDGs and sustainable development.

In addition, the EASIER evaluation model helps advance toward the UNECE’s PfPPP approach, which is focused on delivering value for people as the core objective of PPP projects. In this sense, such value includes engaging stakeholders; increasing access, in particular to the most vulnerable populations; ensuring inclusiveness; having a direct economic impact while preserving the natural environment; and scaling up the impact of the projects by providing scalability and replicability. These are all relevant elements in the context of PPP projects as they contribute to sustainable development. During the different phases of PPP projects, i.e., the design, construction, and operation of infrastructures and services, these dimensions of our EASIER model can potentially be added to have a higher impact on the SDGs. For instance, engaging multiple stakeholders in the design phase of the PPP project will ensure social support and that it will serve the communities affected more effectively. Furthermore, creating mechanisms that allow for the development of new competencies and knowledge during the preparation and implementation of the project to be replicated, or applied to other future projects, also contributes to expanding the positive impact of the PPP. Therefore, the EASIER model allows for a higher standard of quality and impact of PPPs on sustainable development.

Finally, our work has also value for practice, in particular for those involved in activities related to the design and evaluation of projects. Although the evaluation tool has been designed for an ex-post assessment of PPPs’ impact on the SDGs, the EASIER conceptual model can also be of use ex ante. Public administrations, private companies, or financing entities could use the EASIER model ex ante as a reference to define and monitor qualitative and quantitative metrics from the early stages of a PPP project’s design. This would allow PPPs not only to increase their positive contribution to the SDGs, but also to go a step ahead in terms of reaching a higher standard of quality. We believe that the main value of our evaluating tool lies in its holistic perspective and the breadth of the dimensions, touching a wide number of cross-cutting elements. Having six dimensions accounting for economic, social, and environmental aspects, with 53 questions covering a broad range of issues—including PPPs’ contribution to job creation, their impact on increasing access to basic essential services, and reducing air and water pollution, among many others—allows for a comprehensive initial assessment of PPPs’ impacts on the SDGs.

Through the application of the EASIER model to our central case, we also learned several practical elements. For instance, because of the qualitative value or discretion in the scoring, which can result in a relatively high degree of variance, we recommend that a minimum of two evaluators participate independently in the evaluation. Ideally, three evaluators should carry out the questionnaire, and afterward, a shared final assessment should be agreed. An additional element that should be taken into account before starting the evaluation is to define the unit of analysis (e.g., neighborhood, city, region, country) that the PPP being evaluated impacts. Despite these learnings, the evaluation of the PPP’s contribution to the SDGs does not conclude with this article. We consider the EASIER methodology as the first step of a full assessment, probably involving a two-tier or three-tier approach. The EASIER model offers an initial holistic evaluation tool, and after that, one or two specific assessments should be performed in order to have a fuller picture of the impact of a PPP on the SDGs.

Our methodology offered in this article also presents some limitations that can be rectified in future research. First, the coverage of the dimensions is broad, but not in depth, in the sense of the number of indicators that are used to measure specific elements. Second, not all the questions apply to all of the PPP projects, which might create significant differences in the length of the final questionnaire applying to a specific PPP project, depending on the nature of the project. While we have taken steps to ensure the validity of the instrument through a panel of experts who perform a pre-test and estimate the inter-rater reliability coefficients, further validation of the items is required. In this sense, validity could be subsequently enhanced with quantitative approaches such as exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Third, due to the complexity of both PPPs and SDGs and the aim of this tool to be comprehensive and holistic, there is, to some extent, a judgment call in specific questions, which might not be able to be quantified.

Future research could build on these limitations by testing the conceptual framework and evaluating tool in more PPP projects in different sectors, contexts, locations, and/or cultures. The EASIER tool has been tested in the PPP project presented in this article, but the method requires further validation. As more PPP projects will be analyzed using the EASIER evaluation method, the questionnaire offered in this article will probably change slightly. Thus, we encourage further analysis to improve the current model. Nonetheless, and recognizing the current limitations, we advocate for the use of EASIER as an initial evaluation model that can offer a complementary, holistic evaluation tool to assess the contribution of PPP projects to the achievement of the 2030 Agenda of Sustainable Development.