Navigating Governance Tensions to Enhance the Impact of Partnerships with the Private Sector for the SDGs

Abstract

Partnering with the private sector is a key modality in development cooperation to achieve the Sustainable Development Goals (SDGs). Despite their increasing importance, such Public-Private Partnerships for Development (PPPD) experience major challenges in defining, assessing and reporting on their actual impact. This paper explores why, and how this can be improved. We engage in a qualitative synthesis review of academic, gray literature and evaluation reports of public-private programs of development agencies. We identify challenges, tensions and contradictions that affect a proper understanding and assessment of the impacts of such partnerships. The analysis shows that the main challenge in understanding and assessing impacts is the double governance logic that emerges in PPPD monitoring and evaluation (M&E). While M&E functions as an accountability and risk mitigation approach, it should also support collaborative characteristics of PPPDs such as trust and power-sharing, in order to enhance impactful PPPDs. Enhancing the impact of PPPDs for the SDGs requires bridging the divide between (a) result-based, upward accountability monitoring and evaluation approaches and (b) emerging learning, participatory and complexity-based approaches. The paper provides suggestions on how to navigate these governance tensions by using a paradoxical lens.

1. Introduction

Partnerships for the Sustainable Development Goals (SDGs) are perceived as important governance tools to deliver a twofold impact for development: principle-based and goal-based. Firstly, a principles-based approach was adopted at the launch of the SDGs with the ‘5P’ framework: People, Planet, Profit, Peace and Partnering [1]. These five principles are the guiding foundation for all SDGs. They represent a synthesis of principles as discussed in the global arena for the post-World War-II period: from universal human rights principles and the Organization for Economic Cooperation and Development (OECD) guidelines on multinational enterprises, to principles as defined by the United Nations Global Compact. All five principles are equally important and apply basic ethical frames like consequences, duties, rights, virtues and capabilities. ‘Partnering’ can thereby be interpreted as a means to achieve the other four principles [2].

Secondly, the SDG project is also known as ‘governing through goals’ [3]. The 17 SDGs (and 169 targets) present an interconnected set of measurable goals designed to address interrelated challenges and achieve global sustainable development [4]. SDG 17 (partnerships for the goals), thereby, specifically encourages multi-stakeholder engagement in general and effective partnerships, particularly with the private sector. The 2030 Agenda for Sustainable Development requires a massive step-up in resource mobilization and collective efforts for development impact. More than 50 per cent of the financing needed to achieve the SDGs will have to be mobilized through the private sector. Therefore, the 2030 Agenda “calls on business to apply their creativity and innovation to solve sustainable development challenges” [5].

1.1. Problem Statement

In practice, private sector engagement has become a strategic priority across many development agencies. Public-Private Partnership for Development (PPPD) is one modality for actively engaging the private sector in implementing development policy. PPPDs differ in their actor constellation, thematic focus, scope, funding and implementation structures and how development agencies frame them (e.g., public-private development alliances, development partnerships or SDG partnerships). In general, PPPDs—as an approach for private sector engagement in development cooperation—have in common that they create more or less formalized, temporary arrangements between at least a development agency and a private sector actor. Often, PPPDs involve also civil society organizations, knowledge institutions and/or public actors. These actors jointly aim to accomplish long-term social and/or economic development changes, through sharing costs, risks, responsibilities, competencies, and knowledge [6,7,8,9,10]. PPPDs aim to combine complementary logics of each sector: the advantage of private sector such as access to finance, knowledge and technologies, managerial efficiencies and an entrepreneurial spirit with the development concerns, funding and networks of development agencies [11]. PPPDs aim to pro-actively engage the private sector in the implementation of the partnership’s activities, and not only contracting out the implementation to other actors such as civil society organizations. The PPPD should develop a collaborative effort at the governance and the project level. PPPDs differ from more well-known infrastructure or service delivery Public-Private Partnerships (PPPs) in several ways; PPPDs usually work on a short-term program or project basis and aim to identify innovative solutions to longer-term development challenges. PPPDs are not service contracts and often use different governance forms and agreements rather than arrangements that emerge in long-term infrastructure PPPs. To conclude, PPPDs are collaborative governance approaches for addressing development challenges that should ultimately contribute to the SDGs.

Despite the growth of PPPDs, less is known about their actual impact and how to improve this. Participants in partnerships have found it particularly challenging to define, assess and report on their actual impact in a meaningful way [12]. So far, evaluations of development agency partnership programs have found mixed results [13]. The Donor Committee for Enterprise Development summarizes that on the one hand, some programs have not successfully catalyzed wider effects beyond the project. On the other hand, “context analysis to understand market-wide impact and hands-on management have been identified as success factors that are on track to achieve their development targets” [13]. It is in particular the development impact of PPPDs that falls below expectations [14].

Consequently, the discourse on the actual effectiveness of PPPDs as a means to achieve the SDGs and practical improvements is laden with critical accounts [15], often suggesting that the partnership is (a) not adequately—or measurably—addressing the problems for which the partnership was introduced; (b) has engaged an overly dominant private sector partner resulting in excessive or insufficient ambitions that created ‘collaborative complexities’ [16]; or (c) has overly optimistic or superficial claims, subdued responsibilities, or the governance is inadequate. These critical accounts might also explain why—according to the latest assessment of the status of specific SDG partnerships in 2019 by the United Nations Department of Economic and Social Affairs—it was concluded that despite the overwhelming efforts put into partnering around the world, “we are still only scratching the surface in terms of the number, and quality, of partnerships required to deliver the SDGs” [17].

1.2. Research Objectives

We argue that a key factor that explains the impact challenges of PPPDs lies in three types of theoretical and practical difficulties for which this paper aims to provide a contribution.

First, there is a clear need to properly understand and assess the impact of PPPDs. We focus on the critical role that monitoring and evaluation (M&E) approaches play. Extant research shows that M&E practices regularly suffer from a poor design—particularly due to a lack of a clear theory of change. They are often ad-hoc, and fail to produce the required information [5,18]. Promoting improved monitoring and evaluation systems of PPPDs should arguably be a priority to enhance the impact of PPPDs on the SDGs [13]. This ambition resonates well with previous academic research calling for improved impact studies e.g., see [12,19], evaluation frameworks e.g., see [9,12] and M&E practices of PPPDs [18]. Recent research still reiterates that “little is known about the ability of such partnerships to actually produce beneficial societal impact” [20]. Monitoring and reporting on development results of PPPDs is a critical function for accountability, direction, learning and communication purposes [5]. So, challenging PPPDs in their ability to use the M&E function well might have consequences in assessing their impact and further governing and improving the partnership. We perform a qualitative synthesis review of academic and gray literature, and development agencies’ PPPD evaluation reports in order to identify key M&E challenges related to PPPD’s impact.

Second, there is a need to explain the underlying cause for various impact measures in the hybrid governance nature of partnerships e.g., see [21,22]. There exists a double-governance logic in PPPD M&E practices in which M&E functions as a control and risk mitigation approach but should also support collaborative characteristics such as trust building and joint learning. The crucial tension between control and collaboration highlights that the two approaches start from different theoretical assumptions about managing relationships. Control approaches—often rooted in agency theory—assume goal divergence between partners and highlight an adverse partner perspective, which emphasizes control mechanisms in order to prevent opportunistic behavior, but which also might entail high transaction costs. In contrast, collaboration approaches emphasize collective behavior in which a higher value is placed on goal convergence than on self-interest [23]. The potential trade-off between control and collaboration underlying partnership M&E, raises significant governance tensions: can partnerships simultaneously be goals and means, can they simultaneously be aimed at control and collaboration, and what are the implications for the impact understanding, assessment and learning? This raises fundamental criticism and doubts about partnerships as effective governance mechanisms for the SDGs.

Third, there is need for a practical approach to deal with governance-related M&E tensions. We propose a paradoxical lens to better navigate PPPD M&E (and overcome basic governance tensions). The contradictions of partnering are often presented as trade-offs [24] and zero-sum games. Seemingly divergent assumptions accentuate dualities between control and collaboration. Both control and collaboration are however required in partnership governance at the same time for effective impact capturing. A control approach helps to ensure efficiency and effectiveness, while a collaboration approach stimulates participation and empowerment. Control and collaboration are contrasting, yet potentially complementary approaches to governance [25]. A paradoxical lens (as a meta-theory) may be supportive in dealing with the governance tensions associated with effectively capturing the impact of PPPDs [26].

In the remainder of the paper, Section 2 presents the exploratory research design in which we synthesize theoretical and practical insights. Section 3 presents the results of this effort, describing five key challenges related to M&E practices that PPPDs face. Section 4 discusses the underlying governance tensions of PPPD M&E that affects the impact understanding and assessment of PPPDs. We introduce a paradoxical lens to formulate theoretical and practical suggestions for navigating these tensions in order to enhance capturing the impact of partnerships. Section 5 concludes and outlines future areas of research.

2. Research Approach

2.1. Qualitative Research Synthesis

Academic insights on PPPDs are sparse; the phenomenon and its results are mainly discussed in studies developed by the partnering organizations themselves or advisory organizations [26]. Given the lack of a consolidated knowledge base on PPPDs, this paper systematically accumulates and synthesizes information available from scientific and practitioner research on PPPDs. A qualitative research synthesis is an explorative research approach aimed at synthesizing qualitative studies on a certain topic for the purpose of making contributions beyond those achieved in the original studies [27]. A qualitative research synthesis largely presents an inductive approach from which new conceptual understandings can emerge.

Unlike traditional literature reviews or systematic reviews and meta-analyses, qualitative synthesis studies can accommodate differences between questions, research design and the contexts of individual studies [28]. In addition, qualitative synthesis can include theoretical contributions as well as gray literature, including contributions in practitioner journals, conference papers, policy documents and reports from public or private sector organizations [28]. From this perspective, qualitative research synthesis can contribute to the development of actionable knowledge on practical phenomena [28].

2.2. Method of Synthesis

The synthesis is based on three data sources: (1) academic literature; (2) gray literature; and (3) evaluation reports.

(1)

Key insights on the understanding and assessment of the impact of PPPDs in the academic literature were reviewed. The first search focused on the Scopus database as principal search system due to its multidisciplinary nature and its retrieval qualities [29]. The search was limited to the following subject areas: social sciences, agricultural sciences, environmental sciences, business management and accounting, and economics. Since the first donor-driven PPPD programs emerged in early 2000, the search period was defined as being between 2000 and 2020.

We used the following keyword combinations:

Partnership AND “development cooperation” AND impact (21 hits)

Partnership AND “international development” AND impact (77 hits)

Partnership AND “development policy” AND impact (51 hits)

Cross-sector partnership AND impact (56 hits)

The titles of all hits were scanned and when in doubt, the abstracts were read. We only selected publications that discuss partnerships with business involvement in development cooperation, or publications that discuss the impact of cross-sector partnerships in the broader context of sustainable development. We did not include very specific impact areas (e.g., climate change or gender). Articles that focused on public-private partnerships for infrastructure development and intra-sector partnerships (e.g., international NGOs and their local counterpart or government-government partnerships, or research-policy partnerships) were dismissed. Additional articles were found through snowballing techniques (pursuing the references of references and checking citations of respective articles). In total, the search identified 10 relevant publications discussing partnerships with private sector involvement in development cooperation. In addition, we identified 13 publications providing insights into the impact of cross-sector partnerships for sustainable development.

(2)

Knowledge on PPPDs is mainly discussed in ‘gray literature’, i.e., in “the diverse and heterogeneous body of material available outside, and not subject to, traditional academic peer-review processes” [30]. Including gray literature broadened our research scope to more relevant studies, thereby providing a more complete view of available evidence. We applied the same key-word combinations as in the academic literature search in two databases (google and google scholar). In addition, we searched repositories of organizations that are experts in the field of PPPDs (e.g., the Donor Committee for Enterprise Development). We selected 10 key documents that explicitly focused on the impact and results of efforts of partnerships and programs with the private sector of bilateral development agencies.

(3)

The third search focused on evaluation reports of partnership programs of various development agencies. We selected reports that focus on PPPD programs and not on other type of collaborations such as business-to-business programs, or other forms of private sector engagement. In addition, our aim was to capture a wide variety of results from diverse agencies instead of zooming into a specific impact area (e.g., inclusive green growth or gender). It is noteworthy how little evidence of significant evaluations of PPPDs exists [31] or is publicly available. We identified the following three reports that fit our criteria (see

Table 1. Selected Private Partnerships for Development (PPPD) assessment reports.

PPPD Program	Assessment Type	Evaluator	Date
develoPPP.de program (German Federal Ministry of Economic Cooperation and Development)	Evaluation	German Institute for Development Evaluation (Deval) [14]	2017
Facility for Sustainable Entrepreneurship and Food Security (FDOV) (Ministry of Foreign Affairs of the Netherlands)	Mid-term review	Royal Tropical Institute (KIT) [32]	2016
Public-Private Partnerships & Global Development Alliances (U.S. Agency for International Development)	Evaluation	USAID [33]	2008

).

PPPD programs differ in terms of size, focus, design and structure. Similarly, the identified assessments of PPPD programs differ in type (e.g., process, impact, result assessments), purposes (e.g., accountability), depth and methodology. Therefore, our aim was not to compare the specific programs and their outcomes but rather focusing on findings regarding M&E practices for capturing impacts of PPPDs as stated in the evaluation and review reports. The selected reports analyze the program level but build their findings based on assessing PPPD projects of the program portfolio.

All data was analyzed by means of a thematic analysis [34]. In a first step, the key challenges of PPPD impact understanding and assessments were identified in each publication by developing first-order codes. We could identify patterns of these codes between the different publications and assigned second order codes (e.g., focus on direct results, goal alignment challenges, financial and development additionality). This interpretative coding means that we ‘translated’ concepts from one study into another, even if they did not use identical words [35]. In a next step, these second order codes were linked to key characteristics of PPPDs, as discussed in the literature. This allowed us to cluster the second order codes and identify five emerging themes: (1) time-span; (2) ambitions; (3) alignment; (4) added value beyond project results; and (5) relational M&E practices.

3. Findings: Five M&E Challenges for Capturing Impacts of PPPDs

The synthesis analysis identified five interrelated challenges of understanding and assessment of the impact of PPPDs in terms of: (1) time-span; (2) ambitions; (3) alignment; (4) added value beyond project results; and (5) relational M&E practices.

3.1. Time-Span: PPPDs Assess Short-Term Tangible Outputs of Activities Instead of Long-Term Changes

PPPDs are often set up as short-term projects while aiming for long-term relationships and change [8,21]. Developing deeper links and more substantial collaboration then serves as a key factor in enhancing the impact of partnerships. Moreover, collaborative relationships should become longer-term while at the same time remaining flexible. It has been found in practice that some of the best collaborations with business arise from regular and deep engagement between the staff from the development agency and company involved in the partnership [36]. Even though the quality of engagement has become increasingly prioritized over the number of public-private projects as a measure of success, the current aid system does not always facilitate the development of deeper and longer-term links with business [36].

3.1.1. Focus on Immediate Results

In contrast with infrastructure PPPs that are designed as long-term public-private arrangements, PPPDs are often designed as projects of three to five years that implement a series of activities. Monitoring is than operationalized as an accountability mechanism following efficiency principles e.g., see [14]. The short-time frame of the project feed the perception that time and money are too limited to allow for proper impact measurement [37]. Tewes–Gradl et al. identify that “many private sector and donor project managers did not see the value of measuring changes that were far removed from their own sphere of influence and which might not be observable during a project’s lifetime” [37]. Assessments then focus on immediate results; indicators at longer-term outcome level are seldom used; the same applies to indicators at the impact level [14,38]. Consequently, the short-term duration of PPPDs results in unclear development impacts—which in turn can negatively influence mutual engagement and increase transaction costs [33].

3.1.2. Focus on Performance

The emphasis on measurable results and accountability requires monitoring approaches that assess the efficiency of collaborative projects and whether they perform as agreed. These insights help to correct the partnership and ultimately mitigate risks of possible failures. It is therefore observed that in PPPDs, more monitoring (i.e., measurement of activities and baseline performance) is happening than evaluations (measurement of results, effects, outcome or impacts of the PPPD) e.g., see [14,33]. This is similar to other development interventions that are often rooted in performance-based models, which emphasize monitoring at the expense of evaluation [39]. The focus gets therefore easily distracted to short-term indicators of ‘success’ [33]. Academic literature that mainly investigates the performance and efficiency of partnerships, as highlighted by Vestergaard et al. [20] also reflects this perspective. The point of time at which an assessment is undertaken has implications for the findings [40]. “Short-term changes are, however, not good indicators of long-term change” [41].

3.2. Ambitions: PPPDs Have a Transformative Ambition but They Struggle to Capture Multiple Direct and Indirect Changes

Many PPPDs aim to be transformative; i.e., they use distinctive partner contributions to co-create sustainable societal, organizational and individual value [38,42]. Evaluations showed however that PPPDs often have unrealistic expectations [14,33]. PPPDs may use standardized M&E systems of involved organizations, that may fail to fully capture the complexity of partnerships in various contexts.

3.2.1. Various Impact Levels

Academic studies highlight that partnerships are complex and multi-layered [12]. Impact of partnerships can occur at various and multiple levels, such as the broader system and the target group. Impact is usually defined as the longer-term, direct, and indirect effects on whole issues—including the costs and benefits the partnership brings to the wider society [12]. In practice, various impact levels are studied but they are often not comparable. Mainstream research argues that the impact dynamics and the impact on poverty alleviation need to be measured at the level of the beneficiaries [20]. Others argue that long-term changes are often better observed in changed partner relationships, within partner organizations and in changed attitudes of individual participants in a partnership [9,12]. The latter argues that the partnership might add value by affecting how specific partner organizations conduct their core business as a result of their engagement in the partnership. This may translate into longer-term impact on society and the SDGs—even if a concrete partnership project fails.

3.2.2. Focus on Direct Results

PPPDs show positive unintended effects such as new and different relationships developed, changes in the way how work is done, relevant capacity development for future partnerships or project spill over effects that usually involve that project objectives being surpassed. PPPDs also show negative unintended effects, such as loss of reputation [14,33]. Research shows that partnerships struggle to capture and recognize their multiple direct and indirect changes; the focus on tangible results may overshadow recognition of intangible results and longer-term effects [43].

3.2.3. Unclear and Ambitious Objectives

Buse and Harmer [44] (p. 264) refer to a review of 74 Global Health Partnerships which identified that very few partnerships articulated objectives explicitly and measurably. The ambition may be to use comprehensive impact assessments that would attempt to capture all impact pathways. In practice, PPPD evaluations highlighted that some PPPDs have developed overambitious plans that are challenging to realize and ultimately measure the impact [32]. Consequently, postulated causal links between results are often not verifiable, assumptions are too optimistic for the given context, and packages of measures are too ambitious [14]. PPPDs have often multiple objectives. This increases the complexity and need of information gathering. Expected impacts should be estimated for each category of action, which leads to an overwhelming number of indicators and a significant rise in gathering costs. M&E budgets are often limited. So, a comprehensive impact assessment often focuses on some selected cases of the PPPD program—often comprising a collection of opinions by the actors involved and by experts. A meta study of evaluations of Dutch private sector programs found the following justification for a lack of impact data: (a) impact measurement was beyond the scope and budget of the evaluation effort; (b) there was no impact visible due to delayed implementation of projects and transactions; (c) measurement was impaired by a lack of baseline data and (d) the result indicators were too general to measure impact [45]. These findings also illustrate that not all PPPDs may lend themselves to rigorous impact evaluations [46].

3.3. Alignment of Partner’s Accountability Requirements in PPPD M&E System Is a Challenge

PPPDs implement various policy objectives. They must serve both commercial business interests and development objectives. However, public and private actors may have different impact rationales and accountability mechanisms. Private sector actors gain financial returns and access to markets, while development agencies aim to contribute to SDG 17 and mobilize additional resources and capacities [5]. The partners in a PPPD may not be able (or willing) to specify desired development impacts in any meaningful way. As a result, partnerships often lack a theory of change that would underpin the relevance of the objectives of the intervention towards development impact. Monitoring systems, consequently, may instead focus on the inputs and track leveraging effects.

3.3.1. Goal Alignment Challenges

PPPD evaluations show that goal alignment between public development impact and business interests present a considerable challenge. The evaluation of the develoPPP.de program identified a crucial tension between interests of the private partners and the development impact as specified in the programs’ requirements. The evaluation found that companies’ objectives were achieved in all analyzed cases, whereas development objectives were met only in half of the cases, which seriously reduced the added value of the partnerships [14] (p.58). The review of the Dutch FDOV program found in contrast, that most PPPDs were able to strike a relative balance between poverty reduction and business objectives, partly because they did not focus on the poorest of the poor [32].

3.3.2. Diverse Perspectives on Function of M&E

PPPD partners are usually clear that robust M&E systems are important to demonstrate impact to internal and external stakeholders and to taxpayers [31]. However, partners often differ in their perspective on the function of the evaluation, and what and how to assess [37]. The closer the PPPDs activities are to a core business interest, the more engaged the corporation is in M&E [31]. Utilizing public money as leverage for private investments in development creates a form of tension between public accountability requirements and corporate management rationales [32]. The M&E systems of development agencies are often aimed at control and accountability, which can cause a sense of rigidity in the eyes of involved companies [32]. Being accountable for the correct use of public funding can overshadow the appraisal of PPPD results [14]. An overemphasis on accountability may lead to a view on reporting as a duty or burden and not as a means to an end [14,37].

3.3.3. Financial and Development Additionality

Analyzing and measuring development and financial additionality is a key part of development agencies’ accountability requirements. Additionality means that the business would not implement the project without development agency support [47]. Measuring additionality—in particular of partnerships—is complex [5] and “hard to prove and disprove” [32]. Existing evidence and assessment criteria are limited or vague; assessments are often done ex-ante and rely heavily on information provided by applicant business [32,47]. It remains difficult to determine whether PPPDs would not have materialized without development agency (financial) engagement. Development additionality is noted e.g., in terms of gender equality, or improved working conditions [48], or is linked to an increase in scale and speed of project management [32]. Moreover, when adopted, additionality assessment is done for the business involvement only and not for the additionality of the development agency’s engagement.

3.4. Added Value: PPPDs Need to Demonstrate Their Value beyond Project Results

PPPDs are a means to an end. But in international development practice, the distinction between means and ends is often unclear [49]. Attributing results is a general challenge of evaluations. But for cross-sector partnerships, it is particularly difficult to attribute the effect of an intervention when several actors are involved who aim to bring about social change over a longer period of time [41], while operating in complex and changing environments. The OECD [5] suggests that “in the absence of an agreed, cost-effective methodology, measuring impact should focus on assessing the contribution of the collective effort to the partnership outcome”. The impact of the partnership might largely result from the collaborative process, and the synergies between the activities or other types of ‘value added’ by the partnership. Collaboration scholars, therefore, call for a better understanding of how partner relationships influence results in order to improve understanding about the usefulness of the partnership approach [38]. Partnerships can develop various institutional designs [50] and therefore it is necessary to understand which approaches are most effective in which context [37]. Evaluations largely describe the activities and some also describe the direct outputs. They are much less analytical towards whether the desired outcomes are being achieved and how these outcomes and impacts can be attributed to the partnership [33]. Most PPPDs still need to proof whether the partnering approach used for achieving the results was useful and of added value from the perspective of those involved and affected by the partnership’s intervention. In practice, measurement processes rarely reflect on the partnership as an instrument [37].

3.5. Relational M&E Practices: PPPDs Require M&E Practices that Include Equity, Participatory Methods and Flexibility

PPPDs imply a change of what to assess (partnership), for whom to assess (partners), and with whom to assess (with partners) [51]. Partnerships require different designs and metrics than purely contractual relationships; they should be based on inclusion, equity, transparency and mutual accountability.

3.5.1. Equity

Partnering assumes a relationship built on non-hierarchical structures. This creates challenges, in particular for development agencies partnering with business. Although government agencies require traditional upward accountability from the partnership projects they fund, they are also a partner and mutual accountability becomes relevant. It is argued that in order to harness the private sector’s long-term commercial interests for development, the donor-private sector relationship has to become more equal, long-term and strategic [5,31]. Consequently, partnering requires development agencies relinquish some control and predictability and learn how to engage on equal terms with the private sector [13].

3.5.2. Participatory M&E Approaches

The collaborative nature of partnerships challenges how M&E is usually done in development cooperation projects. Integrating participatory approaches into M&E activities is experienced as challenging by PPPDs [33]. Consequently, final evaluations are often not produced as a joint result of the partnership [33]. A vital distinction in their effectiveness depends on the level of trust partners can establish in the partnership. In the Dutch case, the PPPDs experienced M&E frameworks as challenging and complicated and as having high administrative burdens (adding to transaction costs). M&E related communication had to fit into the standardized reporting and monitoring framework of the development agency [32]. The evaluation of the develoPPP.de program, in contrast, highlighted the fact that the monitoring systems were strongly based on trust. The challenge that these projects faced was that they sometimes led to information deficits, which made it difficult to verify the projects’ development activities and results [14] (p. 49).

3.5.3. Flexibility

Conventional evaluation approaches to partnership M&E either lack rigor, or are too narrowly focused and are mostly unable to capture the complexity of partnerships for development. Effective partnerships have been dynamic in adapting activities and strategy to changes in the configuration of the partnership or due to changing context requirements [19]. Partnerships thus require flexible M&E systems. During the partnership realization phase, partners may encounter tensions when predefined monitoring indicators are applied because processes change, relationships are fluid and, therefore, flexibility is required in a partnering approach. PPPDs can become stuck in promises made early on in their funding contract, requiring renegotiations between partners—with serious consequences for the partnering relationship. The review of the Dutch PPPDs identified limited flexibility for PPPDs to conduct changes. This reduces their ability to experiment with and adapt underlying business cases and may lead to risk-minimizing behavior of PPPDs [32]. In contrast, the more trust-based approach identified in the German develoPPP.de program led to project criteria that could be flexibly applied and cover the different needs of companies in various partnering phases [14].

4. Discussion and Approach: Dealing with Two Contradictory Logics

The above findings highlight that PPPD M&E processes need to link two governance logics that influence a proper understanding and capturing of impact (see

Table 2. Differences between control and collaboration approaches to monitoring and evaluation.

Control Approach		Collaboration Approach
Starts from several assumptions about partnering impact and aims to minimize risks that may hamper the achievement of impact	Impact logic	Understands impact as emerging in the process of partnering and dependent on the context
Accountability	Reason	Learning
Assessing a project’s performance	Goal/Rationale	Aims to reflect on whether the ‘right’ will be achieved. Generating insights for improved action. Facilitating mutual learning between actors. Supporting reflexivity
Feedback to (single) organization (often the funder) on how money was spent and whether objectives were reached	Outcome/result	Enhancing a partnership to recognize and respond to change as well as to direct change
Accomplish the needs and requirements of one organization (e.g., funder)Accountability of partners towards this single organization	Ownership	Accomplish the requirements of the collective (and towards beneficiaries)
Financial accountability; project output (predefined indicators)	Focus	Collaborative capacity and relationship
Single organization-led: Principal-agent	Decision-making	Collaboration-led: Stewardship
Single level; result measurement based on predefined indicators; use of established methodologies	Method	Holistic approach; multi-level; open and flexible approach; context-specific
Little/no stakeholder participation;external independent consultant	Involvement	Multi-actor approach based on stakeholder participation
Timely performance informationLess flexibility of monitoring approach	Timeliness	Flexible when learning is required
Use information for program improvement and organizational purposes	Usability of information	Learning for long-term relationship; transparency; sharing of lessons learned to enhance knowledge around partnering
Partnerships may be stuck in their impact promises and face challenges of changing realities (lack of flexibility)	Challenges	Impact assessment may tend to be too loose and unmanageable, resulting in M&E without a clear focus or even learningResource intensive

(Source: partly based on Regeer et al., 2016 [52]).

). Managing these two contradictory logics, however, is also required to enhance the impact of PPPDs. The divide between both logics can be bridged by seeking a compromise between the two. But this is often the actual practice of present PPPDs, which—as argued—may explain for their impact challenges. Following a discussion of rich tradition of organizational research insights, in this section we explore how a paradoxical lens can enable partners to better navigate the different logics and consequently enhance capturing PPPD impacts. The governance tension between control and collaboration is thereby inherent in the hybrid nature of PPPDs [26]. Can M&E practice include a paradoxical lens and deal with these tensions?

4.1. Two (Seemingly) Contradictory Logics and the Emerging Tension

4.1.1. Control Approach

Control approaches assume goal divergence between partners. They highlight an adverse partner perspective which emphasizes control mechanisms in order to prevent opportunistic behavior of partners [25]. M&E in PPPDs following a control logic mainly emphasizes accountability to the development agency. It has a strong focus on collecting timely performance information, measures the result indicators as defined in the planning and design phase of the partnership, focuses on the project activities and outputs, compares results with targets using performance information to make decisions and is often steered by one organization (i.e., the development agency). A control approach requires that impact is defined in the partnership’s influence span; it follows measurable indicators and metrics, mainly at an output or immediate outcome level. A control approach is informed by agency theory, which follows the traditional rationales of accountability rooted in neo-classical economics [53]. Agency theory is premised on the assumption that there is goal divergence between principal and agent, and tight controls and monitoring need to be imposed to eliminate situations of contractors pursuing opportunistic behavior [54]. In summary, control emphasizes contracts and other forms of rules and procedures (e.g., monitoring or penalties) that provide a framework for relationships through which organizations can protect themselves from opportunism and conflict.

4.1.2. Collaboration Approach

At the same time, PPPDs follow a collaborative approach which emphasizes actor diversity and complementarity, a notion of equity and mutuality, a joint goal (and potentially also a joint problem definition, although this might be disputed because of the complex topics that partnerships try to address) [15]. Collaboration approaches emphasize collective behavior, which places a higher value on goal convergence than on self-interest. They are based on the assumption that long-term relationships are developed between actors based on trust, reputation, collective goals and involvement where alignment is an outcome that results from relational reciprocity [23]. In this sense collaboration moves beyond the purely instrumental relationships suggested by classical governance theories (e.g., agency theory or resource dependency theory) [55].

A collaboration approach aiming for transformative impact emphasizes flexibility and learning, is focused on sharing lessons learned and uses tools and measurements that should support relationship-building between partners. From this perspective, M&E comprises complex interactive processes where learning occurs in social relations. Partners monitor their experience, reflecting on this and subsequently analyzing those reflections, and then take action that becomes new experience for further reflection [56]. For an interpretation supported by all parties, a multi-actor assessment team is useful. Flexibility with room for re-interpretation enhances an atmosphere of mutual commitment and cooperation. In this context, from a collaborative perspective, assessment “is understood as learning and becomes a tool for sense-making and probing as much as for finding solutions” [57] (p.37). In conclusion, M&E that follows a collaboration approach emphasizes collaborative learning, aims to involve the target group (i.e., beneficiaries), targets holistic monitoring systems that are multi-level and adaptable if required. This perspective is sensitive to power dynamics, inclusion and exclusion issues, empowerment and disempowerment [41].

From a governance perspective, both approaches follow contrasting logics and reasoning (Table 2), resulting in tensions between learning and accountability which may be considered as incompatible trade-offs [39] and should not be conflated [58]. For example, if the emphasis is (perceived to be) on accountability, assessments may invoke defensive reactions, which may restrict the learning process [59] (p. 202). As our analysis showed, reporting by some PPPDs was experienced as a duty and not as a means to an end [33].

This tension is also highlighted by the fact that assessments following a control function tend to be public—due to transparency criteria imposed upon public agents—and are often performed by external consultants. Partners whose management and implementation activities are monitored are often inclined to defend their actions and achievements. This limits opportunities to learn from evaluations in terms of possible improvements [59]. In a collaborative approach, in contrast, the process of sense-making and interaction should be given room. This may conflict with the accountability function. Another example of how the tension between collaboration and control may manifest itself is how partners appreciate the results. Whether the realized impact and collaboration are perceived as successful may vary depending on expectations. From a control perspective, information from evaluation studies may help in decisions on the continuation of the partnership. Such data is sometimes not intended to be shared with others, and only used for internal improvement. From a collaboration perspective, the information needs to help partners to make sense of the process, and the impact and the lessons learned should also be shared externally to enhance learning from partnerships in general. Partnerships may neglect the fact that there are legitimate differences between partners about findings and may seek consensual conclusions instead of retaining the tension that is productive for their own learning [60].

4.2. Bridging the Divide between Both Logics for Improved Impact Understanding and Assessment

Despite their contradictory logic, PPPDs need both approaches, and they should be complementary. The development agency requires a control approach due to the involvement of public funding. The partnership between organizations—as distinct from traditional contacting out relationships—requires a collaborative approach for adhering to collaborative fundamentals such as equity, mutual benefit and mutual acceptance or responsibilities [24]. The contradictory requirement of control in the form of accountability and collaboration in the form of learning may create a paradoxical tension for PPPD M&E and ultimately impact understanding and impact assessment.

Linking the two logics of collaboration and control leads to conceptual and practical M&E tensions for PPPDs. While accountability and collaboration are juxtaposed, they are also synergistic and interrelated in partnerships. The tension can be framed as a paradox—“contradictory yet interrelated elements that exist simultaneously and persist over time” [61] (p. 382). At the core of paradox theory lies the acceptance of dualities of coexisting tensions where no compromise or singular choice between them has to be made [62].

Literature highlights that control and collaboration in partnering can be powerful and beneficial if harnessed. Reporting and monitoring mechanisms are often highlighted as critical formal implementation structures for ensuring the continuation of on-going efforts [43] but are also required for learning purposes. M&E literature increasingly emphasizes the need to think about multiple forms of accountability (upward, downward, mutual). Reeger et al. developed an integrative approach for reconciling both accountability and learning in one unified evaluation approach for collaborations [52]. This offers an engaging strategy attempt to deal with paradoxes on a longer-term basis and acknowledges paradoxes as a natural condition of work [63].

4.3. Practical Application: Navigating the Governance Tension for Impact Understanding and Assessment

Following this paradoxical perspective, we suggest that partnerships for development require navigating techniques that follow more procedural and learning approaches to assessing and upgrading impact over longer periods [15]. Several suggestions for navigating the control and collaboration tension for increased impact of PPPDs are discussed below. They include and combine the five interrelated themes that were identified as key challenges of PPPDs for understanding and assessing their impacts.

First, development agency and business partners need to align their impact understanding for the respective collaboration. For PPPDs this means determining clear business and development goals from the outset [33]. Partnerships in development cooperation usually emerge from or are initiated by tender processes or direct interaction with development agencies. In this first initiation phase, partners define the (added) value of their collaboration and how they link to the development agency’s policy objectives. An important foundation for impact understanding is therefore defined in this initial partnership stage; all involved must develop a shared understanding of the aspired impact and drivers for the M&E system [33]. In consequence, guiding (navigating) questions focus on the collective aspired impact and aim to prevent tensions that can emerge if motivations of individual partners do not match the jointly agreed impact objective. A possible tool to align different impact logics is the process of jointly developing a Theory of Change (ToC) and developing clarity and mutual understanding about the assumptions underlying the partnership logic. However, although developing a ToC together can support relationship-building, it may not necessarily help to overcome asymmetries in the power relations among actors [64].

Second, development agency and business partners should together define the purpose and the design of the assessment(s) based on whether they can construct appropriate measurement tools. The partnership M&E framework should be developed based on a common decided purpose that is relevant for all partners’ individual accountability and learning needs and requirements and best fits the partnership. Joint goal setting and a commitment of conducting useful evaluations supports trust building and learning about partners’ interests [33]. Such trust-building measures are needed to bring public and private institutional logics closer together [32]. Transparency around needs and willingness to understand the needs and requirements of others are key to aligning the M&E needs and developing an impact assessment that is helpful for the purpose of the partnership and of all partners involved. In addition, it should provide an assessment of the partnership’s contribution to the SDGs [65]. Control-oriented needs may require quick results and success of the partnership, whereas the other partners may aim to learn along the way and appreciate the learning more than direct results. Embedding measurements from the start of the PPPD [37] and engaging key stakeholders early in the process of designing assessments of partnerships [66] helps the PPPD to link internal steering with external reporting and exchange.

Third, process and impact are interdependent in partnerships, and organizing connections between the two types of assessment will put both in perspective [59]. Partnerships risk exclusively focusing either on impact or purely on the process or performance, though during a partnership’s realization, it becomes apparent whether an approach works and whether all specific aspects of the context have been adequately considered or all necessary stakeholders have been involved. Health checks and monitoring may reveal that the partnership approach requires a change while keeping an eye on the societal issue being addressed. Uncovering relational mechanisms such as risk sharing, power, commitment and trust can help to grasp potential or hidden problems of the partnership [67] and lower transaction costs. Unexpected monitoring findings can affect the relationship, process and activities, as they require changing the course of the partnership or even rethinking the approach, actor configuration or even the relevance of the partnership. Making agreements on how to collaborate on adaptation and unforeseen challenges (instead of deciding beforehand exactly what to do in a situation) can satisfy the control perspective, while the collaboration perspective can be used to exchange and joint reflection and learning as a source for addressing possible emerging issues.

Fourth, external exchange and peer-review can help comparisons with other collaborative and non-collaborative initiatives. Comparing and contrasting results can support both accountability and learning. It allows for making sense of findings, provides insights into which factors contributed to the success of the project or indicates created barriers that prevent contributing to societal issues. Comparing partnering approaches requires consideration of the setting and context in which the comparative case operates. Data on the process of partnering and the achievement of impact needs to be an integral part of any plans for replication and scaling of the partnership project. Such information is vital for funding and planning for future activities, and it also helps partners to reflect on the added value of their partnering approach. In addition, linking self-evaluation to an external visit would make the tension productive and prevent external assessors from drawing conclusions that are not considered justified by the partners, a procedure that may enrich internal sense-making through ‘independent’ observations. Thus, both functions may be effectively combined [59]. This process could be supported by a learning partner that supports the generation, access and use of knowledge among PPPD partners for a longer period of time, with the aim of improving the outcomes of the collaboration and developing and sharing lessons that are learned [68].

5. Conclusions and Further Research

Public-Private Partnerships for Development (PPPDs) face several challenges related to impact understanding and assessment. PPPD is an increasingly widespread practical phenomenon which has only been modestly researched for its impact—including possible lessons learned. The effectiveness, not the necessity, of PPPDs remains debated, which points to the importance of effective monitoring and evaluation approaches. This study used a qualitative research synthesis approach for bringing research evidence together from academia and practice to produce an actionable knowledge base [28]. The study synthesized five key emerging themes on M&E challenges related to the impact of PPPDs:

• PPPDs assess short-term tangible outputs of activities instead of long-term changes
• PPPDs have a transformative ambition but they struggle to capture multiple direct and indirect changes
• PPPDs M&E systems are challenged to align individual partners’ accountability requirements
• PPPDs need to demonstrate their value beyond project results
• PPPDs require M&E practices that embrace equity, participatory methods and flexibility

These challenges are caused by the double governance logic of PPPDs. This consequently has implications on M&E practice. Although partnership M&E has a control and risk mitigation function, M&E should also support collaborative characteristics such as trust-building and power-sharing. To enhance the impact of PPPDs on reaching development goals, M&E practice needs to bridge the divide between result-based, upward accountability monitoring and evaluation approaches and emerging learning, participatory and complexity-based approaches. This study elaborates on a paradox approach to address related governance and M&E tensions, by delineating processes through which the tension can be navigated. By doing so, the paper responds to calls from partnership research that highlights the need to shift practitioners’ thinking away from the search for prescriptions and move towards resolutions which are innovative [69]. In line with Vangen (2016), this paper suggests that the paradox lens can support partnership practitioners to recognize the strengths and weaknesses of both control and collaboration approaches for their M&E system and practices [70]. The paper defines several navigating interventions that support a more complex-sensitive M&E [15] and governance structure: aligning impact understanding; agreeing on a common purpose; consider process and impact as interdependent; and engaging in external exchange and peer-review. A paradox approach considers the partnering process as a continuous improvement process in which M&E serve both control and collaboration ambitions.

Avenues for Future Research

This study has several limitations.

First, this paper engaged in a qualitative synthesis study of partnership evaluation reports and key scientific studies. The findings are not based on primary data of actual partnerships or partnership programs. A further limitation of synthesizing studies in order to identify key themes is that critical contextual information is reduced (with some exceptions). Future research could investigate the impact at the (comparative) partnership or partnership program level in order to draw further lessons on more or less effective (national) PPPD M&E practices adding the broader partnership impact literature.

Second, the paper focused predominantly on one specific type of partnerships for the SDGs: Public-Private Partnerships for Development. This is an increasingly important and relevant type of partnership that operates mainly at the interface of aid and trade. However, further research could investigate whether and how similar governance and impact measurement tensions may emerge in partnerships with different actor constellations (e.g., business—non-profit partnerships; multi-stakeholder partnerships) and with various scopes (e.g., global, regional or national) or different purposes (e.g., policy, service delivery, infrastructure, capacity-building and economic development) [6].

Third, this paper concentrated on the impact of development partnerships. The intention was not to consider insights from studies that evaluate other forms of collaboration or collaborative processes e.g., see [71]. An extended perspective may provide additional insights from other collaborative forms and broaden the impact focus. The five key themes that were distinguished in this paper can thereby serve as benchmark for further testing, validation and benchmarking.

Fourth, by performing a review of the literature, this paper has systematically disentangled the impact challenge of PPPDs and conceptually explained the underlying governance tension in the M&E of PPPDs. In addition, it suggests applying a paradox approach to deal with this tension. Future empirical research could explore how PPPD cases actually deal with the paradox in practice and create an inventory of actual strategies applied by PPPDs when facing governance tensions in their partnership M&E. Due to the ‘navigating’ approach that was introduced as a way of dealing with the inherent paradoxes of PPPDs, it seems logical that researchers apply a participatory or action oriented research approach, in which the research follows and interacts with the various phases of the partnership. Witnessing the very large number of PPPDs that have been created in support of the SDG agenda, it should not be difficult to find willing candidates for this effort.