Fostering Learning in Large Programmes and Portfolios: Emerging Lessons from Climate Change and Sustainable Development

1. Introduction

Climate change adds considerable uncertainties and complexities to what are already multidimensional development challenges. These challenges are only likely to increase as the impacts of locked-in temperature rises are felt in countries most at risk, such as the Himalayan region, densely-populated deltas in Asia and much of Sub-Saharan Africa [1]. While there is clear evidence that the current levels of investment into climate change adaptation and resilience building in developing countries is insufficient to meet rising needs [2], investment into research and programme implementation continues to grow. However, effective responses are not just a question of resources: to date, the approaches that many of our organisations are taking have failed to address the complexity and the cross-scalar nature of the challenges that are triggered by climate change in the context of development [3]. This is becoming increasingly recognised at different levels, from community-level organisations right up to international funding agencies, and is prompting new reflections on how to design programming that better reflects the complexity of the challenges it aims to tackle. Scaling up learning-centred programme design and delivery is now seen as an essential part of transitioning toward sustainable and climate-compatible development pathways [4,5].

In response, much practical and theoretical work is being undertaken on the value of reflexive learning as a way of bringing together different knowledge to address the multiple dimensions of complex problems. One of the major themes in this work is better understanding how to take learning beyond the local or individual level into much wider networks of practice. The principles of resilience thinking warn us that organisations must become much more adaptive and dynamic in the ways that they anticipate, interpret and respond to unexpected change [4]. As Boyd and Osbahr note:

Failure to incorporate reflexive learning in the process could manifest itself in misguided policy positions and an inability to assess the changing science of climate change, with serious consequences for practitioners and funding, and missed opportunities for sharing lessons for addressing climate change impacts.

[4] (p. 630)

This need to do things differently has been the subject of much recent discussion within the international development community in particular, but is not yet clearly reflected in most organisations working at the climate and development nexus. New initiatives from funders including the United States Agency for International Development (USAID) [6] and the U.K. Department for International Development (DFID) [7] and large programmes like the CGIAR’s research programme on Climate Change, Agriculture and Food Security (CCAFS) [8] aimed at promoting learning-centred approaches to research and development serve to acknowledge this need and represent early actions towards new approaches. However, many approaches to date have been critiqued for their vagueness and normative treatment of learning. Armitage et al. [9], for instance, argue that the growing popularity of these approaches among development agencies, researchers, government and non-governmental organizations signals that “greater specificity and clarity concerning the meanings and outcomes ascribed to learning is necessary” (p. 96). Along similar lines, USAID’s review of the literature on collaborating, learning and adapting (CLA) highlights the need to “expand the evidence base on the effect, impact and/or contribution of CLA practices to organizational effectiveness and development outcomes” [10] (p. 12), while a recent World Bank report describes how learning processes continue to be viewed by many as “a rather insular or introspective exercise, relevant only to a small number of academics or development practitioners, instead of being a driver of social change towards increased impact” [11] (p. 1). These critiques and the subsequent under-investment into systems and processes that follow from them can mean that while the value of learning processes is recognised on the surface, there is a lack of robust analysis and high level support to drive them to scale [12].

Calls for a transition toward effective learning-centred models of programming, articulated above and elsewhere [12], are particularly salient amid the growing number, size and complexity of programmes aimed at addressing climate change impacts and resilience in the context of development. In particular, the rise of multi-project programmes such as the four examined in this paper, as a model of programming has seen a widening range of collaborators brought together, often across great geographical distances, to work together toward ambitious common objectives [13]. Buffardi and Hearn describe multi-project programmes as:

[T]ypically funded through a single mechanism and address a common, broad theme, such as community resilience or women’s empowerment. They are implemented across different locations by different organisations, and may target different population groups and employ different interventions, but are grouped together under a common set of high-level objectives, often under a single results framework. Importantly, there is an expectation of some level of interaction between the projects.

[13] (p. 4)

These multi-project programmes present novel opportunities and challenges for navigating and learning from complex problems, like climate change, particularly across scales. How, then, can learning processes avoid the pitfalls outlined above, whilst overcoming the challenges of distance, diversity, uncertainty and complexity that these modalities and challenges present? The reflections that follow draw on nested cases of large-scale learning processes with a view to providing new insights on this question.

The review and analysis that form the basis of this paper speak to some of these questions with the aim of providing lessons for future research and programming. We do so by:

(1)

Reviewing the theories that have shaped approaches to reflexive learning in large climate change and resilience-building programmes;

(2)

Conducting a comparative learning review of the design principles, challenges and lessons emerging from efforts to integrate reflexive learning processes into large, highly distributed climate change and resilience-building programmes for development.

We approach this second objective by drawing on evidence from four inter-related (or nested) programmes, all of which are seeking to integrate reflexive learning to inform their practice.

In this study learning processes are understood as cycles of action and reflection, that is, doing and thinking, performing and adapting, aimed at both improving and re-thinking practice. Our framing of learning, as we will discuss below, is influenced by Kolb’s [14] model of experimental learning, which has significantly shaped thinking on reflective practice. By approaching this second objective as a learning review undertaken by people directly involved in the design and implementation of the four programmes under study, the authors see this paper as representing a dimension of reflective practice. As such, the paper may also serve to model how learning processes can be rendered more explicit and brought into wider forums as they unfold; keys to accelerating and expanding action in this field.

We begin by reviewing the case for learning in climate and development planning and programming and linking this to the increased focus on learning at the climate and development nexus. We review some of the dominant theories of learning within the current literature on climate change, resilience and development and identify theories that most closely reflect how learning is framed and supported in the four case study programmes. This review provides the backdrop for our analysis of the four programmes, identifying themes that have stood out across this set of cases. We conclude with a review of key points and questions for future research.

2. Learning as a Unifying Theme in Response to Complexity and Uncertainty

One of the clear challenges that climate change presents to decision-making is the increased uncertainty that it creates. This uncertainty can be attributed to a number of factors, including limits to the current “skill” of climate models and projections and the inherent uncertainties of non-climatic trends that influence GHG emissions and people’s climate vulnerability, such as population and economic growth, national development pathways and political will to take action [15,16]. In specific decision settings (national or local planning, household decision-making, etc.), these factors combine to produce an ever-widening “envelope of uncertainty” that cannot be addressed solely through improvements in the climate sciences [16]. As a result, many scholars and practitioners have shifted their attention toward strategies for how to best make decisions amidst these uncertainties, rather than trying to eliminate them [17,18]. Approaches including adaptive management, resilience thinking, scenario planning and social learning have seen a rise in attention through this shift [18,19,20].

The inherent uncertainty around the future impacts of climate change, paired with the high stakes it bears on decisions related to livelihoods and development are part of what makes it a so-called “wicked problem” for policy and planning. Wicked problems are characterised by their complexity, uncertainty and the divergence of viewpoints and strategies for responding to them. As such, Head argues, tackling such problems requires “new thinking about the multiple causes of problems, opening up new insights about the multiple pathways and levels required for better solutions, and gaining broad stakeholder acceptance of shared strategies and processes [...]. This requires organisational learning and cultural change” [21] (p. 115). Along similar lines, Hurlbert and Gupta [22] term wicked problems like climate change adaptation “unstructured problems”, characterised by the levels of disagreement on the values and norms associated with the issue and on the underlying science that defines it. In such problem settings, they argue, social trust and “triple loop” learning, learning that goes beyond error-correction to rethink underlying values, policies and norms, become key factors for structuring and understanding relationships between problems and solutions.

This appeal to collective reflection, iteration and learning as a means of engaging with complexity and uncertainty is a unifying feature in the literature on resilience [23,24,25], adaptation to climate change [19,25] and disaster risk management [26,27]. To date, much of this literature has focused on learning which engages individuals and collectives (communities, households, governments, networks, etc.) who are managing or governing resources that are exposed to environmental shocks and stresses. As such, there is a strong emphasis on case-based studies of learning at scales ranging from communities to social-ecological systems [19].

More recently, an emerging body of literature has gained prominence, examining the contribution of learning processes to the management and governance of climate change, resilience and sustainability research and programming [4,28,29]. Variously described as “adaptive”, “iterative” and “organisational” learning, this analysis considers how learning, supported through monitoring, evaluation, research and dedicated learning processes, improves programme delivery and management. For Watkiss, Hunt and Savage [28] the focus is on “the management of uncertainty over time, allowing adaptation to develop within a process of learning and iteration” with a view to maximising value for money, while Boyd and Osbahr note the role of learning processes to help “avoid mistakes from the past” [4] (p. 631) and Gonsalves [29] points to improved collaboration and innovation.

2.1. Theoretical Framings of Learning in Climate and Resilience Programmes

Building on our analysis and past reviews of the literature [30,31], three theoretical framings of learning stand out as being widely used in the areas of resilience, climate change adaptation and environmental management, namely adult learning, organisational learning and learning in communities of practice. The framings span different units (or scales) of analysis, from individuals to organisations and wider networks, though the challenge of linking learning across these scales is a recurring theme within the literature [31,32]. Across the framings, there are clear areas of intersection, and of tension, around the nature, process and site of learning; some within the field of adult learning, for instance, challenge the very notion of a “learning organisation” [33].

Adult learning and adult education are themselves vast and dynamic areas of study, rooted in a concern for the ways that learning serves to strengthen individuals and societies. Its aims are instrumental, communicative and transformative, building from one of its founding thinkers’ assertion that “adult education will become an agency of progress if its short-term goal of self-improvement can be made compatible with a long-term, experimental but resolute policy of changing the social order” [34]. One of the main features of adult learning is the central role of lived experience as a source and arbiter of learning [35], with considerable focus on the ways that struggle (either individual or collective) and social action shape this experiential learning [36,37]. Kolb describes learning as “the major process of human adaptation” [14] (p. 32) and, drawing on the work of Kurt Lewin and others, frames learning as a continuous cycle of actions and reflections that lead to changes in understanding and practice. These themes remain central to many of the learning-based approaches to resilience and climate change highlighted above. A critique of Kolb’s (and others’) models of reflective practice, however, is that it provides limited clarity on the nature of the reflection itself. This includes, for instance, whether it is best understood as an individual, cognitive process or an intersubjective, dialogical one [38,39].

Organisational learning shares this interest in how cycles of experience and reflection lead to changes in individuals and collectives. Here, the organisation is the locus of change, with learning contributing to both improvements in current practice, as well as a rethinking of the organisation’s underlying assumptions amidst wider social change and uncertainty; what Argyris and Shön have famously termed “double loop” learning [40,41]. Critics of organisational learning literature, however, argue that it assumes too much homogeneity in interests within an organisation and, in practice, focuses almost exclusively at the level of technical-economic learning [33]. Field suggests that instances of effective learning are probably better described as “shared-interest-group learning”, which occurs at particular times and under particular circumstances where interests align [33].

Departing from this critique of organisational learning is the work of Jean Lave and Etienne Wenger on communities of practice as sites of situated learning that engages actors who have a shared sense of belonging [42,43,44]. For Wenger, communities of practice are at once “born of learning” [42] (p. 230) and potentially powerful sites of social learning. The distinction between a communities of practice framing and a more formal organisational framing is captured in Wenger’s note that “the currency of these systems is collegiality, reciprocity, expertise, contributions to the practice and negotiating a learning agenda; not affiliation to an institution, assigned authority, or commitment to a predefined deliverable” [42] (pp. 243–244). Membership in communities of practice is not uniform. Levels of participation differ, and the interaction between core, active and peripheral group members can stimulate and help transfer learning [45]. With their emphasis on leveraging learning and innovation across organisations and networks (both formal and informal), communities of practice are seen to serve as beneficial collaborative structures for addressing complex and interdisciplinary problems like climate change adaptation [29,45,46].

Communities of practice can be distinguished by three characteristics, which provide them with coherence and create the conditions for collective learning: a sense of joint enterprise; mutual engagement; and a shared repertoire of tools, stories and discourses that belong to the community (see

Table 1. Features of communities of practice and their contribution to learning (adapted from [27,41,42,45]).

	Definition	Contribution to Learning
Joint enterprise	Shared understanding of the goals or expectations that bind members together. Continuously negotiated and to which all members feel accountable.	Determines the level of investment and learning energy within the community.
Mutual engagement	Brings members together across their inherent diversity. Promotes the establishment of norms and collaborative relationships.	Determines the depth of social capital, trust, and reciprocity within the community upon which learning processes can rest.
Shared repertoire	Tools, stories, discourses, born of joint enterprise and mutual engagement, that belong to and define the community. Resources to make meaning of research questions, findings and applications.	Determines the shared ways of understanding that the community can use in reflecting and re-orientating practice.

) [44].

In the section that follows, we set out how these framings have provided a structure for the review of four programme case studies.

3. Study and Methods

3.1. Introduction to the Review

The rationale for this review emerged in early 2016 from a recognition that the authors were working in a set of related global programmes at similar stages of development, all of which had ambitions of promoting a strong learning orientation, and were observing common challenges in promoting learning as a core element of their programme design and delivery. That all four programmes were funded (in whole or in part) through a common government agency (DFID) provided added impetus for comparing experiences. Further, the review offered the authors, all directly engaged in the design and implementation of the programmes in question, an opportunity to engage in the reflective practice that the programmes themselves espouse. The use of learning reviews to document, analyse and communicate learning has been highlighted elsewhere as important within areas like programme design and management, where knowledge tends to otherwise remain tacit and under-documented [47].

The opportunity to conduct a comparative review across interlinked programmes of this scale while they are still in implementation was seen as a unique learning opportunity in itself, with potential lessons for the wider community of climate change and resilience research and practice in international development. While many of the challenges identified through this review can also be found in smaller programmes, the scale and structure of these large multi-project programmes amplifies the issues, but also offers novel opportunities for transferable learning within and beyond individual projects. Moreover, the case studies have been specifically chosen for their nested nature, where two of the programmes in question are actually components of higher-order programmes that are also part of the review (Figure 1). Investigating how learning plays out in the nested programming can bring new insights to the existing body of evidence on learning in large climate change programmes. In particular, the nested case studies offer new scope for comparison, as well as insights on how lessons and strategies might be transferred to higher levels of action at meso- and macro-scales (see [32]).

Table 2. Overview of the four case programmes. DFID, Department for International Development.

Initiative/Programme	Funder(s)	Budget	Implementing Partners	No. of Projects	No. of Countries	Duration	Key Themes
ICF: International Climate Fund	U.K. Government through: DFID (63%); BEIS (35%); Defra (3%) (rounded figures)	£3.87 billion (2011–2015)	Hundreds of partners including multilateral agencies, CSOs, partner governments and other agencies	200+ being implemented	35	From 2011 onwards	Climate change mitigation, resilience, forestry
BRACED: Building Resilience and Adaptation to Climate Extremes and Disasters	DFID through the ICF	£140 million (£110 million currently programmed)	15 implementation consortia and 1 knowledge management consortium, 100+ implementing organisations	15	13	3 years (plus project development and inception phase)	Resilience including: Climate and weather information, social protection, governance and natural resource management, gender
CARIAA: Collaborative Adaptation Research Initiative in Africa and Asia	DFID and International Development Research Centre (IDRC)	CAD$70 million (£43.3 m (conversion rate at time of writing)	4 consortia, 18 core consortium partners and 40+ collaborating institutions.	The 4 consortia are considered the core “projects”	16	7 years (2 years development and inception, 5 years implementation)	Vulnerability in climate change hot spots areas
PRISE: Pathways to Resilience in Semi-Arid Economies	DFID and IDRC through CARIAA	CAD$13.5 million (£8.4 million)	4 core partner organisations; 3 country research partners	7 key projects	6	5 years	Inclusive, climate resilient development in semi-arid lands

provides a more general overview on the size, membership and focus of the four programmes in question to provide a sense of their scale and comparability.

3.2. Methods

In line with comparable multi-case reflections (see [47]), this study took the form of a learning review, rather than hypothesis-driven research. Given the novel features of each of the four programmes, strict comparative analysis would prove extremely difficult. As a result, we have opted for a case study approach to analysis, with the added dimension of using nested case studies to enhance comparability across scales [48]. Comparative case study design is said to be particularly useful in contexts “where the intention is not just to describe a situation, but to analyse and interpret results in such a way that generates questions or propositions for further research” [49] (p. 44). The nested nature of the cases allows for analysis that is both horizontal (across cases) and vertical (from lower- to higher-order programmes) [48].

The starting point for the qualitative data collection and analysis was collecting information on all four case studies through document review, individual observations and a series of reflection sessions. Initial findings were then validated with team members involved the learning activities of each programme. Thematic analysis and categorisation of findings were done collaboratively by the authors across the cases and in stages. These preliminary findings were presented for peer feedback [50] to test their perceived validity and generalisability to other related programmes and contexts.

This first round of documentation and synthesis was followed by a further round of reflections and a process of data reduction and pattern identification [51]. For this, the author team prioritized, scored and grouped themes based on the robustness of evidence, agreement of observations across the cases and perceived novelty and significance as assessed by the authors based on our understanding of the field and our review of the literature. The data reduction and rating exercise led to the identification of eight themes. Through the collective analysis and in line with the case study approach outlined above, findings were structured around common processes to allow comparison and to derive insights about the wider relevance of the themes for learning in multi-project programmes. Here, the features of communities of practice were identified as the most appropriate way of structuring the findings and discussion, as we discuss below.

3.3. Linking Learning Theories to Large and Programmes and Portfolios

In considering the fit of these theories to learning aims and processes in the large and highly-distributed multi-project programmes that were the focus of our analysis, we propose two assertions to guide our review of the case-based evidence; first, that Kolb’s model of experiential learning and reflective practice provides a useful point of reference for establishing what is understood to be a “learning process”. While programmes may have more didactic components aimed at providing formal training or skills-building, these tend to sit apart from the “learning” activities, for instance as capacity building. Further, the form of reflection that tends to be promoted within multi-project programmes is intersubjective rather than individual, building on the perceived benefits of this programming model.

Second, the levels of distribution, variable participation and heterogeneity found within multi-project programmes makes them more appropriately framed as communities of practice than as “learning organisations”. This framing is supported by past reviews of collaborative research and practice on climate and development [29] and related fields [46]. A communities of practice lens also accommodates the fluidity of membership of individuals in nested programmes, where an individual may be a core member of one programme and a peripheral member of another higher or lower-order programme [42].

Taken together, then, reflective practice and communities of practice offer two useful, if imperfect, theoretical framings for first identifying learning processes and then considering how programme design and delivery can best support their emergence in multi-project programmes.

4. Results

4.1. Framing and Situating Learning in Programme Design

As stated above, each of the four multi-project programmes aims to establish, integrate and sustain learning processes and systems within the life of the programme. Across all cases, these learning processes can be seen as models of reflective practice aimed at using experiential learning to inform current practice, increase the effectiveness and impact of their work and, in most cases, generate learning that informs future programmes.

Across all four cases, learning processes have been defined by a specific body that leads on these processes within the programme. In PRISE, this was the Consortium Coordination Unit (CCU), who are themselves members of the consortium; in BRACED, it was the Knowledge Manager, which sits independently of the 15 implementation consortia; in CARIAA, it was the IDRC-based programme team; while with the ICF, it was the Climate Change Compass team (Compass), a separate team commissioned to facilitate and synthesise learning. In most cases, the learning agenda has been designed alongside monitoring and evaluation processes, which is not surprising given the rise of “MEL” (monitoring, evaluation and learning) as a part of international development and wider management processes [52]. The stated aims of the learning activities are set out in greater detail for each of these cases below (

Table 3. Roles, participants and mechanisms for learning in case study programmes

	Stated Role of Learning	Participants	Key Learning Mechanisms
CARIAA	“CARIAA is explicitly designed to capture learning on the process of producing adaptation research and supporting its use in policy and practice. This involves testing the CARIAA Theory of Change through implementation, the findings of which will be used by the CARIAA Team and consortia to inform their program strategies, as well as produce learning on processes of wider interest to the sector […]. The learning framework does not aim at measuring performance for the sake of accountability, but to learn more about how the program model is contributing to a particular set of outcomes” [53] (p. 18).	Programme staff at IDRC; members of all four consortia; CARIAA advisory committee members	Facilitated annual learning review (in-person) with all consortia; thematic working groups (in-person/online); online knowledge-sharing platform
PRISE	Learning is one of the main purposes of information collected through the [monitoring and evaluation] strategy. It should help PRISE members, other CARIAA members and the wider research community better understand the effectiveness of the programme. “Learning will particularly focus on the more uncertain elements of the programme—i.e., the stakeholder engagement and research uptake. Insights from learning will in turn feed back into the programme management cycle” [54].	Consortium members and IDRC programme staff	Annual meeting (in-person); project and country-level meetings (in-person/online); thematic working groups (in-person/online); CARIAA’s knowledge platform
BRACED	“Learning, reflection and iteration are core features of effective knowledge production and management, and inform the way we design and implement all of our activities aimed at building resilience. We do this through: working with Implementing Partners to support action-reflection cycles and document learning around specific implementation processes; conducting evaluation and research within and beyond BRACED to identify the essential building blocks that enable resilient systems to develop and sustain themselves; and testing qualitative and quantitative methods to monitor and measure changes in resilience” [55].	Knowledge manager team; members of all consortia; other resilience initiatives	Facilitated annual learning events (in-person); thematic dialogues (online/in-person); action research by knowledge manager and consortia; online knowledge sharing platform; evaluations
ICF	The Compass programme aims to capture insights from ICF-funded programmes to inform the design and development of programmes that are timely, relevant and effective in tackling climate change and reducing poverty. Ultimately, we want to learn from the experience of the ICF portfolio to inform and improve organisational decision-making. Our approach focuses on understanding why, how, in which circumstances and for which groups of people programmes are more or less effective.	DFID programme managers; policy advisers and other decision-makers; the Compass team	Co-development of learning questions; facilitated collaborative learning approaches to support critical reflection (online/in-person); realist evaluation [56]

).

The descriptions of learning purposes, actors and mechanisms for each of the programmes align closely with the model of intersubjective experiential learning and reflective practice outlined above. Differences are evident around the role and level of independence of “coordinating” bodies that convene or facilitate learning processes; as well as around the use of evaluations a stated input to learning processes or not. We return to some of these points in reviewing the lessons identified in the section below.

4.2. Learning Themes Emerging from Practice

Having established the framing of learning across these four case study programmes, the themes emerging from our learning review focus on the challenges and lessons emerging from early efforts to promote and integrate reflexive learning processes. We categorise these using three key characteristics of communities of practice literature, as outlined in Section 2.1. These characteristics, joint enterprise, mutual engagement and shared repertoire, provide communities of practice with coherence and create the conditions for collective learning.

Some have argued that communities of practice are emergent social phenomena that cannot be “artificially” created; but that, nonetheless, they can be nurtured [46]. Therefore, these themes speak not only to questions of programme planning and design, but to the role of programmes as enabling environments for this emergence. While some of the themes can be seen to relate to more than one key characteristic, each has been situated under the element it most closely supports.

4.2.1. Joint Enterprise

Large multi-project programmes are characterized by diversity, be it geographic, contextual, disciplinary or otherwise, yet work to common objectives. The strength of joint enterprise in agreeing and committing to work toward those common objectives has a bearing on collaboration towards collective learning and implementation activities.

(1) Do not assume a collaborative mind-set:

It is sometimes assumed that researchers and practitioners have a natural inclination towards collaborating with one another on themes of common interest; will share work in progress with peers; and engage in learning and reflection on a regular basis in order to improve the effectiveness and impact of their work. Given the collaborative nature of these programmes and their explicitly-stated learning focus, this was a common assumption within many of the case studies. Our learning review suggests, however, that while many programme partners are indeed inclined to collaborate in principle, it is often seen as an added burden that competes with their “core” responsibilities and is not sufficiently taken into account in their planning and budgeting. Further, despite the widespread push for increased “openness” for sustainable development [57], researchers frequently remain reluctant to share work in progress, particularly with more peripheral members of their programme’s network. In the context of collaboration across the scales of large nested programmes, this can mean that incentives for engaging in learning processes become more critical as the scale of engagement expands.

Recognising that there is likely to be reluctance to engage in learning processes in programmes of this scale, despite the enthusiasm that may be expressed in project proposals, can inform thinking in the programme planning and design stage. This was anticipated in the case of PRISE, for instance, by developing a Memorandum of Understanding signed in the beginning of the partnership that included specific clauses on collaboration, stating, among other things, that: “… the partners agree to collaborate and work together to the highest standard to ensure the successful joint achievement of the overall project” [58].

(2) Understand and create incentives for participation:

If teams leading learning design cannot assume there will be strong intrinsic motivation for partners to collaborate in learning processes, extrinsic incentives become an important way to promote engagement, especially at the outset of a programme. One of the most commonly-assumed incentives in programmes of this nature is financial (either through direct financial compensation, or as a cost recovery for the time invested in learning activities). What has clearly emerged in some of the cases, however, is that financial incentives are sometimes insufficient or simply inappropriate. In the context of diverse and transdisciplinary collaborations, there is not a ‘one size fits all’ incentive structure, and what may provide incentive for one collaborator (the opportunity to document lessons in a peer reviewed publication as an incentive for researchers, for instance) may be perceived as an undesirable burden for another (for example, practitioners who are much more interested in engaging with communities) (see Box 1). This can lead to programmes being effective at incentivising and engaging one cadre of members in learning processes while consistently missing out on others, thereby limiting the sense of joint enterprise.

One way to address this risk is to undertake a review of the individual and organisational incentives or barriers to engagement in learning processes at the outset of programme activities. The learning review revealed that was not undertaken in any of the four cases. Such an incentive review did feature as part of one CARIAA consortium’s partnership initiation processes, however; a missed opportunity for cross-scale learning within the nested programmes. Questions remain from the four cases on how to best engage with members who actively resist being involved in learning processes. Is there space for both ‘carrots’ and ‘sticks’, or does forced involvement of disengaged partners risk undermining the sense of joint enterprise in a learning community? These questions warrant further analysis.

Box 1. Case study Example 1: creating incentives in the CARIAA programme.

For the CARIAA programme, the fact that cross-programme learning is convened by one of the two funding agencies (IDRC) has allowed significant flexibility for the creation of incentive structures on learning and collaboration. One of the main approaches was through the creation of an open-ended “Opportunities and Synergies Fund”, designed in the second year of programme activities to spur more engagement in collaborative processes. To date, this fund has enabled the creation of two significant programme-wide learning processes, alongside a number of other initiatives. Through the fund, collaborating partners can partially fund travel, support from consultants, publishing costs and time requirements for these processes. As noted above, however, the availability of funds alone has not always been sufficient for compelling partners to submit proposed learning activities for funding. The added management and planning burden that must be taken on to access these funds can prove to be a disincentive. IDRC programme staff have therefore worked alongside programme members (particularly more junior members) to support them in developing proposals for these activities.

(3) Set learning priorities and realistic outcomes:

One of the key challenges on designing processes and mechanisms to support learning in programmes of this scale is striking a balance between what is possible and what is feasible. In the four cases, programme budgets are relatively large; there is a large and strong array of partners; and consequently, expectations are high. This can lead to high levels of ambition in terms of the scope and the contribution of the learning processes, or over-promising on what can result from these learning processes. Ultimately, this can have a negative impact on engagement in learning, with partners detracted by demanding plans or systems that do not align with their views of where energy is best invested. It is essential, therefore, to take member priorities as a starting point in scoping learning processes. This can prove challenging in large, multi-project programmes, given the diversity of interests and members, as well as competing priorities from outside actors, such as funders.

In line with theories of learning and adaptive management, experience from the case studies confirms that participants are more motivated to engage in learning and reflection processes if they are confident in that meaningful action can emerge from the process. As such, the design of reflexive processes must also take into account the limits around: re-defining project focus in line with new learning; re-allocating funds or staffing; revising project sequencing and timelines; or modifying partnership arrangements. Learning processes that simply highlight problems that cannot be resolved can ultimately be disempowering for participants [59]. For those coordinating learning, there therefore is a need to secure high-level commitment to acting on results from learning processes and establishing clarity on the barriers to action that might be encountered. Identifying priorities and limits at the earliest stages of a programme can both establish a sense of joint enterprise and focus attention on areas where change is possible and perceived to be relevant. In nested programmes, the limits and possibilities revealed at higher orders of programming can be used to inform the focus of learning activities at lower scales.

4.2.2. Mutual Engagement

Building on from the sense of joint enterprise, mutual engagement in learning processes across the diversity the membership is a necessary, yet inherently challenging condition in programmes of this scale. Sustaining the engagement over a wide geographical distribution also presents clear challenges. Three themes emerged from our reflection in this regard.

(1) Engage early and intensively in setting up learning processes:

Across the four case studies, it was recognised that early engagement of partners or projects is of critical importance to building connections and buy-in to learning processes. This is needed, in part, to generate shared ownership of programme-level aims or intended impacts, part of the community’s sense of joint enterprise; as well as to develop a clear sense of how each project’s activities are relevant to those aims. Perhaps less obviously, however, it is also about securing buy-in within the participating partner organisations, particularly in instances where the programme norms and approaches present challenge the norms of participating institutions. In the context of collaborative and transdisciplinary programmes, this can involve challenging deeply-ingrained assumptions and values, for example between advocacy-based NGOs, research-intensive universities and policy-oriented think tanks [29]. Recognising that mutual engagement across these scales is time and resource intensive; early engagement is critical to establishing the sense of connection and trust that can sustain investment into learning processes (see Box 2).

Box 2. Case study Example 2: leadership changes in the BRACED inception phase.

Programme-wide learning activities in the BRACED programme are convened by its “Knowledge Manager”. The selection and contracting of the Knowledge Manager was done concurrent to selection of the 15 BRACED implementation projects, with an interim knowledge management body assuming responsibility for those functions during the bidding process. Due to delays in contracting, the Knowledge Manager formally started activities after the projects. The implication of this timing was that BRACED implementation projects were later asked to retrofit collective learning processes into their respective workplans, a process that caught many partners off guard and was the source of tension at the outset of programming activities. Managing these tensions required a substantial amount of time and negotiation on the part of the Knowledge Manager and the Implementation Partners. This experience underscores the need for careful sequencing and early initiation of core programming structures to create the right pre-conditions for learning and exchange.

(2) Promote collective ownership of learning agendas:

Learning in programmes of this scale quickly becomes futile if there is no sense of collective ownership, as found in the case studies. Given that learning themes in multi-project programmes may focus around areas of domain or geographical specialism, it can be tempting to encourage partner organisations to each “own” their respective areas of focus in the collective agenda. However, there is a risk that doing so can create organisational silos that restrict wider learning and exchange. Across the case studies, cross-programme working groups on key themes/functions have been used as one mechanism for supporting the integration and emergence of collective ownership that cut across partner organisations. Ultimately, this model of integration across the partnership is critical if the programmatic “whole” is to become more than the sum of what each partner can offer.

Working with organisational champions is an oft-mentioned strategy to facilitate organisational learning and innovation (see [60]). However, in large programmes, leaving this to individual champions involves some risk. Individuals may not be able to drive the process within such a large and distributed process; may not have built up the social capital bring others on board at critical junctures; or may be lost through turnover, leaving critical gaps in leadership. As such, facilitation support to working groups and their champions (in the form of setting up and managing calls or meetings, process guidance and support in sense-making) can help to sustain collective engagement in these processes over a programme’s lifespan.

(3) Invest in facilitated, face-to-face engagement:

Our review reaffirmed that there remains no real substitute for face-to-face engagement for learning, even in programmes of this scale [19]. The amount of progress made in three days of effectively-facilitated interaction can be equal to months of virtual collaboration. Though meetings and process facilitation in the programmes of this scale require substantial investment (in terms of time, personnel and resourcing), when compared to the investment lost to processes that do not deepen engagement and learning, the cost is worth the investment. Given the scale of activity required in programmes of this size (CARIAA learning events have brought together 65–80 participants from over 20 countries, while BRACED Annual Learning Exchanges can involve up to 150 participants over three days), programmes must avoid the temptation to “muddle through” without the effective support mechanisms for such gatherings. This support includes: professional facilitation that builds collective engagement and supports the identification and work toward learning outcomes; documenting process and progress; and a commitment to invest into pre- and post-event activities that maximise the value of the convening. External facilitation is also seen as an important tool for helping to manage the power dynamics that invariably influence (and can ultimately undermine) collective learning processes [46,61].

4.2.3. Shared Repertoire

Once established, a shared repertoire, collective resources born from joint enterprise and mutual engagement, has the capacity to further enhance mutual engagement and bring coherence to a learning community [44].

(1) Identify and regularly revisit themes of shared interest:

One of the challenges to learning from implementation activities within the multi-project programmes examined here is the wide range of themes and contexts within which these activities have been undertaken. This can make identifying common areas of focus from amid the diversity of actions a challenge. The fields of climate change adaptation and resilience also have strong tendencies toward focusing on the context-specific nature of impacts and responses. This tendency, while justifiable, can create challenges for identifying commonalities or generalizable features across specific findings or activities. Doing so involves encouraging and enabling researchers and practitioners to think beyond the bounds of their own projects, which is not always an easy task amidst competing priorities.

Thus, an important task is to identify themes that speak to the interests and priorities of partners (particularly individuals) and to revisit them regularly in light of new developments at project and programme levels, as well as developments external to the programme (world events, new shifts in policy landscape, etc.). A lack of clarity or focus is one of the common points of frustration raised in participant feedback from virtual and face-to-face learning processes in the cases here. While “learning about learning” may be a theme of great interest for some, it is likely to be insufficient for widespread engagement in this field. A related challenge, then, is identifying themes that have sufficient breadth as to be widely relevant, while having enough focus to generate active engagement among a subset of the programme membership (Box 3).

Box 3. Case study Example 3: honest acknowledgement of the limitations of learning in the ICF.

There has been a high level of interest and enthusiasm for learning across the ICF portfolio, but this is balanced with stakeholder’s scepticism of the effectiveness of learning programmes and the limited time individuals have to engage. The ICF learning team, Compass, communicated early with potential participants with regards to the scope of the work, its limits, the processes for identifying learning questions and expectations for engagement. For example, Compass made it clear that learning themes would only be set if they were strategic, portfolio-level and enduring. Programme-specific questions that were not relevant across the ICF would not be considered. Programme managers and policy advisors not only appreciated this honesty, but it also has supported them to understand if, when and how they might engage with the learning work.

(2) Iterate and adapt learning mechanisms:

Much like the emergence of a repertoire of shared learning themes, the development of collective learning mechanisms (the means through which learning activities are undertaken) is a continuous task. While programmes of this scale require significant investment in coordination, it is important not to let the scale of the programme hinder adaptive management. This means reviewing and re-thinking learning mechanisms early and regularly. Rather than striving for a “perfect” set of approaches from the start, it can be more realistic and cost-effective to build on those mechanisms that are generating positive results and scale back investment into elements that do not generate buy-in or yield much in relevant learning.

These processes of adaptive management are not without their challenges, however. We sometimes assume that researchers and practitioners want flexibility and opportunities to review and improve systems that they deem inappropriate, while casting programme funders as rigid and resistant to change. The cases here suggest that this is not always the case. Adaptive management and programming means that budgets and programme practices must have flexibility in response to new learning [62], but funding recipients also want a degree of predictability. Thus, while there is a clear need for adaptive systems and processes to enable learning, the disruptive effects of these processes are seen to carry both opportunities and risks.

5. Discussion

5.1. Multi-Project Programmes as Communities of Practice

We return now to our assertion that a communities of practice framing is well suited to considering how programme design and delivery can best support the emergence of learning in multi-project programmes on climate and development. While this analysis was primarily undertaken as a learning review rather than a testing of this hypothesis, mapping the themes that emerged from the review onto the characteristics of communities of practice offers useful insights on the framing and its challenges in practice (see

Table 4. Mapping learning review themes onto characteristics of communities of practice.

Joint enterprise	Do not assume a collaborative mind-set at the outset of the programme; Understand and create incentives for participation; Set learning priorities and realistic outcomes.
Mutual engagement	Engage early and intensively in setting up learning processes; Promote collective ownership of learning agendas; Invest in facilitated, face-to-face engagement.
Shared Repertoire	Identify and regularly revisit themes of shared interest; Iterate and adapt learning mechanisms.

).

Experiences from the case studies underscored the challenge of promoting joint enterprise and mutual engagement in a time-bound programme setting. If communities of practice cannot be artificially created, but can be nurtured [56], programme planning and design must take on an increased focus on how to best nurture emergent learning-oriented collaborations, rather than on “design and deliver” approaches that are typical of programme management in climate and development. The bodies tasked with facilitating or convening learning processes face the challenge of initiating such a shift in focus.

Our review also highlights a need for those scoping, designing and initiating multi-project programmes to adopt new approaches if they wish to promote the emergence of intersubjective and reflexive learning within a community of practice model. For instance, experience from the four cases reveals that it can take over a year for some new learning systems or practices to get developed, tested and brought into widespread use. For programme funders, this has important implications for establishing the overall timeframes of learning-oriented multi-project programmes, as well as for the sequencing of when certain core functions (such as learning support) are initiated within the programmes.

5.2. Implications for Nested Programmes

One of the noted features of programmes in this review was their nested nature. This nesting, wherein membership, learning and competencies can shift fluidly between higher and lower-order programmes, offers both opportunities and challenges for promoting learning. For instance, while the importance of early and intensive support for mutual engagement was recognised across the case studies, this takes on added complexity within nested programmes, as programmes at different scales are often attempting to establish learning processes and joint enterprise simultaneously. This creates challenges for the sequencing and alignment of emergent processes. Done well, however, the nesting can also offer efficiencies in setting up systems, where partners are able to adopt common approaches, and facilitate the emergence of a shared repertoire of tools and approaches (see Box 4).

Other dimensions of programme design and implementation reveal the same tension between complexity and learning opportunity in nested programmes. Perhaps most significantly, face-to-face meetings across nested programmes offer tremendous opportunities for the transfer of lessons and approaches across scales. In moving from higher order programmes to sub-programmes, they present the chance to introduce overarching approaches and negotiate their fit within nested programmes. In moving upward to higher orders of programming, they offer the opportunity to look at evidence emerging from direct application of concepts and approaches and to explore how similar approaches might work at higher scales of operation. Making the most of these opportunities hinges not only on awareness and ambition, however, but on the programme’s ability to engender a commitment to more complex and intensive forms of interaction within learning processes. Drawing on the principles from communities of practice, this depends on a strong sense of joint enterprise and on mutual engagement.

Box 4. Case study Example 4: learning transfers between PRISE and CARIAA.

For PRISE, annual CARIAA learning events and other cross-consortia meetings are one of the key mechanisms to share learning within and across consortia, and to transfer learning from consortium-level to the CARIAA programme level and back. This has generated a number of concrete outcomes that have shaped both programmes, including new approaches to programme monitoring and evaluation. Here, PRISE first contributed to a collaborative development of CARIAA’s online platform for progress reporting, undertaken as a facilitated face-to-face exercise. PRISE then used the development process as an opportunity to set up its own tool for gathering its own monitoring evidence from partners in programme countries. After a year of use, PRISE shared their revised approach back to the CARIAA level at a programme learning event, with the CARIAA team now using aspects of it to inform a new system for programme-wide learning on research uptake. This process of iterative learning is at the heart of the cross-scale exchange across nested programmes examined in this paper, and the opportunities presented by in-person gatherings have been critical to its effectiveness.

6. Conclusions

Iterative and learning-based approaches to climate change and resilience programming are increasingly being adopted in the context of international development. Paired with the rise of ever-larger “multi-project programmes” expected to deliver impacts at scale, this presents new opportunities and challenges for those who design, manage and implement these types of initiatives. While a significant body of case-based evidence exists for smaller scale and more localised learning initiatives, this paper has tried to contribute to the evidence base on much larger global multi-project programmes, where the task of developing a sense of coherence and collective endeavour can be daunting. We have also sought to draw out how nested programmes offer novel opportunities for learning to move across scales.

While the majority of the themes emerging from this review can be applied to international development programmes beyond the climate change and resilience focus covered here, the complexity and uncertainty of climate change creates an added impetus for having reflexive learning processes sit at the core of programme interventions. This, we argue, calls for a shift in thinking about programme design, with an increased focus on how to nurture emergent learning-oriented collaborations. Undertaking such a fundamental shift in programming approach over the scales of partnership explored in the cases above is no small task. As we have set out above, it involves changes in individual and institutional incentive structures; in programme design; in management principles and practices; and in resource allocations within programmes. It also requires a clear recognition of both the requirements for and the limits to learning-based approaches that are present in programmes of this scale. Drawing on the literature on communities of practice can offer important insights on the kinds of changes needed and the emergent model engagement that programmes might strive for.

Building on this last point, further applied research is needed to better understand these requirements and limits. We have argued, for instance, that nested programmes offer novel opportunities to scale up effective approaches within comparable contexts, but are there limits to the scale at which certain learning processes can be effectively applied? Can programmes reach levels of diversity and distribution that make the sense of joint enterprise impossible to establish? And if so, is this a signal for putting limits on the scale of learning-oriented programme design, or rather a call for altogether different approaches? As investment into research and action on climate change and resilience continue to scale up, these questions will grow ever-more pressing.