Facilitating Collaborative Partnerships in Education Policy Research: A Case of Multi-Stakeholder, Co-Investigation for Monitoring and Evaluation of Education for Sustainable Development

Abstract

Strengthening the research-policy interface is dependent on conducting good research, as well as the appropriateness and applicability of identified policy options. The involvement of relevant stakeholders in collaborative research efforts to co-produce knowledge and recommendations to advance policies is one approach that can arguably improve this interface. This paper provides a practical instance of a research process on education for sustainable development (ESD) to develop a monitoring and evaluation (M&E) framework, which was conducted in the Asia region with participants from seven countries. This research process is presented as a pragmatic case study of how a collaborative research partnership was facilitated, and it examines how the interaction between researchers, policymakers and practitioners can be structured to support mutual learning in the field of sustainability education. The paper examines the wider debates regarding the research-policy interface, and it identifies the learning features that were achieved in this collaborative partnership, as well as the benefits this had for the research and knowledge co-generation. The paper concludes with a discussion of the challenges and issues M&E raises about the relationship between research and policy in ESD and suggests ways to address them.

1. Introduction

Policy research and evidence-based policymaking are essential in the strategic implementation of international sustainability and climate agendas. Its contribution to formulating, developing, and implementing education policy is equally important [1,2]. There has been a significant increase in efforts to strengthen the research-policy (or science-policy) interface and develop productive partnerships for collaborative research between researchers, policymakers and practitioners in order to advance evidence-based policymaking towards the Sustainable Development Goals [3,4,5,6].

Multi-Stakeholder participation and collaborative research partnerships can enhance the effectiveness of education policy research in dealing with context and complexity. In the case of monitoring and evaluation (M&E) of education for sustainable development (ESD), it can increase identification of existing gaps when it is designed to “engender a process of both individual and institutional learning by creating an action-reflection cycle that supports the continual review of implementation and practice” [7]. This learning process “takes place both in action and interaction and focuses on the cognition-action relationship” by which the learning entities “assimilate information and update their cognitions and behaviour accordingly” [8]. It also includes “the cognitive dimension of knowledge and skills, the emotional dimension of feelings and motivation, and the social dimension of communication and cooperation …embedded in a societally situated context” [9]. There are potentials to engage in collaborative research partnerships and knowledge co-production both in the establishment of an M&E framework and in the actual monitoring and evaluation process.

M&E of education supports: (1) Revealing the progress of an implemented education project over time; (2) influencing future education policy and practice; (3) improving decision-making and action; and (4) serving as a means of implementation [10,11,12]. M&E is, thus, a valuable policy tool for implementing sustainability education and ensuring that it achieves expected objectives. In ESD,

“…monitoring and assessment frameworks help: Ensure on-going relevance and effectiveness of ESD efforts, guide planning and reorienting of…programmes; increase understanding of ESD progress, and improve decision-making and action-taking… If participatory evaluative frameworks are used, the process can also inspire and build knowledge among stakeholders nationally and regionally” [13].

Although the importance of M&E of ESD was identified as one of the seven key strategies of the UN Decade of Education for Sustainable Development (2005–2014) [14], it was not given the required level of attention, hence, leading to its limited use and effectiveness [15].

There has been a fair amount of discussion and debate on methodologies and strategies for data interpretation regarding M&E of ESD [16]. However, the collaborative learning and governance processes that underpin effective M&E of ESD have received little attention, and the value of collaborative research in supporting the development of effective M&E frameworks and identification of appropriate ESD indicators has been almost completely overlooked or written off as work to be completed by statisticians. While there has been significant discussion in relation to environmental sustainability and natural resource management regarding the need to enhance collaborative research partnership for the better bridging of the research-policy interface, such discussions have been much less common in relation to educational policy [17,18,19,20,21].

This paper, therefore, presents the case of a practical, collaborative research project to develop an M&E framework on ESD relevant within the Asia region and with participants from 7 different Asian countries in the research process. The research project consisted of three main research components (or research tracks): The first focused on country ESD status reports, the second focused on case studies from local-level ESD initiatives, and the third focused on facilitating a collaborative, multi-stakeholder partnership for co-investigation and knowledge exchange at the research-policy interface. The purpose of the paper is to explore how this diverse partnership and collaborative process was able to facilitate greater interaction between researchers, policymakers and practitioners and improve the overall outputs/outcomes of the research, as well as their relevance for evidence-based policymaking. The aim is to present a reflective case study of the methodology and techniques used to achieve this collaborative partnership and multi-stakeholder investigation (i.e., the primary focus of the third research track), and to share the insights of what were the effective practices in facilitating this collaboration and what benefits this had for the overall impact of the research.

This M&E of ESD research project aimed to facilitate the development of appropriate frameworks to understand the key factors for ESD learning performance and to develop tools/indicators to effectively monitor and evaluate the implementation of ESD in the Asia-Pacific region. The research was designed as a collaborative project and included the participation of government officers, practitioners, and researchers from seven countries in East and Southeast Asia. The structure of the paper is as follows: (1) An introduction; (2) a summary of the three components of the M&E of ESD research project, including the methodology and results; (3) a case analysis of the processes involved in the third research track to establish and facilitate a collaborative research-policy platform with an emphasis on the mutual learning and knowledge co-generation approach [22]; (4) discussions outlining the challenges that could be addressed at the research-policy interface through collaborative research partnerships; and (5) the conclusion.

2. Methods: Summary of the M&E of ESD Research Project

This section presents an overview of the methodology applied and results obtained from the three separate components of a research project to develop a framework for monitoring and evaluation of Education for Sustainable Development. As the main focus of this paper is to explore how the collaborative, multi-stakeholder partnership, which was one of the three main components, benefitted the overall project, this overview will only briefly introduce the full structure of the research project to provide context for the reflective case presented in the next section. This section is not meant to provide an exhaustive description of the research project; hence, the reader is referred to the original reports [7,23,24] for full project details.

Table 1. Research structure and main contributions by project partners. ESD, education for sustainable development. RCE, Regional Centers of Expertise.

Roles of Core Research Team	Establish Research Framework and Parameters	Identify Research Partners	Prepare Reporting Formats	Prepare Facilitation Methods for Consultation	Compile Data from Tracks 1 and 2, Conduct Comparative Analysis, & Prepare Reports
Collaboration Work with Partners	Research Track 1: Status of National ESD Implementation	National ESD Focal points from 7 countries lead reporting with relevant agencies	Focal points complete ESD Country Status Survey & present results		Report 1: ESD Country Status Reports and evaluation of national implementation
	Research Track 2: ESD Practice Cases and Factors of Learning Performance	Focal points from 10 selected RCEs identify and collect information of good practice case	Focal points complete ESD Case Reporting Framework & present results		Report 2: RCE good practice cases and assessment of learning performance in ESD
	Research Track 3: Collaborative consultations on M&E of ESD	An additional 31 experts are invited to participate in consultations with other collaborators	Conduct collaborative review of findings and test of proposals	Review and consultation of preliminary findings by all partners	Report 3: M&E of ESD framework of main factors and important leverage points

shows the research structure of the project and the main contributions by project partners. The project was conducted as a cooperative project led by the Institute for Global Environmental Strategies, the United Nations University—Institute of Advanced Studies, and the UNESCO Asia Pacific Regional Bureau of Education and included participants from seven Asian countries: Cambodia, China, Japan, Malaysia, Philippines, Republic of Korea and Thailand. The overall goal of the research project was to contribute to M&E of ESD through the development of a monitoring framework and ESD indicators for piloting in the Asia region. Specific objectives of the project were as follows: (1) Development of national ESD status reports through data collection and analysis to identify leverage points, success factors and barriers to ESD implementation; (2) collection of case reports on ESD good practice and exploring the linkages between theory and practice in order to develop a framework for mapping ESD learning performance; (3) development of an ESD monitoring and evaluation framework; and (4) drafting of pilot ESD indicators for future application and assessment. The target population of this research was ESD project implementers, policymakers, curriculum developers, teachers, and school administrators. “Policymakers” generally refers to those who are active in making policies and policy decisions, and can include both politicians and government officers. In the research project presented here, this group was represented exclusively by government officers who held active expertise in the area of ESD and were responsible for the evaluation and reporting of its practice.

Regarding methodology, the study involved a mixed-methods design using methodological triangulation and stakeholder review techniques at multiple stages of the research process to secure both consensus and pragmatic validation of findings. The overall methodology included three distinct, although complementary, research tracks. The first research track included multi-country scoping and comparative evaluation to identify the important factors and leverage points for both ESD implementation and practice. This supported the development of the ESD indicators assessment framework. The second research track was based on case studies of exemplar practices, and this allowed for the elaboration of key learning elements and characteristics of ESD learning performance. These two research tracks involved the use of both quantitative and qualitative approaches to draw on multiple knowledge types and various depths of information regarding ESD implementation. The third research track included three expert consultations and two reporting and capacity building workshops that were held over an 18-month period.

Specific methods used in the research are as follows. National ESD implementation was assessed through the collection of surveys completed by national ESD focal points (in collaboration with relevant government departments) in each country. These surveys provided the main quantitative (and quasi-quantitative) data, along with secondary supporting documents, to produce ESD country status reports for the seven countries. These reports aimed to: (1) Review the current implementation capacity of ESD at the national level; (2) identify existing strengths, weakness and gaps of ESD implementation; and (3) and provide recommendations for improving ESD implementation.

A comparative assessment of the country status reports was then conducted to review the common features of ESD implementation capacities (i.e., strengths, weaknesses and persistent barriers) and a capacity analysis approach was used to investigate key leverage points and components of effective ESD implementation. The basic categories of input, throughput and output capacities were used to structure the selective coding of data for this analysis. Additionally, three indicator types, i.e., status indicators, facilitative indicators, and effect indicators, were also considered [25]. The resultant capacity assessment framework provided the lens through which collaborating stakeholders conducted a deeper analysis of existing ESD policies and practices.

Under the second research track, “good practice cases” were collected from ten Regional Centers of Expertise (RCEs) on ESD located in seven countries via a common case reporting framework. These cases represented good practice in ESD in the sense that each Regional Centre of Expertise was selected due to a record of strong implementation, and they were each asked to self-select their project that had proven most successful or effective. Focal points for each regional center of expertise coordinated completion of case reports with the inputs from the members of their center. While such self-selection and self-reporting have an inherent-level of subjectivity (and potential positive bias), it was deemed an important complement to the quantitative and quasi-quantitative nature of the country status reports, and efforts were taken to encourage critically reflective narratives by requiring responses to eight investigatory questions. These rich qualitative narratives supported the examination of key learning features of ESD practice and led to the development of an ESD Learning Performance Framework [26]. For data analysis, two rounds of selective coding were applied. In the first round, factor analysis was applied to identify the key learning aspects and features from each case. During the second round, the learning features were reviewed from the perspectives of different educational theories and pedagogies to support theoretical sampling and grouping of these aspects into meta-categories (i.e., ESD elements). Utilizing these elements as an additional analytical lens, the specific details and aspects of each element were further reviewed and detailed towards creating causal explanations through analytical induction.

The third research track of the M&E of ESD study brought together ESD experts to provide regular review and consultation during the course of the research. This track was able to actively build on the findings from the first two tracks, and it especially worked towards the refinement of a draft M&E of the ESD framework and elaboration of regionally-relevant ESD indicators for future piloting in the region. Outputs from the other two research tracks were also made available for consultation. This allowed participating experts and practitioners to review findings from the research, to consider initial proposals made by the research team for an M&E framework, to test the practical applicability of these proposals, and to provide recommendations for the development of regionally-relevant ESD indicators.

Major Outputs and Results Obtained from the Research

Three main reports were prepared during the course of this research project, as well as an executive summary of the research and a policy brief.

• ESD Status Reports: The seven country status reports present the status of ESD implementation and practice across these countries. Several notable similarities in ESD implementation were observed, and a number of unique features/innovations from individual countries were also identified. A comparative assessment and strategic capacity analysis of the process and system of ESD implementation in these countries was carried out, and this supported the identification of necessary components of effective ESD implementation, the major success factors across the countries, and the persistent barriers to good implementation. Thirty-two factors for effective ESD implementation (categorized as input, throughput and output capacities) were identified across six different sectors: National curriculum, formal education, teacher training, non-formal education, community and civil society, and private sector [7].
• ESD good practice cases and the Learning Performance Framework: Ten exemplar practice cases were collected and documented in the second research report [23]. This report also elaborates an ESD Learning Performance Framework [26] which provides a basis for developing measurable, qualitative learning targets and ultimately indicators to assess educational progress and monitor educational performance at local (classroom) and national levels. Other potential capabilities include informing educational decision-making, aiding curriculum design, course content and teaching pedagogies and facilitating safe and appropriate learning environments.
• Refining the M&E of ESD Framework and elaborating ESD indicators: The M&E of ESD framework initially proposed by the research team became the focus of scrutiny in the last expert consultation with the main goal of elaborating regionally relevant indicators of ESD for future piloting in the region. Originally starting with a research framework that included 55 target areas and 75 questions, the capacity analysis led to the identification of 32 key implementation factors. This final consultation and inputs from diverse stakeholders then led to the elaboration of a three-tiered M&E framework with different actors targeted as the foci/respondents for each tier, i.e., education policymakers, educators and practitioners, and learners/beneficiaries, to provide a more systematic reporting across all levels of ESD implementation and practice.

3. Results: Creating a Collaborative Research Platform for Multi-Stakeholder Knowledge Production

This section addresses the key objective of the paper by exploring how the collaborative process and multi-stakeholder partnership used in this research project facilitated interaction between researchers, policymakers and practitioners; enriched the outcomes of the research; and led to a more effective interface between the research and policy-decision making. While it forms the core of the paper, its arguments and discussions are based on the experiences and research work summarized in the previous section and the collaborative partnership pursued in research track 3 would not have been possible without the work conducted in tracks 1 and 2. In the initial formulation of the research plan only tracks one and two were considered as “real” sources of research data, and the purpose of track three was originally to conduct review, outreach and dissemination. However, both research tracks one and two depended on the cooperation of focal points to complete the ESD country status surveys and the case reporting framework, and as we prepared for the reporting workshops, it became evident that we had the opportunity to utilize the diversity of stakeholders to provide a collective review of the research findings and to ensure the relevance of the project’s recommendations. Due to this, it was in research track three that the primary goal was to develop a multi-stakeholder partnership to achieve methodological triangulation within the project and to enhance both consensus and pragmatic validation of research findings. This paper provides an assessment of the process used during this third research track in order to understand what methods were effective in facilitating this partnership and how collaborative partnerships between researchers, policymakers and practitioners can be used to improve the research-policy interface in the field of sustainability education. This assessment of the benefits provided by the collaborative partnership as part this research process has been conducted only after the completion of the research project, and this is based primarily on a critical reflection of the experiences from this process.

Research track three and the development of a collaborative partnership at the research-policy interface rested on three expert consultations and two reporting and capacity building workshops that were held over the 18-month research phase. An average of 21 ESD experts from international agencies, universities, research institutes, national governments, NGOs, and project implementers in the regional centers of expertise was involved in each event and around 65 persons in total directly contributed (in various manners) to this project. Participatory activities at these events were facilitated through a variety of approaches and methods: (1) Presentations on individual countries’ ESD status; (2) participatory mapping of ESD implementation capacities and review of the ESD capacity assessment framework; (3) SWOT analysis of ESD implementation capacities; (4) presentations of ESD good practice cases; (5) comparative review of ESD learning characteristics and proposed elements, and applied testing of the ESD Learning Performance Framework; (6) collaborative assessment of M&E of ESD priorities and strategic evaluation of the M&E of the ESD framework; (7) structured group discussions on the facilitative tools and policy options for advancing ESD practice; (8) group review of possible indicators and SMART (i.e., specific, measurable, achievable, relevant, and time bound) criteria assessment of feasibility for each indicator; and (9) evaluative activities for participants utilizing the Adaptive Nominal Group Technique.

Collectively, these efforts also facilitated the establishment of a platform for partnership and networking with stakeholders actively exchanging knowledge and experience, as well as working together to harmonize understandings. The collaborative partnership enabled an interactive, learning process in which the practical applicability of ideas and proposals were thoroughly debated and tested. This process of multi-stakeholder investigation was highlighted by the following key aspects:

• Active, multi-stakeholder participation;
• Cooperative inquiry;
• Open dialogue and deliberation; and
• Knowledge co-production and knowledge transfer/acquisition.

The following discussion will focus on the learning processes, their features and examples of how they were utilized in the research activities to achieve the requisite objective.

3.1. Active, Multi-Stakeholder Participation

Active, multi-stakeholder participation occurred throughout the research phase, and the composition of the participants for the organized meetings was crucial to the delivery of the tools and frameworks for which the research was designed. Participation in the research process by policymakers, academics/researchers and practitioners enabled them to clarify their own practical observations of the system, and it subsequently enhanced the pragmatic validation of the research findings [27]. The diversity of expertise and experiences with sustainability education in policy and practice enriched the overall human capacity available during the group interactions. Thus, the participation of experts from international agencies, universities, research institutions, education ministries/departments, NGOs, and the regional centers of expertise in a collaborative effort enriched the research process and its efficacy both through empirical and conceptual influence.

As a fundamental component of human nature which is connected to transformation [28], participation also engendered commonality and furthered networking (Figure 1). By engaging stakeholders in participatory analysis and assessment activities, they were able to provide a more “holistic” testing of the relevance of research findings and proposals by considering their practicality and applicability across different levels of the education systems within numerous countries. This form of participation inherently required a level of compromise from all actors as the proposals for M&E of the ESD framework sought a system that would be functional and valuable both for the policy and practice of ESD.

3.2. Cooperative Inquiry

Cooperative inquiry was supported through collectively engaging participants in reflective processes to review the current status and identify opportunities for improvement. By altering between active and reflective practice to help participants look at experiences from different angles, develop new ideas and test different ways of implementation, this process of cooperative inquiry enhanced the robustness and depth of generated knowledge. Facilitation of genuine collaboration between the members of a cooperative inquiry group can be a challenge, and efforts were taken to secure equal opportunities for sharing and leadership, critical reflection and research cycling, and group problem-solving through addressing implementation barriers and challenges. These served as key assets as they engendered a democratic debate among participants on the framing and definition of the issues at stake [29]. Both order and chaos were also embraced as necessary components of this collaborative discovery process, due to the benefits gained from undergoing periods of confusion and uncertainty [30].

The multi-stakeholder cooperative inquiry introduced ideas from different experts for consideration, created opportunities for collective testing and application of findings and recommendations gathered during the research, and ultimately strengthened the research, due to aspects, including enhanced communication and decision-making, as well as greater development of trust (Figure 1). For example, through the presentations, participants were enlightened on existing education systems and curriculum contents in various countries. This often led to further discussions on the policy, research and practice implications for ESD.

3.3. Dialogue and Deliberation

Dialogue and deliberation constitute the creation of opportunities for reflexive moments and discourse. Positive interdependence and trust-building became exemplar features of this process occurring at the research-policy interface. This component of the collaborative learning process proved important for deepening interactions among participants during the linking of research to policymaking. Thus, the mutual engagement of stakeholders in dialogue during the capacity building workshops, consultations, and activities served to boost their ESD competencies (Figure 1). For example, at first, the project researchers were challenged to provide clarity and simplicity in communicating the results to participants, failure of which could have led to obvious obstacles, including eventual loss of interest in the process. Participants, in turn, were expected to pay critical attention to the entire process with a sense of inquiry and curiosity for understanding, as well as to contribute from their own perspective and experience. However, collaborative research through dialogue and deliberation may also lead to productive disagreements that encourage continued inquiry and discussion instead of consensus [27], and this was particularly observed during the session on refining the M&E on ESD framework and elaborating the ESD indicators using the Adaptive Nominal Group Technique.

Additionally, policymakers appreciated the need for some level of rigor and validity in the research process and the development of specific proposals, instead of the usual “short cuts” and generalization of results that occur regularly, especially in the form of policy briefs. Hence, the deliberative framework of the research established a common format in which participants could present and discuss different features of ESD implementation and practice from various countries in a relatable and comparable manner. For example, there were discussions on how to put mechanisms in place so that multi-level (local classroom, subnational and national) M&E of ESD efforts will be streamlined to address the inherent bias usually found in self-reporting.

3.4. Knowledge Co-Production and Transfer

Knowledge co-production and transfer were supported by allowing for a strong interplay between the research findings, group application and deliberation, and the opportunity for the participants to evaluate the entire research process. The input and feedback from the participatory process were used to further develop, redirect and fine-tune the research as an evolving process. Furthermore, group processing helped to manage knowledge systems and make sense of information.

This type of group learning was observed across most of the research activities. Group meetings led to more concise defining of objectives for M&E of ESD and the development of a robust evaluation framework consisting of selected criteria for systematic M&E of ESD. During the final expert consultation, participants conducted an in-depth review of research findings and recommendations. Through a facilitated process, participants clarified the objectives and target audience of a regional M&E of the ESD process. Group deliberations also led to a significant restructuring of the final M&E of the ESD framework through the inclusion of target actors and audience as a key organizing dimension. Finally, using this new framework, the participants were able to use the list of previously tested indicators and refine the final list to a set of 32 indicators that could be practically measured and had relevance to decision-making across various levels of ESD policy and practice.

The collective learning process, seen to operate at the research-policy interface, showed several notable features (as indicated in Figure 1), including reflexivity, permanence and mutuality. Additionally, there were learning-related changes that occurred during the research process that is attributable to the collaborative learning process. The commonality between these changes was found in aspects, such as the enrichment and deepening of interactions among participants, and the validity and efficacy of research findings. Additionally, enhanced communication and increased collaboration between researchers and policymakers, improved relationships, and trust-building comprise some of the important factors increasing the appropriateness and applicability of the research findings as evidence to inform policymaking. The collaborative partnership also led to a deeper common understanding about ESD and agreement about what can be done to improve its performance, and for the actors involved it also led to increased ESD knowledge, as well as enhanced competencies and skills to support its implementation.

3.5. Reflection on the Benefits of the Collaborative Research Partnership

The workshops and consultations of the research process served to create an interface of interactions between researchers, policymakers, and practitioners for problem-solving and knowledge co-production. These meetings provided the opportunity for initial screening of the research findings and for participants to evaluate the research process, while the feedback received was applied to subsequent research cycles to achieve the desired performance and result. New ideas were generated from different stakeholders that increased the robustness and applicability of research, and opportunities to question/challenge the concepts and thinking used by project implementers were provided. Thus, the workshops helped the core research team and the other participants reach a common understanding. In addition, they provided a forum to discuss, apply and test the proposals and recommendations that were being generated during the course of the research process.

The collaborative research process was designed with the idea to provide benefits both for the research itself and also for the participants in the process, and the workshops and consultation also aimed to build the capacity of participants to understand and work with M&E of ESD. Capacity building was required to encourage greater participation in the discussions on evidence-based policymaking at the research-policy interface [31]. In this light, the monitoring and evaluation process can itself become a learning tool in which actors reflect on current practices and concepts, and they can undertake necessary measures to reorient efforts towards more effective practices when required.

The critical and pragmatic assessment in the final consultation led to a modification of the final M&E framework, while the feedback obtained after capacity-building activities provided the researchers with practical insights into the applicability of their preliminary findings and recommendations. Moreover, the timely availability of the research evidence, which includes the compilation of the existing capacity and good practices in the Asia region contributed to the provision of evidence applicable for use in policymaking. Figure 1 lists some of the positive impacts obtained from the research process.

4. Discussion: Challenges Addressed by Stakeholder Engagement at the Research-Policy Interface

Research and the production of “knowledge” itself are argued to be political practices, and as such research has as much part to play in “making” reality as in reflecting the world around it [32]. Bacchi and Goodwin [32] argue that, “such a stance challenges policy workers cum analysts to reflect critically on the ‘methods’ they use and the categories of analysis they adopt”. In dealing with the science-policy interface, it is important to acknowledge that there is a bi-directional influence, and the boundary between these two spheres remains dynamic and fluid. Thus, the aim of knowledge co-production taking place at this interface must not be to eliminate these influences, but rather to embrace the contextual nature of influence both spheres hold on one another, and through this, the desired results can be “policy-relevant science and science-based policy” [17].

Policymaking, especially in relation to emergent fields of policy, may be understood as a process of social learning [33]. While there are growing calls for efforts to bridge the science-policy interface to address complex socio-ecological challenges related to sustainable development, there is unfortunately limited actual debate on how to achieve this [18]. Weichselgartner and Kasperson propose to, “consider the collaborative production of knowledge as a systematic and emergent inquiry process, embedded in a collaborative partnership between scientists, policy makers, and practitioners for the purpose of generating actionable scientific knowledge” [18].

In the field of education research, the research is inevitably connected directly to both policy and practice, and cannot be separated from these contextual factors. The relevance of education research is also closely linked to how it informs policy and/or practice, but Bridges highlights that the issue of relevance is a challenging concept because what is relevant depends on the target audience, place and time of application [34]. Furthermore, existing research also shows that academic research is low on the list of sources that policymakers use to inform their decision-making, and that they more regularly rely on information from commissions, trusted experts and think-tanks for ideas and evidence [34]. Based on the experience of facilitating a collaborative partnership for education research in this study though, we would propose that involvement of diverse stakeholders in “checking” findings and recommendations improves relevance in a universal manner, aids in aligning priorities for policy and practice, and encourages greater ownership of the findings and recommendations from all involved actors.

Collaboration among researchers, academics, policymakers and practitioners at the research-policy interface of the M&E of ESD research process, however, is not without challenges (see References [16,35,36,37,38] for previous discussions). Weichselgartner and Kasperson identify three general groups of factors that hinder collaborative knowledge production: Functional factors, due to differing needs, objectives, and perspective of actors; social factors that include cultural values and diverse professional forms of practice and communication; and structural factors like institutional settings and standards that can make collaboration difficult or impossible [18]. In the experience of the case presented here, the collaborative partnership did have a positive impact on the functional and social factors that can limit multi-stakeholder engagement at the research-policy interface. Through critically reflective assessments and discussions on the practice of ESD and its monitoring and evaluation, the diverse stakeholders were able to coalesce their objectives and priorities into a more complementary arrangement. Through active participation and knowledge exchange, the actors developed appropriate means to clearly work together and communicate on these matters. While the case presented here did not have a lasting impact on structural factors, it did at least temporarily provide an alternative structure where active collaboration was possible among a diverse group of actors.

In an effort to establish a collaborative research-policy platform for multi-stakeholder participation in M&E for ESD research to support evidence-based policy formation, some potential challenges and possible opportunities to address them are briefly discussed below.

4.1. The Education 2030 Agenda Review and Evaluation Processes

The 2030 Agenda and the Sustainable Development Goals (SDGs) call for follow-up and review processes to draw on both country-led evaluations and quality data. Consequently, governments generally have the primary responsibility for education implementation, which includes putting in place mechanisms for monitoring, accountability and incorporating this into their respective policy and strategies at the national level [11]. However, at the research-policy interface, researchers, policymakers and practitioners should coordinate and use a multi-level approach to address this using ‘corresponding’ research evidence to ensure the applicability and effectiveness of education policy in practice. Clear indicators and targets that are specific and can be measured at the beginning of each project should be established. To be able to do this “Appropriate tools for data collection need to be developed, and the M&E process needs a defined framework for the scope and pace of the work” [35]. Furthermore, ways of translating evidence-based education policies into actual ESD practices on the ground should be pursued, monitored and reported.

4.2. Finding Ways to Conduct M&E and Presenting the Results More Efficiently and Effectively

Deliberation on (a) how to effectively conduct and present M&E results; (b) how to use it to produce a systematic review of ESD implementation for reforms of the curriculum and pedagogy; and (c) how to identify key lessons for later integration and mainstreaming is an important part of the collaborative partnership that helps to strengthen both the usability and relevance of the M&E process. Moreover, including M&E as part of discussions at the research-policy interface to address specific sustainability education targets is important. For example, the provision of safe and effective environments for all is one of the most important targets for the successful implementation of education reforms. To be able to effectively monitor and evaluate this target, policymakers and researchers should make significant inputs during the design and planning stages of its implementation. Furthermore, appropriate science and technology expertise will be crucial to determine architectural and security inputs to achieve socially and physically safe and effective environments for education.

4.3. Need for Policymakers and Researchers to View M&E of ESD Problems with ‘Similar’ Perspectives

Researchers, policymakers and practitioners naturally identify different needs and challenges for M&E of ESD, and therefore, perceive different approaches for this process, but a collaborative partnership in developing (and implementing) an M&E of the ESD framework provides opportunities to appreciate the needs of diverse stakeholders. Contrary to policymakers who have to factor in “entire populations” in designing and implementing their M&E policies, researchers often measure their work against set standards which may be considered as abstract, often limited to “sample size”, and hence, not realistic in real world situations. Bridging this divide can begin at the interface through open and frank dialogue that supports the appreciation of each other’s work and collaboration towards a common objective of implementing quality education.

Despite the complexity underlying M&E, there often remains a preference in decision-making for short and concise statements of findings which can lead to over-simplification and generalization of such complex phenomenon. Additionally, there is a tendency towards making decisions based more on values, beliefs, or interests than specific research findings [39]. For these reasons, complex research results are expected to be packaged in concise recommendations (while simultaneously maintaining the substance of their content). Thus, the use of policy briefs is one-way research can achieve this. However, presenting results from extensive research in a language free of academic jargon can present a substantial challenge for the researcher towards achieving useful dissemination of findings. Enabling pathways for dialogue between policymakers, researchers and other stakeholders, while simultaneously ensuring procedures are more focused on evidence [35] contribute to lowering some of the barriers of dialogue tied to professional jargon and prioritization.

4.4. Consideration of Place and Form of Arrangement for a Collaborative Research Platform

The place and form of arrangement for a collaborative research platform also needs careful consideration. Johansson et al. [40] posit that it is more appropriate when meeting places are arranged in neutral or less intimidating places, especially outside the academy and at a location hosted by the ‘solution beneficiary’, e.g., a municipality, but run by an academic institution to guarantee open exchanges. Roundtables are increasingly seen as a popular interactive and on-going relationship between policymakers and researchers in knowledge co-production and acquisition [41]. Consultations combined with capacity building, as used in this research, provide another popular approach. Additionally, measuring the impacts of the collaboration between the stakeholders at these interfaces would be beneficial, but will also require another set of indicators and different tools for monitoring and tracking progress.

4.5. Communicating ‘Place-based’ Research Results to Support Decision-making

Even though active participation and contribution to policy decision-making are important, researchers’ ability to communicate research results of policy importance to support evidence-based decision-making and in a manner that is informative while not being leading is currently inadequate. However, since the launch of the Education 2030 and SDG 4 on education, a significant increase in meetings between researchers, policymakers and practitioners at conferences and policy processes to identify appropriate approaches to implementing as well as monitoring and evaluating education has been noted. It is, therefore, imperative that the development of M&E systems and tools that are tailored for regions, countries and local areas are discussed at corresponding research-policy interfaces. It is also necessary that researchers adapt their messages to what the people actually need, hence seeking inputs from practitioners and policymakers is critical.

5. Conclusions

Collaborative research partnerships can support the effectiveness and quality of ESD, provide an important policy tool for its practice, and bridge the research-policy interface. Multi-stakeholder participation in research, fact-finding and pragmatic testing are also essential for sustainability-related education initiatives, including SDG 4 (and across all of the Sustainable Development Goals), and this can be deepened through partnerships for M&E which benefit from the inherent iterative cycle of regular review and improvement of implementation.

Implementing collaborative partnerships for M&E of ESD necessitates a colossal undertaking to identify appropriate indicators; collect, manage and evaluate essential data; and ensure timely assessment so adverse results may be quickly resolved. However, its achievement will largely depend on the involvement of relevant stakeholders comprising of policymakers, education-related government officers, curriculum developers, and education practitioners in efforts to develop appropriate interfaces to achieve the needed knowledge and resource co-production to advance policy and practice of sustainability education.

This study set out to present a reflective case study of the methodology and techniques used to achieve a collaborative research process on M&E of ESD conducted in the Asia region. This case aims to demonstrate the dynamics of interactions between researchers, policymakers and practitioners regarding how they can influence each other at the research-policy interface, and therefore, enrich the outcomes of the research and support evidence-based policy formulation. Participants collaborated through a series of interactive sessions and discussions that covered (a) review of the findings from the ESD country status reports; (b) examination of ESD good practice cases; (c) development of an ESD learning performance framework; and (d) refining the M&E of the ESD framework. In addition to identifying and elaborating the ESD target indicators for piloting in the region, the participants considered additional ways in which their knowledge collaboration could contribute to meeting the key objectives of the research.

The utilization of a collaborative partnership at the research-policy interface had positive implications for the development of the M&E framework. Compared to an alternative where the core research team had only conducted the research with little interaction or participation from key stakeholders, important aspects of local contextualization, culture appropriateness, and comparative effectiveness would have been largely overlooked. The reference and evaluative tools considered in the research were significantly improved, due to the participation and insights of a diverse stakeholder group. The enhanced results from this collaboration fed into the development of a regional ESD policy process document and were well cited in the final report of the UN Decade on ESD [15].

Efforts to achieve the Sustainable Development Goals can be substantially strengthened through evidence-based policymaking which is dependent on robust and dynamic M&E systems, but even more important and recognized as its own goal in SDG 17 is the need for partnerships to achieve effective implementation for the goals. This depends on policies, plans and practices whose components are the outcome of collaborative interaction between diverse stakeholders through a combination of governance principles, scientific evidence and persuasion at the research-policy interface. The case presented here highlights how a facilitative platform to support the collaborative investigation and mutual exchange among diverse stakeholders can provide a valuable approach towards developing partnerships that can transcend the research-policy interface and transform it into the social learning process for sustainable development that is both reflective and active in nature.