3.1. Categories of Objectives
Six categories of objectives were distilled from the Grounded Theory Analysis. These categories comprise all objectives mentioned in the interviews and in the selected GEA documents. The objectives mentioned in interviews represent the perspectives of different stakeholders, while the GEA documents represent the official objectives. Given their diverse backgrounds, the interviewees described particular objectives in very different ways. For example, different interviewees discussed their own personal objectives or motivations for engaging with a GEA process, their personal interpretations of the official objectives or the objectives of others, or the perspective of the institution, organization or government they were representing. Moreover, in both interviews and GEA documents, objectives were discussed somewhere along a spectrum between being targeted to a specific meeting and being overarching objectives transcending the stakeholder engagement activities to confer benefits on the GEA process as a whole or even on society more broadly. This often coincided with the extent to which an objective was described in a pragmatic sense, using practical and precise wording and reflecting directly on outcomes, as opposed to a more conceptual or theoretical description. These characteristics are used to describe the Results below and to highlight the different viewpoints within categories of objectives.
The category of objectives mentioned most often was the importance of providing a source of information (discussed by 86 interviewees and found in both GEA documents, for a total of 88 mentions). The second most often mentioned category was fostering a dialogue (76). These are followed in frequency by the category of objectives to improve communication and understanding (71), and the category to create a sense of ownership (67) respectively. Lastly, the categories of objectives to exert control over the process (52) and to facilitate learning (47) were mentioned the least often by interviewees and in GEA documents. These six categories are presented in the following subsections (Section 3.1.1
, Section 3.1.2
, Section 3.1.3
, Section 3.1.4
, Section 3.1.5
and Section 3.1.6
, respectively), each of which includes specific examples to highlight the diversity of perspectives on objectives which underlie the categories. For each subsection, a brief overview of selected literature is included in order to bring in pertinent ideas which complement or go beyond the empirical material.
The extent to which each category of objectives was mentioned in interviews or in GEA documents in conjunction with a specific method of stakeholder engagement is shown in Figure 1
, where the four lines represent the four methods detailed in Table 1
. Figure 1
shows the absolute number of sources (interviews or GEA documents) which mentioned a method and category of objectives together at least once.
3.1.1. Source of Information
The most prominently mentioned category of objectives in interviews and in GEA documents alike was that stakeholders should be engaged with in order to provide a source of information to the assessment. There were two overarching ways of describing this category of objectives in interviews, which were mentioned evenly across all groups and which were sometimes both mentioned by one interviewee. The first and most commonly mentioned standpoint is pragmatic in the sense that it refers to actually writing the report, emphasizing primarily the role of stakeholders as authors and other scientific experts. One government representative involved in the IPCC stressed the need to include information from “a mix of disciplines involving social science, natural science, economics, technology, and others” to make the findings more policy-relevant. A scientific expert who participated in GEO-5 stated that “you just can’t do the assessments without access to stakeholders’ information”. One target audience member not involved in either assessment also took this perspective, explaining that you need authors with “different science and research background to present for understanding, for discussion, different interpretations of science findings”.
The other standpoint mentioned prominently within this category of objectives is that GEAs should aim to diversify the sources of information to the process beyond scientific information alone. Two interviewees involved in GEO-5 scientific expert panels described the need to engage stakeholders from different “cultural backgrounds”, and representatives of “private sector, civil society, [and] holders of traditional knowledge” for example. A government representative involved in IPCC WGIII AR5 stressed that assessments should explore a “plurality of perspectives of the problems”. A target audience member eloquently described the need to move “beyond a view that the only kind of expertise which is valid to bring in to assessments is from formal science and accredited peer-review publication towards opening up to a wider range of stakeholders and a wider range of forms of knowledge […] to contribute to the kinds of solutions that might emerge”. One interviewee with experience as an author in both GEO and IPCC assessments explained the need for diversifying sources of information “as a validation of whether solutions are actually feasible”, important in light of the shift towards solution-orientation. However, one interviewee involved in IPCC WGIII AR5 warned that “for any assessment [it would be] very, very risky to include people when in the end you rely on information that are not peer reviewed”, suggesting that “if you want to have broader stakeholder involvement then you have to set up a research project”.
In GEA documents, this category of objectives was linked directly to all four of the engagement activities described in Table 1
, and in particular focused on diversifying the sources of information. For example, the IPCC WGIII AR5 selected authors and participants for the scoping meeting based on “the following criteria: Scientific, technical and socio-economic expertise, including the range of views; geographical representation; a mixture of experts with and without previous experience in IPCC; gender balance; experts with a background from relevant stakeholder and user groups, including governments” [25
]. In GEO-5 regional consultations, an official objective was to engage “geographically representative and gender balanced regional groups” [24
In the literature, this category of objectives has also often been discussed with regards to diversifying the sources of information to an assessment. Similarly to what was found in GEA documents, this includes the engagement of authors from diverse disciplines, institutional affiliations, geographic locations, and with a variety of areas of topical expertise [9
]. However, stakeholders can also provide information in other roles stemming from a broader diversity of worldviews, beliefs and knowledge systems [7
]. For example, non-scientific stakeholders could contribute information based on cultural beliefs or experiences to the problem framing and to analyzing the potential success or failure of solution options, including co-benefits and side effects [47
The category of objectives to foster a dialogue was common across interviews and was the second most commonly mentioned category in the material analyzed overall, but was actually not mentioned explicitly in GEA documents. Within this category, interviewees focused on two points in particular. Firstly, some interviewees characterized this category of objectives in an overarching sense, as an objective which applies to stakeholder engagement over the whole course of the GEA process and has benefits transcending the process itself. This standpoint was mainly described by government representatives and target audience members. For example, one target audience member stated bluntly that “we lose if we do not discuss, we win if we discuss”, while another advocated taking a “more deliberative approach which explicitly acknowledges that there are alternative views on what the problem is and why it matters”. Some interviewees focusing on the overarching nature of this category of objectives described dialogue as leading to a broader, desirable outcome. For example, a government representative involved in IPCC WGIII AR5 described how dialogue processes within a GEA could help “bring new actors into the broader discussion, reset priorities, re-frame the agenda”, while a government representative from GEO-5 argued this could help “deal with lack of trust from some countries”. Another government representative for IPCC WGIII AR5 described how regional-scale dialogues in particular could “build up a regional common understanding on issues, and then move forward to a global process” while a target audience member explained that “when you have the chance to discuss […] this interaction is a positive thing towards collaboration at the regional level”.
A second standpoint within this category of objectives had to do with describing dialogue as an objective with more practical implications for reaching the targeted goals of a specific meeting or engagement activity. This perspective came up primarily during interviews with authors and other involved scientific experts. For example, a GEO-5 author saw a high value in promoting “interdisciplinary discussions to clarify the scientific findings and methodology” in regional consultations. Regarding the SPM negotiations, an expert involved in the publication of GEO-5 felt that striving to foster dialogue would help to overcome “misunderstandings between policy makers and scientists”. An author involved in the IPCC WGIII AR5 explained that in some cases which come up during the SPM negotiations “there is no scientific way of saying that this way of representing data to carry this political message is better than that way—you can say it’s marginally better but you can’t say it’s right or wrong”. In these situations, this interviewee argued, a broader dialogue must be fostered since science alone cannot justify the choice of presentation.
The main emphasis in the literature when addressing dialogues at the science–policy interface focuses on an open, back-and-forth flow of information between different actors [49
]. This can often be observed in practice in GEAs [10
], even though the official GEA documents do not explicitly state that fostering a dialogue is an objective. Similar to interviewees, some researchers have argued that dialogue can build trust and mutual understanding, and clarify representations of problems, impacts and potential solutions [9
]. Others have emphasized in particular the importance of fostering dialogue in situations not necessarily requiring consensus as an end point [31
]. Fostering dialogue, and ensuring that stakeholders feel they are not only listened to but have a more meaningful role in an ongoing discussion, can contribute to strengthening the legitimacy and salience of the assessment report, important criteria for success in GEAs [2
3.1.3. Communication and Understanding
The third category of objectives for engaging with stakeholders in GEAs, mentioned most often by government representatives, is to improve the communication and understanding of the main messages and findings of the assessment. There were three general ways of describing this objective in the interview material, all of which have a relatively pragmatic basis. Firstly, and most commonly, this category of objectives was described with regards to how different findings were worded and presented. From this standpoint, the pragmatic end goal is to ensure that information shared with target audiences is clear and comprehensible to them. One IPCC WGIII AR5 author described “the act of translation to a policy context” as a crucial objective of the SPM negotiation, further explaining that “policy makers can help shape that language in a way that communicates the ideas better”. A GEO-5 author stated that “we do not all speak the same language, we need to translate the researchers language into the stakeholders language”. In an unrecorded interview, a government representative involved in IPCC WGIII AR5 explained that some of the technical language often used by scientists in the GEA process can be difficult to understand for many policy makers, in particular in less developed countries. However, some interviewees felt that too much emphasis on communication could be detrimental to scientific credibility, in particular when negotiating the SPM document. For example, many interviewees from all groups felt that the findings in the SPM were “watered down” as a result of the negotiation process. However, as one GEO-5 author pointed out, this is a “trade-off—by going through with that [negotiation] the report gets more visibility, but some information gets lost”.
A second line of thinking focuses on the types of outputs produced. From this standpoint, a major aspect of engaging with stakeholders with the objective of improving communication and understanding is to determine which types of products are most appropriate and useful for different audiences. Here, the practical goal is to produce these appropriate and useful products in order to increase the likelihood that they will actually be used. This perspective was mentioned most prominently by authors. For example, one GEO author suggested that “if they want to have an impact on teaching, they should provide PowerPoint slides”, which could be best designed together with experienced teachers to be used more easily in classrooms. An IPCC WGIII AR5 author described how the GEA should tailor-make “presentations in some developing countries, so that policy makers can understand better the issues”.
Finally, a third line of thinking has to do with the strategic dissemination of GEAs’ findings in order to exert an influence on global governance more broadly. This was most often mentioned by government representatives with regards to engaging with stakeholders to secure a place for the GEA in milestone global environmental governance events. For example, one government representative in GEO-5 described the role of the assessment with regards to the United Nations Conference on Sustainable Development (Rio+20), stating that “the more [GEO-5] findings are shared the more a wide spectrum of the stakeholders gain the support for consequential actions in the right directions to reach agreements at the international stage”. An unrecorded interview with an IPCC government representative involved since the first assessment report stressed the connection between the IPCC reports and the United Nations Framework Convention on Climate Change (UNFCCC) Conference of the Party meetings, a major target audience for the IPCC and a means of exerting influence.
In GEA documents, this category of objectives was mentioned most prominently with regards to the SPM negotiations. In GEO-5 this was done quite directly. For example, an important objective of the GEO-5 SPM was to effectively “communicate the findings of the GEO-5 assessment and maximize accessibility of GEO-5 information [to target audiences], including in terms of format and languages” [22
] (p. 8). This is very much in line with the first standpoint focusing on the act of translating scientific findings. GEO-5 also strove to “make use of a range of multi-media and tools and innovative approaches” to reach a diverse target audience [22
] (p. 8), more in line with the second standpoint on the types of outputs envisaged. In IPCC WGIII AR5 this objective was mentioned more indirectly. For example, government representatives involved in the IPCC review process and SPM negotiation must provide “integrated comments on the accuracy and completeness of the scientific and/or technical content and the overall scientific and/or technical balance of the drafts” [23
] (p. 16). This would in part involve ensuring that the wording and framing are appropriate for the target audience (in this case, for governments).
In the literature, discussions about communication and understanding primarily relate to the efficient and effective transmission of information to target audiences. The literature stresses that while scientists involved in GEAs are experts in their fields, they are not necessarily experts in communication, requiring input from a broader group in order to improve understanding. This includes in particular ensuring that GEA messages fulfill a demand from end-users [5
] and are framed appropriately [42
]. Another crucial aspect is the role of stakeholders in interpreting information, making often complex and technical findings more usable [52
], a highly pragmatic rationale akin to those which came up during interviews regarding this category of objectives.
The fourth category of objectives is to build up a sense of ownership over the GEA process or its products amongst target audience members. While not mentioned in official GEA documents, interviewees who discussed ownership saw this category of objectives as very important. Interviewees often use the term ownership interchangeably with buy-in. Ownership was chosen as the heading for this category because, as some interviewees described it, ownership can be a means of encouraging stakeholders to buy into (or believe) a GEA process or products, whereas buy-in on its own does not necessarily require engagement.
There were two general ways of describing this category of objectives in the interview material, both mentioned primarily by authors and other scientific experts as well as by government representatives to some extent, and both referring primarily to building a sense of ownership amongst target audiences and in particular governmental actors. Firstly, interviewees talked about the objective to build up a sense of ownership as a unique way that GEAs in particular can increase impact. One government representative involved in IPCC WGIII AR5 described how “the degree of authority and ownership is something that other good reports, like World Bank or other similar global reports, don’t quite have”. An author involved in GEO-5 described how “people who are part of the process become ambassadors for the results, and that’s how the assessment would increase its impact”. One scientific expert involved in the production of IPCC WGIII AR5 stated that “government buy-in is the critical part of the whole process […] that’s what gives the whole thing weight and allows it to have credibility above and beyond normal scientific work”. Many interviewees linked this in particular to the SPM negotiation (see Figure 1
). One scientist with many years of experience working with GEO assessments stated directly that “by having these really negotiated versions, governments feel a sense of ownership […] and do something with the report”.
The second way of describing this category of objectives was a more pragmatic perspective regarding how exactly the objective of fostering ownership amongst different stakeholders might take place in a GEA. One author involved in GEO-5 felt that this was a central concern for future GEAs, asking “how do we download it to the country level how do we get buy-in from policy makers and politicians?” One government representative involved in IPCC WGIII AR5 felt that “the larger buy-in is there when all countries can participate”. A producer involved in GEO-5 explained how particular strategies can help in building ownership, but come with trade-offs: “focusing on policy successes probably did some good in achieving some buy-in from policy makers, but maybe the trade-off was that problems most in need of policy attention were downplayed or not featured that strongly because they didn’t have success stories in them”.
In the literature, building a sense of ownership is often linked to the GEA having a pragmatic impact or influence along the lines of the first perspective above, with the assumption being that stakeholders who are involved are more likely to feel ownership and in turn are more likely to actually use the end products [2
]. The most prominent example is the SPM negotiation, where the engagement of government representatives is thought to increase their sense of ownership and ultimately increase the influence of the final document on policy decisions [54
3.1.5. Exerting Control
The fifth category of objectives involves different actors exerting some form of control over the GEA process or its products. This category of objectives was never explicitly mentioned in official GEA documents, but still came up in interviews to some extent. Very few interviewees described themselves as holding this objective (rather discussing their observation that others exerted control), and all descriptions referred to a specific instance of a stakeholder or group exerting control.
By far the most common examples given by interviewees had to do with government stakeholders exerting control over the SPM document. Both scientific experts as well as government representatives themselves discussed this standpoint. For example, one scientific expert who participated in the IPCC WGIII AR5 SPM negotiation meeting felt that governments were “trying to skew the science in their favour”. A government representative involved in that same meeting noted that many other governments “were exerting a certain influence which was based on the political priorities of their countries”. A high-level scientific expert involved in GEO-5 explained that “political influence had too much influence on the Summary for Policy Makers”. One author wrote in a public letter after the IPCC WGIII AR5 SPM negotiation meeting that “nearly all delegates in the meeting demonstrated the same perspective and approach, namely that any text that was considered inconsistent with their interests and positions in multilateral negotiations was treated as unacceptable” [55
]. More broadly, one government representative involved in GEO-5 observed that there has been a push for “much more governmental control of all the sessions”, while a GEO-5 producer reflected that “you see governments wanting more and more control over the assessment process”, implying that government control may actually be increasing. As explained briefly in Section 3.1.3
above, government control is often seen as negative, for example when the final SPM becomes too “watered down” as a result of their involvement, sacrificing scientific credibility. Thus, increasing government control would not likely be welcomed by many expert scientists. However, many scientists do still see the value of relinquishing some control in particular in order to foster more ownership or buy-in. For example, one highly experienced individual who has participated as an author and producer in both GEO and IPCC assessments explained that while “the negotiation process eliminates a lot of the stuff which the scientists consider important, at least it’s some common point that has the buy-in from some policy makers”, also indicating a trade-off with the objective of building a sense of ownership.
In addition, there were a few cases where scientists exerted control. One author involved in GEO-5 described how they observed cases where “authors seemed to want to push what went into these final chapters as a function of anticipated funding needs”. Another expert with a long history of engaging with GEAs including all GEO assessments recalled numerous cases where “people [are] coming in with their pet projects and favourite subjects”, insisting these are included in the assessment. However, at least one scientific expert who has been involved in many GEA processes made the case that scientific control did not have to be a negative thing. This interviewee stated that “a lot of time you really have to sneak stuff in that you feel is a greater good”.
In the literature, control has long been linked with power relations, for example as used as leverage during debates, in particular where competing values play a strong role [56
]. Control has been described as being exerted both between and within stakeholder groups. For example, some individual governments or negotiating blocks may be more powerful than others [57
], or some scientific disciplines may be taken more seriously in debates [28
]. In general, non-scientific stakeholders exert comparatively less control over GEA processes, at least in part due to continuing emphasis on quantitative and science-based knowledge in place of other perspectives [45
The final category of objectives identified is learning. Learning via stakeholder engagement, though not mentioned explicitly in GEA documents, was mentioned to some extent during interviews, in particular by target audience members who had not actually been involved in GEO-5 or IPCC WGIII AR5. These interviewees spoke primarily about the high potential for learning during GEAs, how learning could or should be an objective. One target audience interviewee stated bluntly that “knowledge exchange and learning should become one of the main objectives and one of the main results of this kind of exercises”. Another described how GEAs are unique in bringing together such a diversity of actors at the global scale, stating that “people can learn a lot from each other, across different scales, different government levels, but also among government levels”. Another still focused on how learning could take place by building connections to other groups, stressing that “the focus should be on the communities that are built, you know, the social networks that are cultivated”.
Expert authors also mentioned learning as an objective of stakeholder engagement, though to a lesser extent. Scientists had a different overall perspective on learning, focusing more on learning by doing. For example, one author involved in GEO-5 explained how “working with people from different backgrounds, disciplines, countries—I think we all learned something and gained capacities through the process”. Other authors confirmed this statement and described, for example, learning to work in an interdisciplinary environment, learning how to think collectively, or learning about problems and potential solutions from entirely different perspectives. Another GEO-5 author described the value added of learning through GEAs as opposed to other channels, explaining how “in those [stakeholder] meetings, you could gather a lot of information and feelings and ideas that are not in the research papers”. A highly experienced IPCC WGIII AR5 author stressed that GEAs provided an important opportunity for learning amongst early career researchers, who often “have gone on to become scholars in their own right, writing fabulous stuff—think about what a great training ground it is”.
Government representatives also mentioned learning to some extent, but focused more on what they needed to learn, emphasizing the demand-side of this category of objectives. One government representatives involved in IPCC WGIII AR5 described how governments wanted to learn “what’s new since the last report”. A government representative involved in GEO-5 described how “we learn from China, Japan and others in the Pacific Region where they have similar problems and much better approaches”.
However, while learning was generally described positively, there were still reservations. One government representative involved in IPCC WGIII AR5 in particular described how “governments have better and better understood their respective constraints and what is at stake for each nation and where they are situated globally and the things they can do, they have learned and learned and learned”. This interviewee went on to explain that despite the clear evidence that learning has taken place over many years of IPCC reports, there was still a lack of concrete policies in place, implying that learning may simply not be sufficient on its own.
Described in the literature in a pragmatic sense, learning occurs through an uptake of relevant and comprehensible information from the products of a GEA, and can be facilitated by participation [43
]. Stakeholder engagement can engender social learning and behavioral change at multiple levels and can also translate upwards to organizational learning, in GEAs and in other processes at the science-policy interface [59