The Evolution of Communicating the Uncertainty of Climate Change to Policymakers: A Study of IPCC Synthesis Reports

The Intergovernmental Panel on Climate Change (IPCC) reports on climate change have served to alert both the public and policymakers about the scope of the predicted changes and the effects they would have on natural and economic systems. The first IPCC report was published in 1990, since which time a further four have been produced. The aim of this study was to conduct a content analysis of the IPCC Summaries for Policymakers in order to determine the degree of certainty associated with the statements they contain. For each of the reports we analyzed all statements containing expressions indicating the corresponding level of confidence. The aggregated results show a shift over time towards higher certainty levels, implying a “Call to action” (from 32.8% of statements in IPCC2 to 70.2% in IPCC5). With regard to the international agreements drawn up to tackle climate change, the growing level of confidence expressed in the IPCC Summaries for Policymakers reports might have been a relevant factor in the history of decision making.


Introduction
One of the main sources of information about climate change-that is, about the current weather and climate data assessments and the science involved, the vulnerabilities associated with that change and the different strategies of adaptation and mitigation, and with a global world perspective-is the Intergovernmental Panel on Climate Change (IPCC), an international body of the United Nations that was established in 1988 by the World Meteorological Organization (WMO) and the United Nations Environment Program (UNEP). [1,2] The most widely read section of the IPCC reports is the Summary for Policymakers [3] and these have served to alert both the public and policymakers about the scope of predicted climate changes and the effects they would have on natural and economic systems.
The first IPCC report was published in 1990, since which time a further four have been produced. In the first stage of report preparation, a team of internationally recognized experts with extensive peer-reviewed research experience in their field, produce a detailed draft document setting out the current state of scientific knowledge according to the peerreviewed literature. Once this draft report has been externally reviewed, an Executive Summary aimed at policymakers, and based closely on the scientific text, is drawn up with input from government representatives.
The IPCC reports gather the results of three working groups: Working Group I (WGI) examines the physical science underpinning climate change and its possible future evolution; Working Group II (WGII) assesses the impacts of climate change and the vulnerabilities of natural and economic systems to it, including options for adaptation; and Working Group III (WGIII) focuses on climate change mitigation, exploring methods for limiting or preventing greenhouse gas emissions [4]. The climate projections made over the thirty years since the first IPCC report was published have been highly accurate with regard to the predicted increases in CO2 levels and temperature, although they have proved to be underestimates in terms of observed sea level rises [5][6][7].
Scientific knowledge has evolved considerably over this thirty-year period, as have the ways in which it is communicated, both to professional audiences and to the public at large. Communicating science to the general public is always challenging [8] (Seacrest et al. 2007), but in the case of climate science there are a number of particular difficulties related to how the level of scientific evidence is expressed, the degree of agreement among experts and researchers on each topic, the level of confidence in scientific methods and data, and the likelihood associated with the results of the mathematical models used to predict future changes [9].
During these three decades of IPCC climate reports, the level of certainty about the scientific assessments given to the public has increased, though there is not much literature on this evolution.
At any time during these years, once the current and latest scientific information available from the scientific community has been released in peer-reviewed scientific papers and the most relevant incorporated into every IPCC report, those scientific assessments are used by policymakers to draw up treaties and protocols for tackling climate change. How ambitious these protocols have been would depend, at some point, on policymakers' perceptions regarding the degree of urgency and the need to take specific action. It is worth noting here that, according to one recent study, the tone of the probabilistic language used by the IPCC tends to be conservative, with the majority of climate findings being assigned an intermediate level of confidence [10] which may in turn favor conservatism and inaction among policymakers.
By examining how the level of confidence, agreement, evidence and likelihood in the climate science described in the IPCC Summaries for Policymakers has evolved over the past thirty years, we can track the level of uncertainty in the information communicated to policymakers. As already noted, communicating climate science poses a particular set of challenges, not least the fact that the audience for the IPCC reports comprises government agencies, the media and, to a lesser extent, the general public, each of whom may interpret the results in light of their own special interests, focusing on elements which differ from those which scientists wished to highlight [11,12]. Consequently, both the structure of the reports and the kind of language used will also be key factors in getting the message across.
Some authors have studied the readability of IPCC reports and have compared those produced by different working groups with the Summaries for Policymakers [3] importantly, they found that not only has readability not improved significantly over time, but it has also actually decreased between the two most recent reports (IPCC4 and IPCC5).
Other studies have examined, across several different countries and languages, the public's interpretation of the probabilistic statements used in IPCC reports [13]. Although the language in which the report was read was a factor, the authors found that laypeople tend to interpret the level of certainty of statements as being around 50%, much less extreme than the reports' authors intended. Decision making depends on the level of certainty on the premises and information given to the subjects. [14].Difficult decisions need well informed subjects, especially when uncertainty is involved in the knowledge related to decision making. This problem is well known in the fields of medical decision making and ethics [15,16]. The level of certainty, and the manners in which it is communicated to the decision makers, plays an important role in whether the subjects decide to take action. [17] In IPCC5 and based on the guidance note for authors [18], the certainty of scientific statements is ranked using qualifiers of evidence (from limited to robust), agreement (from low to high), confidence (from very low to very high) and likelihood (from exceptionally unlikely to virtually certain). Although some authors have raised doubts about the treatment of uncertainties in the Fifth Assessment Report(AR5) [19] the report does enable a detailed analysis of the level of confidence associated with the statements it contains, since the corresponding expressions appear in italics and in brackets. [10] analyzed the report of WG1 in IPCC5 and counted the frequency of qualifiers for all statements made, assigning a numerical value so as to calculate a mean score for the report as a whole. They concluded that the way in which probabilistic language was used by the IPCC favored inaction among policymakers.
The aim of the present study is to analyze the content of IPCC Summaries for Policymakers in order to determine the degree of certainty associated with the statements they contain. We also examine changes over time in the level of confidence expressed in each of the reports that have been made available to policymakers and society in general.

Materials and Methods
The source material for this study is the Executive Summaries that have been approved by governments and which contain a Summary for Policymakers of the key conclusions reached by each of the working groups. The specific sources consulted for our analysis were as follows: 1.
Executive Summary First Assessment Report (FAR) [20] In addition to the Executive Summary itself, the FAR also includes a Summary for Policymakers setting out the conclusions reached by the three working groups.

2.
Summary for Policymakers WG I/II/III Second Assessment Report, (SAR) [21] This Summary for Policymakers contains a chapter by each working group.

3.
Synthesis Report-Summary for Policymakers Third Assessment Report (TAR/AR3) [22] This Synthesis Report includes a Summary for Policymakers written in a question-andanswer format, with no clear differentiation between the findings and conclusions of the different working groups.
Synthesis Report-Summary for Policymakers Fifth Assessment Report, AR5 [24] In addition to a Synthesis Report, AR4 and AR5 include a Summary for Policymakers that describes the findings and conclusions of the three working groups.
The first IPCC report was published in 1990 (prior to the internet age and hence in paper format), with a supplement appearing in 1992. It is descriptive in tone and designed to be informative for a nonspecialist audience, with very few references to the level of certainty of the data and projections, although the presumption is that it represents the scientific consensus of the day [25]. This latter aspect generated considerable political and scientific rise to numerous counter statements and denials with regard to the report's content [26]. We do not consider this report in our analysis of changes over time in the level of uncertainty, although we do discuss its communicative aspects.
Our analysis is based on the official and original English version: for each report we perform a content analysis of all sentences containing expressions related to the certainty of the scientific statement being made, without taking into account the text of figure legends or tables. The content analysis uses the categories (qualifiers) described in AR5, which is the report that most specifically describes the treatment of uncertainties. Thus, this report served as the basis for establishing the levels of uncertainty of statements made in previous reports, where the definition was less developed, and which are summarized in Table 1 (the complete version can be found in Annex 1). The table shows some of the equivalences between the different reports for each level of uncertainty.
In AR4, the criteria for uncertainty qualifiers are largely the same as in AR5 and they are also shown in italics, but also appear in other certainty qualifiers that are less standardized than in AR5. Thus, it is necessary to read the text in order to identify references to the level of evidence, agreement, certainty and likelihood, and match them to one of the four levels of uncertainty. This is a limitation of the content analysis which we have sought to minimize by applying the criteria shown in Table 1, thus enabling comparison of these statements of confidence. In the case of AR3 and SAR, uncertainty is expressed in similar but not identical terms. It is therefore necessary again to study the text of both reports in order to identify the degree of uncertainty associated with different statements, and thus be in a position to compare the results of all four reports in terms of how different levels of uncertainty are expressed.
In order to perform a numerical analysis of results, we counted the number of statements corresponding to each level of certainty in each of the reports studied. Each of the five categories of uncertainty was assigned a numerical value from 1 to 5, where 1 is Very Low, 2 Low, 3 Medium, 4 High and 5 Very High [10]. This enabled us to create a rank order for the different levels of uncertainty.
In a second stage we introduced a further analytic variable, "Call to action", which refers to the level of action implied by the most confident statements, compared with that for statements expressed with a low or medium level of confidence [19]. Thus, we defined "Call to action" as the sum of statements made with "Very High + High" levels of certainty, whereas "No call to action" was defined as the sum of statements made with "Medium + Low + Very Low" levels of uncertainty. Studying changes over time in the level of uncertainty expressed in the Summaries for Policymakers may help to shed light on the varying levels of action and inaction shown by governments in the context of international protocols. [27].

Evolution of the Structure and Communicative Aspects of IPCC Reports
The structure and format of the content included in the Summary for Policymakers of the five reports has changed considerably over time. There are also notable differences in the length of the summaries; the shortest is 20 pages for AR4 and the longest 38 pages for SAR, including a separate Summary for each of the three working groups. Table 2 summarizes these differences.

2007
Fourth Assessment Report (AR4) Summary for Policymakers, Synthesis Report Fragments highlighted in bold.
Graphs and tables Level of likelihood and confidence in italics 20 p.

2014
Fifth Assessment Report (AR5) Summary for Policymakers, Synthesis Report

Key fragments in boxes
Graphs and tables All levels of uncertainty shown in italics and confidence also in parentheses 32 p.
The communication of the state of scientific knowledge on climate change began with the first IPCC report (1990/92), and it was this document that began to define most of the concepts that would become familiar over the following thirty years. [28,29]. Interestingly, the notion of scientific "consensus" is not a defining feature of this report [25].; in fact, the only reference to it is when the lead author of the WG1 summary states that it was not fully achieved among participating authors. In terms of its style, the report is written in direct and plain language and it includes many definitions and explanations of phenomena, alongside projections for the future. Although the document proved to be controversial, both scientifically and politically, it was also groundbreaking and made an impact worldwide [25]. Perhaps its most original contribution was to begin the process of gathering the findings of peer-reviewed research into summary reports that were then approved by representatives of governments from around the world, hence the name of the body responsible for the report: Intergovernmental Panel on Climate Change [30].
The second report (1995) is the most extensive as it includes a separate summary by each of the three working groups. Those by WGI and WGIII consist of extensive text broken down into chapters, while that by WGII includes tables and figures among the text, although all three summaries are aimed primarily at nonspecialist audiences. The report contains numerous conditional statements and also contemplates the possibility of "surprises", due to the fact that climate changes are, by their nature, "difficult to predict". The process of consensus among the various authors who drew up and subsequently revised the texts begins to more clearly defined in this report [31].
The third report (2001) is structured in the form of nine questions that were formulated in consultation with the Conference of the Parties (COP) to the United Nations Framework Convention on Climate Change (UNFCCC). These questions were based on submissions by governments and were approved by the IPCC. The text of the report uses different colors and bold format, although the latter serves to separate parts of the text rather than to highlight key points. The report includes statements of both confidence (in parentheses) and likelihood, although it also contains numerous expressions that qualify the available knowledge on the various topics addressed.
The fourth report (2007) contains various sections, each with a clear heading, and uses bold text to summarize some of the statements made previously in the text. These statements are direct and assertive. We did not consider these statements in our analysis, unless they included specific reference to the level of confidence, which, in this document, is indicated in italics.
The fifth report (2014) contains highlighted conclusions that appear in boxes and act as a lead-in to each section. Once again, we only considered these statements if specific reference was made to the level of uncertainty. The overall text of this report is highly structured, and statements and predictions are accompanied by the corresponding level of confidence and likelihood (always in italics).
Over time the IPCC reports have adopted a more direct tone and more graphics and visuals [32] when summarizing the key conclusions of the working groups, and the assessment of uncertainty has been incorporated without making the texts more complex, thanks to the use of clearly defined and easily identifiable terms (in italics) to indicate the level of confidence and consensus. Indeed, the way in which uncertainty has been treated has evolved considerably since the first IPCC report, not least through the publication of several guidance documents aimed at report authors [33]. By the time of the third report (TAR: IPCC, 2001) some formal guidelines for the treatment of uncertainty were available [34], while the authors of the fourth report (AR4: IPCC, 2007) had access to the more specific Guidance Documents on Uncertainty Communication [35]. This was further developed in the Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties [18], which established a clear set of criteria for expressing the degree of certainty associated with the findings and conclusions presented in this report (AR5: IPCC, 2014). By contrast, the first two IPCC reports were much more imprecise in their treatment of uncertainty-this was especially the case in the first report (FAR; IPCC, 1990IPCC, , 1992.

Evolution of Uncertainty
Our analysis considered a total of 837 statements distributed unevenly across the reports (from 383 in IPCC2 to 87 in IPCC4), each of which was rated from 1 to 5 according to the level of uncertainty expressed. As we will see, the trend over time is towards greater uncertainty in the statements made. Table 3, the largest number of uncertainty-related expressions, 383 (46% of the total), are found in IPCC2, a lengthy text with only a few figures (in the summaries produced by WGII and WGIII) and which makes frequent use of conditional statements. The report with the second highest number of such expressions (23% of the total) is IPCC5 (AR5), and most of these statements are accompanied by the corresponding level of uncertainty. Next comes IPCC3 (AR3), with 21% of the total number of uncertainty-related expressions. Finally, the report with the smallest proportion of such expressions (10%) is IPCC4 (AR4), which is also the shortest and a document containing numerous tables and figures, which were not considered in our analysis. In terms of the proportion of statements at each level of uncertainty the most notable shift occurs between AR3 and AR4. Up until AR3, statements are most commonly expressed with a medium level of uncertainty, whereas from AR4 onwards the highest proportion of statements corresponds to the higher certainty level; in addition, the percentage of statements of very high certainty in AR4 and AR5 is at least equivalent to or even higher than the proportion expressed with a medium level of uncertainty. One possible reason for this increase in certainty is that climate science really took off around the turn of the century, as is illustrated by the number of articles being published on the topic of climate change [36]. A quick search of the publication databases shows that the number of articles labeled "climate change" increased from 2500 in the year 2000 to 27,000 in 2019 (Scopus), or from 20,000 to 78,000 over the same period according to Google Scholar (see Annex 2). The other factor that may account for the shift in certainty level is that it coincides with the introduction of clearer criteria for the treatment of uncertainty by the lead authors of IPCC reports [37].

As seen in
If we analyze the uncertainty indicators according to the two broad categories we defined earlier, namely "Call to action" (Very High + High confidence) and "No call to action" (Medium + Low + Very Low confidence), we can gain an idea of the extent to which the different reports may have served to encourage action on the part of policymakers (see Table 4). If the data from WGI or certain recommendations made by WGII or WGIII with regard to adaptation or mitigation are presented with a very high or high level of confidence (defined here as implying a "Call to action"), then policymakers may be more motivated to take action than would be the case for statements made with very low, low or medium levels of confidence ("No call to action") [38][39][40]. Table 4. Grouping of statements for each report according to whether they implied a "Call to action" or "No call to action". It can be seen in Table 4 that up until AR3 the level of confidence associated with the content of reports remained oriented towards "No call to action" (56.9% of statements), whereas a strong shift towards "Call to action" is observed in AR4 (68.9% of statements expressed with high or very high confidence). In other words, while the majority of statements in the summaries of SAR and AR3 did not conclusively constitute a "Call to action" on the part of policymakers, this situation had clearly reversed by the time of AR4, and in AR5 over 70% of statements are associated with a high or very high level of confidence.
Applying the criterion of mean level of uncertainty to the IPCC Summaries for Policymakers, it can be seen that the mean rating for the level of certainty increases progressively from 3.09 in SAR to 3.89 in AR5 ( Table 4). The problem with assigning a numerical value to the level of certainty expressed in the reports is that the way in which they are written favors the midpoint of 3 for qualifiers. The authors tend to include those statements for which there is agreement or consensus, and also greater likelihood, and hence the midpoint of 3 has greater weight. [41]. It should also be noted that this numerical rating of certainty varies much less significantly across the four reports (from 3.09 to 3.89) than do the percentages reflecting the number of statements made with a high or very high level of confidence, those implying a "Call to action" (from 32.8% to 70.2%).

Discussion
The dangers posed by climate change are not tangible nor visible in day-to-day life, this poses a well-known paradox, the Giddens's paradox [42] that leads decision makers to wait until the dangers are so visible that are inevitable. Adding uncertainty to the scientific assessments and projections about climate change adds tension to the communication of those outcomes [43]. Scientific uncertainty may be used to stimulate political and social controversy, and to try to show to the public an untrue lack of scientific consensus [44,45]. Many stakeholders track the process of decision making, from governments to the general world population, including all kinds of international organizations and corporations, with many different points of view and perspectives [46,47] The IPCC reports and the degree of certainty expressed in those assessments may be perceived very differently depending on the observer's educational background, and between observers from different countries [48,49].
Science, facts and knowledge about climate change and how to respond to the challenges it poses to humanity, are a complex political, economic and also linguistic [50] and communication issue that has evolved through pacts, protocols and treaties during the last 30 years [51][52][53][54][55].
In recent times, especially after the Paris agreement, and the "post truth politics" in several countries, the communication of IPCC scientific assessments and mitigation solutions communication has entered a new era more committed to specific and active action [56].
Knowing the high complexity of the decision making and the international political agreements, we have drawn a timeline that will track the publication of IPCC reports in relation to the main climate change agreements (see Table 5 and, especially, our analysis of "Call to action".
The background to these initiatives is to be found in the 1980s, a decade that saw the emergence of environmental movements and the start of increased interest in climate change research [51], which we have already commented on. The increase in public and scientific awareness about climate change was echoed in greater involvement on the part of political authorities, both national and international, and led ultimately to the establishment of the Intergovernmental Panel on Climate Change in 1988.
The first IPCC report (1990) drew together the literature predicting global warming as a result of fossil fuel emissions and it reflected the predominant view among scientists that urgent action was needed to avoid the damaging impact of climate change. This resulted in the creation of the United Nations Framework Convention on Climate Change (UNFCCC) and the signing of an international treaty in 1992. The UNFCCC established the Conference of the Parties as its main decision-making body, under whose aegis the Kyoto Protocol would later be signed in 1997 [52,53]. The second IPCC report, published in 1995, was much more cautious in its statements of current scientific knowledge about global warming and climate change. When the Kyoto Protocol was adopted in 1997, two years after publication of the SAR (in which 67.1% of statements classified as "No call to action" in our analysis), it only had 84 signatories and did not garner the support of several major powers including the USA, China and Canada. The lack of national and international bodies' commitment caused great disappointment to the chair of the IPCC [54].
Regarding the third IPCC report (2001), "Call to action" was still fairly weak (43% of statements in our analysis), and it was not until 2005 that the key requirement of the Kyoto Protocol was achieved, namely for binding targets by no less than 55 Parties who in total accounted for at least 55% of global carbon dioxide emissions. It thus took eight years for the Kyoto Protocol to come into force [55].
Our analysis of the fourth IPCC report (2007) indicated a much stronger "Call to action" (68.9% of statements were high or very high confidence). By the time of the Conference of the Parties 15 (COP15) in Copenhagen in 2009, it was hoped that either a new agreement would be reached, or the Kyoto Protocol would be renewed and extended beyond its scheduled expiry in 2012, although the latter did not ultimately come about. At the time of expiry, the Kyoto Protocol had 192 signatories [56,57].
Finally, AR5, published in 2014, continued to make a stronger "Call to action", with 70.2% of statements made to a high or very high level of confidence. One year later, the Paris Agreement was adopted by 192 parties (out of a possible 197 signatories) in December 2015 [58][59][60].

Conclusions
The level of certainty in the scientific assessments published in the different IPCC reports since 1990 has increased as the published knowledge about climate and climate change has become more robust and widespread.
Our content analysis of the degree of certainty associated with the scientific assessments of these summaries for policy makers shows a progressive increase over the years, and this is especially apparent when statements are grouped into two categories: those expressed with a high or very high level of confidence versus those of medium, low or very low confidence. The proportion of the former group has increased from 32.8% in IPCC2 to 70.2% in IPCC5.
There are a great number of factors of all kinds that tackle with the politics of climate change governance: but viewed from a historical perspective, in which the main climate change agreements can be time lined to the publication of the various IPCC reports, the relevant growing level of confidence in climate science, markedly in the last two reports, and the corresponding need for policymakers to take action, may have contributed to the more widespread adoption and ratification of these agreements. Indeed, our analysis suggests that the level of certainty expressed in the reports is likely to have been a relevant factor in the process of decision making that led to international protocols and accords aimed at mitigating or reducing the impact of climate change.
In sum, the increasing scientific consensus that is reflected in higher levels of confidence and the stronger likelihood of model predictions, and the communication of this information to society, has encouraged stronger action and greater commitment on the part of governments to the mechanisms of international cooperation designed to tackle climate change.