1. Introduction
The opening sentences of the Summary for Policymakers in the IPCC 2014 Synthesis Report state that “Human influence on the climate system is clear, and recent anthropogenic emissions of green-house gases are the highest in history. Recent climate changes have had widespread impacts on human and natural systems” [
1]. Those who wish to be prepared to face these new challenges need the appropriate knowledge and skills [
2]. For this reason, the United Nations Educational, Scientific and Cultural Organization (UNESCO) launched the Climate Change Education for Sustainable Development programme within the framework of the UN Decade of Education for Sustainable Development (2005–2014) as a key programme in Education for Sustainable Development [
3]. Climate change education is designed to enable people to understand the causes, processes, and impacts of climate change [
4].
In the climate change school project described here, 16–18-year-old students engaged with a climate-relevant issue over a period of three to six months. Supported by meteorology students and climate research scientists, they prepared and presented a comprehensive lecture (including written research work) on a climate-related issue. Within the framework of the project, they also participated in scientific lectures on climate research and remained in constant contact with science personnel. In this way, the project contributes to climate education, promotes scientific discussion of climate-relevant topics, and gets schools actively involved in climate research. As part of a general science-based education, schools teach (or should teach) epistemological beliefs—that is, a basic understanding of the genesis, justification, and distribution of knowledge—as a necessary grounding for participation in society [
5]. By doing so, an understanding of the
nature of science can be promoted and a scientific approach can be trained. In addition to basic scientific education, geography teaching is intended to develop students’ understanding of scientific ways of thinking and working, as well as the associated uncertainties [
6].
The aim of the present study was to evaluate the climate change school project in terms of epistemological beliefs and attitudes regarding climate science. The evaluation employs a scale measuring epistemological beliefs, as well as two further scales addressing the relevance of climate change in society and career prospects in the field of climate research. The study examines changes in epistemological beliefs regarding climate change among grade 11 and grade 12 students who have participated in a science-based climate project over a period of several months. After outlining the conceptual basics and theoretical framework, the empirical approach is described, followed by results and a discussion of the findings.
2. Conceptualization of the School Project as a Collaboration Between Schools and Climate Scientists
According to the educational standards for geography, the central task of geography teaching is to address phenomena and processes of current geographic and geoscientific relevance—such as climate change—and to develop strategies for action with students, based on well-founded expert knowledge, judgement, and problem-solving competences [
7]. The school project Climate Change aligns with these aims. The project was established by a teacher and a climate scientist to promote science communication. Teachers are made aware of this project through further training courses, where they can make initial contacts with the project team. High school students spend about six months working on a climate-relevant issue (1–3 students per group), creating climate maps based on climate data models, and researching scientific sources. The students are supported at all times by climate scientists and meteorology students (meteorology students are far advanced in their studies; they are either undergraduates or graduating students) who conduct workshops, help to create maps, and recommend relevant literature. They visit schools a few times and are available to answer questions via email. The selection of participating schools was based on the time of the study. The participating schools were selected exclusively after the beginning of the study, since the project could not have taken place yet due to the pre-tests. The project enables participating students to learn more about climate change in their own pace.
The project’s core elements are as follows:
Cooperation with climate scientists;
Insight into scientific research methods (climate modelling and visualization);
Application of scientific methods to project work;
Promoting argumentation skills excursions to research institutes; and
Presentation of results to classmates, teachers, and scientists.
In addition, the project promotes the following skills described by Mochizuki and Bryan [
3]:
Critical thinking, systemic thinking, and problem-solving competences;
Dealing with uncertainty;
Analysing, producing, and evaluating information; and
Information retrieval, media use, and technology.
The project topics encompass multiple areas of the natural and social sciences. This interdisciplinarity extends to teaching practices, as the project is integrated in geography, biology, physics, chemistry, and computer science courses.
5. Results
This section describes and compares the quantitative data from the pre- and post-tests, followed by qualitative evaluation of the guideline-based interviews.
5.1. Quantitative Analysis
Figure 2 shows the results for the epistemological belief scales and attitudes to climate change and career in climate science. The
x-axis indicates mean values for each scale and the
y-axis indicates mean values on the five-point Likert scale.
The effect strengths were calculated using Cohen’s d, as shown in
Table 2. If the calculated value is d < 0.2, the effect strength is low, d < 0.5 represents an average effect strength, and d > 0.8 indicates a large effect strength [
35]. Cohen’s d is defined from −∞ to +∞. For d < 0, the effect is negative; d > 1 indicates a large effect.
The source scale values change nominally from 2.54 in the pre-test to 2.48 in the post-test (p = 0.568). Mean values ranging from somewhat disagree to neutral reflect a slightly critical view of statements made by scientific authorities. From the respondents’ point of view, not everything said by scientists is necessarily correct or certain. A value of d < 0.2 indicates only a small effect.
On the certainty scale, respondents’ belief that knowledge is provisional and variable is indicated by the high pre-test value of 4.12 and the post-test value of 4.17 (somewhat agree) (p = 0.572), so rejecting the perfection of knowledge. Here, too, the values remain quite stable over the two measurement points and there is only a small effect strength of d < 0.2.
Values on the development scale change from 4.24 (pre-test) to 4.38 (post-test), indicating that scientific knowledge is considered subject to development and change. This shows a tendential change (p = 0.022), with values ranging between somewhat agree and fully agree and effect strength in the middle range.
The justification scale also shows a trend change (p = 0.027), with a pre-test value of 4.25, as compared to 4.35 post-test, and in the range between more likely to agree and fully agree. As the scale refers to the authority of scientists and multiple ways of acquiring knowledge, respondents tend to believe that scientific theories are based on different results and that scientists can change their minds. The effect strength here is also in the middle range.
Values on the relevance of climate change scale increase only slightly. The pre-test value is 4.17, as compared to 4.33 in the post-test (somewhat agree) (p = 0.067). Both are high values, indicating the topic’s perceived importance. Effect strength here is again in the middle range.
Pre- and post-test data for the career scale reveal differences between the two measurement points. While the pre-test value (2.81) is in the neutral range, interest increases in the post-test, with a value of 4.33 (somewhat agree) (p = 0.000). This indicates strongly increased interest in working in the field of climate research. The Cohen’s d value of 1.881 indicates a strong effect strength.
In summary, values on four scales remain high (somewhat agree to agree) while one scale remains stable and low (somewhat disagree) and one changes strongly (from neutral to somewhat agree).
5.2. Qualitative Analysis
The interview guide covered the four scales of epistemological beliefs used in the questionnaire. One or two questions were asked in relation to each scale, giving students an opportunity to respond more fully than in the multiple-choice questionnaire. The use of an introductory open question served as a warm-up to prompt insights on the topic of climate change. The answers focused on the drastic effects and complexity of climate change, which were variously described as ‘destructive’ (II: 4), ‘drastic’ (III: 6), ‘worse than imagined’ (III: 21) and ‘far-reaching and diverse’ (IV: 5). According to one participant, “industrialised countries—the polluters—do not have problems with, for example, low coastlines, as in developing countries. The countries that are actually to blame will get the most out of climate change” (III: 8).
The complexity of the issue was addressed by one student in the following terms, “you wouldn’t expect so many and such strong factors as are actually involved in climate change”. The same student expressed astonishment that “there are still people who say ‘that’s not man-made; there’s no such thing as that’” (I: 6).
Source of knowledge scale: This question asked students to what extent their view of the source of climate data had changed as a result of their participation in the school project. Almost all mentioned the considerable effort involved in data collection and modelling, speaking of a ‘meticulous process’ (IV: 16) that is ‘time-consuming’ (IV: 21) and of computers that ‘take months to calculate a data set’ (IV: 23). In this context, one student addressed the ongoing development of the models, “[...] if everything has to be reconsidered now, it will probably take another few years until you have all the data and then something will probably have changed again that you will have to recalculate. So, I think the computers will never stand still” (IV: 25). The possibility of working with ‘real’ modelled climate data surprised the students, “And it was really easy to go through the website and then just download this dataset and do the maps with this program. Well, it was a lot easier than I had imagined; I thought all this data was locked and that only certain people could access it or something” (III: 19).
Certainty of knowledge scale: This question addressed the extent to which participation in the school project had shown students the possibilities and limitations of modelled climate data. In this context, some students stressed that they had learned little about the climate system and climate data and their generation in school lessons, “At least in class I didn’t really learn that much about the climate in general” (I: 30) and “we didn’t work so much with climate data” (I: 29). Another group addressed the increase in knowledge through participation in the school project, “the field of vision has increased enormously” (II: 24); “one simply thinks quite differently about the influences of climate change” (II: 25); and “before, one hardly referred to data; because we have now done this in school, a lot has already changed there” (II: 26).
One student related the limitations of climate data to missing data sets, “Well, if you create the climate maps and there are no values, then there was simply a white area on the map [...]” (III: 40). Another referred to the uncertainties of the scenarios, “the limitations are that there are only these scenarios, and you don’t know which one will occur. [...] Only one scenario can be considered at a time” (III: 33).
Development of knowledge scale: This scale captured the extent to which students’ sense of the changeability and further development of climate models had changed following participation in the project. The student groups from both schools more or less agreed that there was only a small increase in knowledge about climate models, either because they “searched for more about facts about my personal topic than about the general topic” (I: 41) or because “I knew about modelling even before this semester and what is possible in principle, and my knowledge has not actually increased” (II: 34). The uncertainties of model calculations were also acknowledged, “Subject 2 has just mentioned that climate model 8.5 is now being changed again. This means that you are never actually at a point where you know that this is exactly what is likely to happen in the future. This is always a path that never ends. I would say this ongoing processing of the data is never absolute” (IV: 44); “But I think you should consider that any of these scenarios is theoretically possible, so you should talk about all of them” (IV: 57).
Justification of knowledge scale: The scale refers to the extent to which students have become more aware of how to arrive at scientific knowledge. Students associated this question with their online search for scientific articles. Much of the students’ homework consisted of literature research and only part of the work related to climate data. This explains their focus when answering this question, that they have learned “...to work with reliable sources and to refer to scientific results, and to get as good as they can into the matter they are researching, without relying on sources like Wikipedia or something like that but relying more on scientific institutes if they really want to work on something” (IV: 69). Some students spoke about measuring and observing data, “On the one hand, we collect data that we can collect everywhere in the world,” and “[...] through the behaviour of animals and plants, through their observation...” (IV: 69), (I: 43).
No explicit questions were asked about the relevance of climate change and careers in climate science scales, as these did not relate to epistemological beliefs. However, some of the content of student answers was relevant, which is why they are also listed here.
Relevance of climate change scale: The urgent need for good climate policy and more environmentally conscious attitudes was evident in some student responses, “[...] that it is now the last way to do anything and that now, through the Paris Accord [...] perhaps we are taking a step in the right direction” (I: 22); “I would say that in order to stop climate change, it is important that everyone looks at themselves and not just at others or at politics or other countries like the USA, where everything is even worse. Just ask yourself: What can I change in my everyday life to stop climate change or at least make a contribution [...]” (III: 21). One student concluded that “for [me], the most important insight was that everyone has to change something himself” (III: 25).
Career in climate science scale: While no explicit questions were formulated for this scale either, answers could be inferred. No student expressed a direct interest in a career in climate research, but their visits to the university and the German Climate Computing Center(DKRZ) remained memorable, “Yes, we have also heard this lecture from DKRZ (other B agree) about the complex computers they use and how long it takes to calculate (IV: 21); and “we were recently at the University of Hamburg and there I was also at such a researcher, climate day. (B4 agrees that he was there as well.) And they told us something that was very interesting, actually” (I: 9).
6. Discussion
This study represents a first step in evaluating the Climate Change school project, focusing on the extent to which participation changes the students’ epistemological beliefs and possible career perspectives.
Of the four scales related to epistemological beliefs, three remained at a stable high level (certainty, development, justification) and one remained at a stable low level (source). The relevance scale also remained at a stable high level while the career scale values increased from pre-test to post-test (d < 0.3; p < 0.10).
Values for the
source scale indicate students’ slightly critical position regarding statements by scientific authorities. In that context, the TIDE model of Muis, Bendixen and Haerle [
17] distinguishes between general, academic, and domain-specific areas.
As the questionnaire related specifically to natural scientists, these answers can be assigned to a specific domain, but different domains influence each other and contribute to academic and general beliefs. Early education (school) initiates the development of the individual’s academic epistemic beliefs, which are also socially constructed and context-bound [
17]. It would be interesting to investigate whether values from other domains (e.g., statements by politicians, journalists, or academics themselves are viewed more or less critically, and whether they change throughout the life course.
The fact that values were ‘only’ within the slightly critical range can be explained by the theory of cognitive division of labour. As most people are not in a position to fully understand expert knowledge, it is important to deal appropriately with this lack of knowledge, and certain skills must be learned in order to evaluate second-hand assertions [
16]. For this reason, it would be interesting to ascertain what the participants were thinking when they answered the question, as a lack of personal expertise means that one must rely in some way on statements made by scientists. Unfortunately, the question was differently interpreted by the students in the interview in terms of the considerable time and technical effort involved in data generation rather than the transfer of knowledge by authorities. In this respect, the qualitative data provide no further insights. On the other hand, it is generally necessary to evaluate sources as well as statements within the framework of possibilities if one is to avoid accepting everything at face value. The form of communication also plays a decisive role, but this is beyond the scope of the present discussion.
According to Lederman, Abd-El-Khalick, Bell and Schwartz [
33], the possibilities and limitations of scientific data are reliable and permanent, but never absolutely certain, and this is discussed in the second interview question (
certainty scale). Some participants recognised the limitations of the data and the process was also worked out by a group, showing awareness of the variability of model data and the importance of different scenarios. This is also reflected in responses to the questionnaire. Using Panoply (a map visualisation program) and MSCM (dynamic climate modelling software) allowed the students to handle different scenarios, deciding on one or comparing different scenarios.
The third interview question (development scale) focused on the fact that climate models are only projections and that generated climate data are always fraught with uncertainty. Most groups initially related the question to existential knowledge of climate models and spoke of a slight increase in knowledge, as they already knew that models project the future climate. The longer the question was discussed, however, the closer the students came to the core of the question. The ongoing development of climate models was described by one student as a path that never ends. Another participant indirectly addressed the diversity of models and the repetition of modelling runs.
At this point, it is worth noting the students’ excursion to the German Climate Computing Centre (DKRZ), which left a lasting impression. Supercomputers occupying an entire floor of the building model climate data day and night and model runs sometimes last several months. A lecture by a scientist as part of this excursion referred, among other things, to the developments in raster resolution and additional parameters in recent decades, including the increased use of physical and chemical parameters in the newer models. These models are complex and reflect the inherent uncertainty of the climate system. The willingness and ability to engage with uncertainty and to deal with it ‘productively’ are fundamental prerequisites for learning [
37] and are building blocks in students’ understanding of science.
The final interview question addressed the justification of scientific findings gained through observation, experimentation, and modelling and how they can be confirmed or refuted at any time [
32]. The high values in the quantitative data (4.24 and 4.38) confirm the perceived importance of diverse opinions and of scientific methods of gaining knowledge as referred to above. As noted by Urhahne and Hopf [
11], those who pursue scientific knowledge are more interested in the subject and more motivated to perform; they have a higher subject-specific self-concept and use learning strategies that are cognitively demanding. (It would be interesting to test this idea in another study, but this is beyond the scope of the present discussion.) The interview question was related more to literature search than to the generation of scientific findings. The reliability of serious sources was emphasised and positive reference was made to the increased ability to research scientific content. These outcomes were supported by the school project and are generally positive.
The importance of climate change to the students was reflected in the stable high approval ratings on the
Relevance of Climate Change scale. Active, long-term, and in-depth examination of a topic not only imparts specialist knowledge but, at its best, creates situational interest. According to Krapp [
24], this can also promote individual interest, which can be interpreted as a positive motivational disposition towards a certain field of knowledge or action. The classification of climate change as an important and relevant topic can also lead to increased interest in pursuing a career in climate science, possibly because of personal contact with scientists and increased familiarity with the field within the framework of the school project. The students visited a climate research institute, worked alongside scientists, and were exposed to high-performance computers, giving them an insight into the professional work of the climate researcher.
As described by Meyer [
38], these can be characterised as extra-curricular learning venues that support the pursuit of cognitive, instrumental, social-communicative, and affective learning goals. In the interviews, the students referred repeatedly to the excursion to the German Climate Computing Centre (DKRZ), confirming that the visit was memorable and aroused their interest. These places of learning are notable as encounters with an authentic person in the scientist’s mediation of science, [
39]. In this context, Braund and Reiss [
40] argued that “authentic school science should provide experiences that are more in line with the sorts of activities that scientists and technologists do in the real world of science and that such experiences should include student-directed tasks and more open-ended enquiries.” The increased interest in a possible career in climate research can be linked, among other things, to the intensive exchange with climate scientists and the accompanying insight into this field of research.
6.1. Assessing the Project
The Climate Change project also brings education for sustainable development into schools. The 17 goals of Education for Sustainable Development adopted by UNESCO in September 2015 articulate major challenges for humanity. SDG 13 focuses on “measures to combat climate change and its effects” [
2]. To that end, learners should work towards stated objectives in the cognitive, socio-emotional, and behavioural domains [
41]. Each of these domains is subdivided into five detailed objectives. On closer inspection, it becomes clear that participation in the Climate Change School Project addresses several of these objectives.
The learner understands the greenhouse effect as a natural phenomenon caused by an insulating layer of greenhouse gases [cognitive learning objectives].
The learner is able to understand their personal impact on the world’s climate, from a local to a global perspective [socio-emotional objectives].
The learner is able to anticipate, estimate and assess the impact of personal, local and national decisions or activities on other people and world regions [behavioural learning objectives] [
2].
The Climate Change project contributes both to deeper discussion of climate-relevant issues and to the development of skills within the framework of Education for Sustainable Development. Participation. The school project also promotes other key interdisciplinary competences considered ‘transversal, multifunctional, and context-independent’ [
2]. The understanding of the “nature of science” and the scientific handling of data will continue to be promoted.
The four dimensions of science understanding (source, certainty, development, and justification) [
19] are promoted by participation in the school project. The relevance of epistemological beliefs lies in their relevance for learning and knowledge acquisition in formal educational contexts, as well as for informal learning and processing scientific knowledge in everyday life [
19].
6.2. Limitations of the Project
One limitation of the present research is the small number of participants (n = 36) in the quantitative part. This prevents any reliable statistical generalisation about the understanding of science at a wider level. Future studies should increase the sample size to enhance generalisability. In the present case, the use of Cohen’s d value facilitated evaluation of effect strength.
As respondents differed in their interpretations of the interview questions, which were based on the questionnaire scales exploring epistemological beliefs. It is reasonable to ask whether the questions were formulated too imprecisely or whether the respondents could not cope with the complexity of the questions. We asked, for example, for the possibilities and limitations of modeled climate data. Some students answered that they missed datasets they need for their project, so they did not come up with the idea that the question did not refer to their project, but to climate data in general. So, there was no dealing with the complexity of the climate system and the uncertainties (population growth, dealing with renewable energy, etc.). As such, we suspect that the two issues are linked and for that reason, no conclusion can be drawn as to whether participation in the project promotes understanding of science as a whole. The consideration of selected scales shows only a deeper understanding of aspects of climate change, data generation, and the limits of climate models. During the interviews, it was noticeable that the questions frequently had to be repeated at the participants’ request, but for reasons of comparability they were not reformulated or explained. This suggests that the questions were unclear and possibly too complex. In the future, the questions should be distributed in printed form throughout in order to make them easier to understand.
Many variables influence the formation of science understanding (e.g., teacher personality, lesson design, teacher knowledge, topic, group dynamics) and, as these cannot easily be controlled, it is not practicable to include an experimental control group in this setting. Although we have collected other data from upper school geography lessons on climate change (where students did not participate in the school project), we have not included these data as a control group because of the difficulties of variable control. It is important to acknowledge that students can respond well without participating in the project, but these data lie beyond the scope of this study.
It is also important to note that the content of the epistemological belief scales was not explicitly taught, but formed part of the project only implicitly. In a study of sixth graders’ views on the nature of science, Khishfe and Abd-El-Khalick [
42] found that “an explicit and reflective inquiry-oriented approach is more effective than an implicit inquiry-oriented approach in enhancing sixth graders’ views of the target NOS aspects.” On that basis, it is reasonable to ask how the project concept might be adapted to explicitly promote appropriate epistemological beliefs. In addition, the accompanying regular geography lessons on climate change might usefully be more closely coordinated with such a project, with parts perhaps conducted by project members (e.g., meteorology students).
The Climate Change school project will be run over a longer period of time (approximately six months) in parallel with regular geography lessons, which will of course consume more teaching time. The pupil-oriented approach (in which the main topics are chosen by the students themselves) enhances interest in learning about this important contemporary topic. The Fridays for Future movement, in which many students participate, confirms the relevance of climate change for students and the data on a potential career in climate science further confirm that authentic learning creates interest and attention.