The Effectiveness of Education for Sustainable Development

Perhaps the most important issue in our time is how to sustain our planet’s resources, while developing wealth and well-being for a growing population. This monumental task has been defined in the concept of sustainable development (SD). During the last few decades the world communities have agreed upon addressing SD through international treaties. As a response Education for Sustainable Development (ESD) has been launched as an answer to cope with sustainability. However, empirical studies are a missing link in the discourse around ESD, where decisions and implementation strategies are heavily based on policy recommendations and gut feelings by practitioners. We used data from 2413 students in grades 6, 9, and 12 from 51 schools across Sweden to study the effectiveness of ESD. In line with the current debate on the definition of ESD, we quantified the extent to which teaching can be labeled as holistic and/or pluralistic. Through a series of descriptive analyses and the estimation of structural equation models, our results indicate that ESD can indeed impact on student outcomes in terms of their sustainability consciousness. The results of this study reveal the key role ESD plays in addressing SD, paving the way for a more sustainable future.


Introduction
Perhaps the most important issue for survival in our times is how to sustain the natural environment and our planets resources at the same time as develop wealth and well-being for a growing population.This monumental task has been defined in the concept of sustainable development (SD).During the last few decades the world communities have, under the umbrella of the UN, agreed upon jointly addressing SD.As a response, Education for Sustainable Development (ESD) has been launched as one of the key answers to dealing with sustainability.The underlying idea of ESD is to empower students with sustainability competences through a holistic interdisciplinary perspective of content and pluralistic learner-centered democratic teaching strategies.ESD has been adopted globally as a consequence of the UN Decade for Education for Sustainable Development (DESD;2005-2014), which has reshaped curricula worldwide.In spite of a global commitment to ESD as a teaching approach, there is very little empirical evidence for (1) the extent to which ESD is implemented in classrooms, and (2) the effects ESD has on student outcomes (such as their knowledge, attitudes, and behavior towards SD).The present study reports on a large scale investigation that tests these two key questions.

Sustainable Development
The concept of sustainable development (SD) seeks to combine environmental concerns with social and economic development.The concept became familiar to the public in the 1980s with the report Our Common Future, also known as the 1987 Brundtland report.Until then, environmental protection had been the focus, arising from salient environmental problems such as pollution.With the concept of SD, it was emphasized that environmental problems should be dealt with in relation to social and economic issues.SD is, in today's society, an overarching goal of balancing between the well-being and improved lives of people globally in space and time, while at the same time preserving natural resources and ecosystems [1].This is an ambitious goal that seems to evade and constantly move further distant at the horizon.Nevertheless, this is a goal that international community managed to agree upon and work towards.Common policy documents have been decided upon and implemented internationally and nationally in most countries [2].
The United Nations raised SD on the global agenda in order to improve actions for a sustainable future.UNESCO has outlined a number of sub-themes to the environmental, social, and economic dimensions of SD: (1) environmental perspectives: natural resources (water, energy, agriculture, biodiversity), climate change, rural development, sustainable urbanization, disaster prevention, and mitigation; (2) socio-cultural perspectives: human rights, peace and human security, gender equality, cultural diversity and intercultural understanding, health, HIV/AIDS, and governance; and (3) economic perspectives: poverty reduction, corporate responsibility and accountability, and market economy [3] (pp. [18][19][20][21].
Actions for a sustainable future can be considered at the level of governments, businesses, organizations, and citizens.In the current paper we investigate the level of the individual.From an educational perspective all outcomes and assessments are conducted at the individual level, and the steering documents for the Swedish school system regarding ESD are also supposed to be evaluated at the individual level.Hence if investigating the effects of ESD in Swedish school it should reflect the overall aim of the organization and therefore we zoom in on the level on the individual.
It is widely accepted that most instances of deteriorating environmental conditions are caused by human behavior.Drivers of sustainability problems are rarely the result of malicious intent, but rather the consequences of the lifestyles of billions of humans.Schultz [4] argues that SD is "a goal that can only be achieved by changing behavior".The action of that behavior can work at different levels of the society from the individual personal life style, to political decision making, or business closures etc. and thus have different powers in making differences for sustainable development.Moreover, behaviors in favor of SD are restrained by structural and cultural barriers as shown by Isenhour [5].Nevertheless, the behavior of individual citizens is a construct that has been studied extensively in the social sciences, and while scholars may not agree on the most applicable empirical model to explain its causation, there is a large consensus that is correlated to psychological characteristics of individuals, such as attitudes and knowledge.Especially relevant is the study of these constructs in young people, at the onset of their lives as active citizens and leaders of tomorrow [6], of consolidating habits, behavioral patterns [7], and the formation of identity and interpersonal relationships [8].In the current study, we focus on students' sustainability consciousness or SC [9,10].The concept of SC is a composite notion, unifying contents in environmental, social, and economic issues, as well as psychological constructs relating to knowledge, attitudes, and behaviors relating to such issues.The concept of SC was developed in close relation to the UNESCO sub-themes outlined.To reach a transition towards a sustainable future, critical ingredients for change at the level of individual citizens include a better understanding of, more positive attitudes towards, and behaviors in line with the principles of SD.The question then to be answered is how to accomplish this ambitious goal?How can we change the path of our future?One of the most pronounced answers to this question in the last decade has been: education.
Educational efforts aim for empowering students' with action competence in order to deal with such critical ingredients for change.Action competence is considered as being closely related to knowledge of action possibilities, confidence in one's own influence, and a willingness to act [11].SC, with its different psychological components builds on this definition by Breiting and Mogensen [11].Thus, the concept of SC is regarded to reflect students' sustainability action competence.

ESD as a Response to the Call for SD
By educating citizens, especially young generations within the formal schooling system, the hope has been to effectively address the issue of SD [12].This hope led to the launch of the UN Decade of Education for Sustainable Development (DESD), overseen by UNESCO.During the Decade, ESD was launched as an approach to teaching and learning that promotes SD.ESD has grown from an idea into a global movement [13], and the understanding of what it is or should be has evolved during the DESD.The UNESCO definition reads: "Education for Sustainable Development means including key sustainable development issues into teaching and learning; for example, climate change, disaster risk reduction, biodiversity, poverty reduction, and sustainable consumption.It also requires participatory teaching and learning methods that motivate and empower learners to change their behavior and take action for sustainable development.Education for Sustainable Development consequently promotes competencies like critical thinking, imagining future scenarios and making decisions in a collaborative way" [14].
Within this definition we can see traces of two essential features of ESD: the first deals with content, the second with pedagogy.These two are well recognized in the literature: "ESD continues to grow both in content and pedagogy and its visibility and respect have grown in parallel" [13] (p. 2).As seen in the UNESCO definition, ESD contents cover diverse disciplines: climate change, poverty reduction, consumption etc., and interdisciplinarity is crucial to understand the complexity of SD issues [15,16].In ESD literature this notion is labelled holism or a holistic approach since it includes multiple perspectives on content.ESD emphasizes the necessity to include all three dimensions (environmental, social, economic) and focus on their interrelationship, as well as interactions over time and space [17,18].The holistic perspective of SD recognizes that social and cultural factors are often the cause of environmental problems, and there is often a conflict of interest between economic, social, and environmental goals of individuals as well as societies [19,20].Such conflicts do not only span over disciplines, they also tie into local and global perspectives as well as past, present, and future generations [1].In their outline of ESD as a teaching tradition Öhman [21] identifies three essential aspects of holism: connect environmental, social, and economic dimensions of SD issues, integrate their past, present, and future implications, and focus on their local, regional, and global nature.
The second essential feature of ESD deals with the process of teaching and learning.ESD focusses on the development of skills and action competence for sustainability [22].This pedagogy has been labelled pluralism [23,24], and is characterized by a striving to acknowledge and engage different perspectives, views and values when dealing with SD issues.The underlying idea is that the complexity of such issues and conflicts of interests make it impossible to teach predefined solutions.Instead of teaching the "right" answers and response to SD, ESD focusses on reflecting on issues [23].This progressive and transformative pedagogical approach develops students' critical evaluation of alternative perspectives and calls for learner-centered teaching strategies (e.g., critical thinking, participatory decision making, value-based learning, and multi-method approaches; [24][25][26][27], and social learning [28].Essentially, ESD aims to facilitate learning in such a way that learners understand the world based on their own observations, and develop competences to take action for sustainability.
Holism and pluralism are often seen as intricately intertwined.Learning is thought to take place through the pluralistic interceptions of social, economic, and environmental perspectives [29].These two essential features of ESD have been recognized as challenges for teachers and schools that strive to implement ESD [13,30,31].
In the literature there is an emphasis on encouraging holism and pluralism for deliberation about ESD.It is argued that unless ESD and the discourse on SD stay open to opinions and debates of educators, it risks becoming indoctrination [32][33][34][35][36]. Hence holism and pluralism in education are suggested as essential to promote SD, and are therefore in the central focus of our current study, in which we focus on investigating the implementation of ESD in the context of formal education in Sweden.
At the same time, critique against the effects and appropriateness of ESD has been advocated.Kopnina [37] argues that the pluralism and holism embedded within ESD may in fact confuse the teacher and the student of the problems and contradictions inherent in the SD paradox.Kopnina claims that the holism and pluralism induce a focus on economic and social issues leading to anthropocentric views, turning the students away from ecocentrism and the problems of the environment that need to be solved in order to reach SD [37].Kopnina [38] (p.700) states that: "plural perspectives on ESD can lead practitioners into an essentially anthropocentric paradigm which can be counter-productive to the effort of fostering environmentally concerned citizenry".Also, Jickling [33] discussed normative questions that need to be recognized as important areas of inquiry regarding education and ethics, implying that ESD as a deliberative approach might be difficult to implement.Wals [39] talks about the problem of using the deliberative method of ESD (holism and pluralism) to solve the normative problems of reaching SD as defined from expert-views, therefore it is not evident that ESD can or will enhance peoples willingness to consciously solve the sustainability problems of this planet, which will be investigated in this study.
Although there has been critique ESD has been implemented as a common discourse all over the world, governmental and non-governmental agencies support schools in the implementation of ESD.Often this takes the form of certification system in which schools are encouraged to integrate ESD into their daily educational practice through certificates that can be earned.Research on the effects on the SD-outcomes of students due to the implementation of ESD are scarce and much needed, as illustrated by the UNESCO recommendation of "the need for more research, innovation, monitoring and evaluation to develop and prove the effectiveness of ESD good practices" in the DESD final report [40].

The Context of ESD in Sweden
In Sweden, the compulsory school system consist of grades 1-9, starting with the year children become seven years old and ends the year they turn 16.Students leaving grade 9 have completed compulsory school, and are entitled to apply for a three-year upper secondary school education.Upper secondary education offers a mix of vocational and academic programs [41].Both the compulsory school system and upper secondary education received new curricula in 2011 [42].For compulsory school the curriculum is designed so that students should have achieved certain capabilities and knowledge requirements in grade 6 and in grade 9.For upper secondary there are also competences and knowledge requirements that need to be achieved for each subject area.
The concept of SD is not included as a separate subject in the Swedish school system.However, SD is included as parts of the descriptions of all subjects in the curricula.All teachers are therefore expected to include sustainability issues in their teaching.Teachers should also enable students to train subject specific capabilities for the future.Moreover, SD is described as one of the comprehensive perspectives in education that Swedish schools have to include in their work.The idea of the DESD was to implement ESD at all levels of education.As a consequence, there are clear linkages between the Swedish curricula and the international DESD documents [3,14].
There have been few political initiatives to support schools with the implementation of SD teaching and learning during the DESD, except for its infusion into in the new curricula.Therefore, schools have relied heavily on supporting organizations and authorities.The eco-schools project [43] is an international program which aims to raise student awareness of issues associated with SD.In Sweden, the non-governmental Keep Sweden Tidy Foundation [44] administers the eco-school green flag certificate and develops programs attuned to the needs of the Swedish school system.Every year, schools send a report of their activities to the Keep Sweden Tidy Foundation, which then gives feedback to the schools on their progress.If schools do not send in reports they will lose their green flag certification.Independently of the eco-school certification system, the Swedish National Agency for Education hands out the award of School for sustainable development, which Swedish schools from preschool to upper secondary school can apply for [45].Schools have to apply for a new award every three years.In 2007 the World Wide Fund for Nature (WWF) initiated another initiative in Sweden; the three-year school development project called School on sustainable way [46].Schools that participated in the school development project have to a high extent continued their work on implementing sustainability into their education in the spirit of the project.Finally, The Global School, (a section of The Swedish Council for Higher Education) offers professional development for teachers, school leaders, and policy makers with a focus on globalization and ESD [47].

Research on the Effectiveness of ESD
While ESD is being implemented in formal education all over the world, large scale empirical research on its effects and effectiveness are scarce and strangely absent in the literature.Typically, studies that delve into the effects of ESD have a focus on its environmental dimension.In 2010, Coertjens, Boeve-de Pauw, De Maeyer, and Van Petegem [48], published a study that zoomed in on the effects of classroom practices on the environmental attitudes and knowledge of grade 9 students in Flanders, using the nationally representative PISA 2006 data.While not at the core of the study, the results do show that students in those schools of which the school leader indicates that they participate in school-wide educational programs that focus on environmental issues, report more positive environmental attitudes than students from schools for which the school leader indicated no such participation.Though statistically significant, the effect is reported as very small.Pirrie, Elliot, McConnell, and Wilkinson [49] surveyed schools participating in a certification program focusing on managerial and educational aspect of the environmental dimension of SD (the eco-schools project).One of the aims of their study was to assess the perceived educational effectiveness of this project in Scotland.Students were asked whether (to their own opinion) the school's participation in the project has had an impact on their environmental concern and behavior.Given the likelihood of the elicitation of social desirability through such questions, the very positive response of the students to this question is to be taken with care.
Other studies surveyed the effects of environmental education on students more directly by comparing the knowledge, attitudes, motivations, behaviors, and other outcomes between students that are or are not in schools that participate in eco-school certification programs.Hallfredsdottir [50] shows that while students in Icelandic eco-schools know more about environmental issues, they do not have more positive attitudes.Similar results are seen in the study of Krnel and Naglic [51] in Slovenia: students in schools that do participate in the eco-schools project know more about the environment than students in schools that do not.Their attitudes and behaviors however are not affected by the certification program.Ozsoy [52] confirms the cognitive effect of participation in the eco-school program in Turkey, but-in contrast to other scholars-they do see a higher intention for pro-environmental behavior.A major limitation of these studies is, however, that they are built on samples of often only a few schools, and so it is doubtful that their results can be attributed to the schools' participation in the certification program.
Large scale studies are scarce, but do exist.Boeve-de Pauw and Van Petegem [53,54] surveyed 50 schools in Flanders, half of which were experienced participators in the eco-school certification program.These studies confirm the cognitive effect: again, the results suggest an increase in the students' knowledge (and associated values) but not in their attitudes and behavior (and associated values).A large scale study about the Brundlandt certification program for Canadian schools [10] showed no impact on the environmental behavior of students.In recent large scale studies in Sweden [9,55], small positive effects were observed in the sustainability consciousness of grade 6 and grade 12 students in ESD-certificated schools, but negative effects in that of grade 9 students.
Overall, these small and large scale comparative studies are almost always conducted to evaluate programs for environmental education.Beside the authors' [9,53,55] no large scale studies of the effects of ESD can be found in the literature, and the few existing studies in EE seem to suggest that while schools' participation in a certification program sometimes results in their students knowing more about the environment or sustainable development, it rarely results in them caring or acting more.What these studies also have in common is that they focus only on the outcomes of certification programs, and that they are cross-sectional by design: they compare groups of students at one single moment in time rather than following cohorts of students and study how their outcomes change over time (and how that change is or is not related to what is happening in the school).While such a cross-sectional design can be used to compare schools, it also requires footnotes to be placed on the validity to make conclusions about the effects of EE and ESD, since none of the studies provide information about (a) the route students have made, and (b) what is actually happening in the schools.It might, for example, be possible that a school joins a certification program because concerned members of the school team acknowledge that their students have low sustainability attitudes and see the certification as way to deal with this issue.
The certification then becomes a proxy of low attitudes rather than of high ESD implementation (especially in the early stages).Such a scenario might explain negative effects of a certification program, like the ones identified by Olsson et al. [9].Of course many alternative scenarios are plausible, and very likely each school has its own specific set of reasons to join certification programs.This stresses the need to go beyond studying ESD outcomes, and comparing them between certificated and non-certificated schools, if we want to learn more about the effects of ESD.This has, to our knowledge, not yet been done in a large quantitative study although ESD has been implemented worldwide for over a decade.This is the gap to be filled by this study.
While a comparative approach does allow pinpointing differences among schools, it is not very informative when it comes to explaining the reasons behind such differences.An even more appropriate research approach could be longitudinal, though such studies are often both a logistical and financial challenge.We can also move forward by focusing on outcomes simultaneously as we focus on the educational process.Cincera and Krajhanzl [56] showed that it is not the schools' participation in a certification program that will have an impact, but rather the students' participation in the decision-making processes at school.Their results show that students who perceive higher participation in such processes also have higher levels of action competence.In an earlier study, Cincera and Makova [57] showed that the lack of effects on outcomes of a certification program in the Czech Republic was connected to problems with its implementation within the school.These results underscore the arguments raised above to nuance the usefulness of ESD-certificates as proxy for actual ESD, and stress the importance of not just studying outcomes but also what is happening in the classroom.As argued above, especially relevant to answer the call for evidence-based practice in the DESD final report [40], and to judge the effectiveness of ESD, is the study of the occurrence of the holistic approach of ESD to content matter and the pluralistic approach of ESD to teaching, in relation to students' sustainability consciousness.The current study aims to do exactly that, within the context of formal education in Sweden.

Research Questions
With our current study we aim to go beyond using the ESD-certifications as a proxy for ESD.We propose four research questions to dig deeper into the effects of ESD in the context of Swedish formal education.We focus on students' perceptions of the occurrence of ESD in grades 6, 9, and 12.Our main interests in research questions I -III are therefore the occurrence and (grade specificity of the) effects of the holistic ESD approach to contents and the pluralistic ESD approach to teaching: I. To what extent is ESD (holism and pluralism) happening in the classroom?II.What is the effect of ESD (holism and pluralism) on students' sustainability knowingness, attitudes, and behavior?III.Is the effect of ESD (holism and pluralism) on students' sustainability knowingness, attitudes, and behavior grade specific?
We also zoom in on the effect of schools' participation in ESD-certification programs, but rather than connecting such participation directly to the outcomes at student level, we connect it to their perceptions about the occurrence of the two ESD-dimensions.Our fourth research question, therefore, reads: IV. Do ESD-certifications facilitate ESD (holism and pluralism)?

Sample
Participants for this study were selected from grade 6 (ages 12-13), grade 9 (ages 15-16), and grade 12 (ages [18][19].At these ages they have received goals according to the Swedish compulsory school curriculum and the final year of the upper secondary school.The study reported here is part of a larger investigation, in line with the established practice of studying the implementation of ESD in the Swedish school system by comparing two sets of schools: one that has received an ESD-certificate and the other without such a certificate.The current study is based on data from both sets, but we do not primarily focus on differences between them (as we did in earlier studies [9,55]).
In the first step of the sampling procedure, 26 certified ESD-schools were selected.These schools were identified through registers compiled by certifying organizations [42,44,46,47].In the second step we selected 25 non-certified schools.These were matched to the certified schools based on socio-economic factors and students grades extracted from the Swedish SALSA (Skolverkets Analysverktyg för Lokala SambandsAnalyser: The National Agency for Education's analysis tool for context analysis [58]) and SIRIS (Skolverkets Internetbaserade Resultat-och kvalitetsInformationsSystem.(The National Agency for Education's online information system on results and quality [59].))databases.Table 1 gives an overview of relevant numbers for each of the grades.We invited students from one or two class groups in each grade in all the schools to participate to avoid that responses from students in a single school should excessively bias our results.Overall, the sample included 2413 students from 51 schools across Sweden, with a mix of big and small schools as well as schools in urban and rural areas.Data was collected in 2013 during teaching hours and always with a researcher present in the classroom to ensure that all the participants received the same instruction.Our sample can be considered a healthy representation of schools in Sweden.
Table 1.The number of participants, the sex ratio (boys:girls), response rates, and missing data of each grade and for the whole sample.

Variables
Student perceptions of the ESD at school.We developed items to tap into ESD based on its definition as a selective teaching tradition by Sandell et al. [26].The items were specifically developed to reflect the two central features of ESD: (I) a holistic approach to content and (II) a pluralistic approach to teaching.Holism was surveyed through the students' perceptions about the extent to which contents are covered regarding environmental, economic, and social interrelations, as well as geographical relationships (local, regional, and global), and relationships and consequence over time (past, present, and future).Pluralism is reflected in items based on the central idea that no single ethic is given priority in the teaching, which implies that the standpoints and values of all people are treated as equally relevant.As a consequence, the learning process focuses on critical inspection of alternative viewpoints.Since democracy is a central aspect of the pluralistic approach in ESD, our items are also concerned with the extent to which students are allowed to influence the contents of teaching and learning.Since this was the first time these items were used in analyses, we tested the construct validity through confirmatory factor analysis (CFA), with excellent model fit (RMSEA = 0.054, CFI = 0.989, and TLI = 0.982).Table 2 presents the seven items tapping into the students' perceptions about ESD in their school.The respondents marked their answers on a five-point Likert scale: (1) never, (2) seldom, (3) sometimes, (4) often, and (5) very often.A "don't know" option was available.The items were introduced in the survey by the following statement: "The following claims focus on your experiences in the classroom".

ESD-dummy.
A variable discrimination between the schools that do and do not participate in an ESD-certification program was included to be able to estimate effects of participation, and differences between the schools.
Student outcomes.The dependent variables in this study are the students' sustainability knowingness, attitudes, and behaviors.We surveyed theses constructs through the concept of sustainability consciousness that reflects the outcomes of ESD and students' action competence in issues concerning sustainability.The instrument is based on the work of [60], and was further developed theoretically and validated empirically by Olsson et al. [9] and Berglund et al. [55].The concept of sustainability consciousness taps into students' knowingness (K), attitudes (A), and behavior (B) in relation to the sub-themes to the environmental (ENV), social (SOC), and economic (ECO) dimensions of SD.Each of the 50 items in the SC instrument relates to a combination of either K, A, or B to ENV, SOC, or ECO.The knowingness items are concerned with what the students know or acknowledge being necessary pre-requisites for SD.The attitude items tap into the extent to which the students evaluate SD statements as positive or negative.The behavior items ask about the respondents' actions related to SD. Sample items are: "Sustainable development requires that we humans reduce all sorts of waste."(K-ENV); "I think that it is important that people in society exercise their democratic rights and become involved in important issues."(A-SOC), "I avoid buying goods from companies with a bad reputation for looking after their employees."(B-ECO).The respondents marked their level of agreement or disagreement to the statements on a five-point Likert scale, ranging from 1 (Strongly disagree) to 5 (Strongly agree).For each item, a 'Don't know' option was available.In this study we use the students' overall mean scores on K, A, and B. Their responses therefore reflect their sustainability knowingness, sustainability attitudes, and sustainability behaviors, across all three dimensions of SD.The three constructs show good internal consistency: K (19 items, α = 0.84), A (14 items, α = 0.80), B (17 items, α = 0.80).The complete surveys can be consulted freely online [61].

Analyses
To be able to answer our research questions we developed a short instrument to tap into the two dimensions of ESD (holism and pluralism) identified by Sandell et al. [26] and Öhman [21].Through confirmatory factor analyses we validated our scale these for these two key constructs.Next, to answer the first research question, we provide descriptive statistics for the two ESD dimensions both at the item level and at the level of the construct.To estimate the effect of the two dimensions on the student outcomes, we built a structural equation model using the entire dataset (so grades 6, 9, and 12 together).While the two ESD dimensions were inserted into the model as latent constructs, the three sustainability consciousness dimensions were discrete scale scores.The effects of the first on the latter were initially estimated linearly, and then non-linearly by including quadratic estimates.The model was then reconfirmed using the datasets per grade, so as to answer our third research question.Finally, the fourth question was answered by including a dummy variable, discriminating between certified schools and non-certified schools, into each of the grade specific models.All analyses were performed using the statistical software package Mplus [62].To evaluate the model fit, multiple fit indices were used with minimum values around 0.95 for the CFI and TLI index, and a value of 0.06 or less for the RMSEA [63].Where necessary, modification indices (MI) were used to identify how to improve the model fit based on error covariance between items within the same factor [64].Since the data was ordered categorically by nature, the WLSMV estimator was used with delta parameterization [62].The SEM analyses were not estimated through a multilevel approach, but the errors for the nestedness of students within schools were corrected for their hierarchical dependency through the Mplus type = complex command.
Once fit-indices were acceptable with all factors included in same model, the estimations for interrelationship between the constructs were estimated.First, the relations between the different dimensions of ESD were fitted, next those between the dimensions of SC (knowingness, attitudes, and behavior) were included, and then the effect of the first on the latter, and finally the dummy variable for certifications was introduced.Interaction effects between independent variables were tested but, since they did not improve the model significantly with only a marginal decrease in the BIC-value they were not withheld in the final model.

Results
To answer our first research question, we present descriptive statistics (means and standard deviations) based on the raw data of the students' answers to the seven items tapping into their perceptions of their teachers' approach to teaching.These items were scored on a Likert scale from 1 (never) to 5 (very often), with a neutral 3 (sometimes) in the middle.Table 2 presents results both at the items level and at the level of the construct.Overall the table shows that neither the holistic ESD approach to contents, nor the pluralistic approach to teaching are common-place in Swedish classrooms, with scores only moderately higher than a neutral 3.
The table also shows that students perceived the holistic approach to content as happening more frequently than the pluralistic approach to teaching.An item standing out from the crowd is the one dealing with the students' participation in the decision making of topics that will be covered in the classroom, students indicate that this rarely happens.The results also illustrate that both holism and pluralism occur-in the perception of the students-more frequent as students enter higher grades, with a general pattern of the lowest raw means in grade 6, and the highest in grade 12.This increase is also reflected in the mean scale scores for both constructs, and -as illustrated in Figure 1-the increase is statistically significant.2 indicate that holism and pluralism are not common practice in the Swedish classrooms, the standard deviations (resulting in coefficients of variation of up to 38%) illustrate that there are meaningful differences in the answers of the students.This variability allows us to use the data to answer the second research question.The results of the SEM analyses used to test the effect of holism and pluralism on the different outcomes at student level are reported in Figure 2. The standardized regression coefficients shown in this figure illustrate that both ESD dimensions have a positive effect at the student level.Small effects are observed of holism and pluralism on the sustainability attitudes of students: an increase of one standard deviation in either of the two ESD dimensions results in an increase of about 11% of a standard deviation in attitudes.Medium effects are observed, respectively, for the effect of holism on sustainability knowingness, and of pluralism in sustainability behavior.These effects represent an increase of about 22%-24% of a standard deviation in response to an increase of one standard deviation in the respective ESD dimension.

Figure 2.
Effects of holistic approach the contents and pluralistic approach to teaching on the sustainability knowingness, attitudes, and behavior of students in grades 6, 9, and 12.All estimates are significant at p < 0.01.Non-significant effects are not shown in the figure .Figure 2 presents the results of SEM analyses using data from grades 6, 9, and 12.To study the grade specificity of the results presented in this figure, we estimated models using only the data of the respective grades.The estimates for the standardized regression coefficients for each of the grades are presented separately in Table 3.The table furthermore presents the standardized regression coefficients of the quadratic terms of holism and pluralism.The quadratic variables were included in the models to test for non-linearity of the effects if significant, and where they provided a better description of the model for the data, they were withheld in the model and presented in Table 3.As can be seen in the table, the overall pattern shown in Figure 2 (an effect of holism on knowingness and an effect of pluralism on behavior) is confirmed for each of the grades, though the effect sizes differ.The effects on sustainability attitudes were not significant in the grade specific models, and therefore not included.The same overall picture was found when we ran models using the domain specific items only (ecological, social, and economic).While all these thematic models showed satisfactory fit to the data, the main focus of the current study in on the overall, holistic, approach of sustainability consciousness.
Overall, we see linear effects for holism on knowingness that increase in size with increasing grade.While in grade 12 we show a linear effect of pluralism on sustainability behavior, the effect is non-linear (quadratic) in grades 6 and 9.The quadratic term is positive in grade 6 (indicating a concave curve) and negative in grade 9 (indicating a convex curve).Figure 3 presents a graphic representation of the curves for grades 6, 9, and 12.These results provide an answer to our third research question as they show that the effects differ both in size and in shape across the grades.
Our final research question deals with the effect of schools' participation in ESD-certification programs on the extent to which students perceive that the holistic ESD-approach to content and the pluralistic ESD-approach occur happen in the classroom.To estimate these effects in each of the grades, we include a dummy variable (discriminating between schools that do and do not participate in ESD-certification programs) into the grade specific models that were built to answer research question three.Table 4 reports the estimates for the standardized regression coefficients of this dummy variable is each of the three models.As can be seen both in this table and in Figure 4, the certification has no effect on the perceived occurrence of either of the two ESD dimension in grade 6.In grade 12, the results show positive effects of participating in a certification program: a small increase in pluralism and a medium increase in holism.In grade 9, the dummy variable for certification has a negative effect, with students in certified schools reporting equal perception about the occurrence of holism but lower occurrence of pluralism in the classroom.Table 3. Grade specificity of linear and non-linear (quadratic) effects of holistic approach to contents and pluralistic approach to teaching on sustainability knowingness, attitudes, and behavior.All estimates are significant at p < 0.01, non-significant estimates are marked ns.

Discussion
The aim of the current study was to investigate the effectiveness of ESD, in the context of formal education, in terms of promoting the sustainability consciousness of adolescents.In addition we also investigated the effects of ESD-certifications.However, we deliberately deviated from common practice in studies on the effects of ESD certification-system, which is to connect the participation in such programs directly to outcomes at student level [51,54].As argued in the introduction, comparing the outcomes of students in schools that do and do not participate in such programs can uncover differences but not explain them.In our current study we therefore connected the students' perception regarding the extent to which ESD occurs in their classroom to student outcomes, regardless of whether their school participates in certification programs or not.This allows us to draw conclusions about the effectiveness of the ESD rather than of the participation in certification programs.To our knowledge, this is the first large-scale study to address the effectiveness of ESD in this innovative approach.
It is important to underscore here, that our current study also distinguishes itself from common practice in the field of sustainable education by focusing on the concept of sustainability consciousness [9,55], which brings together notions of knowingness, attitudes, and behaviors across the three dimensions of sustainable development (the environmental, social, and economic).In this study we furthermore developed a novel instrument to tap into the two essential traits of ESD as defined by UNESCO and scholars in the field of ESD research [13,21,29,30]: the holistic approach to contents and the pluralistic approach to teaching.Our results establish this measurement instrument as valid and reliable.Our results can, furthermore, be seen at two levels of interpretation.The first is of a descriptive nature and goes into current practices of ESD in Swedish schools.It describes the occurrence of ESD in classrooms in grades 6, 9, and 12, and goes into differences that are observed between schools that do and no not participate in ESD-certification programs.The second level of interpretation deals with the potential of ESD.It explores effects and their grade-specificity of holism and pluralism on the outcomes at student level.

The Occurrence and Effects of ESD
Our results show that neither holism nor pluralism occurs convincingly much in the perception of students.This fact questions whether ESD as a teaching approach has been really accepted and implemented by Swedish teachers during the UN Decade for Education for Sustainable Development.
The strong rhetoric at policy level [3,14] seems not to have large effects, in terms of adoption, on the practice of teachers.Overall, the holistic approach to contents occurs more frequent than the pluralistic approach to teaching according to our results.We do observe an increase of both holism and pluralism as the students responding to the questions are older, with the highest prevalence of both features in grade 12 and the lowest in grade 6.Moreover, though the UNESCO definition of ESD emphasizes that both features should go hand in hand, our results show that they are only mildly correlated.All in all, holism seems to be more common practice, without necessarily being connected to pluralism.If we look at the results concerning the effects of ESD on student outcomes, and follow Schultz's [4] reasoning that SD can only be achieved through behavioral change, the pluralistic approach is suggested as desirable.Indeed, while a more holistic approach to content results in increased sustainability knowingness it is a more pluralistic approach to teaching that results in more sustainability behavior.We need to recognize, though, that holism has-to a lesser extent-also a positive effect on the students' sustainability behavior.
The results from our structural equation model thus show that when teachers integrate the environmental, social, and economic dimensions of sustainability issues, as well as focus on their past, present and future, and on their local, regional and global nature, students gain a better understanding (i.e., and increased knowingness) of the complexity of SD.At the same time, as teachers invest more into teaching through pluralism, allowing for different viewpoints and opinions to be addressed, to critically reflect on what is being learned, and for students to participate in the decision making around which topics are addressed in the classroom, the students report more frequent sustainability behaviors.These results might seem straightforward, but up until now no empirical studies have been able to demonstrate the direct effects of ESD, and those who have attempted to do so have sometimes even shown negative effects [9].Our current results, and novel methodological approach, are therefore a clear step forward as they illustrate that ESD can make a difference.We thus contribute to answering the call in the DESD final report for an evidence base for efforts in the implementation of ESD [40].
A main conclusion of our findings is that ESD works.ESD as a teaching approach induces sustainability consciousness among the students.However, as described previously, the students seem not to experience pluralism to a very high degree indicating the difficulties in implementing this approach, one reason for this could be the normativity paradox, i.e., to teach ESD deliberatively but still strive for the predefined goal of SD, as discussed by Wals [39].Further studies investigating these difficulties are definitely called for.Moreover we do not know to what extent the students' increased sustainability consciousness helps them in taking action against structural barriers in society such as the culture of consumerism as discussed by Isenhour [5].It would be of great interest to confirm whether the self-reported behavior as found in this study is put into action in real situations.If not confirmed, the effectiveness of ESD could be challenged because if the students are incapable of putting their increased sustainability consciousness into action, SD is unlikely to be reached.
The critique raised against ESD approaches by Kopnina [37,38] suggesting that the focus on economic and social issues would lead students into anthropocentric world views turning them away from eco-centrism cannot be tested in a coherent way by our instruments.What we can say is that as students experience more holism in their teachers' approach to teaching, they also report a greater knowingness of environmental issues, and as they experience more pluralism they also self-report increased behavior in response to environmental issues.These results indicate that the hypotheses of Kopnina [36,37] are not supported by our results.

Grade-Specificity of Effects
We initially modeled the effects of ESD on student outcomes across our entire sample of students from grade 6, 9, and 12 simultaneously.The next step was to reconfirm the structural equation model for each grade separately and test the grade-specificity of effects.Our results show that each grade has a different slope for its effects.For the holism-knowingness effect, grade 12 students show the steepest slope, indicating that they benefit more from the same increase in holism than students in grades 6 and 9.The grade differences in the curves for the effect of pluralism on sustainability behavior are more complex, confirming the complex nature of the concept of pluralism.While for grade 12 our results show a linear increase in sustainability behavior as the students report that pluralism occurs more in their classroom, for students in grades 6 and 9, this relationship is non-linear.
In grade 6, our results show a concave curve (with a negative estimate for the β of the quadratic term) with the effect of pluralism on sustainability behavior increasing rapidly at first, then reaching a maximum, and finally turning negative (as can be seen in Figure 3B).This suggests that students in grade 6 initially adopt sustainability behaviors more quickly than their older peers, in response to more pluralism in the classroom.There is however a maximum for pluralism that these 11 and 12 year-old kids can handle.More pluralism in the classroom will result in less structured and more open approaches to teaching.Learning environments of this kind might be too demanding and novel for sixth graders, resulting in a drop in the effectiveness of pluralism in terms of promoting their adoption of sustainability behaviors.For grade nine students, we see an opposite shape for the effect of pluralism of sustainability behavior: the curve is convex (with a positive estimate for the β of the quadratic term).This suggests that more effort is needed to achieve a comparable effect of pluralism for 15 to 16 year old students.Often adolescents of this age are, within the context of formal education, perceived as difficult teenagers that are less motivated than students of other ages [65].A recent study on differences in sustainability consciousness among adolescents [66] confirms that indeed these students report the lowest knowingness, attitudes, and behaviors in relation to SD.Our current results confirm these findings, since at the mean values for the dependent variables shown in Figure 3A,B, the curve for grade 9 students is the lowest.At the same time, our results show that these adolescents should not too hastily be considered a lost cause or an audience too preoccupied with themselves to be open for ESD; the convex curve shows that while at first more effort is needed for pluralism to promote sustainability behaviors in grade 9 students, as the efforts increase so does the effect size, with grade nine students showing the highest reported sustainability behavior at the high end of the pluralism continuum.Rather than being regarded as a problematic group, these 15-16 year old students can thus be seen as a group with the potential to excel in sustainability behavior when their learning environment is highly pluralistic.
Overall, the grade specificity of the effects underscores that there is no single solution for ESD, since students from different grades respond differently to different intensities of holism and pluralism in the classroom.This confirms the complexity of ESD, and the importance of ESD implementation approaches designed specifically for the intended target audience.Overall, across the grades, we do observe positive effects of holism on sustainability knowingness and of pluralism of sustainability behavior.The earlier discussed moderate occurrence of holism and especially pluralism can therefore be seen as call to action for the teachers in Sweden, and the organizations that support them in developing their professional skills for tackling SD in the classroom.

ESD-Certification Programs
The most widespread of such organizations (both governmental and non-governmental) establish certification programs that often depart from an accountability perspective [52].These programs award schools with certificates if they implement SD and ESD into their daily practice.Several studies have compared schools that do and do not participate in such programs in terms of the students' outcomes in EE and ESD, but often they show no or even negative effects [9,51,52,54].Since these studies have not accounted the reasons behind the participation of the school in the certifications, it is difficult to generalize their results into statements about the effectiveness of EE and ESD.In the current study we estimate the effects of schools' on participation in certification programs on student perception on the occurrence of holism and pluralism in their classrooms.While for grade six, we observed no effect (equal holism and pluralism), we do see positive effects in grade 12: students in certified schools report more pluralism as well as more holism.The effects on holism is twice as substantial as that on pluralism, indicating that in grade 12 the participation in certification programs results more strongly in a classroom practice that shows increased attention for the interconnections between the environmental, social, and economic dimension of SD, the past, future and present, and the local, regional, and global nature of SD issues.The impact on pluralism is, though statistically significant, smaller: students indicate only a mild increase in the extent to which different viewpoints and critical reflections occur in the classroom, as well as that own opinions are encouraged and participation in decision making around what is being read in class.Again, if we consider Schultz's [4] central argument, than certification programs should focus more heavily on supporting schools and teachers in developing skills for a pluralistic approach to teaching, and to a lesser extent also for the holistic approach of ESD to dealing with content matter.Currently, our results based on the respondents from grade 12 show that, in Swedish schools, we are seeing the opposite effect, with increased holism rather than pluralism in the classroom.
Again, the students in grade nine deviate from this pattern; here we observed no increase in holism due to participation in certification programs, and a decrease in pluralism.Students in certified schools report that their teachers create a less pluralistic learning environment in their schools.This adverse effect of the programs might be connected to the argumentation above, that the common perception of grade nine students, is that they are a difficult age group to work with and they are pre-occupied with themselves more than students at any other age.As illustrated through the grade specific impact curves of the effects of holism and pluralism on student outcomes, this reasoning does match to our results, but as argued above, efforts to ESD, and especially pluralism in the ninth grade classroom have the potential to generate larger effects than in any other grade.Certifications programs, and ESD-effectiveness, would therefore benefit from including support of ninth grade teachers specifically in introducing or increasing a pluralistic approach to teaching.
We were unable to show effects of either of the ESD features on sustainability attitudes in the grade specific models.This might suggest that attitudes are not affected by ESD.More plausible, however, is that the lack of effects is a methodological artifact rather than a genuine finding.As suggested by Boeve-de Pauw, Jacobs and Van Petegem [67], attitude statements in the sustainability consciousness instrument might be too easy to agree with, resulting in both a ceiling effect and low variability among respondents.This combined with the drop in sample size when only data from students in a specific grade is used to estimate the structural equation model might explain the non-significance of effects of ESD on sustainability attitudes.More research and further development of the instrument seems needed to draw conclusions here.

Conclusions and Recommendations
Our study also revealed that the connection between the holistic approach of ESD in dealing with content matter and the pluralistic approach of ESD to teaching are in practice not as correlated as the UNESCO definition and research [21,29] proposes them to be.Future research efforts could focus on the effectiveness of ESD in approaches where the correlation between holism and pluralism are a dependent variable.A hypothesis in this context could be that in classrooms where holism and pluralism are connected more strongly and teaching and learning thus is more in line with the UNESCO definition of ESD, it is also more effective in terms of promoting students' sustainability consciousness.Future endeavors focusing on the effectiveness of ESD also need to consider other concepts than students' sustainability consciousness, such as time perspective [68], risk perception [69], self-determination [70], and self-efficacy [71].There is indeed a growing body of recent literature that supports the legitimacy of such concepts as important outcomes of ESD [72].
An important conclusion that can be drawn from our results is that they show empirical evidence for the effectiveness of ESD.To our knowledge this is the first time this has been done in a large scale study.As illustrated in the final report [40] the DESD has resulted in the launch of an increasing amount of educational initiatives under the flagship of ESD, a similar increase in research on their outcomes and effectiveness is, however, missing.Empirical studies are a missing link in the discourse around ESD, where decisions and implementation strategies are heavily based on policy recommendations and gut feelings by practitioners.The results we present in our current study are an important step forward to answering the call in the DESD final report and for the successful implementation of ESD.The measurement instruments we have used in this study could, if adapted for use in a self-evaluation context, prove valuable tools for schools and supporting organizations to monitor and steer efforts in the implementation of ESD.We can conclude that our novel approach to studying the effectiveness of ESD has produced valuable new insights.While our results establish that ESD has potential to be effective, they also show that there is no single fix-it-all solution, and that educational practice needs to be attuned specifically to the audiences they are targeted at.Our results thus confirm the key role that ESD can play in addressing SD and securing a sustainable future for the generations to come.

Figure 1 .
Figure 1.Grade differences in mean scores for holistic approach to contents and pluralistic approach to teaching.* marks significance, all at p < 0.01.

Figure 3 .
Figure 3. Grade specificity of effects for (A) holistic approach to content on sustainability knowingness; and (B) pluralistic approach to teaching on sustainability behavior.The blue lines represent grade 6, the green ones grade 9, and the brown ones grade 12.

Figure 4 .
Figure 4. Raw mean scores for holism and pluralism in schools that do and do not participate in ESD-certification programs.* marks significant differences (p < 0.05).

Table 2 .
Descriptive statistics (means and standard deviations) for the ESD scales and items.

Table 4 .
Effects of participation in the ESD-certification programs on perceived holism and pluralism in grades 6, 9, and 12. * marks significant differences (p < 0.05).