Next Article in Journal
Nā Kilo ʻĀina: Visions of Biocultural Restoration through Indigenous Relationships between People and Place
Next Article in Special Issue
Relationships between Student Personality Traits, Mobbing, and Depression within the Context of Sustainable Tourism Education: The Case of a Faculty of Tourism
Previous Article in Journal
Mixed Intercropping of Wheat and White Clover to Enhance the Sustainability of the Conventional Cropping System: Effects on Biomass Production and Leaching of Mineral Nitrogen
Previous Article in Special Issue
Observation of the Effect of Gender on Children’s Concept of Motion; Sustainability Issue

Sustainability 2018, 10(10), 3366; https://doi.org/10.3390/su10103366

Article
Critical Thinking Development—A Necessary Step in Higher Education Transformation towards Sustainability
Institute of British and American Studies, Faculty of Arts, University of Presov, 08001 Prešov, Slovakia
*
Author to whom correspondence should be addressed.
Received: 9 August 2018 / Accepted: 16 September 2018 / Published: 20 September 2018

Abstract

:
Education for sustainable development has been addressed by professionals, authorities, and a number of research studies in the last decades, and yet the results are not clearly visible. Teachers as agents of a new mindset do not seem to understand the ways of approaching raising awareness of the issues of global challenges. This study investigated the possibility of raising the extent of applying critical thinking to the problem-solving issues of a group of student teachers in their initial training. The sample consisted of 48 student teachers. Both qualitative and quantitative methods were used: namely, a questionnaire, content analysis, and focus group interviews. An archive of the students’ materials was also used for content analysis. In the data analysis, inter-rater reliability, as well as parametric (t-test) and non-parametric tests (Mann–Whitney U test) were applied. The results of the study provide evidence of the positive impact of the use of case studies in teaching pre-service teachers on the development of their critical thinking skills. It can be concluded that it should not be taken for granted that teachers automatically know how to develop the competencies that are necessary for sustainable development (SD). Instead, teacher education institutions need to incorporate thorough training that focusses on education for sustainable development (ESD) into the entire teacher programme in order to make sure that the teachers leave their initial training well prepared for guaranteeing Sustainable Development Goals (SDGs).
Keywords:
education for sustainable development; key competencies for sustainable development; critical thinking; reflection; teacher education

1. Introduction

Education for sustainable development (ESD) is not a new concept; it has been developed as part of a general sustainability discourse in which education plays an important role [1]. Teachers hold the key to the development of future generations, and need to master the tools for unlocking the potential of young people. However, this, requires the systematic training of teachers, since being able to function within ESD means not only becoming aware of specific critical issues connected to the future development of human survival, it also involves mastering key competencies [2] that are applicable to a wide variety of situations within which the learning process takes place.
In 2015, the 2030 Agenda for Sustainable Development was adopted by the United Nations (UN) General Assembly [3] and 17 Sustainable Development Goals (SDGs) were identified in order “to secure a sustainable, peaceful, prosperous and equitable life on earth for everyone now and in the future”, and they address issues that are being considered key barriers in sustainable development (SD) such as “inequality, unsustainable consumption patterns, weak institutional capacity, and environmental degradation” [2] (p. 6). What seems to be critically important in the 2030 Agenda is the general consensus that it is not merely knowledge and understanding that citizens need to have; they also need to acquire a set of key competencies that will enable them to actively participate in taking over responsibility for the world [2]. It becomes clear that the competencies that are highlighted in this document cannot be taught through individual actions, but instead need to be transversal and applicable in a variety of contexts and for a variety of purposes.
This paper focusses on selected key competencies identified within the concept of ESD—critical thinking—and the process of embedding it into initial teacher training courses. Teacher education represents a unique space for bringing about learning and raising the awareness of future educators of how important it is to develop these competencies with their learners. Since, as the document further states, these competencies cannot be taught, but they need to be learnt through active participation and experience [2], the initial training of teachers should incorporate opportunities for the future teacher to master the essence of generating conditions for their development. Critical thinking has been selected because of its wide applicability into the education system, and also because the education system very often supports the replication and memorisation of facts, which is knowledge without engaging students actively in the process of discovery.
Our concern regards the ability to apply a critical thinking approach to the problem-solving situations of student teachers who have not experienced such an approach themselves, and the possible improvement of this ability after an intervention based on critical thinking principles.

2. Key Competencies and Their Place in Teacher Education

The reasons for introducing the 2030 Agenda for Sustainable Development [3], which was adopted by the General Assembly of the United Nations in 2015, was to increase the pressure on educational institutions to modify their approaches and learning outcomes in order to secure addressing the global challenges faced by the whole of humankind today. Education has been a high priority for United Nations Educational, Scientific and Cultural Organization (UNESCO) since 1992, and it was brought into focus again in 2005 by the UNDESD (United Nations Decade on Education for Sustainable Development 2005–2014) declaration, with guidelines and recommendations for the implementation of ESD principles. However, it became clear that in order to address the complexity of these global issues, “new skills, values, and attitudes that lead to more sustainable societies” [2] (p. 1) need to be acquired. Organization for Economic Co-operation and Development (OECD) countries have also come up with an initiative [4] to support the growth for sustainability and identification of the competencies that hold key importance for handling future challenges. Rychen and Salganik [5] (p. 5) highlighted the importance of “noncognitive factors such as attitudes, motivation, and values”; however, these are not always appropriately addressed within the educational context. Cebrián and Junyent [6] explained that in order to foster sustainability skills, it is important to apply integrative and interdisciplinary teaching via problem solving, critical thinking, action competence, and systems thinking.
Taking into account that teachers should be the guarantors of the processes that facilitate the development of understanding and awareness of sustainability from an early age, they need to know how to create opportunities for their learners to acquire and develop the competencies that they will need to take an active role in the world. Therefore, it is the task for the teacher education institutions to equip student teachers with the skills that are necessary for a competency supportive approach.
Competency in general can be understood as an important skill that is needed for a particular job (https://dictionary.cambridge.org/dictionary/english/competency). Barth et al. [7] characterised competencies as “dispositions to self-organisation, comprising different psycho-social components, existing in a context-overlapping manner, and realising themselves context-specifically” (p. 417). Similarly, Simonton [8] saw them as skills constituting an essential component of certain performance or achievement in connection to a specific domain, and Klieme, Hartig, and Rauch [9] (p. 9) described them as “context-specific cognitive dispositions that are acquired by learning and [are] needed to successfully cope with certain situations or tasks in specific domains.”
However, as Barth et al. [7] claimed, there is no absolute agreement on what a key competencies list should look like, and over the last decades, the understanding of competencies and their role in sustainable education varied [10,11,12,13]. UNESCO, in their new document Education for Sustainable Goals: Learning Objectives (2017), made it clear which competencies they considered to be key ones. As they stated, these should “include cognitive, affective, volitional and motivational elements” [2] (p. 10), which means that they do not only cover the learning outcomes on the cognitive level. As Rychen and Salganik [5] (p. 1) proposed, “measur[ing] traditional notions of academic achievement and skill development, such as reading and mathematics skills” can be based on practical considerations; these skills will empower people in gaining social and economic success. Wiek et al. [14] came up with a list of suggested key competencies in 2011, and they made a clear distinction between key competencies (systems thinking, anticipatory, normative, strategic, and interpersonal), and what they addressed as regular academic competencies such as critical thinking, communication, pluralistic thinking, data management, etc. However, they implied that these competencies should serve as the foundation of academic sustainability education [14].
Based on the first proposals, UNESCO suggested the following set of key competencies to be applied in any sphere of education:
  • Systems thinking competency: the ability to recognise and understand relationships; to analyse complex systems; to think of how systems are embedded within different domains and different scales; and to deal with uncertainty.
  • Anticipatory competency: the ability to understand and evaluate multiple futures—possible, probable and desirable; to create one’s own visions for the future; to apply the precautionary principle; to assess the consequences of actions; and to deal with risks and changes.
  • Normative competency: the ability to understand and reflect on the norms and values that underlie one’s actions; and to negotiate sustainability values, principles, goals, and targets, in a context of conflicts of interests and trade-offs, uncertain knowledge, and contradictions.
  • Strategic competency: the ability to collectively develop and implement innovative actions that further sustainability at the local level and further afield.
  • Collaboration competency: the ability to learn from others; to understand and respect the needs, perspectives, and actions of others (empathy); to understand, relate to, and be sensitive to others (empathic leadership); to deal with conflicts in a group; and to facilitate collaborative and participatory problem-solving.
  • Critical thinking competency: the ability to question norms, practices, and opinions; to reflect on own one’s values, perceptions, and actions; and to take a position in the sustainability discourse.
  • Self-awareness competency: the ability to reflect on one’s own role in the local community and (global) society; to continually evaluate and further motivate one’s actions; and to deal with one’s feelings and desires.
  • Integrated problem-solving competency: the overarching ability to apply different problem-solving frameworks to complex sustainability problems and develop viable, inclusive, and equitable solution options that promote sustainable development, integrating the above-mentioned competences [2] (p. 10).
However, the application of an approach developing the above listed competencies requires new paradigm thinking about the role of education and the roles of individual participants [15]. The key role of the teacher needs to be in recognizing the potential abilities of the learners and facilitating the growth of understanding the values embedded in ESD. As Connel et al. [16] highlighted, the development of competencies depends on the interaction between individual potential abilities and actual experience in a domain.

2.1. Reflection and Critical Thinking in Teacher Education

Teacher education in different countries covers a wide variety of areas within a variety of forms. It includes lectures, seminars, teaching practice, tutorials, etc., with the main aim of allowing student teachers to get as close to their future profession as possible. Teacher training programmes in general try not to offer only a one-way transfer of knowledge or examples of good practices with the hope that student teachers would simply memorise what needs to be done in a particular situation.
Student teachers need a lot space for thinking, experimenting, reflecting on past experience, conceptualising, and planning in order to become as functional and effective as they possibly can. However, the process of internalising new facts goes through a fixed system of schemata where they—consciously or not—choose what resonates with their inner principles. This decision-making process is operated by the system of beliefs that they have acquired in their life so far. The practices they have experienced have become models that work, whether they as learners were satisfied with them or not. Borg [17] stated that a belief is not necessarily a concept we are aware of; it can be held even unconsciously. However, it is usually accepted by the person as true, and as such, it plays an important role when deciding how to act or understand certain ideas. Belief systems seem also to be crucial in decision making [18,19,20], because of the time spent within the educational system as a learner—i.e., as a person being educated, as a receiver of the treatment—in comparison to the time spent in teacher education. Richards and Lockhart [18] illustrated this in numbers, comparing 3060 days of observation as a learner to 75 days of classroom experience during teacher preparation programmes.
The way of “deconstruction” of already existing system of beliefs in order to open the doors to new principles is rather demanding and complicated by student teachers needing to first become aware of their underlying guiding principles, and then critically evaluate their habitual expectations, and act upon revised viewpoints [21]. One of the ways that they can start searching for the reasons why they did something in class or why they chose a certain approach in solving certain problems, is to apply reflection [22,23] with the aim of disclosing the reasons for certain actions and enabling student teachers to learn from their own experience [24]. Even though reflective practice has many definitions and specifications [25], and would even vary from institution to institution, there has been an agreement that the role that reflection plays in teacher education is irreplaceable, and reflection as a skill is one of the most important competencies that student teachers need to master [26,27].
At the same time, the development of reflection corresponds with the development of critical thinking competencies. By becoming aware of one’s own thoughts, feelings, or motifs, student teachers become open to understanding their learners’ viewpoints and transfer the approach of applying reflective thinking into their own classes. This goes hand in hand with the development of critical thinking and “the ability to question norms, practices, and opinions; to reflect on own one’s values, perceptions, and actions; and to take a position in the sustainability discourse” [2] (p. 10). Attention to the development of both reflective skills and critical thinking seems to be crucial, since as Paul and Elder [28] very rightly suggested, teachers are only able to develop their learners’ critical thinking to the extent that they themselves are able to apply it. Their understanding of critical thinking as “the process of analysing and assessing thinking with a view to improving it” [28] (p. 7) indicates that the evaluation and rethinking of viewpoints or even belief systems are the key principles in order to succeed in the act of moving forward towards new understandings of concepts and overriding quick associational judgements or automatic, reactive decision-making systems [29].
In order to support education for sustainable development, it seems absolutely crucial to make student teachers aware of ways that they can become open to the evaluation of other viewpoints, and moreover, how important it is to apply such approaches in their own teaching. Critical reflection as a skill needs to be developed from the early years of schooling, since it appears to be an irreplaceable tool for understanding how people are “conditioned and confined by the socio-cultural structures they are operating in” [30] (p. 101). Pedagogical approaches allowing the processes should involve students as agents of personal change in order to empower them as future agents of change in the broader context. Teacher education usually creates some space for student teachers to immerse themselves into the process of reflection, whether through individual tasks or collaboration. One way of supporting these tasks can be the use of tools, such as a portfolio, which in its nature engages student teachers in evaluation and self-evaluation through reflecting on past experience and planning for realistic goals [31]. However, the reflection should be used systematically during the entire training, since as Gün [32] indicated, the incidental flashbacks would not lead towards desirable outcomes, and would generate only vague feelings where what we need is some evidence and clear, tangible data. These could be later used to support argumentation and the disclosure of inner motifs of particular acts. It is necessary to encourage students to “focus on the content of the justification, dialogue and transformative learning” in their reflective writing [33], what more student teachers need to build their understanding of new ideas and conclusions on clear evidence rather than a vague feeling of previously accepted behaviours.

2.2. Critical Thinking as One of the Key Elements for Higher Education Transformation towards Sustainability

Filho [34] concluded that “if children and adults are to develop interdisciplinary, systems-based knowledge of the natural and built environments and the skills to participate actively in developing a sustainable society and economy, education for sustainability should be infused into more subject areas and at all grade levels”. Lozano et al. [35] claimed that “incorporating some material or creating a standalone introductory SD course could appear as a relatively simple starting point for institutions. However, such steps tend to result in the students learning and studying for that particular course, but not being able to integrate SD principles into their professional life.” They [35] mention several approaches for how to incorporate sustainability into the higher education curricula, e.g., coverage of some environmental issues into an existing course, developing a specific sustainability e-course, and developing an undergraduate or a graduate programme for sustainability and others.
Change in higher education towards sustainability is not an easy task [36]. SD can be learnt in a linear way, but it is crucially important that learners understand and accept it, i.e., they have to critically perceive it. Lozano et al. [37] found that “the implementation of SD in higher education institutions (HEIs) has been compartmentalised and not holistically integrated throughout the institutions”. Thomas [38] discussed how important it is to give learners the opportunity to learn to think, i.e., to learn “how to think” rather than “what to think”. This is especially important in the groups of pre-service teachers, as those graduates will pass this experience and skill onto their learners. Introducing models for ESD implementation, Lang (in Thomas [38]) defined critical thinking and reflective learning as one of the key components that allow for transformational change. Ceulemans and De Prins [39] described a manual and method for the integration of SD in higher education curricula, and it can involve both in-service and pre-service teachers.
Filho et al. [40] claimed that transformation in higher education towards sustainability should encourage interdisciplinary and transdisciplinary approaches, the integration of theory and practice, the individual commitment and development of synergic actions in groups, ethical discussions and reflections, and the adoption of critical thinking. They see, similarly as Lang [38], critical reflection as the key concept in transformational learning. Filho [34] stressed that “Education for Sustainability (EfS) is meant to be a lifelong learning process that leads to an informed and involved citizenry having the creative problem-solving skills, scientific and social literacy, and commitment to engage in responsible individual and cooperative actions“.

3. Materials and Methods

3.1. Background of the Study

In Slovakia, it is obligatory at elementary and secondary schools to integrate so-called cross-curricular topics in their curricula; they should be an integral part of the courses. The topics cover areas such as financial literacy, environmental education, multicultural education, media education, personal and social development, the protection of health and life, traditional culture, transport and traffic education, project writing and presentation skills, etc.
The research focused on integrating global issues integration into teacher training study programmes in Slovakia, and found that the significance and importance of education for sustainable development (ESD) is not integrated in teacher training curricula. Discussions with teacher trainers and students uncovered a relatively very low level of understanding of the issue [41].
In the last decade, content and language integrated learning (CLIL) has become popular in Slovakia, and the integration is mostly done by language teachers.
If teachers have to teach SD, they have to understand and believe it. Understanding principles of the three pillars of sustainability—environment, economy, and society—is important to create conditions for global change in opinions about the issue. In this sense, we believe that currently, it is especially the group of pre-service teachers who have to understand its importance and gain the knowledge so that they can transfer it into their teaching.
This study focusses on the possibility of increasing the extent of critical thinking execution in pre-service teacher training within the use of case study analysis. It appeared to be a suitable tool for the development of critical thinking in student teachers, and the systematic use of this tool was anticipated as a possible opportunity to intensify the extent of use of this key competence. We understand higher-order thinking skills to be necessary abilities and capacities for the teachers in practice who have to solve problems ad hoc every day, considering the consequences of their decisions and actions.
Case study analysis was introduced as a regular part of the teacher training programme in the academic year 2017/18. All of the participants (n = 48) were students of the English language teacher training programme at the University of Presov in Slovakia.

3.2. Design

A mixed-method approach was applied. Qualitative data on student teachers’ reflections in the reflective parts of the teaching practice portfolios were gathered during two semesters. Data gathered from reflections from teaching practice were analysed with respect to description, evaluation, justification, dialogue, and transfer [33]. Scores from the critical thinking questionnaire [42] (pp. 98–99) presented the quantitative data and were related to students’ reflections. Focus group interviews were used to explore and obtain more knowledge and evidence on students’ attitudes towards using case studies as a tool for critical thinking development, and at the same time look for more information that could help the researchers interpret data gathered from the critical thinking questionnaire. The main research questions were:
  • What is the level of students’ awareness and beliefs about critical thinking?
  • Does the application of case studies in teacher training programmes increase the critical thinking skills of pre-service teachers?
  • How do students perceive their critical thinking skills before and after the intervention?

3.3. Sample

The sample consisted of:
  • Forty-eight English language student teachers (undergraduates in 2016/17) from the University of Presov in Slovakia. Students were sampled when the opportunity arose. They were aged between 21–28 years, and there were 39 females and nine males. The students were in the first and second year of their Master’s studies.
  • Forty-eight teaching practice reflection portfolios that comprised reflective writings of 2016/17 English language major students from the University of Presov in Slovakia.
  • Archived teaching practice portfolio reflections that comprised reflective writings (between 2011 and 2016) where students reflected on their teaching practice individually. Students wrote teaching reflections after the first and second semesters. Out of each type, 75 pieces were randomly selected. This is the material where students were expected to use higher-order thinking skills, such as critical thinking, not only to describe the lessons they taught, but also to analyse the lessons and suggest ways of improving their teaching.
  • Ninety-six case studies analysis (gathered in 2017/2018).

3.4. Instruments

The Critical Thinking Toolkit (CriTT) [42] (pp. 98–99) consisting of 27 items with a 10-point scale (1—strongly disagree, 10—strongly agree; ordinal data) with three factors, namely confidence in critical thinking (17 items), valuing critical thinking (six items), and misconceptions (four items) specified was used to explore students’ awareness and beliefs about critical thinking. It is important to stress that the CriTT measures beliefs and attitudes about critical thinking. The confidence in critical thinking factor measures the respondent’s confidence in critical thinking; the valuing critical thinking factor measures the extent to which the respondent distinguishes the importance of critical thinking; the third factor, misconceptions, measures the avoidance of critical thinking and/or misconceptions of critical thinking.
Quantitative content analysis was applied to analyse students’ reflections on their teaching practice. Coding categories for evaluating reflections in teaching practice portfolios were applied based on the study realised by Poldner et al. [33]. Two researchers were involved to increase the internal validity and objectivity in coding the responses and data interpretation. The statistical tests (inter-rater reliability) to test the agreement between the raters were run individually for the results before intervention and after intervention measurements. The inter-rater agreement was 0.742 before intervention, and 0.786 after intervention in evaluating reflections in teaching practice portfolios. Values between 0.75–0.99 were considered to be excellent, and values between 0.50–0.74 were considered to be good.

3.5. Procedure

All of the first-year master students take two semesters of an obligatory English as a foreign language (EFL) methodology course with teaching practice. Each student analysed six case studies during two semesters (three case studies per semester). During semesters, students discussed and analysed different case studies (based on topics related to the syllabus, including short-term and long-term planning, interaction and rapport, misbehaviour, learner autonomy, using authentic materials, and teaching learners with student educational needs). Students had time to read and think about each case in advance. During the lessons, they were given a set of questions that they should consider. Various interactional patterns were applied; however, they mostly worked in groups with a teacher monitoring them and facilitating the process. Class discussions were usually moderated by the teacher. For homework, they had to individually write a case study report with three compulsory parts, namely: (1) context description; (2) available alternatives (their strong points and threats); and (3) their decision and recommendation.

4. Results

Data management and analysis were performed using IBM SPSS Statistics version 23, and significance levels were set at the 5% level.

4.1. Critical Thinking Questionnaire

Participants in the sample filled in the Critical Thinking Toolkit [42] (pp. 98–99). The term English Language Teaching was used in question number four, where the specification of discipline had to be added at the beginning of the research. The three areas of critical thinking, valuing critical thinking, and misconceptions were evaluated individually. We expected that students would express a neutral attitude generally with a lower value (below five) for critical thinking and a higher value for valuing confidence in critical thinking (above five). The values reached in all three categories were surprisingly similar. The sample distribution was normal (for all three variables). The basic descriptive statistic information (the median, mode, and standard deviation for all three variables and total results) is summarised in Table 1.
The following tables (Table 2) bring the frequency tables for all three factors—confidence in critical thinking, valuing critical thinking and misconceptions—that were calculated as average values, as there was not an equal number of questions per category, which means that the totals per category were not comparable.
The range of values for different categories was interesting. The smallest range was in the category of confidence in critical thinking (1.47), and the biggest range (3.5) was reached in the category of misconception. The box and plot graphs (see Figure 1) display the range, minimum, and maximum values, as well as quartiles.
The data indicate that the group was not homogeneous concerning avoidance of critical thinking. On the other hand, they were homogenous concerning their confidence in their critical-thinking abilities.

4.2. Teaching Practice Reflection

To increase the objectivity of the evaluation and reliability of the measures, two raters evaluated the teaching practice reflections (that were a part of the teaching practice portfolio with the set standard form). To assess the consistency of measurements made by raters, inter-rater reliability was calculated by taking an average of both raters’ measurements. The coders applied the categorisation according [33] to the following categories and subcategories: Description (Description of Pupils’ Actions, Description of Student’s Actions, Description of Context of Situation, Description of Pupils’ Goal, Description of Student’s Goal or Task, Description of Feedback Mentor, Description of Student’s Opinion); Evaluation (Evaluation of Students’ Action, Evaluation of Students’ Goals, Evaluation Without Content, Evaluation of Pupils’ Results and Behaviour, Evaluation of Mentor, Evaluation of Students’ Expectations); Justification (Justification of Student’s Action, Justification of Pupils’ Goals, Justification of Student’s Goals, Justification of Student’s Choice of Situation, Justification of Student’s Own Opinion); Dialogue (Dialogue with Student Mentor, Dialogue with Student with Theory, Critical Reflection of Student′s Assumptions or Actions); Transfer (Transfer Student’s Specific Learning). First, the frequencies were calculated, and then, they were recalculated to percentages (as a ratio of all of the instances that were found in individual reflections). We were interested in the total number of items and issues recorded by students and the number of recorded items per category. The description was evaluated as the category where students did not need to apply critical thinking skills, whereas transfer was considered as a category that needed critical consideration and logical conclusions.
The statistical tests were run individually for all of the categories. The paired sample t-tests were conducted for three variables (description, evaluation, justification) after checking the normal distribution of variables and confirming the null hypothesis. The following Table 3 shows the main characteristics of the variables description, evaluation, and justification before (des_PER, Eval_PER, Just_PER) and after the intervention (Des_PER2, Eval_PER2, Just_PER2).
Studying the data before and after intervention, a two-tailed paired samples t-test revealed the number of description before intervention (m = 55.487; s = 7.716) and after intervention (m = 50.027; s = 4.021), t (74) = 5.237, p ≤ 0.05, which was the expected result. An increase was observed in the category evaluation, which was again an expected result. (m = 36.560, s = 6.289 before intervention and m = 41.913. s = 4.178; after intervention, t (74), s = 6.420). A statistically significant difference was not observed in the variable justification (p = 0.397).
The Wilcoxon signed ranked test (see the Table 4) was applied in the categories for dialogue (Dia_PER) and transfer (Transf_PER) as a non/parametric alternative to one sample t-test. Both tests resulted in confirming the null hypothesis (as in both cases, p ≥ 0.05).
The results indicate there is no statistically significance in the difference between the variables dialogue and transfer; still, we observe a slight increase in both categories.
To elucidate the effectiveness of the case studies on critical thinking, we separately conducted an analysis of 2017/18 students’ reflections, i.e., reflections of those students who during their EFL course worked with case studies. During the first measurement, they reached similar values as their older colleagues. However, comparing the results of the second measurement, in the case of the 2017/18 students, we observed a statistically significant difference in all five categories (see the following Table 5), and the gains in those categories were bigger.
The parametric paired samples t-test (see Table 6) was run for the variables with normal distribution in the evaluation category.
A comparison of two groups (archived reflections and 2017/18 reflections) was necessary to study the possible influence of introducing a case study in our context. We ran the tests to compare the data gathered ((1) ex-students who did not work with case studies; and (2) the group of 2017/18 students who worked with case study during their EFL classes).
The test results indicate that there is a statistically significant difference between the 2017/18 students and 2011–2016 students in three variables, namely: evaluation, justification, and dialogue (the differences are highlighted in Table 7).

4.3. Focus Group Interviews

Thirty participants of the study sample (out of 48) were randomly divided into three focus groups (10 participants each). The interview schedule comprised structured and open questions to identify and explore students’ attitudes towards using case studies as a tool for the development of their critical thinking, and its strong and weak points in connection to teaching practice. Moreover, the researchers tried to examine whether a change in their thinking occurred and whether this change can be attributed to the application of case studies. All three focus groups lasted up to 60 min. The matrix protocol was used and further analysed.
The recorded and transcribed data were divided into several categories: Expectations, Experience, Shift, and Perceived Efficacy. The choice of the categories was given by the need to identify student teachers’ own perception of their critical thinking competency and their understanding of the importance of being able to apply a critical thinking development approach in their own classes, and at the same time, the categories were focused on the process and transformation of their own competency.
In the category of Expectations, the student teachers expressed initial confusion and a rather low level of interest at the beginning of the research. Some even admitted that they were worried that it would complicate the accomplishment of the methodology course. The reason for this worry was stated very clearly: it was the lack of prior experience with tasks focusing on any kind of analysis conducted in the past. Student teachers were not able to define clearly what was meant by critical thinking; they just had blurry ideas, but were not able to speak about it with confidence. They also admitted they had never heard about education for sustainable development, and could not figure out any possible role that they could play in it in the future.
The category of Experience aimed at finding out student teacher’s perceptions about the process of going through the systematic training in applying critical thinking for case studies. Student teachers commented on their first attempts, and described them as poor attempts to describe the problems stated in the case studies without deeper immersion. When they compared their first results and the last results, they stated that they were more proud of the results especially when they compared their own analysis and the analysis of other students. Collaboration was highly appreciated by the respondents, and they admitted that they would probably not reach such good results without collaborating with their peers. In general, they all appreciated the experience, and highlighted that the results of this experience are transferable into other subjects and areas, and they expressed their belief that they had acquired important skills for their future profession.
Very similar results were gathered in the category of Shift, where the aim was to identify what exactly changed in how student teachers approached the tasks. Student trainees in unison agreed that their thinking has changed over the period of the intervention. Due to the step-by-step guidance of the trainer in the initial phase of the intervention, they started to understand how they needed to approach the task, and later they were able to do it on their own. Most of the respondents stated that they find it as their “second nature” now, and do it automatically. Some of the respondents added that they even noticed that they had a tendency to apply it in their own classes during the teaching practice.
Last but not least, the category of Perceived Efficacy supported the above stated data by suggesting that the student teachers feel more oriented in how they would develop the critical thinking of their own students, and that the concept of education for sustainability with the development of key competencies was clearer in its substance. However, this category also revealed that the training provided is not sufficient, and the student teachers expressed the desire to receive more focused instruction on how to support ESD. Even though they understood the core of the idea, they still felt very limited in regard to how they could contribute to this development as future professionals.

5. Discussion

The results of the critical thinking questionnaire indicated that students perceived their critical thinking skills neutrally or slightly positively. The same can be said about how they valued critical thinking skills. The questionnaire also brought important feedback about the study programme and teachers’ demands on students, especially regarding item number five (“Critical thinking is essential in higher education”), number nine (“You cannot get a good degree without good critical thinking skills”), item number 16 (“All my lecturers expect me to think critically”), and number 27 (“I think critically in lectures”). Students in our sample reached the value six in item numbers five, 5.33 in item number nine, 4.54 in item number one, and only 4.29 in item number 27. Frequently, teachers complained about the not very active approach of students, who had low autonomy, but on the other hand, students felt that they did not need or were not expected to think critically, or offer their own opinion or interpretations. The highest score was reached in item number 25, which could be because all of the students studied language, and thus were trained in reading techniques and strategies.
The focus group discussions analysis allowed us to study results in depth. Most students (22 out of 30) claimed that they frequently simply copied what was on the board/screen during the lectures, and teachers do not force them to interact during the lectures. There is a number of teachers who make the text of lectures available (either by email or they upload them to the learning management system LMS Moodle). A number of students claimed that this allowed them to follow the lectures more actively without the need to make long notes, especially if the texts were available in advance. However, there was a group of students who claimed that the texts were almost the only source that they used for their study, and they could pass with lower grades.
Participants positively accepted the use of case studies; frequently, they used the words “helpful”, “useful”, “authentic”, and “beneficial” to describe it; on the other hand, they sounded sceptical, and expressed fear that each situation in real life is different and “schemes do not work”. They appreciated that they were “led to consider alternative and possible impacts”, and they claimed that it offered them the possibility to “realise how many factors have to be considered” in each situation.
As to the analysis of teaching practice reflections written in the period 2011–2016, the mean difference in the category description was 5.40 with t = 5.237, and based on the two-tailed sigma, we reject the null hypothesis and suggest that there is a statistically significant difference as to the category of description. To be more specific, the number of descriptive instances was lowered, which was expected. A decrease in all but two subcategories was observed. The two subcategories dealt with students (opinion and their choice of situation). This finding can be evaluated positively, as students’ focus moved from learners to students. A statistically significant difference was also observed in the category evaluation, where the increase of the mean was 5.35. The three other categories did not change significantly after the intervention.
The same analysis was realised with the teaching practice reflections written in the period 2017/2018, and the first measurements brought similar results (compare Table 2a, Table 3a and Table 4a, Table 5a). However, in the second case, we found a statistically significant difference in all five categories.
Comparison of both groups (see Table 7) indicates there is a statistically significant difference in three categories, namely, evaluation, justification, and dialogue. We understand this result to be very important, as it indicates that the application of case studies led towards more evaluative reflection with a focus also on the justification of learners’ and student’s action, goals, dialogue with mentors, and theory, but also teacher training. We perceived that students in the focus groups discussion expressed that they could feel the change in the way they thought about their teaching, and they attributed this change to a large extent to case studies analysis during the semesters.
It had been already mentioned that case study pedagogy/methodology has been applied in various jobs’ preparation. Our study results are comparable to the findings e.g., of Huang et al. [43], who found that case studies as a hospital-based teaching strategy encouraged the development of critical thinking skills and developed the dispositions of nurses. Duffy and Stone [44] presented similar results when their experiment results (they used the California Critical Thinking Skills Test to measure different domains of critical thinking) proved the development of deduction and interference skills, as well as recording better time scores after using case-study learning.

6. Limitations

Further data collection is required to determine exactly how case study methodology affects the critical thinking of pre-service teachers. The study itself has several limitations. The sample came from only one university and study programme, limiting the feasibility of the generalization of results to graduates of other teacher training programmes. That it is a longitudinal study is in a certain way a limitation in itself, as there are various other factors (uncontrolled variables) that might have influenced the results. To increase the data reliability, focus group discussion was applied to evaluate the possible causes of the change. Naturally, in accordance with Cohen et al. [45] (p. 179), it needs to be acknowledged that qualitative data such as those presented in this study do carry a certain degree of bias as to “the subjectivity of respondents, their opinions, attitudes, and perspectives”. However, this, meant that the data were not fully anonymised, which might have influenced students’ reactions and answers (even though they were assured that it could not influence their study results). The replication of the study can also bring further validation of the results.

7. Conclusions

This study attempted to measure the possible impact on the level of critical thinking applied in a teacher training programme by student teachers before and after intervention based on case study analysis. The study was based on the premise that teachers play an extremely important role in the process of achieving SDGs in education. As suggested by other research studies [46,47,48], teacher education needs to respond to the changing world by acknowledging SDGs and incorporating them into programmes with an interdisciplinary approach as well as insisting on the approaches that train student teachers and help them develop those key competencies that they are later going to apply in their own classrooms.
Student teachers in the study underwent an intervention in which they intentionally focused on their own critical thinking development and expressed their personal viewpoints in focus group interviews after the intervention was completed. The results of the study suggest that case study analysis develops critical thinking; it especially fosters and facilitates evaluative, reflective, and self-reflective skills. Those skills are especially important in the teaching profession, and students realising its strength gives us hope that they will advance this experience in their own teaching practice.
As can be concluded from the focus group interviews, the student teachers stumbled over the tasks focusing on a certain level of analysis, and they admitted that they had a tendency to approach the tasks without much thought; instead, they applied the steps to which they were accustomed. With the trainer’s intervention after the individual stages of the case study analysis, they were forced to investigate within the community the tools for approaching the task differently. What seems to be most promising, based on the results of the study, is the change that the participants experienced and described in the interviews. The last stage of the case study analysis was handled with much more ease in comparison to the first encounter with the task. Yet, it has to be admitted that this easiness does not necessarily guarantee that student teachers will automatically apply those approaches in their own classrooms with their students. However, it can be suggested that by a clear focus within the teacher training programme and with the aim of raising awareness, future teachers can be much better equipped for their profession with a positive change in their belief system.

Author Contributions

The authors have contributed equally to this work.

Funding

This research was funded by research project KEGA 065PU-4/2016.

Acknowledgments

The authors would like to express their gratitude to the anonymous reviewers for their valuable and constructive comments on an earlier version of this paper that greatly contributed to improving its final version.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Evans, N.; Stevenson, R.B.; Lasen, M.; Ferreira, J.-A.; Davis, J. Approaches to embedding sustainability in teacher education: A synthesis of literature. Teach. Teach. Educ. 2017, 63, 405–417. [Google Scholar] [CrossRef]
  2. United Nations Educational, Scientific and Cultural Organization (UNESCO). Education for Sustainable Development Goals: Learning Objectives; UNESCO: Paris, France, 2017; Available online: http://unesdoc.unesco.org/images/0024/002474/247444e.pdf (accessed on 1 July 2018).
  3. The United Nations (UN). Transforming Our World: The 2030 Agenda for Sustainable Development; UN: New York, NY, USA, 2015; Available online: http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E (accessed on 5 June 2018).
  4. Organization for Economic Co-operation and Development (OECD). Definition and Selection of Competencies (DeSeCo). 2016. Available online: http://www.oecd.org/education/skills-beyond-school/definitionandselectionofcompetenciesdeseco.htm (accessed on 21 July 2018).
  5. Rychen, D.S.; Salganik, L.H. (Eds.) Key Competencies for a Successful Life and Well-Functioning Society; Hogrefe & Huber Publishers: Ashland, OH, USA, 2003; ISBN 0-88937-272-1. [Google Scholar]
  6. Cebrián, G.; Junyent, M. Competencies in Education for Sustainable Development: Exploring the Student Teachers´ Views. Sustainability 2015, 7, 2768–2786. [Google Scholar] [CrossRef]
  7. Barth, M.; Godemann, J.; Rieckmann, M.; Stoltenberg, U. Developing key competencies for sustainable development in higher education. Int. J. Sustain. High. Educ. 2007, 8, 416–430. [Google Scholar] [CrossRef]
  8. Simonton, D.K. Expertise, competence, and creative ability: The perplexing complexities. In The Psychology of Abilities, Competencies, and Expertise; Sternberg, R.J., Grigorenko, E.L., Eds.; Cambridge University Press: New York, NY, USA, 2003; pp. 213–238. ISBN 978-0-521-00776-4. [Google Scholar]
  9. Klieme, E.; Hartig, J.; Rauch, D. The Concept of Competence in Educational Contexts. In Assessment of Competencies in Educational Contexts; Hartig, J., Klieme, F., Leutner, D., Eds.; Hogrefe & Huber Publishers: Ashland, OH, USA, 2008; pp. 3–22. ISBN 978-0-88937-297-9. [Google Scholar]
  10. Burke, J.W. Competency Based Education and Training; The Farmer Press: London, UK, 1989; ISBN 1-85000-1. [Google Scholar]
  11. Bowden, J.; Marton, F. The University of Learning: Beyond Quality and Competence in Higher Education; Kogan Page: London, UK, 1998; ISBN 0-7494-2292-00. [Google Scholar]
  12. Sipos, Y.; Battisti, B.; Grimm, K. Achieving transformative sustainability learning: Engaging heads, hands and heart. Int. J. Sustain. High. Educ. 2008, 9, 68–86. [Google Scholar] [CrossRef]
  13. Willard, M.; Wiedmeyer, C.; Flint, R.W.; Weedon, J.S.; Woodward, R.; Feldman, I.; Edwards, M. The Sustainability Professional: 2010 Competency Survey Report; A Research Study Conducted by the International Society of Sustainability Professionals; ISSP International Society of Sustainability Professionals: Pittsburgh, PA, USA, 2010; Available online: www.sustainabilityprofessionals.org/sites/default/files/ISSP%20Special%20Report_3.10_final_0.pdf (accessed on 22 June 2018).
  14. Wiek, A.; Withycombe, L.; Redman, C.L. Key competencies in sustainability: A reference framework for academic program development. Sustain. Sci. 2011, 6, 203–218. [Google Scholar] [CrossRef]
  15. Wals, A.E.J. Sustainability in higher education in the context of the UN DESD: A review of learning and institutionalization processes. J. Clean. Prod. 2014, 62, 8–15. [Google Scholar] [CrossRef]
  16. Connell, M.W.; Sheridan, K.; Gardner, H. On Abilities and Domains. In The Psychology of Abilities, Competencies, and Expertise; Sternberg, R.J., Grigorenko, E.L., Eds.; Cambridge University Press: New York, NY, USA, 2003; pp. 213–238. ISBN 978-0-521-00776-4. [Google Scholar]
  17. Borg, M. Teachers’ Beliefs. ELT J. 2001, 55, 186–187. [Google Scholar] [CrossRef]
  18. Richards, J.C.; Lockhart, C. Reflective Teaching in Second Language Classrooms; Cambridge University Press: Cambridge, UK, 1996; ISBN 978-0521458030. [Google Scholar]
  19. Woods, D. Teacher Cognition in Language Teaching; Cambridge University Press: Cambridge, UK, 1996; ISBN 0-521-49788-4. [Google Scholar]
  20. Williams, M.; Burden, R.L. Psychology for Language Teachers. A Social Constructivist Approach; Cambridge University Press: Cambridge, UK, 1997; ISBN 0-521-49880-5. [Google Scholar]
  21. Cranton, P. Understanding and Promoting Transformative Learning: A Guide for Educators of Adults, 2nd ed.; Jossey-Bass: San Francisco, CA, USA, 2006; ISBN 978-0787976682. [Google Scholar]
  22. Gough, D. Weight of Evidence: A framework for the appraisal of the quality and relevance of evidence. Res. Pap. Educ. 2007, 22, 213–228. [Google Scholar] [CrossRef]
  23. Pollard, A.; Collins, J. Reflective Teaching: Effective and Evidence-Informed Professional Practice; Continuum: London, UK, 2008; ISBN 978-0-8264-9340-8. [Google Scholar]
  24. Dewey, J. How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process; D.C. Heath: Lexington, MA, USA, 1933; ISBN 978-0669200249. [Google Scholar]
  25. Walsh, S.; Mann, S. Doing reflective practice: A data-led way forward. ELT J. 2015, 69, 351–362. [Google Scholar] [CrossRef]
  26. Marzano, R.J.; Boogren, T.; Heflebower, T.; Kanold-McIntyre, J.; Pickering, D. Becoming a Reflective Teacher; Marzano Research Laboratory: Bloomington, IN, USA, 2012; ISBN 978-0-9833512-3-8. [Google Scholar]
  27. Hrevnack, J.R. Guided development of reflective thinking in the observations of classroom teachers by pre-service candidates. Acad. Educ. Leadersh. J. 2011, 15, 81–93. [Google Scholar]
  28. Paul, R.; Elder, L. (Eds.) Critical Thinking Competency Standards: Standards, Principles, Performance Indicators, and Outcomes with a Critical Thinking Master Rubric; Foundation for Critical Thinking: Tomales, CA, USA, 2005; ISBN 978-0-944583-30-2. [Google Scholar]
  29. Facione, P.A. Critical Thinking: What It Is and Why It Counts; California Academic Press: San Jose, CA, USA, 2015; Available online: http://www.insightassessment.com/pdf_files/what&why2010.pdf (accessed on 30 July 2018).
  30. Tilbury, D. Environmental education for sustainability: A force for change in higher education. In Higher Education and the Challenge of Sustainability: Problematics, Promise, and Practice; Corcoran, P.B., Wals, A.E.J., Eds.; Kluwer Academic: Dordrecht, The Netherlands, 2004; ISBN 978-0-306-48515-2. [Google Scholar]
  31. Bullock, D. Learner self-assessment: An investigation into teachers’ beliefs. ELT J. 2011, 65, 114–125. [Google Scholar] [CrossRef]
  32. Gün, B. Quality self-reflection through reflection training. ELT J. 2011, 65, 126–135. [Google Scholar] [CrossRef]
  33. Poldner, E.; Schaaf, M.V.; Simons, P.R.; Tartwijk, J.V.; Wijngaards, G. Assessing student teachers’ reflective writing through quantitative content analysis. Eur. J. Teach. Educ. 2014, 37, 348–373. [Google Scholar] [CrossRef]
  34. Filho, W.L. Towards the Promotion of Education for Sustainability. Rev. Educ. 2009, 33, 33–277. [Google Scholar]
  35. Lozano, R.; Ceulemans, K.; Seatter, C.S. Teaching organisational change management for sustainability: Designing and delivering a course at the University of Leeds to better prepare future sustainability change agents. J. Clean. Prod. 2015, 106, 205–215. [Google Scholar] [CrossRef]
  36. Aktas, C.B.; Whelan, R.; Stoffer, H.; Todd, E.; Kern, C.L. Developing a university-wide course on sustainability: A critical evaluation of planning and implementation. J. Clean. Prod. 2015, 106, 216–221. [Google Scholar] [CrossRef]
  37. Lozano, R.; Ceulemans, K.; Alonso-Almeida, M.; Huisingh, D.; Lozano, F.J.; Waas, T.; Hugé, J. A review of commitment and implementation of sustainable development in higher education: Results from a worldwide survey. J. Clean. Prod. 2015, 108, 1–18. [Google Scholar] [CrossRef]
  38. Thomas, I. Critical Thinking, Transformative Learning, Sustainable Education, and Problem-Based Learning in Universities. J. Transform. Educ. 2009, 7, 245–264. [Google Scholar] [CrossRef]
  39. Ceulemans, K.; Prins, M.D. Teachers manual and method for SD integration in curricula. J. Clean. Prod. 2010, 18, 645–651. [Google Scholar] [CrossRef]
  40. Filho, W.L.; Raath, S.; Lazzarini, B.; Vargas, V.; Souza, L.D.; Anholon, R.; Orlovic, V. The role of transformation in learning and education for sustainability. J. Clean. Prod. 2018, 199, 286–295. [Google Scholar] [CrossRef]
  41. Jančovič, J.; Penfold, S. Global Education in Pre-Service Teacher Training in Slovakia: Needs and Recommendations; Človek v ohrození: Bratislava, Slovakia, 2017; ISBN 978-80-89817-14-6. [Google Scholar]
  42. Stupple, E.J.; Maratos, F.A.; Elander, J.; Hunt, T.E.; Cheung, K.Y.; Aubeeluck, A.V. Development of the Critical Thinking Toolkit (CriTT): A measure of student attitudes and beliefs about critical thinking. Think. Skills Creat. 2017, 23, 91–100. [Google Scholar] [CrossRef][Green Version]
  43. Huang, Y.C.; Chen, H.H.; Yeh, M.L.; Chung, Y.C. Case studies combined with or without concept maps improve critical thinking in hospital-based nurses: A randomized-controlled trial. Int. J. Nurs. Stud. 2012, 49, 747–754. [Google Scholar] [CrossRef] [PubMed]
  44. Duffy, L.N.; Stone, G.A. Case Study Learning: A Tool for Teaching Critical Thinking. Available online: https://pdfs.semanticscholar.org/presentation/4d34/b4d5cab374f45073b394fe9cec828c0f65b7.pdf (accessed on 25 June 2018).
  45. Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education; Routledge: Oxon, UK, 2011; ISBN 0415583357. [Google Scholar]
  46. Hughes, M. (Ed.) Teaching and Learning in Changing Times; Blackwell: Oxford, UK, 1996; ISBN 9780631192787. [Google Scholar]
  47. Darling-Hammond, L.; Bransford, J. (Eds.) Preparing Teachers for A Changing World: What Teachers Should Learn and Be Able to Do, 1st ed.; Jossey-Bass: San Francisco, CA, USA, 2007; ISBN 978-0-787-99634-5. [Google Scholar]
  48. Summers, M. Education for sustainable development in initial teacher training: Issues for interdisciplinary collaboration. Environ. Educ. Res. 2005, 11, 623–647. [Google Scholar] [CrossRef]
Figure 1. Box plots for variables confidence in CT (CRT_avg), valuing CT (VCT_avg), and misconception (MIS_avg).
Figure 1. Box plots for variables confidence in CT (CRT_avg), valuing CT (VCT_avg), and misconception (MIS_avg).
Sustainability 10 03366 g001
Table 1. Descriptive statistics for variables confidence in critical thinking (CT) (CCT_avg), valuing CT (VCT_avg), and misconception (MIS_avg).
Table 1. Descriptive statistics for variables confidence in critical thinking (CT) (CCT_avg), valuing CT (VCT_avg), and misconception (MIS_avg).
Statistics
CCT_avgVCT_avgMIS_avgTotal_avg
NValid48484848
Missing0000
Median5.6185.6675.7505.648
Mode5.5885.833 a5.7505.556 a
Std. Deviation0.3090.4660.7510.251
Minimum4.8824.6673.7504.815
Maximum6.5296.6677.2506.111
a Multiple modes exist. The smallest value is shown.
Table 2. (a) Frequency table for confidence in CT (CRT_avg) variable. (b) Frequency table for valuing CT (VCT_avg) variable. (c) Frequency table for misconception (MIS_avg) variable.
Table 2. (a) Frequency table for confidence in CT (CRT_avg) variable. (b) Frequency table for valuing CT (VCT_avg) variable. (c) Frequency table for misconception (MIS_avg) variable.
(a)
CCT_avg
FrequencyPercentValid PercentCumulative Percent
Valid4.88212.12.12.1
5.05912.12.14.2
5.11812.12.16.3
5.17612.12.18.3
5.35336.36.314.6
5.41248.38.322.9
5.47136.36.329.2
5.52924.24.233.3
5.588816.716.750.0
5.64748.38.358.3
5.70636.36.364.6
5.76536.36.370.8
5.82448.38.379.2
5.88236.36.385.4
6.00012.12.187.5
6.05912.12.189.6
6.06312.12.191.7
6.11812.12.193.8
6.17612.12.195.8
6.23512.12.197.9
6.52912.12.1100.0
Total48100.0100.0
(b)
VCT_avg
FrequencyPercentValid PercentCumulative Percent
Valid4.66724.24.24.2
4.83312.12.16.3
5.00024.24.210.4
5.167510.410.420.8
5.33324.24.225.0
5.500714.614.639.6
5.667714.614.654.2
5.833816.716.770.8
6.000816.716.787.5
6.16712.12.189.6
6.33324.24.293.8
6.50012.12.195.8
6.66724.24.2100.0
Total48100.0100.0
(c)
MIS_avg
FrequencyPercentValid PercentCumulative Percent
Valid3.75012.12.12.1
4.25012.12.14.2
4.75036.36.310.4
5.00036.36.316.7
5.250714.614.631.3
5.500714.614.645.8
5.750816.716.762.5
6.00048.38.370.8
6.25012.12.172.9
6.500714.614.687.5
6.75024.24.291.7
7.00036.36.397.9
7.25012.12.1100.0
Total48100.0100.0
Table 3. (a) Descriptive statistics: categories description, evaluation, and justification. (b) Paired sampled test: categories description, evaluation, and justification.
Table 3. (a) Descriptive statistics: categories description, evaluation, and justification. (b) Paired sampled test: categories description, evaluation, and justification.
(a)
Paired Samples Statistics
MeanNStd. DeviationStd. Error Mean
Pair 1Des_PER55.487757.7160.891
Des_PER250.027754.0210.464
Pair 2Eval_PER36.560756.2890.726
Eval_PER241.913754.1780.482
Pair 3Just_PER6.099754.4630.515
Just_PER25.556753.3700.389
(b)
Paired Samples Test
Paired DifferencestdfSig. (2-Tailed)
MeanStd. DeviationStd. Error Mean95% Confidence Interval of the Difference
LowerUpper
Pair 1Des_PER–Des_PER25.4609.0301.0433.3837.5385.237740.000
Pair 2Eval_PER–Eval_PER2−5.3537.2200.834−7.014−3.691−6.420740.000
Pair 3Just_PER–Just_PER20.54245.5100.636−0.7251.8100.852740.397
Table 4. (a) Descriptive statistics: categories dialogue and transfer. (b) Wilcoxon signed ranks test: ranks of categories dialogue and transfer. (c) Wilcoxon signed ranks test: categories dialogue transformation.
Table 4. (a) Descriptive statistics: categories dialogue and transfer. (b) Wilcoxon signed ranks test: ranks of categories dialogue and transfer. (c) Wilcoxon signed ranks test: categories dialogue transformation.
(a)
Descriptive Statistics
NMeanStd. DeviationMinimumMaximum
Dia_PER751.0241.4870.0005.882
Transf_PER750.8301.7000.0007.500
Dia_PER2751.2251.5470.0005.263
Transf_PER2751.2791.6770.0005.556
(b)
Ranks
NMean RankSum of Ranks
Dia_PER2–Dia_PERNegative Ranks21 a21.71456.00
Positive Ranks24 b24.13579.00
Ties30 c
Total75
Transf_PER2–Transf_PERNegative Ranks16 d18.72299.50
Positive Ranks25 e22.46561.50
Ties34 f
Total75
(c)
Test Statistics g
Dia_PER2–Dia_PERTransf_PER2–Transf_PER
Z−0.694 h−1.698 h
Asymp. Sig. (2-tailed)0.4870.090
a Dia_PER2 < Dia_PER; b Dia_PER2 > Dia_PER; c Dia_PER2 = Dia_PER; d Transf_PER2 < Transf_PER; e Transf_PER2 > Transf_PER; f Transf_PER2 = Transf_PER. g Wilcoxon Signed Ranks Test; h Based on negative ranks.
Table 5. (a) Wilcoxon signed ranks test: ranks of the categories of description, justification, dialogues, and transformation. (b) Wilcoxon signed ranks test: categories of description, justification, dialogues, and transformation.
Table 5. (a) Wilcoxon signed ranks test: ranks of the categories of description, justification, dialogues, and transformation. (b) Wilcoxon signed ranks test: categories of description, justification, dialogues, and transformation.
(a)
Ranks
NMean RankSum of Ranks
Des_PER2–Des_PERNegative Ranks41 a23.00943.00
Positive Ranks2 b1.503.00
Ties5 c
Total48
Just_PER2–Just_PERNegative Ranks7 d10.1471.00
Positive Ranks34 e23.24790.00
Ties7 f
Total48
Dia_PER2–Dia_PERNegative Ranks4 g7.6330.50
Positive Ranks27 h17.24465.50
Ties17 i
Total48
Transf_PER2–Transf_PERNegative Ranks2 j10.0020.00
Positive Ranks21 k12.19256.00
Ties25 l
Total48
(b)
Test Statistics m
Des_PER2–Des_PERJust_PER2–Just_PERDia_PER2–Dia_PERTransf_PER2–Transf_PER
Z−5.675 n−4.659 o−4.263 o−3.590 o
Asymp. Sig. (two-tailed)0.0000.0000.0000.000
a Des_PER2 < Des_PER; b Des_PER2 > Des_PER; c Des_PER2 = Des_PER; d Just_PER2 < Just_PER; e Just_PER2 > Just_PER; f Just_PER2 = Just_PER; g Dia_PER2 < Dia_PER; h Dia_PER2 > Dia_PER; i Dia_PER2 = Dia_PER; j Transf_PER2 < Transf_PER; k Transf_PER2 > Transf_PER; l Transf_PER2 = Transf_PER. m Wilcoxon signed ranks test; n Based on positive ranks; o.Based on negative ranks.
Table 6. (a) Descriptive statistics: category of evaluation. (b) Paired sample t-test: category of evaluation.
Table 6. (a) Descriptive statistics: category of evaluation. (b) Paired sample t-test: category of evaluation.
(a)
Paired Samples Statistics
MeanNStd. DeviationStd. Error Mean
Pair 1Eval_PER36.32125486.5844430.950382
Eval_PER239.21187486.9629661.005018
(b)
Paired Samples Test
Paired Differences
MeanStd. DeviationStd. Error Mean95% Confidence Interval of the DifferencetDfSig. (Two-Tailed)
LowerUpper
Pair 1Eval_PER–Eval_PER2−2.89 +4.6350.669−4.237−1.545−4.320470.000
Table 7. Mann–Whitney U test.
Table 7. Mann–Whitney U test.
Hypothesis Test Summary
Null HypothesisTestSig.Decision
1The distribution of Des_PER2 is the same across categories of groupIndependent samples
Mann–Whitney U Test
0.275Retain the null hypothesis.
2The distribution of Eval_PER2 is the same across categories of groupIndependent samples
Mann–Whitney U Test
0.005Reject the null hypothesis.
3The distribution of Just_PER2 is the same across categories of groupIndependent samples
Mann–Whitney U Test
0.003Reject the null hypothesis.
4The distribution of Dia_PER2 is the same across categories of groupIndependent samples
Mann–Whitney U Test
0.000Reject the null hypothesis.
5The distribution of Transf_PER2 is the same across categories of groupIndependent samples
Mann–Whitney U Test
0.108Retain the null hypothesis
Asymptotic significances are displayed. The significance level is 0.05.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop