Engagement SPIRALS in Elementary Students: A School-Based Self-Regulated Learning Approach

Abstract

SPIRALS program was designed in accordance with the inclusive Universal Design for Learning approach and the social cognitive model of self-regulated learning. This project aims to promote cognitive, emotional, and behavioral engagement among elementary students, and especially those at high risk of dropping out (such as Roma students). To test the efficacy of the program, an intervention study was performed for four months within a truly inclusive learning environment, involving the whole classes. A quasi-experimental pre-test/post-test design with a control group (n = 63) and an experimental group (n = 57) was used. The dependent variables were student engagement (cognitive, emotional, and behavioral), academic self-concept, perceived climate of support, reading comprehension, and academic performance. Students in the experimental group exhibited statistically significant improvements compared to the control group in six of the seven dependent variables analyzed (behavioral and emotional engagement, academic self-concept, perceived climate of support, reading comprehension, and academic performance). Further, the magnitude of differences tended to be larger in student engagement, perceived climate of support, and reading comprehension than in academic self-concept or academic performance. These results provide evidence supporting the usefulness of intervention programs aimed at promoting student engagement, specifically among students at higher risk of failure or dropout.

1. Introduction

Schools are currently facing the challenge of adopting an inclusive perspective when attending to students, who are increasingly diverse in terms of capabilities, motivations, and ethnic and cultural backgrounds; ensuring furthermore, that they feel listened to, included, and valued [1,2]. In this regard, Europe has recently set an objective for sustainable development focused on ensuring inclusive, equitable, and quality education [3]. However, educational practice does not always fit this inclusive model, as it focuses more on mitigating students’ needs than in valuing the difference and developing effective strategies to address that diversity so that all students can access quality learning together [4,5]. Recent literature indicates the need to develop an inclusive, intercultural curriculum, as well as to improve educational resources and teacher training on interculturality and inclusion [6,7,8,9]. Nonetheless, teaching intercultural contents on a regular basis is a pending issue for schools to solve, especially when it comes to recognize and value Roma culture [10,11,12]. Although there are several projects towards Roma children’s educational inclusion [13], Roma people’s contribution to society at large is not yet integrated into the school curriculum, despite being the largest minority in Europe, but also one of the most impoverished, segregated and persecuted through history [14]. Since they arrived in Spain around six centuries ago, Roma children have been initially excluded from education and then segregated in special schools. Finally, they have been integrated into regular schools but still adopting an intervention model that considers Roma pupils as problematic, which can prevent them from accessing the same educational opportunities as other students have [10,15,16]. In this regard, it is important to ensure the engagement of students, families, and other stakeholders with the school by listening and accomplishing their real aspirations and needs, even transforming the school into a learning community [11]. We believe that fostering Roma students’ school engagement is likely to help them grow their competencies and increase their educational attainment, which will lead to better human development outcomes.

Another issue that hinders the fulfillment of the above-mentioned sustainable education goal is the latest data on early school dropout, particularly high for groups at risk of social exclusion. The percentage of 18 to 24-year-olds who drop out of school without completing secondary education in Spain is 17.9% (Europe mean score of 10.6%) [17]. This national data is worrying, due to the impact of early school leaving on youth social inclusion. However, this reality is particularly concerning among Roma youth, whose high rates of early school dropout (around 70%) are in sharp contrast with the rest of the population [18]. Roma people’s low engagement with school may be due to their high levels of poverty, discrimination, and school segregation [14,19].

In order to address this situation, various studies have been developed to improve students’ engagement [12,20,21]. However, the programs reviewed did not examine the training on self-regulating skills in truly inclusive learning environments. In this sense, the SPIRALS program was designed to fill this gap (its name in Spanish is ESPIRALES, which is an acronym for School program of intercultural promotion, learning regulation, reading comprehension and study habits). The main goal of this study is the evaluation of the effectiveness of this program to improve the cognitive, emotional, and behavioral engagement of elementary students within the school context.

1.1. Engagement and School Dropout Framework

A progressive fall in student engagement is usually associated with school dropout processes [22,23,24], which can be predicted even from elementary school [25], and it is more evident in the transition between elementary and secondary school [26]. Engagement may be understood as students’ thoughts, emotions, and behaviors concerning their school experiences, along with their commitment to their educational goals [27]. Engaged students tend to use skills to self-regulate their learning process (cognitive dimension), and they usually identify with the school and value education (emotional dimension). Moreover, these students are prone to attend class, participate actively, and make clear efforts to construct meaningful learning (behavioral dimension) [27,28]. Student engagement can be negatively affected by the presence of risk factors and how school and students deal with them (e.g., school staff’s capacity to detect and tackle problems early, and students’ ability to seek support) [24]. Thus, one of the possible trajectories of disengagement from school describes a downward spiral, originating in an accumulation of socio-educational disadvantages, even from early infancy, in the school or family context [24,29], which is something that happens to Roma children [16]. Students have less access to opportunities for quality learning when they attend schools in disadvantaged neighborhoods [29]. These schools have few resources, and teachers typically have little teaching experience and low academic expectations for their students [29,30]. Additionally, students in these schools are likely to show low socioeconomic status and various school-related problems, such as poor performance and low school engagement.

The cyclical Participation-Identification model by Finn [22] provides an essential theoretical framework for this study. According to this model, the student who is not very involved at the behavioral level (e.g., irregular attendance, noncompliance with rules, lack of attention or effort in class) is likely to obtain negative school results (e.g., low achievement, curriculum gaps, grade retention). These lead to them distancing themselves emotionally from academic activity and the school, reinforcing the cycle of emotional and behavioral disconnection. This theoretical model also helps us to understand how students build their reading competence from cycles or spirals of cumulative development [31,32].

Finally, poor school practices can influence school disengagement; for instance, when schools do not promote core skills such as the use of self-regulation strategies, which are often associated with better reading competence and academic performance [31,33,34] or seeking help and support [24,35]. Nevertheless, given that learning self-regulation skills are not generally taught explicitly in class, the more disadvantaged students, who usually lack knowledge on these strategies [31], find it more challenging to involve themselves effectively in the school context. That is the case of some Roma students, according to Mitjavila [36].

In addition to the abovementioned, schools are often slow to react to the signs of disengagement or intervene from a punitive or negative perspective, which feeds school disengagement in groups that are particularly disadvantaged or discriminated against [24]. In this regard, educational practices persist addressing Roma students’ needs, blaming them and their environment for their failures [37,38,39]. Furthermore, they are usually separated from their peers in segregated classrooms, placed in compensatory programs that do not bridge over their learning gaps, or in special education schools [15,19,38]. These aspects can lead to stigmatization and can harm their identity, self-esteem, and self-concept, and can hinder inter-ethnic relationships, and reinforce the cycle of disengagement [38,40].

In this cycle of disengagement, according to the logic of Finn’s model [22], attending segregated schools or classrooms, along with high levels of school absence [13,18,19], are related to Roma students’ low behavioral participation in quality school experiences. This repeated non-attendance makes it harder to acquire core skills, such as reading, in which Roma children from impoverished backgrounds exhibit significantly lower levels than their non-Roma peers from the beginning of elementary education [41]. Thus, learning gaps that they accumulate turn into low school performance [15], against which they react by emotionally disengaging from academic learning and the school, which accentuates the spiral of failure.

A perceived lack of support from family, teachers, and classmates can also affect Roma students’ engagement. For instance, a mismatch between family and school about values, beliefs, and practices (marriage at a young age, starting in the family workplace at an early age, and schedules and routines which are incompatible with school commitments) can make their emotional and behavioral engagement difficult [12,39,42]. Similarly, emotional engagement can be affected by the low value placed on education beyond basic numeracy and literacy by family or community members [10,42], and by socio-emotional pressure that can be brought to bear on them if they achieve academic success [39,43].

1.2. Promoting Roma Children’s School Engagement

To encourage successful school careers for Roma students, the school needs to become a truly intercultural, inclusive learning environment where teachers set positive expectations, and socio-affective relationships are built on commitment, constructive dialogue, and mutual respect and recognition [16]. In this sense, various studies have shown that the promotion of a positive climate of support, especially by teachers [27], is essential to encourage student engagement in general [44,45], and Roma students’ engagement in particular [46]. Moreover, perceived support is positively associated with academic self-concept, which acts as a mediating variable associated with school engagement [47]. The latter is positively associated with school achievement [48,49,50]. Thus, teachers can encourage Roma students’ academic and social success by promoting their language and culture in the classroom, which could strengthen their self-confidence, their identity, and their traditions, preventing the school environment from being perceived as something in opposition to their community [38,51]. The development of an intercultural curriculum could also contribute to the eradication of prejudiced, stereotypical attitudes, and better intercultural understanding among their classmates [52,53].

Soriano and colleagues [21] implemented the Luz Violeta (Purple Light) values educational program in a group of Roma adolescents in the south of Spain. This program was adapted to the Roma culture and delivered by Roma teachers in an extra-curricular context. In the sessions, they promoted reflection via a narrative and enhancement of the conflictive relationships between Roma and Moroccan adolescents. The results showed that, compared to the control group, feelings of self-realization, self-concept, and self-esteem increased for the experimental group. The program Yellow’s Trials and Tribulations by Rosário et al. [20] also used narratives to encourage reflection; in this case, on self-regulation of learning and its usefulness. The results of the study showed that the intervention managed to improve the cognitive and behavioral engagement of a sample of Roma children in the fourth year of the elementary school in Portugal. Another program that worked with Roma children was Knock! It’s Time to Learn! [12], which, similarly to Soriano et al. [21], involved Roma culture, appealing to young Roma to wake up the children in their community and take them to school, which rose their behavioral engagement and their academic performance.

1.3. The Present Study

None of the programs above noted trained self-regulation and intercultural skills in truly inclusive learning environments with different student profiles. Nor did they consider positive relationships with teachers and peers, engagement, and reading competence together, which are key aspects for school success according to the Check and Connect program [32,54]. Consequently, the SPIRALS program was built in an attempt to fill this gap. It was designed following the inclusive approach of Universal Design for Learning (UDL) [55] to encourage the development of cognitive, emotional, and behavioral engagement in elementary school students, and especially the engagement of those at high risk of dropping out (such as Roma students). The UDL approach defends that it is the learning environment that exhibits deficits, not the students, whose learning is often hindered by schools’ inflexible curriculums. For that reason, it focuses on providing students with various forms of representation of content, action and expression, and engagement, so that learners acquire resources and knowledge, but also become strategic and motivated [55,56].

Based on the above, the SPIRALS intervention program was built and implemented using a quasi-experimental design with a control and an experimental group, in various classes of fourth-grade students (9–10 years old) in the North of Spain. The aim of the program was to encourage students’ cognitive, emotional, and behavioral engagement (particularly Roma students) through the development of the skills needed to set engagement-success spirals, within and outside the school. The backbone of SPIRALS is the story-tool Yellow’s Trials and Tribulations [57,58], and the theoretical framework is the social cognitive model of self-regulation of learning from Bandura [59] and Zimmerman [60,61]. The objective is to provide students with declarative, procedural, and conditional knowledge about the use of self-regulatory strategies (cognitive engagement). Students are expected to be actively and systematically responsible for the regulation of their emotions, thoughts, and behavior in order to construct meaningful learning and increase their interest in learning [62,63], thus, SPIRALS aims to encourage students on this path to school engagement.

In order to test the efficacy of the program, an intervention study was carried out, for four months, with a quasi-experimental pre-test/post-test design using a control group and an experimental group. The study was performed in a truly inclusive learning environment with the whole classes. The dependent variables were school engagement (cognitive, emotional, and behavioral), academic self-concept, perceived climate of support, level of reading comprehension, and academic performance. In this study, a general hypothesis was formulated:

Following the intervention, the students in the experimental group, compared to the control group, will show higher school engagement (cognitive, emotional, and behavioral), academic self-concept, perceived climate of support, reading comprehension, and academic performance. The differences between the control and experimental groups will be larger in school engagement (cognitive, emotional, and behavioral), and perceived climate of support and reading comprehension than in academic self-concept and academic performance.

2. Materials and Methods

2.1. Participants

The intervention program was carried out for four months with students in the fourth grade of various public elementary schools in the Principality of Asturias (North of Spain). Firstly, a list of elementary schools in the central area (urban) with a high percentage of Roma students was provided by the Ministry of Education of the Government of the Principality of Asturias, Spain. These schools were invited to participate in the educational project. Secondly, among the schools that accepted the invitation, those in which there were no Roma children enrolled in fourth-grade classes were excluded. Lastly, from the total that met the inclusion criteria indicated (nine), three schools were selected to participate in this study, attending to their interest to participate. One of the classes of each school was randomly selected and randomly assigned to both conditions (control or experimental). Although initially, the total sample was composed of 126 students, finally, only 120 completed the two data collection moments. Six students did not fulfill all data; reasons are as follows: not having completed the measurement instruments on either occasion or not attending all sessions of the intervention program. Finally, the sample was composed of 57 students in the control group and 63 in the experimental group. The control group (three classes) was made up of 57 students (47.2% girls) aged 9 to 10 years (M = 9.17, SD = 0.379). Most were non-Roma students (75.5%), and the remaining 24.5% were from other minorities (11.6% were Roma, the other 12.9% from other minority groups). The intervention classes (three classes), with a high level of inclusivity, had a total of 63 students, of whom 42.9% were girls. Almost two-thirds (65.1%) were non-Roma, while 17.5% were Roma children. Their ages ranged from 9 to 11 years old (M = 9.21, SD = 0.481).

This study was carried out in accordance with the recommendations of the Asturian Regional Ministry of Education, which approved the project that enabled us to collect the data and implement the intervention program in the schools that agreed to participate (number of registration CD0CR40000176440033005N). School administrators gave their consent for the participation of both teachers and students, who were volunteers and did not receive any financial reward. At the beginning of the academic year, via a school meeting and in writing, all parents were informed and gave their consent for their children to participate in the activities to be undertaken by schools throughout the year, including their participation in our intervention program. Data was processed in compliance with the Spanish Personal Data Protection law (15/1999). For this purpose, the intervention expert, who was the only person with access to participants’ personal data, signed an agreement to keep it confidential. The procedures applied in the study were developed in accordance with the recommendations of the Ethics Committee of the University of Oviedo and the Helsinki Declaration of 1975 and 1983.

2.2. Variables and Measures

We performed a quasi-experimental study in which the independent variable was the group (control vs. experimental) and the dependent variables were student school engagement (cognitive, behavioral, and emotional), perceived climate of support, academic self-concept, reading comprehension, and academic performance. Except for reading comprehension and academic performance, all the items (see Appendix A) were rated using a Likert scale (1 = never to 5 = always) to make it easy for elementary school children to understand. To study the construct validity (one-dimensionality), confirmatory factor analyses were carried out. The items used to measure the different constructs in this study were translated and adapted by experts to the Spanish population. Prior to the statistical analysis, the items that were inverted were recoded so that high values would indicate a higher level of the variable being measured.

2.2.1. School Engagement

The cognitive, emotional, and behavioral dimensions of school engagement were measured in accordance with the model from Fredricks et al. [27].

• Student cognitive engagement. Cognitive engagement was operationalized as (a) the use of self-regulation learning strategies, and (b) the perceived usefulness of these strategies, as in Núñez et al. [34]. The use of self-regulated learning strategies was evaluated using the Self-regulation Processes of Learning Inventory–IPAA, based on the model from Zimmerman [60,61], which provides information on the three phases of the process of self-regulation of learning: planning, execution, and evaluation (α = 0.78). The perceived usefulness of these strategies was measured using nine items from the questionnaire Instrumentality of self-regulation of learning (α = 0.84). Construct validity is adequate, both in the case of IPAA (χ² = 29.27; df = 27; p = 0.35; CFI = 0.98; RMSEA = 0.03) and instrumentality (χ² = 20.93; df = 18; p = 0.28; CFI = 0.98; RMSEA = 0.04).
• Student emotional engagement was operationalized as (a) the value placed on education and identification with the school [22], and (b) the attitude towards school tasks. The value placed on education was measured using five items adapted from the Valuing of school education subscale by Wang et al. [64]. Identification with the school was evaluated using three items adapted from the School belonging subscale [64] and the School identification subscale [50], both designed by Wang and colleagues. Students’ attitudes towards school tasks were evaluated via two items adapted from the scale Emotional engagement by Skinner et al. [65]. Both reliability (α = 0.73) and construct validity (χ² = 44.34; df = 31; p = 0.06; CFI = 0.95; RMSEA = 0.06) are acceptable and good, respectively.
• Student behavioral engagement was operationalized as (a) adaptive behavior (referring to good classroom behavior and attendance), and (b) involvement in academic tasks and learning both in and out of school. Adaptive behavior was evaluated using five items adapted from two subscales by Wang et al.: four from the School compliance subscale [64], and one from the School participation subscale [50]. Involvement in tasks and learning was evaluated using six items adapted from the Behavioral engagement scale by Skinner et al. [65] and the Attentiveness subscale by Wang et al. [64]. Both reliability (α = 0.85) and construct validity (χ² = 57.78; df = 39; p = 0.03; CFI = 0.95; RMSEA = 0.06) are acceptable and good, respectively.

2.2.2. Academic Self-Concept

This was measured using the students’ responses to four items taken from the SDQ-II (Self-Description Questionnaire) from Marsh [66]. Similarly to other studies [35,67], the reliability in our study is high (α = 0.89), and the construct validity (χ² = 1.7; df = 1; p = 0.19; CFI = 0.99; RMSEA = 0.07) is good.

2.2.3. Perceived Climate of Support

This was operationalized as the perception of positive relationships with (a) classmates, and (b) the teacher. Ten items were used, which were taken from the subscales of academic support, emotional support, and perceived social efficacy from Patrick et al. [49]. The reliability of the measure is high (α = 0.89), and the construct validity is acceptable (χ² = 61.59; df = 30; p = 0.00; CFI = 0.92; RMSEA = 0.08).

2.2.4. Reading Comprehension

This was measured using two different texts (one for pre-test, the other for post-test) to avoid the learning effect. Both texts were taken from PROLEC-R (Evaluation battery for reading processes, revised) by Cuetos et al. [68], which evaluates reading skills in children from the first to the sixth year of elementary school. Students’ responses were recorded on a scale of 0 to 1 (0 = incorrect; 1 = correct) by two members of the research group, resulting in substantial inter-rater agreement (k = 0.78).

2.2.5. General Performance

Data on performance was collected in the different subjects (percentage of subjects passed, numerical grade in each subject) including the subject of Spanish Language (numerical grade) during the three terms of the school year the program took place (T1, T2, and T3). T1 and T3 were the initial (pre-test) and final (post-test) evaluations, T2 was an intermediate evaluation.

2.3. Procedure

An intentional sample was selected from state-funded schools attended by Roma students. The authorities at each school were contacted by an email that included a summary of the project. The project was then presented to the management of each school, and once the teaching staff approved it, the school agreed to collaborate in the study.

Two preparatory meetings were held to coordinate with the teachers involved. Teachers were given a manual with guidelines for each session of the intervention, and an adapted version of the story Yellow’s Trials and Tribulations [69] to be able to perform it in class. Finally, the teachers attended two three-hour training sessions on theoretical and practical aspects of the program regarding interculturality, inclusion, and self-regulation of learning.

The program comprised ten intervention sessions (described in the corresponding section), and two assessment sessions (pre- and post-test) in which the students completed an individual questionnaire about the variables to be measured. The research assistants, who had a psychological and teaching education background, attended a training session focused on the intervention program and on how to conduct the assessment sessions. After delivering the questionnaires, the research assistants explained the instructions to the children, read each item aloud, and gave simple examples to ensure students’ understanding. The intervention took place for four months, during ninety-minute Spanish Language lessons, with the whole class group and the collaboration of the researcher and the teacher. In order to ensure the program was followed accordingly to the protocol, both completed a rubric that assessed the extent to which the sessions had followed the plan. In addition, the first author was present during the sessions to ensure that the program was followed appropriately. Lastly, in order to deepen our understanding of the impact of the program on Roma children and their classmates as well as on their teachers, their perceptions about the intervention were collected through interviews and satisfaction questionnaires throughout and at the end of the program. Some of these quotes are included in the discussion to illustrate findings.

2.4. The Intervention Program

The intervention was based on the PLEE self-regulated learning model by Rosário [70], itself based on the previous work by Zimmerman [60,61]. According to the model, students follow a cyclical process made up of three interdependent phases (Planning, Execution, and Evaluation), which reproduce the self-regulatory dynamic.

The acquisition of this self-regulatory competence is influenced by contextual variables, such as the learning environment [59,60,61], and it must be promoted through direct instruction, modeling, and guided and autonomous practice [57]. The story-tool Yellow’s Trials and Tribulations [57] is aimed at children aged 6 to 10 years, and it follows a vicarious narrative methodology. Discussing the story encourages identification with the characters and training on self-regulated learning strategies. This story [69] narrates Rainbow Colors’ journey to find their lost friend, Yellow, and describes the self-regulation strategies that the Colors use to solve the problems they encounter successfully. For instance, some of these issues (e.g., laziness, fear, disobedience) are characters (Laziness, Fear, Disobedience) who attack children’s behavior, preventing them from attaining their goals at home, at school or in other contexts. Thus, the story promotes the development of strategies to identify these issues and overcome them through self-regulation, as well as the acquisition of fundamental rules and values for life in society (e.g., effort, empathy, cooperation, friendship).

In order to foster a positive climate of support and emotional engagement, various methodological innovations were added to the core program [57,58]. The sessions were delivered during Spanish Language classes, and with the entire class group. Finally, to facilitate vicarious learning and student motivation, the story was adapted to be performed, so that the students could experience it, and group discussion was encouraged about what the story was telling the children and the self-regulation strategies involved. In addition, some intercultural contents were included in the story and the exercises to deepen and reflect on the richness of minority cultures, focusing on the contribution of Roma people, the largest and most ancient minority in Spain. By introducing Roma culture, it was expected to increase Roma students’ cultural pride within the school context, and promote a truly inclusive, respectful, warming climate, opened to intercultural dialogue. In this regard, some Roma children acted as narrators of the Rainbow Colors’ journey to best transfer this knowledge to the whole class.

The sessions were structured in a similar way: introductory activity, dramatized reading and group reflection on the chapter of the day, working in small groups on activities related to Spanish Language, to self-regulated learning, and on occasion, to Roma culture. The initial activity included reviewing homework and a summary of the previous chapter. Below,

Table 1. Contents and self-regulating strategies of some sessions from the SPIRALS program.

Session 1. I’m Cool!
Activities	Welcome and commitment to the rules. Self-awareness chart “You are cool”. Presentation of narrators (Ana and her Roma friends). Dramatization of chapters 1 and 2. Spanish Language worksheet (reading comprehension, reflection, and knowledge). Homework: If we are responsible, everything works well. What have I learned?
Skills:
Self-regulation of learning	Strategies: self-evaluation (characteristics, tastes, motivation to go to class, completing tasks in class and at home); seeking social help for a problem in the story (Yellow has gone missing) and in real life; self-consequences (lack of rules).
Intercultural	Introduction to Roma history and languages.
Social and civic	Reflection as an instrument for analyzing and finding solutions. The importance of positive interdependence and co-operative work. The importance of rules and responsibility. Creation of a climate of support and trust. My relationship with my teachers and classmates. Positive approach to problems.
Linguistic	Dramatization of literary texts. Spoken and written questions about the story. Language knowledge: adjectives.
Session 3. My secret is PLEE!: I PLan, I Execute, and I Evaluate
Activities	Review of tasks. Summary of the story. Dramatization of chapters 5 and 6. Spanish Language worksheet (reading comprehension, reflection, and knowledge). I use PLEE for everything, what about you? (an origami chicken). Homework: PLEE helps me to be a better student. What have I learned?
Skills:
Self-regulation of learning	Definition of planning, execution and evaluation (PLEE), and examples in the story (collecting ants) and in real life (application to origami and to being a better student). Strategies: establishing objectives (dividing objectives into smaller tasks and managing time using a planner); physical and psychological structuring of the environment when studying; looking up information in the dictionary.
Social and civic	Reflection as an instrument for analyzing and finding solutions. The importance of positive interdependence and co-operative work.
Linguistic	Dramatization of literary texts. Spoken and written questions about the story. Language knowledge: using the dictionary.
Session 5. When I read, I enjoy it, and I learn!
Activities	Review of tasks. Summary of the story. Dramatization of chapter 8. Spanish Language worksheet (reading comprehension, reflection, and knowledge). Why is it important to read? Homework: When I read, I enjoy it, and I learn. What have I learned?
Skills:
Self-regulation of learning	Using PLEE to solve problems in the stories (Perseus kills Medusa and rescues Cassiopeia) and in real life (solving a word search puzzle). Applying PLEE to reading. Strategies: self-evaluation of my reading habits; physical and psychological structuring of the environment when studying; organizing and finding information and monitoring (word search, reading); self-consequences (benefits of reading)
Intercultural	Valuing and respecting the diversity of cultures. Positive evaluation of the Roma people. Their history. The Roma flag and anthem. Roma languages. Roma traditions and values. Stories of other cultures.
Social and civic	Reflection as an instrument for analyzing and finding solutions. The importance of positive interdependence and co-operative work. My relationship with my teachers and classmates. Empathy in personal relationships.
Linguistic	Dramatization of literary texts. Spoken and written questions about the story. Application of strategies for understanding texts. Development of reading habits. Language knowledge: linguistic and cultural diversity in the surroundings.

gives a summary of the content and self-regulation strategies of some of the sessions.

2.5. Design and Data Analysis

Six fourth-year elementary school classes were selected pseudo-randomly. Half of the classes were randomly assigned to the treatment program and the other half to the control. Before beginning the study, various tests were carried out to confirm whether the levels of behavioral engagement, cognitive engagement, emotional engagement, perceived climate of support, academic self-concept, reading comprehension, and overall performance were equivalent between the students in the control and experimental groups. These prior measures were labeled Z₁, Z₂, Z₃, Z₄, Z₅, Z₆, and Z_7. At the end of the study, similar tests were performed for each of the dependent variables, labeled Y₁, Y₂, Y₃, Y₄, Y₅, Y_6, and Y₇. This produced a randomized pre-test/post-test clustered design. In cluster-randomized trials, small and large groups of individuals (rather than individuals themselves) are randomly assigned to experimental conditions, and individuals from the same clusters are repeatedly measured on two or more occasions over time. This design is a natural choice for testing many educational research questions.

There are numerous options for analyzing data with this type of design, and no gold-standard approach exists. However, likelihood-based mixed-effects models (multivariate and univariate) provide suitable general analytical frameworks to determine whether changes in response differ between treatment groups. When the units of analysis are randomized to the treatment groups and the baseline values are obtained before any interventions, adjusting for baseline is appropriate [71]. Most empirical studies report ANCOVA to be often the most appropriate statistical method for adjusting for baseline covariates when analyzing randomized studies with continuous outcomes [72]. When different mean values of baseline for the treatment and control groups are not expected, both analysis of variance (ANOVA) and ANCOVA models will provide unbiased estimates of the treatment effect. However, using ANCOVA based on the observed data is known to produce unbiased estimates of treatment effects in the presence of baseline imbalance. In this study, the statistical treatment of the data used univariate and multivariate ANCOVA models. More specifically, we used one-way mixed MANCOVA analysis (the class factor was considered to be random) to check the overall principal effect of the program on the dependent variables which denoted various different indicators on the engagement construct (i.e., behavioral engagement, cognitive engagement, and emotional engagement). The one-way mixed ANCOVA was used to examine the effects of the intervention program separately on each of the other four remaining dependent variables (perceived climate of support, academic self-concept, reading comprehension, and overall performance). It was assumed that these represented indicators of different psychological constructs.

We used a two-stage strategy for data analysis. In the first stage, we established whether prior performance (baseline) significantly explained post-test performance (which led to testing whether the regression slopes were null). If the slopes were null, no covariates would need to be included in the model, and fitting a one-way mixed MANOVA/ANOVA model would be enough. If the slopes were non-zero, that would lead to the second stage in order to verify whether the regression slopes were homogeneous (that is, demonstrating whether the relationship between prior and final performance was similar across the different treatment groups). If the parallelism assumption were accepted, all interest would focus on the MANCOVA/ANCOVA sub-model with parallel slopes; otherwise, focus would be on explaining the interaction between covariates and treatment groups using the Johnson-Neyman technique.

Data was analyzed using SAS (Version 14.3) Proc MIXED. The Proc UNIVARIATE and Proc GLIMMIX modules were used to check whether the data was approximately normal and whether covariance matrices were equal between groups.

3. Results

3.1. Descriptive Analysis

Prior to conducting the analysis, we examined the distribution of data from the different samples for the baseline and post-baseline responses across treatment groups. The extent of skewness and kurtosis for the variables in the model, as well as the sample size, means and standard deviations are presented in

Table 2. Descriptive statistics for measurements obtained from baseline and post-baseline across treatment groups.

Experimental Group
	Z1	Z2	Z3	Z4	Z5	Z6	Z7
Sample size	63	63	63	63	63	63	59
Mean	3.9754	3.9550	4.1825	4.2222	3.7024	5.6349	7.2261
Standard deviation	0.6830	0.6094	0.3859	0.4821	1.0529	1.8948	1.5631
Skewness	−1.6327	−0.9815	−0.9798	−0.0437	−0.6411	−1.1413	−0.2028
Kurtosis	3.6683	0.3479	1.7225	−0.7048	−0.5746	0.8619	−1.1448
	Y1	Y2	Y3	Y4	Y5	Y6	Y7
Sample size	63	63	63	63.	63	63	62
Mean	4.1458	4.0107	4.3651	4.4508	3.9643	7.2540	7.0687
Standard deviation	0.6186	0.5917	0.4322	0.4899	0.8945	1.1909	1.6363
Skewness	−1.4712	−0.7996	−1.5273	−1.2575	−0.9771	−1.8149	−0.1273
Kurtosis	3.6354	0.0643	2.9529	1.5077	0.7486	3.0379	−1.2050
Control Group
	Z1	Z2	Z3	Z4	Z5	Z6	Z7
Sample size	54	54	54	54	54	54	55
Mean	4.1817	3.9698	4.2389	4.3519	3.8611	5.8333	6.7836
Standard deviation	0.4494	0.5663	0.5395	0.5901	0.9055	1.7129	1.3521
Skewness	−0.3582	−0.7478	−1.8571	−1.2611	−0.7207	−0.9240	−0.3306
Kurtosis	−0.2139	0.8527	4.0841	1.2855	0.0264	0.7328	−0.4059
	Y1	Y2	Y3	Y4	Y5	Y6	Y7
Sample size	56	56	56	56	56	56	57
Mean	3.9836	3.8446	4.1375	4.0696	3.7679	6.3571	6.8821
Standard deviation	0.5109	0.6720	0.5643	0.7565	0.8840	1.4824	1.4589
Skewness	−0.5173	−0.7171	−1.3025	−0.9270	−0.5908	−1.4474	−0.2631
Kurtosis	−0.1298	−0.1228	1.6483	0.7419	−0.4852	4.6732	−0.4272

Z1/Y1 = Behavioral engagement; Z2/Y2 = Cognitive engagement; Z3/Y3 = Emotional engagement; Z4/Y4 = Perceived climate of support; Z5/Y5 = Academic self-concept; Z6/Y6 = Reading comprehension; Z7/Y7 = General performance.

. As the table shows, skewness and kurtosis values were generally within the range ± 1.5 and ± 4.0, respectively, which is considered a reasonable approximation to the normal curve. Data with a skew above an absolute value of 3.0 and kurtosis above an absolute value of 8.0 is considered problematic [73]. We concluded that the values for skewness and kurtosis were within allowable limits for each of the covariates and dependent variables, so continued with the analysis given that the assumptions of the model appeared tenable.

3.2. Analysis of Covariance

3.2.1. Null Regression Curve Test

The first stage of the modeling strategy tested the null slope hypothesis; in other words, verifying whether the covariates should be eliminated from the analysis. To determine whether covariates were significant in the analysis, a type III sums of squares model without intercept was created. This model included treatment groups and interactions of treatment groups with the covariate prior performance. The information from this analysis allowed us to consider regression slopes for each level of treatment variable and evaluate whether they were null.

Table 3. Estimators of interaction parameters obtained in the first modeling stage after creating a regression model without intercept.

Effect	Estimate	S.E.	DF	|t Value|	Pr > |t|
Experimental group × Behavioral engagement	0.7379	0.0622	108	11.87	<0.0001
Control group × Behavioral engagement	0.8303	0.1064	108	7.81	<0.0001
Experimental group × Cognitive engagement	0.6512	0.1036	108	6.29	<0.0001
Control group × Cognitive engagement	0.6890	0.1208	108	5.70	<0.0001
Experimental group × Emotional engagement	0.6058	0.1368	108	4.43	<0.0001
Control group × Emotional engagement	0.5555	0.1063	108	5.23	<0.0001
Experimental group × Perceived climate of support	0.7203	0.1173	108	6.14	<0.0001
Control group × Perceived climate of support	0.8856	0.1049	108	8.44	<0.0001
Experimental group × Academic self-concept	0.7167	0.0593	108	12.08	<0.0001
Control group × Academic self-concept	0.8035	0.0748	108	10.74	<0.0001
Experimental group × Reading comprehension	0.2059	0.0843	108	2.44	0.0162
Control group × Reading comprehension	0.3763	0.1061	108	3.55	0.0006
Experimental group × General performance	1.0045	0.0441	105	22.77	<0.0001
Control group × General performance	0.9227	0.0515	105	17.92	<0.0001

S.E. = Standard Error; DF = Degrees of Freedom; Pr> = Probability.

shows the test results for secondary effects. All the tests were significant, leading to the conclusion that the slopes were not null for the seven covariates.

3.2.2. Parallelism Regression Slope Test

When using ANCOVA/MANCOVA designs to examine differences in treatments, in addition to model assumptions and non-null slopes, one must have univariate/multivariate parallelism. To test the hypothesis of multivariate parallelism with the indicators in the engagement construct (i.e., Y₁, Y₂, Y₃), we must check that the interactions between the treatment groups and the covariates are simultaneously zero. Similarly, the indicators for the remaining constructs were examined separately.

Table 4. Results of ANCOVA/ MANCOVA to evaluate parallelism regression lines.

MANCOVA Analysis
Effect	dfN, dfD	F	Pr > F
Var	3, 6	3.52	0.0885
Var×Z1	3, 325	20.59	0.0001
Var×Z2	3, 315	13.25	0.0001
Var×Z3	3, 315	3.59	0.0141
Var×Gr	3, 6	0.66	0.6069
Var×Z1×Gr	3, 315	2.76	0.1548
Var×Z2×Gr	3, 315	1.31	0.2523
Var×Z3×Gr	3, 315	0.91	0.4356
ANCOVA Analysis
Z4				Z5			Z6			Z7
Effect	dfN, dfD	F	Pr > F	dfN, dfD	F	Pr > F	dfN, dfD	F	Pr > F	dfN, dfD	F	Pr > F
Z	1, 108	104.1	0.0001	1, 108	253.6	0.0001	1, 108	18.5	0.0001	1, 105	807.6	0.0001
Gr	1, 4	3.0	0.1590	1, 4	2.8	0.1708	1, 4	5.6	0.0773	1, 4	3.3	0.1420
Z×Gr	1, 108	1.1	0.2959	1, 108	0.8	0.3641	1, 108	1.6	0.2111	1, 105	1.5	0.2304

df_N = Numerator degrees of freedom; df_D = Denominator degrees of freedom; Pr> = Probability; Gr = Groups; Var = Dependent variables; Z₁ = Baseline for behavioral engagement; Z₂ = Baseline for cognitive engagement; Z₃ = Baseline for emotional engagement; Z₄ = Baseline for perceived climate of support; Z₅ = Baseline for academic self-concept; Z₆ = Baseline for reading comprehension; Z₇ = Baseline for general performance. The same in the following tables.

shows the overall test of parallelism not to be significant. Univariate parallelism tests led to the same conclusion.

Given parallelism and the fact that the covariates were significant in the analyses, the next test of interest was to determine whether there were significant differences between the conditions. We began by testing the equality of mean vectors for the three dependent variables referring to engagement (i.e., behavioral engagement-Y₁, cognitive engagement-Y_2, and emotional engagement-Y₃) between the two treatment groups using one-way mixed MANCOVA. The top panel of

Table 5. Results of MANCOVA model with parallel slopes.

MANCOVA Analysis
Effect	dfN, dfD	F	Pr > F
Var	3, 6	8.13	0.0155
Var×Z1	3, 327	25.02	0.0001
Var×Z2	3, 327	14.26	0.0001
Var×Z3	3, 327	7.22	0.0001
Var×Gr	3, 6	9.92	0.0097
ANCOVA Analysis
Behavioral engagement				Cognitive engagement				Emotional engagement
Effect	dfN, dfD	F	Pr > F	Effect	dfN, dfD	F	Pr > F	Effect	dfN, dfD	F	Pr > F
Z1	1, 109	202.3	0.0001	Z2	1, 103	75.4	0.0001	Z3	1, 111	41.7	0.0001
Gr	1, 4	13.5	0.0213	Gr	1, 4	2.5	0.1589	Gr	1, 4	6.7	0.0319

df_N = Numerator degrees of freedom; df_D = Denominator degrees of freedom; Pr> = Probability.

shows that the null hypothesis of equal means or treatment effects when the three dependent variables are considered simultaneously was rejected [F(3, 6) = 9.92, p = 0.0097]. Given that there was a significant difference between groups considering all the engagement variables simultaneously, we were interested in determining whether variables Y₁, Y₂, and Y₃ made similar contributions to the analysis of group differences. As the bottom panel of Table 4 indicates, there were significant differences between the experimental group and control group for behavioral engagement [F(1, 4) = 13.50, p = 0.0213] and emotional engagement [F(1, 4) = 6.70, p = 0.0319]. However, there were no significant differences between the experimental group and control group for cognitive engagement [F(1, 4) = 2.50, p = 0.1589]. As the overall test was rejected, these individual tests were protected: F tests, i.e., each test was evaluated at the nominal alpha level.

Finally, a one-way mixed ANCOVA model was used to determine whether there were differences between the experimental group and control group for each of the remaining dependent variables considered separately (i.e., perceived climate of support-Y₄, academic self-concept-Y₅, reading comprehension-Y₆, and general performance-Y₇).

Table 6. Results of ANCOVA model with parallel slopes.

ANCOVA Analysis
Perceived climate of support				Academic self-concept			Reading comprehension			General performance
Effect	dfN, dfD	F	Pr > F	dfN, dfD	F	Pr > F	dfN, dfD	F	Pr > F	dfN, dfD	F	Pr > F
Z	1, 109	107.8	0.0001	1, 109	262.0	0.0001	1, 109	16.9	0.0001	1, 106	830.9	0.0001
Gr	1, 4	31.0	0.0051	1, 4	10.0	0.0342	1, 4	16.3	0.0157	1, 4	7.7	0.0502

df_N = Numerator degrees of freedom; df_D = Denominator degrees of freedom; Pr> = Probability.

shows that there was a significant difference between the experimental and control groups in perceived climate of support [F(1, 4) = 30.95, p = 0.0051], academic self-concept [F(1, 4) = 9.98, p = 0.0342], reading comprehension [F(1, 4) = 16.26, p = 0.0157], and a marginally significant difference in general performance [F(1, 4) = 7.69, p = 0.0502]. Figure 1 shows graphically the differences observed between the control and experimental groups in the dependent variables (the means are adjusted by the covariates, for both the multivariate and univariate cases).

4. Discussion

The aim of this study was to examine the effect of a psychoeducational intervention on elementary school students’ engagement, as well as on students’ perceptions of social support, academic self-concept, reading comprehension, and academic performance. The program attempted to strengthen these aspects through the dramatization and discussion of a story, while at the same time, working on self-regulation, reading, social, and intercultural skills.

The results confirm, almost totally, our general hypotheses. Data was positive for six of the seven dependent variables analyzed: behavioral and emotional engagement, academic self-concept, perceived climate of support, reading comprehension, and academic achievement. In contrast to expectations, no statistically significant differences were found regarding cognitive engagement. In addition, differences between the experimental and control groups regarding the variables in analysis (e.g., emotional engagement) were larger than those for the other variables (e.g., self-concept). The magnitude of those differences tended to be larger for school engagement (except for the cognitive dimension), perceived social support, and reading comprehension than for academic self-concept or academic performance (particularly the latter).

Prior research has indicated the importance of working on cognitive, emotional, and behavioral engagement, as a whole, in order to improve academic performance and prevent disengagement and school dropout [27,50], even in elementary school [32,54]. Our findings support the idea that educational intervention produces positive effects in elementary school student engagement in emotional and behavioral dimensions regardless of ethnic or cultural origin. However, current data did not show the expected results in cognitive engagement. This finding was surprising, given the fact that the intervention was based on a social-cognitive theoretical background [59,60,61], and carried out in Spanish Language classes; two keys for maximizing the efficacy of self-regulation learning programs according to Dignath et al. [33]. Nonetheless, this may be explained by the short duration of the program [34], the limited number of sessions [20,74], and the scarce training on self-regulated learning of the teachers delivering this program [33]. Despite that, the trend of improvement seen in this variable is promising and might be able to be confirmed with a larger sample.

The behavioral dimension of engagement has been the most widely studied due to its observable nature and its positive association with academic performance [27]. In this regard, our results are in line with those by Rosário et al. [12,20], showing that intervention improved students’ behavioral engagement and academic performance, although the improvement in the latter was marginal. This may be due to the fact that students did not participate in the program for long enough to develop and transfer self-regulated learning strategies to their other schoolwork. In addition, the students may not be used to thinking autonomously and discussing the process between themselves and the teacher.

With regard to the third engagement dimension, our findings are consistent with those by Wang and Eccles [47], in that teaching a curriculum that is relevant to students’ interests and goals increases their emotional engagement and their academic self-concept.

Addressing content of other cultures, in particular Roma culture, in the classroom within the framework of a school subject and with the whole class group, is one of the strengths of this intervention program. This educational opportunity made it possible for all the students to be able to think together about the contribution of minority cultures to society in general, particularly Roma culture, which has historically been made invisible [53,75].

Previous studies have shown the benefits of including intercultural content both for students from minority and non-minority groups [8,76]. On the one hand, it augments minority students’ cultural pride, which is related to better school performance [77], and better self-concept [21], which is in line with the findings from our study. This is reflected in the comments from some Roma students who participated in the program, who felt valued: “We also saw my family’s history, and the history of the Roma flag. (…) And something I loved was having my roots explained” (Raúl, aged 11).

In this regard, the study by Dimitrova et al. [77] with Roma students, confirmed the importance of exploring ethnic identity and the production of positive socialization experiences to help students understand their roots, improve their life satisfaction, and their academic performance.

On the other hand, the program contributed to better intercultural understanding, which turned into significant improvements in the perceived climate of support, and increased emotional engagement. In this respect, various studies have found that establishing positive relationships with teachers and classmates, in addition to improving behavioral engagement, has a positive influence on the emotional dimension in the whole group [32,47,78]. Giving the same opportunities to all the class to benefit from the program produced the ideal conditions to encourage the construction of a positive academic identity, as well as strengthening significant interethnic relationships likely to increase the feelings of belonging in the class group [2], improving academic self-concept and school engagement [47]. These results were also achieved in our study. Various non-Roma students’ and teachers’ comments recognized the value of Roma history and culture for their own development and to encourage positive coexistence at school:

[One important lesson] was that you must respect each other and that no matter where you are from, Roma or non-Roma, we are all equal (Misael, aged 9).

We would like to highlight that the Roma students have felt valued by the rest of their classmates, which has produced true educational inclusion. In addition, all the children have learned to identify their emotions, their problems, and healthy ways to approach them. (…) We have also gained knowledge about Roma culture that we did not know, and we have enjoyed experiencing the adventures of looking for Yellow alongside our students (Montse and Telma, teachers).

One of the particularly positive results of applying the program was the perception of support from classmates and teachers. This educational change can be attributed, at least in part, to the use of dramatization, a methodology that has also been successfully used in other studies to improve the development of linguistic skills [79], encourage inclusion and reduce low performance and dropout among Roma students [80].

4.1. Implications for Practice and Policy

The results from this study provide evidence supporting the usefulness of intervention programs aimed at promoting student school engagement, specifically in students at higher risk of failure or dropping out. In addition, findings showed the importance of implementing these programs in whole class groups, during school hours, along with other activities, producing truly inclusive education.

Although this might occasionally require flexibility to integrate self-regulation skills, and social and intercultural skills in the context of a school subject, this strategy has been shown to benefit all students. In addition, the engagement of all the students in the program was achieved, thanks to the implementation of activities from a UDL perspective, a strategy that means using different materials, media, and resources of expression and action, and innovative teaching methods, such as dramatizing a story [55]. One of the keys to promoting the construction of engagement-success spirals by all students, particularly those from disadvantaged ethnic and cultural minorities, is encouraging their participation in quality educational experiences [3,27,30] in which they feel included and valued. In order to achieve this goal, it is essential to turn the school into a truly inclusive, intercultural learning environment, which entails profound organizational and curricular changes [16]. Thus, it is essential to assume that all students can learn and succeed together within the regular classroom [8,11], always counting on the management and teacher staff [24,48,81].

In addition, the classroom must become a space promoting not only the acquisition and application of skills, or the evaluation of levels of achievement, but also the development of self-regulation strategies [55]. These would allow the students to make the most of each learning experience, constructing the necessary skills for academic success, as well as the establishment of positive relationships with themselves and others [33,34,35].

The school curriculum must include content that, in addition to being important from an academic point of view, connects to students’ personal, social, and cultural realities [55]. This is particularly important among students from ethnic minorities [7], such as Roma pupils. The latter often feel a significant disconnection between what is taught at school and what they learn within their families and their community, and dissonance that complicates their emotional and behavioral engagement with the learning promoted at school [39,43]. In order to bridge over these contexts, it is crucial to incorporate Roma culture into the school curriculum and teacher training [10,11,12], not only to recognize its value but also to generate a warm climate. Indeed, teachers’ and students’ knowledge of minority cultures incentivizes intercultural understanding, encouraging the reduction of prejudices and stereotypes often suffered by minority students, especially Roma [8,52,75]. School plays a critical role in the promotion of positive views of the Roma community and other minority groups, as well as in the formation of the citizens of tomorrow so that they can build a more equal and inclusive society for all. All considered, our findings suggest that it is possible to promote students’ sustainable development by increasing their school engagement (cognitive, emotional, and behavioral levels). This way, students are likely to be ready to assume responsibility for their learning process and become the protagonists of evident efforts to change their school behaviors and their lives. Hopefully, these efforts will have a positive impact on the institution itself and the education system at large.

4.2. Limitations and Future Research

Although the data in this study was collected via a sufficiently robust design and analyzed with statistical rigor, current findings must be taken cautiously. Readers should take into consideration that the program was applied to three classes, without follow-up measures, and the measuring instruments were mainly self-reports.

However, in this study, it was not possible to examine the progression of the dependent variables among Roma students, mainly due to the low numbers of Roma participants. It would be interesting to examine this in future research with larger samples. It would also be useful to extend the intercultural content in the intervention program to include other cultures, and work with teachers from subjects other than Spanish Language so that the self-regulation strategies could be applied in all subjects, which would further encourage them to be transferred. Likewise, it would be interesting to understand the effect of the intervention over longer time windows; for instance, using longitudinal or repeated measure designs. Finally, solely using quantitative or qualitative research approaches may not be sufficient. Future research could consider using mixed designs, because this methodology would enrich the conclusions as a consequence of the qualitative analysis of the intervention process itself and of the results gathered from different perspectives (e.g., students, teachers, pedagogical teams). Usually, the purpose of the qualitative phase is to help explain the results of the quantitative phase. Therefore, it would be recommendable to combine quantitative and qualitative research methods to gather a more comprehensive insight into the impact of the entire process of intervention. Unfortunately, in the present study, since a mixed study was not scheduled, we can only provide comments from participants that, despite being valuable, lack the validity and reliability of a true mixed study.