Teachers’ Beliefs About the Consequences of Grade Retention: Scale Validation and Differences Across Individual and School-Level Factors

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

(1) Use of Cronbach's Alpha: Inappropriate for ordinal data and categorical data that are not normally distributed.

(2) Use of ANOVA: Inappropriate for ordinal and non-normal data; non-parametric methods are recommended.

(3) Lack of Measurement Invariance Analysis: No evidence to ensure the validity of comparisons between groups.

(4) Lack of Independent Ethics Approval: No mention of approval by a university or external ethics committee; reliance solely on state review may be inadequate.

(5) Accessibility of Data: It is especially important that readers have easy access to the dataset, which should be explicitly addressed in the manuscript.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review your manuscript. I believe that the attempt to validate the Beliefs on Consequences of Retention (BCR)-scale is of high quality and adds valuable insights to the grade retention literature. The paper is well-written and follows a clear structure, which makes it easy to follow. Overall, I believe the work aligns with the current state of the art, and I would recommend a minor revision. Below, I outline several specific suggestions that, in my view, would strengthen the manuscript.

 

Abstract

* For me, it was not clear what the E/CFA framework specifically entails when I was reading the abstract. It is explained in the article, but the authors could opt to ommit the framework from the abstract for clarity purposes. After all, the abstract already mentions that it is a validation of the internal consistency, reliability, and unidimensionality.

 

Introduction

* page 1: "Likewise, its application largely depends on the perceptions about grade retention of those who decide how to manage heterogeneity within educational systems and schools" - I believe that a reference to the work of Dupriez et al. (2008) on heterogeneity management models would be considerate here, as they conceptualized grade retention as a differentiation mechanism.

* page 2: "Exploring these beliefs in greater depth can help reveal the dynamics underpinning grade retention practices within an educational system". I do not understand entirely what is meant by this sentence. Is it perhaps possible to clarify?

* page 2: I like the historical overview of Chilean policy-making with regard to grade retention. Well done!

* page 3: "Overlooking potential negative consequences, such as school disengagement, worsening behavioral issues, and impacts on self-esteem and self-concept, all in the pursuit of improving the academic performance of those who struggle the most". The section on non-cognitive outcomes of grade retention should be expanded or, at the very least, provided with sources. Which studies specifically state the noted effects on school disengagement, worsening behavioral issues and the impact on self-concept?

* page 3: "This perception is related to Chile's highly valued meritocratic discourse (PNUD, 2017)": is the idea of meritocracy exceptionally prevalent in Chile? After all, meritocracy is a widespread ideology across the western world (and beyond). Is it that much more common that it would lead to different teacher's beliefs on grade retention, in comparison to other countries? Are there cross-national studies that compare these beliefs across different countries? In any case, a bit more historical background on why meritocracy seems to be more common in Chile than elsewhere may be appropriate.

* page 3: "This logic overlooks other reasons that may lead to lower grades and thus to repeating a grade, such as illnesses and other adverse childhood experiences, including material hardship, family and domestic dysfunction, and exposure to violence, among others (Hinojosa et al., 2019)". If the authors wish to do so, they could mention the distinction between internal and external causes (or attributions) of school failure.

* page 3: "However, they will also consider potential adverse effects, such as peer rejection and increased behavioural problems". What does the research say on peer rejection and increased behavioral problems? Try to substantiate these claims immediately by backing them up with adequate sources. This also pertains to the earlier comment that non-cognitive outcomes of grade retention are mentioned only briefly but not substantiated enough. That is important, as the validated scale contains several items on these non-cognitive items.

* page 4: "(...), greater importance will be given to the opportunities this mechanism may offer students who need to improve their development". I do not understand entirely what is meant by this sentence. Could the authors perhaps clarify this?

* page 4: "It could have negative effects on psychological, economic, health, and even social terms in the short, medium, and long term (Goos et al., 2021)": see above. What kind of effects? And why? How does this contrast with the overly optimistic attitudes of teachers?

* page 4: "As a result, perceptions of this mechanism that align with these criticisms acknowledge the multiplicity of factors that could be key in perpetuating social stratification, which practices like grade retention may generate, making it more difficult for students with less social and cultural capital to improve and advance in their educational trajectories". I believe this sentence could be made less complex, which will enhance its readability. Splitting it up may be a valid option.

Materials and data analysis

- page 6: "Additionally, teachers were asked whether the availability of complementary support could determine grade retention, as the new Decree 67 explicitly states that students repeating a grade must receive academic support". Very interesting. What kind of complementary support?

- page 6: I like the clear, transparent step-by-step explanation on how the authors performed their factor analysis.

- page 7: Small spelling erorr, a redundant 'the': "(..) and the of the grade retention rate in the first cycle of primary education (grades 1 to 4)"

Results and discussion section

- page 8: "The dataset contains 4,297 observations, corresponding to all 4th-grade teachers from Chilean schools who thoroughly answered the questions (...)". What is meant by 'thoroughly answered'? How is this operationalized? Rephrasing may be a valid option, as 'thoroughly' carries an unnecessarily subjective loading. 

- page 8: "Of these respondents, approximately 88% are women, and about 12% are men". I assume the gender imbalance is representative for the overall Chilean teacher population in primary education, as this would be in line with most other western(ized) countries. Nonetheless, this could be explicitly stated for the reader. Was the non-response rate higher for men? The fact that female teachers in primary education are more optimistic on grade retention may be due to a selection effect of male teachers. The group of male teachers is so small that it can be assumed particular groups of men decide to teach in primary education, for instance more progressive men, men who are less status-oriented and men who are less prone to follow social roles. I let the authors decide if that is something noteworthy (as it may be out of scope for this particular article), but this may be different from teacher gender differences in secondary education.

- page 10: "Finally, the construct that we will call Beliefs on Consequences of Retention scale (BCR scale) comprises six items". Something that is noteworthy, is that the omitted items are also beliefs on consequences of retention. Could the authors perhaps reflect on the observation that the name of the scale remains rather broad? Is rephrasing perhaps an option, so that it does not pertain to items that are no longer part of the final scale? In summary: what binds the items that does exclude the other items?

- page 11: was there a check for multicollinearity? After all, schools with a high/low retention rate may also be schools with a distinct school population, a distinct program, from a distinct geographical area, et cetera. This is an important limitation.

- page 12: Including the significance levels in Table 4 would provide a clearer understanding of the statistical results and their implications

- page 12: "Additionally, the unidimensional structure was tested through a CFA, which demonstrated a good fit for the items comprising this construct". Yes, that is true. But what do the excluded items then reflect?

- page 12: I think it may be important to note that the noted differences are significant but small. After all, that is in line with other studies on the school retention composition.

- page 13:  "In contrast, teachers in schools with low retention rates may have less supportive perceptions of retention, possibly due to a stronger focus on alternative educational strategies to manage student heterogeneity". It would be helpful to provide examples of alternative educational strategies here.

- page 13: "In line with the notion of negative reinforcement, teachers operating under this logic may overlook the potentially negative consequences of grade retention, such as emotional distress, disengagement from school, and damage to self-esteem". These are factual statements on non-cognitive outcomes of grade retention. Hence, I believe it is a good idea to back these statements up with studies that show the effect of emotional distress, self-esteem or self-concept and disengagement from school.

- page 13: "On the other hand, belief systems applied to education are the ideas that teachers, students, or institutions hold about teaching, learning, and knowledge (Wolf & Brown, 2023). These systems influence how teachers teach and interact with students, how students learn, and how the educational environment within the classroom is structured". Santos et al. (2023) found that teacher's beliefs on grade retention are a reflection of their central pedagogical beliefs. The authors could opt to mention this as well.

 

At last some comments, in no particular order:

- It becomes clear soon why Chile is a good, suitable case study for your undertaking. Decree 67 is explained clearly, and the outcome of the degree will be of interest to scholars in the field beyond the national context. It will be interesting to see how the overoptimistic teacher attitudes clash with the top-down approach that the Chilean government currently seems to implement (if I understand it correctly). What I miss, however, is an oversight on the prevalence of grade retention. It is stated that current data are hard to find because of government intervention, but what do historical data say?

- In addition to this, the literature study also lacks an insight in the precise decision-making process on grade retention in Chile. Are teachers the sole decision-maker to retain a given student or not? Or do parents, scores on standardized tests, parents, etc. also play a role? Do teachers decide individually or in group? Is there a teacher council of some sorts? This is important, as the specific decision-making process tells us something about the potential link (or lack thereof) between teacher's beliefs and grade retention rates. I believe such a reflection would further strengthen the paper.

- Can the authors reflect on the potential use of the scale in secondary school? Should the scale be confined to primary school research, or is a wider application possible? What are potential challenges and pitfalls in this regard?

- In the discussion section, I miss a segment on how to deal with the resistance that teachers feel against the scientific state of the art on grade retention and 'a more holistic, inclusive approach on education', as the authors stated. Can we change the beliefs of teachers? Should we? And, if so, how? Should policy-makers take into account the overoptimism of teachers? If so, how? Some of these questions could be answered by the authors in the discussion section.

- References should be uniform. Right now, some are capitalized while others are not.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Title: Teacher-Perceived Beliefs of the Consequences of Grade Retention in Primary Schools: Adaptation and Validation of the Grade Retention Survey

This is a very relevant topic. The sample that is drawn (4297 teachers from the fourth basic level of Chilean schools) is very important.

Teachers' beliefs regarding their students' repetition. This is a theoretical construct that is difficult to consider homogeneous. That is, it is not possible to differentiate beliefs about one aspect or another in relation to the academic performance of students for the next course, emotional implications of students who repeat, ..., family implications, ...

On the other hand, the work of adapting and validating a scale involves carrying out a methodological process, which begins with the justification of why that scale is used and not others, with a theoretical analysis of the factorial construct of the instrument, the translation of the original scale, and the application of the scale to a representative sample.

Next, the reliability and statistical validity of the questionnaire is assessed. This is a factor analysis, that is, assessing how the observed variables or items are grouped into factors or theoretical constructs. These theoretical constructs are those that should be able to explain the belief construct. Exploratory Factor Analysis is the first step, identifying the factors that can explain the main construct.

Confirmatory Factor Analysis allows us to verify (validate) this structure. It is advisable to replicate the cycle of Exploratory Factor Analysis – Confirmatory Factor Analysis as many times as necessary. Finally, it will be necessary to verify the configural , metric, scalar and strict invariance of the instrument to verify the theoretical consistency of the proposed model or factor structure. There must be theoretical consistency of the proposed statistical model.

In my opinion, the comparative analysis of means can justify the development of differentiated scales taking into account different factors. This can justify the use of ANOVA or Student 's t test (depending on the number of categories of the variable used as a comparator, so if the variable is male or female, it would be reasonable to use Student 's t test ).

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Although I do not fully agree with the methodological decisions, I think it is important that arguments have been incorporated in this regard. The work denotes an important improvement and is a contribution to the subject being addressed.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Some of the problems that were pointed out in the first review still persist.

Among the most relevant are:

1)      The need to clearly explain the research objectives and the revision of the title (see guidelines in the discussion section).

2)      Include a section detailing the participants

3)      A section identifying the variables used is still necessary.

4)      Indicate the R package used. Identify the analysis techniques that have been performed with STATA and those that have been used in the statistical package in R.

Comments for author File: Comments.docx

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 3

Reviewer 3 Report

Comments and Suggestions for Authors

Dear authors, thank you for the opportunity to read your work. Thank you for your consideration and good work.