Testing the Feasibility and Impact of Train-the-Trainer Delivery for a Peer Tutoring Reading Programme in Chile
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsStrengths Highlighted by Reviewers
- This study examines a beneficial reading program in the community.
- The questionnaire instrument in line 379 is quite relevant.
Areas Requiring Revision
- Please explain in line 11 of the abstract the feasibility measurement instrument, which should not rely solely on teacher opinion but rather measure student ability with a clear reading instrument.
- Please explain the improvement in children's outcomes in the preliminary study in line 70 and its relevance to this study, especially if the same instrument or reading material is used.
- If students' paired reading ability is measured through self-correction or are corrected, as stated in line 107, then the research results should reflect this. Similarly, the indicator "pairs discuss the text, ask questions about it, and reflect on their behaviors during paired reading" should appear in the research results.
- Please explain in more detail the instrument for the Spanish version of the Paired Reading program, in line 166.
- The three research questions in line 179 are yes-no questions, although they require extensive explanation or statistical analysis.
- Research question number 2 on line 182 should explain how to improve children's reading outcomes with clear indicators.
- If question 3 on line 184 is an opinion, then the results of this opinion are irrelevant, just as public opinion about whether food is tasty or not tasty cannot be used as a benchmark for whether people should eat or avoid certain foods.
- Please explain the reading material instrument on line 211.
- Although the questionnaire on line 379 is quite relevant, the reading instrument and reading evaluation are not clearly outlined as per the Paired-Reading Program standards.
Comments for author File: 
Comments.pdf
Author Response
We thank the reviewer for these helpful comments and have addressed these in the manuscript.
Comment 1: Please explain in line 11 of the abstract the feasibility measurement instrument, which should not rely solely on teacher opinion but rather measure student ability with a clear reading instrument.
RESPONSE 1: We have clarified the fact that a reading assessment was used to measure student reading ability. Results indicate the Paired Reading train-the-trainer programme (Latin American Spanish version) is feasible to implement in elementary schools in high poverty areas in Chile, and capable of improving children’s reading ability as measured by an independently designed standardized reading assessment.
Comment 2 : Please explain the improvement in children's outcomes in the preliminary study in line 70 and its relevance to this study, especially if the same instrument or reading material is used.
RESPONSE 2: We clarify further the difference between the prior version of Paired Reading which included ‘developer led’ delivery, and the present study which designed a train the trainer model of delivery.
This study builds on previous findings of using Paired-Reading in the UK, and on findings from the first Latin American Spanish version of Paired-Reading, where training was delivered by the programme developers to teachers in schools in Colombia. Findings indicated that this programme could be transferred to Spanish language contexts in Latin America, and positive improvements of approximately two additional months were detected for children’s reading (Author removed, 2023). It is clear given the crisis in reading attainment in this region, that a sustainable and scalable solution is required to improve reading standards for children and young people, using cost-effective interventions.
Drawing on the literature about scaling programmes through delivery by trained trainers who are not programme developers, a workshop and cascade train-the-trainer (TTT) model was designed during this study to deliver Paired-Reading in Latin America. We present here the findings of this matched study, including six Grade 6 classes and 98 children, to understand whether TTT delivery of the Paired-Reading programme in Chile was implementable and capable of improving children’s outcomes and detecting similar effects in reading as the previously tested develop-led version of Paired reading.
Comment 3: If students' paired reading ability is measured through self-correction or are corrected, as stated in line 107, then the research results should reflect this. Similarly, the indicator "pairs discuss the text, ask questions about it, and reflect on their behaviors during paired reading" should appear in the research results.
RESPONSE 3: Given the small nature of the study with limited funding, the effectiveness of the Paired Reading technique on student reading was measured using an independently designed standardized online reading test which was low cost and with high reliability. Therefore we now mention in the limitations section the need to undertake evaluation using student’s Paired Reading observations to assess various elements and behaviours of Paired Reading including pairs reading together, tutee self-correction, tutor correction, and questioning.
Comment 4: Please explain in more detail the instrument for the Spanish version of the Paired Reading program, in line 166.
RESPONSE 4: This study tested the Spanish version of the Paired Reading programme which included a teacher manual and student resources for use during the reading sessions (Author removed, 2023).
Comment 5: The three research questions in line 179 are yes-no questions, although they require extensive explanation or statistical analysis.
RESPONSE 5: The research questions are kept quite succinct although the methods section then explains the measures which used to answer the research questions.
Comment 6: Research question number 2 on line 182 should explain how to improve children's reading outcomes with clear indicators.
RESPONSE 6: We have clarified Research question 2 as follows: Do children who engage in the process of Paired-Reading improve their reading outcomes as measured by an independently designed digital standardized reading test?
Comment 7: If question 3 on line 184 is an opinion, then the results of this opinion are irrelevant, just as public opinion about whether food is tasty or not tasty cannot be used as a benchmark for whether people should eat or avoid certain foods.
RESPONSE 7: We have removed question 3 from the manuscript.
Comment 8: Please explain the reading material instrument on line 211.
RESPONSE 8: We have now included this in the manuscript - The ‘five finger test’ includes the following process: Once the tutee selects a book they believe will interest them, they open the book on a random page and place five fingers onto the page. The tutee then attempts to read the words under the fingers. The tutee then repeats this for another four pages. If they can read all the words on the five pages, the book selected is too easy. Tutees are advised to choose a book where they make between about 2–3 errors (out of the 25 words read).
Comment 9: Although the questionnaire on line 379 is quite relevant, the reading instrument and reading evaluation are not clearly outlined as per the Paired-Reading Program standards.
RESPONSE 9: We have added more descriptive information regarding the Dialect assessment platform, including its theoretical background, areas assessed, duration, measures reported, and duration. We also explain how the test flowchart adjusts based on student performance
Reviewer 2 Report
Comments and Suggestions for AuthorsThe manuscript titled "Testing the feasibility of train the trainer delivery for a peer tutoring reading programme in Chile" demonstrates the Paired Reading train-the-trainer programme (Latin American Spanish version) is feasible to implement in elementary schools in high poverty areas in Chile, and capable of improving children’s reading ability. These findings have some implications for education in Chile. However, these contributions are not adequately highlighted in the current manuscript. I would suggest the authors revise and improved the manuscript carefully. The main comments are listed below.
1.The use of professional terms lacks consistency and clarity throughout the manuscript. Further refinement is therefore recommended to ensure alignment with academic conventions. Part of specific expressions to be revised is as follows:
(1) Abbreviations should be spelled out in full at their first occurrence. (Line18, 33,... ) All abbreviations must be clearly defined with their full terms upon first occurrence in the manuscript.
(2) Please verify the accuracy of the original terms corresponding to the abbreviations. For example, randomised trial (RCT) or randomised controlled trial (Line 53...).
(3) Terms for which abbreviations have been defined do not need to be repeated in full in later sections (line 142:train-the-trainer (TTT) delivery, line 283: randomised controlled trial, line 184 ... )
(4) A relatively large number of abbreviations are utilized in the manuscript, and the authors are encouraged to reduce their use to enhance the paper’s readability. Specifically, it is suggested that the full expressions be used for abbreviations that appear only once in the manuscript.
Authors are requested to carefully review the use and expression of abbreviations throughout the manuscript.
Abstract
2.The keywords should be modified and improved.
Introduction
3.Compared with previous studies, beyond the difference in the study region, it would be helpful to further clarify what the core innovations of this research are.
4.The significance and necessity of the current study need to be further elaborated to better demonstrate its value to the field.
Methods
5.It is worth considering whether placing the research questions in the Methodology section is appropriate. Additionally, the writing of the Methodology section should follow certain academic conventions and logical flow. I would recommend the authors reorganize the Methodology section in a more coherent logical order. The overall structure of the manuscript may benefit from adjustments to enhance the clarity of the views presented.
- Some sentences are too long to comprehension. For example, Line 230-234.
7.Small participant sample sizes are unsuitable for testing main effects, it would be valuable to elaborate on how the scientific rigor of the effect size testing in this study is ensured.
- It is necessary to clarify class grouping and matching in the Methodology section.
Results
9.It is necessary to specify in the Results section what type of analysis of variance (ANOVA) the authors employed.
10.It is important to note that pre-test score differences existed between the experimental and control groups. Even when these pre-test scores were analyzed as a covariate, the effects of statistical regression cannot be overlooked. In general, scores tend to show greater improvement when moving from low to moderate levels, whereas advancing from moderate to high levels is relatively more challenging. This pattern may have influenced the study’s results. More explanations are needed for this issue.
12.Effect size constitutes a critical outcome in this manuscript, and it is essential for the authors to provide additional clarification on two key aspects: first, how the effect size was calculated, and second, its scientific rationale in the context of this study.Furthermore, relevant references supporting the chosen effect size calculation method should be explicitly included.
Discussion
13.The discussion section does not contain much content and has not yet fully highlighted the value of this study. The theoretical depth requires further enhancement.I would suggest adding further discussions to enrich this section.
Conclusion
14.The conclusion should summarize the main findings of this study.
#Additional Comments
1.Numerous statements in the article require revision.
2.Some inappropriate spacing is present throughout the manuscript.
3.Please carefully check the format of the “References” section.
4.Some issues about “Author removed ”!
Author Response
We thank the reviewer for these helpful comments and have addressed these in the manuscript.
Comment 1: Abbreviations should be spelled out in full at their first occurrence. (Line18, 33,... ) All abbreviations must be clearly defined with their full terms upon first occurrence in the manuscript.
RESPONSE 1: Completed throughout the manuscript (eg PISA, RCT, TTT etc)
Comment 2: Please verify the accuracy of the original terms corresponding to the abbreviations. For example, randomised trial (RCT) or randomised controlled trial (Line 53...).
RESPONSE 2: Addressed in the manuscript throughout.
Comment 3: Terms for which abbreviations have been defined do not need to be repeated in full in later sections (line 142:train-the-trainer (TTT) delivery, line 283: randomised controlled trial, line 184 ... )
RESPONSE 3: Done
Comment 4: A relatively large number of abbreviations are utilized in the manuscript, and the authors are encouraged to reduce their use to enhance the paper’s readability. Specifically, it is suggested that the full expressions be used for abbreviations that appear only once in the manuscript.
RESPONSE 4: Done
Comment 5: Authors are requested to carefully review the use and expression of abbreviations throughout the manuscript.
Response 5: Done
Comment 6: Abstract -The keywords should be modified and improved.
RESPONSE 6: Done
Comment 7: Introduction -Compared with previous studies, beyond the difference in the study region, it would be helpful to further clarify what the core innovations of this research are.
RESPONSE 7 : We have clarified the key difference between this study and previous versions of the Paired Reading programme, as being the development and testing of a train-the-trainer delivery model in preparation for scalability.
Comment 8: The significance and necessity of the current study need to be further elaborated to better demonstrate its value to the field.
RESPONSE 8: This is addressed more clearly now in the introduction.
Comment 9: Methods -It is worth considering whether placing the research questions in the Methodology section is appropriate. Additionally, the writing of the Methodology section should follow certain academic conventions and logical flow. I would recommend the authors reorganize the Methodology section in a more coherent logical order. The overall structure of the manuscript may benefit from adjustments to enhance the clarity of the views presented.
RESPONSE 9: We thank the reviewers for the helpful comments. In response we have places the theory of change and the research questions at the end of the introduction, prior to the methods section.
The methods section has been restructured as follows:
- Introductory paragraph to explain the overall methodology
- Ethical approval
- Inclusion criteria, sample and allocation
- The intervention
- Analysis plan
Comment 10: Some sentences are too long to comprehension. For example, Line 230-234.
RESPONSE 10: Sentence length in this section has been addressed
In addition to the guidance included in the teacher manuals and during teacher training about book selection, during this study, the research team helped the librarians select books. Books were selected as suitable for the students working in pairs, based on ranges of Lexile measures for Chilean grade 6 students. Librarians also ensured that there would be ample variability in topics, difficulty level, and genres. Book boxes were provided for each participating classroom.
Comment 11 .Small participant sample sizes are unsuitable for testing main effects, it would be valuable to elaborate on how the scientific rigor of the effect size testing in this study is ensured.
RESPONSE 11: We agree, but this was a feasibility study. The small sample size gives a number of limitations including pre-test differences, sampling error and we have modified the document. To this effect we have made it more clear that the aim was not to test main effects, but to establish an effect size to help inform a larger subsequent trial that would test main effects. P3 line 102.
Comment 12. It is necessary to clarify class grouping and matching in the Methodology section.
RESPONSE 12: We have included further clarification in the manuscript now on page 5/6: Grade 6 students who received the treatment were therefore matched with Grade 6 students from the same school from the previous academic year who had not received Paired Reading and had received usual teaching, and who had completed the same reading measure.
Comment 13. Results -It is necessary to specify in the Results section what type of analysis of variance (ANOVA) the authors employed.
Response 13: We have added the words one-way and added a reference to IBM P8, line 347.
Comment 14. It is important to note that pre-test score differences existed between the experimental and control groups. Even when these pre-test scores were analyzed as a covariate, the effects of statistical regression cannot be overlooked. In general, scores tend to show greater improvement when moving from low to moderate levels, whereas advancing from moderate to high levels is relatively more challenging. This pattern may have influenced the study’s results. More explanations are needed for this issue.
Response 14: We agree. We have added this discussion as a limitation (but please note this study was not designed to measure main effect, a larger study will be required to do that).
P13 559 onwards
There are other limitations of the study that present statistical challenges. The sample size was small. This may have contributed to pre-test differences. Either way there is a risk that pre-test differences alter the way in which a sample respond to an intervention and this can result in sampling error (Twisk, Bosman, Hoekstra et al., 2018). Minimisation on the basis of pre-test in a larger trial would overcome this. This pre-test differences are problematic and this trial should not be taken as oproviding a definitive analysis of main effects.
Comment 15: Effect size constitutes a critical outcome in this manuscript, and it is essential for the authors to provide additional clarification on two key aspects: first, how the effect size was calculated, and second, its scientific rationale in the context of this study.Furthermore, relevant references supporting the chosen effect size calculation method should be explicitly included.
Response 15: We have included this in the method and the results. We have made reference to the Cochrane handbook. We have included effect sizes for using both the Cohen and Hedges formulae. In fact Hedges’ g and Cohen’s d returned very similar results (g=+0.66, d=+0.67). Even here d was rounded up to 0.67 and g was rounded down to 0.66.
Comment 16: Discussion - The discussion section does not contain much content and has not yet fully highlighted the value of this study. The theoretical depth requires further enhancement. I would suggest adding further discussions to enrich this section.
Response 16: We have revised the discussion section to highlight the value of this study.
Comment 17: Conclusion -The conclusion should summarize the main findings of this study.
Response 17: We have revised the conclusion section.
Comment 18: #Additional Comments
1.Numerous statements in the article require revision. Revisions complete
2.Some inappropriate spacing is present throughout the manuscript. Checked
3.Please carefully check the format of the “References” section. Checked
4.Some issues about “Author removed ”!
Responses to comments 18:
1.Numerous statements in the article require revision.
RESPONSE: Revisions complete
2.Some inappropriate spacing is present throughout the manuscript.
RESPONSE: Revisions completeChecked
3.Please carefully check the format of the “References” section.
RESPONSE: Revisions completeChecked
4.Some issues about “Author removed ”!
RESPONSE: Revisions complete - We have removed author references where possible - In particular we have removed the following references:
(Author removed, 2018) Author removed (2018). Peer Tutoring Manual (2018). (6th ed.) Queen's University Belfast. https://pure.qub.ac.uk/en/publications/peer-tutoring-manual-2018
(Author removed, 2021) Author removed, (2021). Manual: Tutoria entre Pares – Misma Edad (2021). (& ed.). Queen’s University Belfast. https://pureadmin.qub.ac.uk/ws/portalfiles/portal/324742340/Manual_Lectura_entre_Pares_Misma_Edad_2021_.pdf
However, the other references have been included as they are the best evidence to illustrate the points made. We would be glad to exchange these references with another if the reviewer can suggest a better substitute.
Reviewer 3 Report
Comments and Suggestions for AuthorsThis is a dense paper with a large volume of data and which tackles a very interesting topic. The paper is well-designed and it has impact on future research in the field.
Comparisons with similar studies carried out in the UK and Colombia reflect the sustainability of the study; entail the manipulation of large volumes of data and can improve the format of future studies.
Observations:
-I would be a good idea to separate the Introduction from the Literature Review;
-Reread the Conclusions: the verb to demonstrate occurs three times in two sentences;
- Enlarge the Conclusions in order to have a balanced structure of the paper.
Author Response
We thank the reviewer for their helpful observations which we have addressed as follows:
Comment 1: I would be a good idea to separate the Introduction from the Literature Review;
Response 1: We have separated this in the manuscript.
Comment 2: Reread the Conclusions: the verb to demonstrate occurs three times in two sentences;
Response 2: We have revised the conclusion
Comment 3: Enlarge the Conclusions in order to have a balanced structure of the paper.
Response 3: We have revised and enlarged the conclusion.
