Embedded Information Problem-Solving Instruction to Foster Learning from Digital Sources: Longitudinal Effects on Task Performance
Round 1
Reviewer 1 Report
The paper is a comprehensive and reliable study over interesting and important topic. Paper is well structured and well written. The methodology applied for research is well selected and described. Despite of some problems in designing control group for the research, the sample is justified and well structured. The results are comprehensively presented and properly used for interpretation. The literature of the subject is well covered and the results are discussed with similar studies. The conclusions are meaningful from scientific as well as from practical point of view.
There are some minor concerns about the paper especially referring to the selection of experimental and control group but only some of them are addressed in the final chapter of the paper (initial IPS skills of students) while some remain unaddressed (p.e. teaching program / teachers approach differences in schools hosting control and experimental group, or performance of students from different schools). There are also some ideas that could be addressed and perhaps explained by authors. For example, there is no definition what is considered “long-term” instruction. But the drawbacks presented above does not undermine the overall quality of the paper.
One sentence should be also devoted to the illustration of the results that is mostly done in tables. Perhaps, using figures more would be a good solution to get better perception of the results among readers.
Author Response
We would like to thank the reviewer his/her comments and suggestions to improve the paper.
Next, we provide a response to all reviewer's comments and suggestions.
Reviewer's comment: There are some minor concerns about the paper especially referring to the selection of experimental and control group but only some of them are addressed in the final chapter of the paper (initial IPS skills of students) while some remain unaddressed (p.e. teaching program / teachers approach differences in schools hosting control and experimental group, or performance of students from different schools).
Response: We have tackled this suggestion by adding the next paragraph in section: 4.1. Limitations and implications for future research (lines 570-573)
Furthermore, our research did not study the impact of pedagogical variables, such as school teaching approaches or school teaching programmes, on students’ results. In future studies, possible differences of the participant schools in key variables related with teaching should be monitored and considered.
Reviewer’s comment: There are also some ideas that could be addressed and perhaps explained by authors. For example, there is no definition what is considered “long-term” instruction.
Response: We have tackled this suggestion by adding the next paragraph in section: 1.2.3. Long-term instruction (lines 204-206).
Long-term instruction for learning has been considered as the instruction that lasts over a quarter of the academic year [55], or even as the instructional course that may take place over two or three weeks [56].
Reviewer’s comment: One sentence should be also devoted to the illustration of the results that is mostly done in tables. Perhaps, using figures more would be a good solution to get better perception of the results among readers.
Response: We have tackled this suggestion by adding Figure 4 and give another illustration of the results.
Furthermore, we have thoroughly revised the language and spell check.
Reviewer 2 Report
The research is interesting and its presentation is appropriate.
The instrument (whole-task IPS instruction) is adequately explained, but nevertheless the levels of validity and reliability are not mentioned (point 2.3).
Author Response
We would like to thank the reviewer his/her comments and suggestions to improve the paper.
Next, we provide a response to all reviewer's comments and suggestions.
Reviewer's comment: The instrument (whole-task IPS instruction) is adequately explained, but nevertheless the levels of validity and reliability are not mentioned (point 2.3).
Response: We have tackled this suggestion by adding the next paragraph in section 2.5. Measurements (lines 385-390).
The study considered reliability and validity issues. For variables 1 and 2, two raters, familiar with both the IPS tasks and the materials, coded 15% of all the answers protocols. Interrater reliability computed on this subsample of protocols yielded a Cohen’s Kappa higher than .80. One rater scored the remaining protocols. For variable 3, each participant essay was reviewed by two raters. Discrepancies were solved using a consensus-based approach. Member checking is a well-established procedure to build up “trustworthiness” in qualitative research [67].
Furthermore, we have thoroughly revised the language and spell check.
Reviewer 3 Report
The research is performed at a high scientific level. It has significant scientific interest for readers and is relevant. It can be published as follows.
Author Response
We would like to thank the reviewer his/her positive comments about our paper.
We have thoroughly revised the language and spell check.
