Ascertaining the Online Learning Behaviors and Formative Assessments Affecting Students’ Academic Performance during the COVID-19 Pandemic: A Case Study of a Computer Science Course
Round 1
Reviewer 1 Report
The study presents the ascertain the variables affecting students’ academic performance for one computing course. It is interesting topic, and used correlation and regression Analysis between learning behavior Variables and the Total. If the following points could be improved, it would be better.
(1) how about the method that is applied to other courses not just Algorithm course?
(2) How about questionnaire or question-based experiment for students’ interests, prior knowledge and so on.
Author Response
Comment 1: How about the method is applied to other course, not just Algorithm course
Answer: Thank you for your comment. One of the reasons why this course was selected as the same instructors were involved in teaching the two cohorts. Another reason is the course is a core unit involved programming concept and implementation where the students were required to learn the data structure concept and coding together. The course reflects both theory and implementation of a computer science course. Finally, this course assessment type includes formative and summative assessments which meet the research conceptual framework and the findings/results could be used to answer the research questions.
Comment 2: How about questionnaire or question-based experiment for students’ interest, prior knowledge and so on.
Answer: Thank you for your comment. “Questionnaire for students’ interest, prior knowledge and so on” was stated as future work in the paper which could be further investigated with a different research objective and scope.
The research used the LMS log taken in 2020 during the COVID-19 outbreak, to ascertain the online learning behaviours and formative assessments affecting students’ academic performance during the COVID-19 pandemic. The two cohorts of students had completed the computer science program.
Reviewer 2 Report
I would recommend peering review the article with an English native speaker due to some grammar inconsistency.
In studies involving endusers, ethical clearance should be obtained through a University Ethical Committe not through "written requests to the university and the course’ lecturer" (p. 5).
The article is well-referenced but as the authors mention is limited to a specific course of computer science.
Time spent on assignment is not detailed in table 3.
Conclusions are general, and no mention is made about the overload of work for the teacher if individual feedback for formative assessment is provided.
Author Response
Comment 1: Peer review the article with English native speaker due to some grammar inconsistency.
Answer: Thank you for your comment. Proofread was performed by a third reader.
Comment 2: In studies involving end users, ethical clearance should be obtained through a university ethical committee not through “written requests to the university and the course lecturer”.
Answer: Thank you for your comment. The sentence has been rephrased. “The data used in this research is ethically obtained with written requests to the University Human Research Ethics Committee.”
Comment 3: The article is well-referenced but as the authors mention is limited to a specific course of computer science.
Answer: Thank you for your comment. The authors decided to report a specific course since the prior researchers have pointed out that single-course predictive models are generally effective in modelling students’ performances; and, the models are not transferable to other courses (Riestra-Gonz´alez et al., 2021).
Comment 4: Time spent on assignment is not detailed in table 3.
Answer: Thank you for your comment. The error has been rectified. The detail was misaligned in table 3.
Comment 5: Conclusions are general, and no mention is made about the overload of work for the teacher if individual feedback for formative assessment is provided.
Answer: Thank you for your comment.
The conclusions given are as follow “These findings suggest that lecturers could predict final course performance based on formative assessments and online learning behaviours during the course. Therefore, lecturers could provide adequate interventions and feedback to students for the duration of the course. The present study highlights that the gap of the predictive power between formative assessments’ results and online learning behaviours for academic success is small.”
The authors have completed a thorough check of the paper and the literature. It is unclear how “the overload of work for teacher if individual feedback for formative assessment is provided” should be included in the conclusion as it was not mentioned in the literature and in this study.
Reviewer 3 Report
Overall, this is an interesting study about academic success in online learning. That said, there were several areas to improve on. First, I wanted more information about why this course was chosen. What is it about a computer science course that is interesting or relevant to look at? Second, the authors do address the limitations of the study, particularly how there is no data about how students performed in the course before the shift to online learning. This to me is a big oversight and one that should be addressed earlier in the paper. That said, I liked how the authors are looking at technological literacy as a way to better understand online learning. That is key.
Another concern was that the authors had a lot of variables they were looking at in this study. These didn't seem to come from the literature review, so I would encourage them to discuss why those variables were chosen earlier in the paper. For example, why is it important to look at the frequency of students' accessing materials?
The findings are not surprising: time management and active participation are predictors of student success. The suggestion is for instructors to monitor these aspects to ensure student success. I believe the authors could go further here. What other ways might we think about active participation? Some suggestions could be really useful for pushing instructional methods forward.
Author Response
Comment 1: First I wanted more information about why this course was chosen. What is it about a computer science course that is interesting or relevant to look at?
Answer: Thank you for your comment. One of the reasons why this course was selected as the same instructors were involved in teaching the two cohorts. Another reason is the course is a core unit involved programming concept and implementation where the students were required to learn the data structure concept and coding together. The course reflects both theory and implementation of a computer science course. Finally, this course assessment type includes formative and summative assessments which meet the research conceptual framework and the findings/results could be used to answer the research questions.
Comment 2: The authors do address the limitations of the study, particularly how there is no data about how students performed in the course before the shift to online learning. This to me is a big oversight and one that should be address earlier in the paper.
Answer: Thank you for your comment. The classes were conducted face to face before COVID -19 as mentioned in the Overall Context section of the paper. The data related to students’ academic performance before the shift to online learning and the academic staff involved are not available to the authors. The student performances before COVID -19 could have provide more insight before the online learning. Comparative study of student performance before covid-19 and during covid-19 is not the aim of this study. It could be further investigated with a different research objective in future.
Comment 3: Another concern was that the authors had a lot of variables they were looking at this study. These didn’t seem to come from the literature review. So I would encourage them to discuss why those variables were chosen earlier in the paper.
Answer: Thank you for your comment. The justification of the variables used were presented in the research framework section of the paper and the variables were discussed in the Results and Discussion section as well.
Comment 4: what other ways might we think about active participation? Some suggestions could be really useful for pushing instructional methods forward.
Answer: In the Conclusions section of the paper, the authors suggested to incorporate qualitative research approach to further explain the categories/predictors identified including active participation.
