Background Similarities as a Way to Predict Students’ Behaviour

Round 1

Reviewer 1 Report

The work is well presented, it is novel in the Spanish context and the applications of the study can be very useful in on-line training processes.

The SAKAI platform is precisely described and the algorithms selected for the study are duly justified.

The cited authors are relevant and the justification and bibliographic review are adequate.

However, some questions can be raised for improvement:

- Describe in depth the mortality of the sample, reasons why it has been reduced.

- Explain the generalization problems derived from the context and the sample.

- Improve the appearance of the graphs and images selected in the paper.

 

Author Response

Dear reviewer, thank you so much for your kind and thorough review. Following, we provide some responses to your concerns. Do not hesitate requesting any further information. Best regards. Daniel Burgos

==

However, some questions can be raised for improvement:

REVIEWER COMMENT (RC): Describe in depth the mortality of the sample, reasons why it has been reduced.

RESPONSE (RS): A further explanation has been included in the section "Limitations of the study". An explanation has been included in the section "Background"

RC: Explain the generalization problems derived from the context and the sample.

RS: A further explanation has been included in the section "Limitations of the study". An explanation has been included in the section "Background"

RC: Improve the appearance of the graphs and images selected in the paper.

RS: the images have been improved, and the two last images have been removed, in line with the further editing of the text

Reviewer 2 Report

Overall, the paper could only provide some supports for Sakai. The methods used in the paper are not new or innovative. The paper needs to have a more structured focus. Decide if you want to make it be technical-focused or educational-focused. For either case, please adopt a proper writing style.

Here my thoughts:

the first paragraph that talks about Sakai has no citation. For the claims made in that paragraph, you need to cite proper documents. lines 55-57: provide sample citations for each of the mentioned goals. the introduction requires extensive editing. Several grammatical mistakes. there are a couple of lines that are disconnected from the text. For example lines 84-85 "For instance, research by Siemens & Baker (2012), was performed to investigate how data mining techniques can be successfully used for adaptive learning." has no connection to the previous or next sentence. it can be safely removed. furthermore, the Siemens & Baker (2012) says nothing about Sakai you need to use proper terms. We don't' have the term "game learning environment" (line 98-99). the introduction and background sections are not doing a good job in preparing readers. In my opinion, you should start with an overview of EDM and just listing some of the most frequently applied features of EDM. Then talk about Sakai and how it's different/similar compare to other open-source LMs like Moodle. Then point out to the gap in the literature on Sakai and pave the road for this study. line 140, in "He proposes two..." who is "He"? the paper talks about the kind of data that were extracted (i.e., Table 1) but there's no justification as to why those data deemed relevant and appropriate. in line 221, what is "r34t45t45"? there are no information about the sample. there are no information about the cognitive style questionnaire used in this study and it's not clear why author(s) chose MBTI. There has to be a methods section describing all this information. the part on comparing different methods (Tables 2-3) is not serving any useful purpose in this paper. lines 328-334: it seems that the levels of success were determined but the model. I would argue that this can potentially result in an arbitrary classification of achievement. All universities have pre-defined categories of academic performance that reflect their standards and policies. This should be used as criterion for success. lines 342-344: what is the basis for this claim "Therefore, the teacher could adopt materials that oppose the cognitive styles, that is, they could try introducing Intuitive and Perceivers cognitive style that might be beneficial to “other” cluster of students"? What teaching-learning theory supports this? the graphs in figure 3-6 are not user-friendly (i.e., hard to read and follow). Please other forms of representation. At least, start with a clear explanation of one graph and then move to other ones.

Author Response

Dear reviewer, thank you so much for your kind and thorough review. Following, we provide some responses to your concerns. Do not hesitate requesting any further information. Best regards. Daniel Burgos

==

REVIEWER COMMENT (RC): Here my thoughts: the first paragraph that talks about Sakai has no citation. For the claims made in that paragraph, you need to cite proper documents

RESPONSE (RS): Cavus, N., & Zabadi, T. (2014). A comparison of open source learning management systems. Procedia-Social and Behavioral Sciences, 143, 521-526.

RC: lines 55-57: provide sample citations for each of the mentioned goals

RS: Gronlund, N. E. (1998). Assessment of student achievement. Allyn & Bacon Publishing, Longwood Division, 160 Gould Street, Needham Heights, MA 02194-2310.

RC: the introduction requires extensiveediting. Several grammatical mistakes. there are a couple of lines that are disconnected from the textFor example lines 84-85 "For instance, research by Siemens & Baker (2012), was performed to investigate how data mining techniques can be successfully used for adaptive learning." has no connection to the previous or next sentence. it can be safely removed furthermore, the Siemens & Baker (2012) says nothing about Sakai you need to use proper terms. We don't' have the term "game learning environment" (line 98-99)

RS: these two references show a required background to understand EDM and Sakai. Both references relate to these two basic foundations of our research. In the section "Background" a number of techniques and theories on the paper's topic are presented to support its research ground. All are connected. We have improved the connectors and writing flow, when needed

RC: the introduction and background sections are not doing a good job in preparing readers. In my opinion, you should start with an overview of EDM and just listing some of the most frequently applied features of EDM. Then talk about Sakai and how it's different/similar compare to other open-source LMs like Moodle. Then point out to the gap in the literature on Sakai and pave the road for this study

RS: some editing has been carried out, when found appropriate. A contextualisation about LMSs and adaptive learning is also included, along with the references:

De Bra, P., Pechenizkiy, M., Van Der Sluijs, K., & Smits, D. (2008, June). GRAPPLE: Integrating adaptive learning into learning management systems. In EdMedia+ Innovate Learning (pp. 5183-5188). Association for the Advancement of Computing in Education (AACE).

De Bra, P., Smits, D., Van Der Sluijs, K., Cristea, A. I., Foss, J., Glahn, C., & Steiner, C. M. (2013). GRAPPLE: Learning management systems meet adaptive learning environments. In Intelligent and adaptive educational-learning systems (pp. 133-160). Springer, Berlin, Heidelberg.

RC: line 140, in "He proposes two..." who is "He"?

RS: corrected. "These two researchers..."

RC: the paper talks about the kind of data that were extracted (i.e., Table 1) but there's no justification as to why those data deemed relevant and appropriate.

RS: these inputs are the basic features of UNIR's methodology framework

RC: in line 221, what is "r34t45t45"? there are no information about the sample

RS: a reference mistake from the reference manager. Proper citation inserted

Chin, W. W., Salisbury, W. D., Pearson, A. W., & Stollak, M. J. (1999). Perceived cohesion in small groups: Adapting and testing the perceived cohesion scale in a small-group setting. Small group research, 30(6), 751-766.

RC: there are no information about the cognitive style questionnaire used in this study and it's not clear why author(s) chose MBTI

RS: in page 5 there is a direct reference to MBTI and the cognitive approach: "The MBTI has 100 forced-choice items that are part of the four bipolar scales, that is, EI (Extraversion-Introversion), TF (Thinking-feeling), SN (sensing-intuition), and JP (Judging- perception). Combining the four dimensions result in 16 different types of cognitive functioning". Redaction modified accordingly

RC: There has to be a methods section describing all this information. the part on comparing different methods

RS: thanks for the suggestion. We consider that methods are described along the paper in a sufficient manner; our focus is on the study and interpretation of the tests

RC: (Tables 2-3) is not serving any useful purpose in this paper

RS: they show the default paraments using RapidMiner (table 2) and the pre-processing values (table 3); both are key to understand the AI applied to this study

RC: lines 328-334: it seems that the levels of success were determined but the model. I would argue that this can potentially result in an arbitrary classification of achievement

RS: the level of success leans on the combination of the student's performance, the implemented model and the support layer provided to the participants. We believe that pages 2 and 4 support this statement properly

RC: All universities have pre-defined categories of academic performance that reflect their standards and policies. This should be used as criterion for success.

RS: thanks for the recommendation

RC: lines 342-344: what is the basis for this claim "Therefore, the teacher could adopt materials that oppose the cognitive styles, that is, they could try introducing Intuitive and Perceivers cognitive style that might be beneficial to “other” cluster of students"? What teaching-learning theory supports this?

RS: supported by the following references:

Allinson, C. W., & Hayes, J. (1996). The cognitive style index: A measure of intuition‐analysis for organizational research. Journal of Management studies, 33(1), 119-135.

Reeve, J. (2009). Why teachers adopt a controlling motivating style toward students and how they can become more autonomy supportive. Educational psychologist, 44(3), 159-175.

Riding, R., & Rayner, S. (2013). Cognitive styles and learning strategies: Understanding style differences in learning and behavior. David Fulton Publishers.

RC: the graphs in figure 3-6 are not user-friendly (i.e., hard to read and follow). Please other forms of representation. At least, start with a clear explanation of one graph and then move to other ones.

RS: selected graphs have been improved or editing, accordingly

Round 2

Reviewer 2 Report

There is still a lot of editing required to make this paper worthy of publication. For example, line 85 "There are a number of survey and researches on the EDM (Educational Data Mining) that exists today" should read "There are a number of surveys and studies on the EDM (Educational Data Mining) that exist..." The comment have been taken at face value or not been taken seriously at all. I still don't see any theoretical framework being used to convey this study. The Gronlund, N. E. (1998) citation is a textbook on teaching classroom assessment for pre-service teachers. It talks about the relationship between assessment and instruction, how to make and plan for assessment and how to develop summative assessments in the classroom. It has nothing to do with "course adaptation and learning recommendations based on students’ learning behaviour". there's still not enough information about the sample or adequate information about the measurement tools used to collect data in this study.

Author Response

Dear reviewer,

following your comments we have carried out an extensive revision of the paper, and we have addressed all the concerns sent a few days ago

We sincerely hope to meet your expectations

Best regards,

Daniel Burgos

==

COMMENT FROM THE REVIEWER (CR):

There is still a lot of editing required to make this paper worthy of publication. For example, line 85 "There are a number of survey and researches on the EDM (Educational Data Mining) that exists today" should read "There are a number of surveys and studies on the EDM (Educational Data Mining) that exist..." The comment have been taken at face value or not been taken seriously at all.

RESPONSE TO THE REVIEWER (RR): An extensive, external and native proof-reading service has been hired for the occasion. I hope that the reviewer finds this in-depth editing in line with his or her expectations. We apologize for any misleading impression abou how serious we take our 5-year research work or how it is properly presented

COMMENT FROM THE REVIEWER (CR):

I still don't see any theoretical framework being used to convey this study. The Gronlund, N. E. (1998) citation is a textbook on teaching classroom assessment for pre-service teachers. It talks about the relationship between assessment and instruction, how to make and plan for assessment and how to develop summative assessments in the classroom. It has nothing to do with "course adaptation and learning recommendations based on students’ learning behaviour"

RESPONSE TO THE REVIEWER (RR): Gronlund’s research is focused on assessment, and we take assessment as a tool to learn from the student’s behaviour, to produce personalised recommendations. We also take Gronlund’s input as an innovative approach by the time of his publication, in 1998, a very early stage on the Internet, and with no competitive LMS on the market. The reviewer’s point is right about the Gronlund’s reference alone does not seem enough. We provide additional, up-to-date foundation references to the approach: Salehi, M., Kamalabadi, I. N., & Ghoushchi, M. B. G. (2013). An effective recommendation framework for personal learning environments using a learner preference tree and a GA. IEEE Transactions on learning technologies, 6(4), 350-363. Sosnovsky, S., & Brusilovsky, P. (2015). Evaluation of topic-based adaptation and student modeling in QuizGuide. User Modeling and User-Adapted Interaction, 25(4), 371-424. Bhat, A., Kamath, D. M., & Anitha, C. (2019). A Novel Deterministic Framework for Non-probabilistic Recommender Systems. In Emerging Technologies in Data Mining and Information Security (pp. 85-97). Springer, Singapore. Lorenzen, S., Hjuler, N., & Alstrup, S. (2019). Tracking behavioral patterns among students in an online educational system. arXiv preprint arXiv:1908.08937.

COMMENT FROM THE REVIEWER (CR):

there's still not enough information about the sample or adequate information about the measurement tools used to collect data in this study.

RESPONSE TO THE REVIEWER (RR): In lines 140-148 we have inserted a specific description about the sample profile and the tool, so that the reviewer and the future readers can contextualise the research better.

Round 3

Reviewer 2 Report

Although I still think it can be improved but I appreciated the effort made by the author(s) to enhance the manuscript. The ultimate goal is to publish appropriate works that could inform both theory and practice.