Vygotsky’s Systemic Perspectives on Managing the Risk of Student Failure in Technology-Enhanced Learning Design

Response to Reviewer 1's Comments

1. Summary

Dear Editor,

Thank you for allowing us to submit a revised draft of our manuscript titled “Vygotsky's Systemic Perspectives on Managing the Risk of Student Failure in Technology-Enhanced Learning Design” to the MDPI on Applied Sciences. We sincerely appreciate the time and effort you and the reviewers have dedicated to evaluating our manuscript. We are grateful for the insightful comments and constructive feedback, which have helped us improve the quality of our work. This manuscript has addressed all the issues raised during the review process. To facilitate the review, we have highlighted the revisions in red. Our response letter includes:

(1)        A point-by-point explanation of the revisions made in the manuscript,

(2)        Our responses to the reviewers' comments.

We appreciate your consideration of our revised manuscript and await your feedback. Please find the detailed responses below and the corresponding revisions/corrections highlighted/in track changes in the re-submitted files.

2. Questions for General Evaluation

Reviewer’s Evaluation

Response and Revisions

Does the introduction provide sufficient background and include all relevant references?

Must be improved

It is improved. Revisions have been made to enhance the background.

Is the research design appropriate?

Must be improved

It is improved. We have made revisions regarding the RQs and highlighted the research scope.

Are the methods adequately described?

Not applicable

We have added a “Methods” section as suggested.

Are the results presented?

Must be improved

It is improved. We have revised the relevant section to enhance clarity.

Are the conclusions supported by the results?

Must be improved

It is improved. We have made revisions to ensure that conclusions are supported by the results.

3. Point-by-Point Response to Comments and Suggestions for Authors

Comment 1: This theoretical paper applies 'Vygotskian methods and theory to propose ways of managing the risk of failure (students' failure to learn?) within a design approach that incorporates Learning Analytics language in a Backward Design process'. It is a theoretical paper. As such, it would benefit from greater clarity of the concepts that the authors are trying to connect in the paper, specifically Learning Analytics, Learning Design, Design Thinking (?), and Backwards Design (?).

Response 1:

Thank you for your thoughtful feedback. Although “design thinking” was mentioned, in the revised version, we have updated the text to give more focus on Learning Analytics and Backward Design as core elements in the proposed design approach. We have revised the text and the structure to ensure that these concepts are clearly defined and their connections better articulated. Your feedback is very helpful, and we believe these changes will enhance the clarity and coherence of the paper.

 “BWD is a goal-directed design methodology that has pedagogical roots in constructivism [41]. It aims to improve the quality and effectiveness of design towards the achievement of desired learning outcomes. The three principal stages of BWD are:

1.         Identifying desired learning results: The teacher identifies the essential understandings, knowledge, and skills to be mastered by the students. The learning object reflects “big ideas” that are “transferable”, and the focus is on higher-order desired learning outcomes.

2.         Determining acceptable evidence: The teacher identifies student evidence of achievement towards the desired learning outcomes and the methods by which this evidence will be collected and assessed. Emphasis is placed on authentic performance tasks and the use of a variety of assessment techniques.

3.         Planning learning experiences and instruction: The teacher makes decisions on the teaching methods, learning events, and strategies that will meaningfully inform the design and sequence of the learning activities. Activities are aligned with specific learning objectives and are supported by learning resources.” [lines 450-465, p.11]

Comment 2: In the conclusion, the authors state that the paper presents a stochastic analysis and conceptual synthesis with Vygotsky's theory and methods as a philosophical anchor. The research is limited to exploring elements of the design process that may contribute to managing the risk of student failure and improving the effectiveness of technology-supported educational scenarios.

The RQs are:

RQ1: In what ways can Vygotsky’s concepts and methods (re)form a design approach for managing the risk of student failure in technology-enhanced learning scenarios?

RQ2: What are the Vygotskian implications in addressing the relationship between the risk of failure and the design of technology-enhanced learning scenarios?

Neither RQ mentions Learning Analytics or backward design.

Response 2: Thank you for your thoughtful feedback. We agree. In the revised we have, accordingly, modified the RQ1 to mention Learning Analytics and Backward Design. In addition, we have modified our statement to make it clearer. Thank you again for your valuable suggestions.

“RQ1: In what ways can Vygotsky’s concepts be combined with the methods of backward design and learning analytics to (re)form a design approach for managing the risk of student failure in technology-enhanced learning scenarios?” [lines 114-116, p.5]

“This paper presents a theoretical analysis and conceptual synthesis of Vygotsky's work and existing knowledge from the fields of LD and LA with a focus on managing the risk of student failure in TEL environments. The research is limited to exploring elements of the design process that may contribute to managing the risk of student failure and improving the effectiveness of technology-enhanced learning scenarios; it does not consider other factors, such as demographic or psychological, that may influence the risk of student failure, nor does it consider broader socio-cultural/economic factors that may influence risk management processes in different educational systems.” [lines 789-797, p.20]

Comment 3: Many ideas, concepts, and technical terms are mentioned without being fully explored or, in some cases, correctly understood. From the references and text, I fail to see any connection to Design Thinking, even though it is mentioned as a keyword.

Response 3: Thank you for your insightful comments. After careful consideration, we agree with your observation that some of the ideas, concepts, and technical terms were not fully explored or clarified in the text. In response to your feedback, we have removed these concepts to ensure the focus remains clear and well-defined. Additionally, we have replaced 'Design Thinking' with 'design approach' in the keywords to better align with the text and to more accurately reflect the purpose of the paper. We appreciate your valuable input and believe these changes have strengthened the clarity of the paper.

Comment 4: I cannot get a clear sense of how the authors have used Learning Analytics (LA), how LA is currently applied to assess student risk of failure, and how this can be improved based on the article's contribution to the field.

Response 4: Thank you for your valuable feedback. We appreciate your comments regarding Learning Analytics (LA). We have revised the text as well as relevant figures to provide a clearer explanation of how LA is currently used in the context of assessing student risk of failure and how this can be improved based on the article's contribution to the field. We have also provided a practical example. Your feedback is helpful, and we believe these revisions will enhance the overall clarity and contribution of the paper.

“These techniques are based on the use of several algorithms to create predictive models of at-risk students and to generate early warning systems allowing for early intervention and support [3].” [lines 48-50, p. 2]

 “Several studies have linked LD activity type data to data from student learning behavior [9]. As an example, time spent on communication activities in blended and online environments is a major predictor of academic retention. Moreover, LD activities have contributed to predicting a significant proportion of LMS behavior. This means that a large proportion of student behavior in a technology-enhanced learning environment can be determined by how teachers design their courses” [lines 81-87, p. 2]

 “Although risk management processes emphasize data-driven technologies to support evidence-based decisions, research to demonstrate their connection with pedagogy is needed. This paper focuses on exploring design decisions that can benefit the learning process and help teachers manage the potential risk of student learning in technology-enhanced learning environments. The risk of student failure is determined by the level of acquiring desired learning outcomes. The likelihood of such risk may be influenced by several factors, some of which may be psychological or demographic and related to the individual’s broader socio-cultural environment. The purpose of this research is to suggest ways in which teachers might ensure the conditions for students to learn within the educational (socio-cultural) environment.” [lines 103-112, p. 3]

  “LA techniques and methods have been used to analyze learning behavioral data generated from these interactions (such as time spent, number of views, posts, or activities completed, grades/scores) to investigate factors that influence the risk of student failure in technology-enhanced courses. Several algorithms have been used to develop predictive models. Predictive modeling is the most used practice in assessing student performance and detecting students at risk for early intervention and support. The results of a literature review on the use of LA in predicting at-risk students indicate that student behavioral and academic data are the main predictors of student performance and at-risk students [24].” [lines 303-311, p. 8]

 “However, more detailed and practical research that illustrates which LD decisions can best guide LA techniques is needed [31]. Most studies tend to investigate LA during or after the implementation of the design and without the teacher’s initiative-taking participation at the design time.” [lines 349-352, p. 9]

  “Engagement and feedback assessment activities also have the potential to identify students at risk of failure, predict academic performance and support early interventions to enhance student’s learning [39].” [lines 400-412, p. 10]

 “The integration of LA elements in the design process could serve as a language for the teacher to analyze possible signs of student risk in object-oriented learning activities. A proposal for this, following, is illustrated in Figure 5.” [lines 468-470, p. 11]

  “For instance, choosing a page link for a video-watching activity can give basic metrics, such as the number of views, while a quiz for a self-assessment activity can offer more metrics, such as number of views, scores, number of attempts, number of submissions, number of feedback viewed. If the teacher has planned a forum activity, one concrete metric is the number of posts. The low number of posts may indicate low participation or engagement.” [lines 482-487, p. 12]

 “Α practical example of how the proposed design approach can be operationalized is shown in Figure 11, in which the design of a learning scenario, part of the undergraduate course 'Management Information Systems, is depicted. This course is delivered at the University of West Attica in Greece and for the e-learning part, the Moodle platform is used. The SMART learning objective for this unit of learning was for students to be able to design in groups a new information system they will propose for an organization or business by analyzing at least five functional and non-functional requirements by the end of the second month of the semester. A set of formative assessment activities was designed to provide guidance and targeted feedback on the desired outcome before final exams and final project deliverables. To promote authentic and active learning, we leveraged problem-based learning principles in a collaborative context. Thus, the sequence of the planned learning activities was based on exploration, discussion, creation, and self-reflection/ co-reflection. The vertical lines show the connection of each learning activity to the corresponding Moodle resources, as well as the metrics and indicators that we considered that may be of value for risk identification. The metrics presented in this figure emerged after reflecting on the interaction possibilities of the Moodle resources we selected to identify important signs about students' performance and learning behavior. Based on these metrics, we concluded on two main risk indicators: low participation and low performance.” [lines 627-649, p. 16]

 Figures in the revised manuscript: Figure 2, Figure 5, Figure 6

 References added:

3. Bañeres, D.; Rodríguez, M.E.; Guerrero-Roldán, A.E.; Karadeniz, A. An Early Warning System to Detect At-Risk Students in Online Higher Education. Appl. Sci. 2020, 10, 4427, doi:10.3390/app10134427.

9.          Toetenel, L.; Rienties, B. The Virtuous Circle of Learning Design and Learning Analytics to Develop Student-Centered Online Education. In The Routledge International Handbook of Student-Centered Learning and Teaching in Higher Education; Routledge, 2020 ISBN 978-0-429-25937-1.

Added Figure: Figure 8. Α practical illustration of the proposed approach for the design of a technology-enhanced learning scenario.

Comment 5: Furthermore, the underlying social-cultural theories, e.g., Actor Network or Activity Theory, should be defined and exemplified in the form of a conceptual framework.  As it stands, the paper introduces several concepts that are poorly defined, not properly introduced, and remain unconnected - see, for example, the list of abbreviations:

AI Artificial Intelligence

BWD Backward Design

CoI Community of Inquiry

EDM Educational Data Mining

HCD Human-Centered Design

HCAI Human-Centered Artificial Intelligence

HCI Human-Computer Interaction

HCLA Human-Centered Learning Analytics

HMPI Human-Machine Pair Inspection

LA Learning Analytics

LD Learning Design

LMS Learning Management System

PLA Predictive Learning Analytics

TEL Technology-Enhanced Learning

VLE Virtual Learning Environment

ZPD Zone of Proximal Development

Comment 5: Thank you for your thoughtful feedback. We appreciate your comments regarding underlying social-cultural theories. After careful consideration, we have revised the text to better define and introduce Activity Theory, making sure it is more connected to the purpose of the study.

Additionally, we have removed from the list of abbreviations the ones that we recognized may have confused or made the main concepts of this paper less accessible. We believe these revisions will improve the clarity and coherence of the paper, and we appreciate your helpful guidance in this process.

“At the core of the learning design is the (student) activity. The concept of activity is captured in Vygotsky's ideas through the Activity Theory. Activity Theory describes human actions as parts of purposeful activities of a collective nature that involve learning. Activity is an object-oriented action mediated by cultural tools and signs. In Vygotsky's basic model of mediation, three nodes are observed: the subject (human) uses tools to achieve an object. The object is the motivation for the activity. The activity is mediated by tools, and the process of the subject working towards an object using tools brings about an outcome [43]. This model was extended to consider the socially mediated nature of the activity and the roles of other individuals in the division of labor. In the context of the learning process, students act on the learning object (learning objective) using the services and tools (learning resources) of the environment (e.g., LMS) through which they interact with the learning content, with the teacher, and with their peers. Learning resources frame learning activities and are crucial for learner interaction and engagement. The process of a student working towards a learning goal brings about a learning outcome. The risk of student failure lies at this point. According to Vygotsky, the desired learning outcomes must be in the student's ZPD. To promote student activity (and thus learning) in the ZPD, it is appropriate to design with a focus on the learning objectives. BWD is a goal-directed design methodology that has pedagogical roots in constructivism [41]. It aims to improve the quality and effectiveness of design towards the achievement of desired learning outcomes. The three principal stages of BWD are:

1.         Identifying desired learning results: The teacher identifies the essential understandings, knowledge, and skills to be mastered by the students. The learning object reflects “big ideas” that are “transferable”, and the focus is on higher-order desired learning outcomes.

2.         Determining acceptable evidence: The teacher identifies student evidence of achievement towards the desired learning outcomes and the methods by which this evidence will be collected and assessed. Emphasis is placed on authentic performance tasks and the use of a variety of assessment techniques.

3.         Planning learning experiences and instruction: The teacher makes decisions on the teaching methods, learning events, and strategies that will meaningfully inform the design and sequence of the learning activities. Activities are aligned with specific learning objectives and are supported by learning resources.

The integration of LA elements in the design process could serve as a language for the teacher to analyze possible signs of student risk in object-oriented learning activities. A proposal for this, following, is illustrated in Figure 5.” [lines 436-470, p. 11]

“AI       Artificial Intelligence

BWD    Backward Design

HMPI   Human-Machine Pair Inspection

LA        Learning Analytics

LD       Learning Design

LMS     Learning Management System

TEL      Technology-Enhanced Learning

ZPD     Zone of Proximal Development”

 [line 886, p. 22]

Comment 6: The paper never refers back to the RQs. There is no operational description of how the RQs are answered - and in the end, they aren't.  I fail to see how the authors performed a stochastic analysis (on what).

Comment 6: Thank you for your insightful feedback. We appreciate your comments regarding the research questions (RQs). We have made revisions to ensure that the RQs are better integrated and addressed more clearly.

Additionally, we understand your concern regarding the term 'stochastic analysis.' After careful consideration, we have replaced the term with a more appropriate description. We believe these revisions will help to improve the clarity and coherence of the paper.

 “This paper presents a theoretical analysis and conceptual synthesis of Vygotsky's work and existing knowledge from the fields of LD and LA with a focus on managing the risk of student failure in TEL environments.”

 “Based on our previous work [71], we propose a design approach that guides the teacher design decisions on key LD elements under the principles of BWD to promote the conditions for quality and effective learning experiences. In this approach, LA semantic elements are incorporated to support teachers’ monitoring of potential risks when implementing technology-enhanced learning scenarios. Most risk management processes emphasize data-driven technologies to support evidence-based decisions. The risk of student failure in a learning scenario is a result of predictive models. However, the effectiveness of learning depends on decisions made by the teacher during the learning design process. This paper focuses on investigating design decisions that can benefit the learning experience and help teachers manage the potential risk of student learning in technology-enhanced learning environments. The risk of student failure refers to the failure of students to learn and is determined by the level of acquisition of desired learning outcomes. The following research questions of this paper have been answered accordingly:

RQ1: In what ways can Vygotsky’s concepts be combined with the methods of backward design and learning analytics to (re)form a design approach for managing the risk of student failure in technology-enhanced learning scenarios? Vygotsky's legacy offers important theoretical foundations for establishing a design approach that incorporates semantic elements of LA into a BWD process and aims to manage the risk of student failure in technology-enhanced learning scenarios. BWD principles have pedagogical roots in constructivism and can support the design within student ZPD. In this context, ZPD focuses on the development of an individual's higher-order skills through guidance and collaboration. It ensures that learning objectives are tailored to what students can achieve through support, guidance, and feedback and that they promote the development of abstract thinking and self-regulation. Assessment is used to promote active learning, collaboration, and self-reflection. The variety of assessment methods helps the teacher to gather meaningful data for understanding the learning process within the ZPD. Integrated LA semantic elements can work as a language, enhancing the teacher's ability to construct risk indicators, promote the conditions for a productive learning environment, reflect on decisions, and improve the effectiveness and quality of the design. Vygotsky’s theory stresses the importance of the sociocultural environment and mediation in cognitive development. Learning activities promote active learning through social participation and collaboration between students and their peers and teachers. The use of tools is essential to mediate learning. In the context of technology-enhanced environments, these tools are the learning resources with which students interact with their peers, teachers, and the learning content. The appropriate selection of such tools ensures the conditions for meaningful engagement in the learning environment and influences the learning experience. BWD and LA can be used as the language of the teacher in the design environment.

RQ2: What are the Vygotskian implications in addressing the relationship between the risk of failure and the design of technology-enhanced learning scenarios?

Addressing the relationship between the risk of failure and the design of technology-enhanced learning scenarios is dialectical and dynamic. Vygotsky’s theory emphasizes the role and importance of ZPD, interaction, and mediating tools of the environment in cognitive development. The risk of failure in the process of designing a technology-enhanced learning scenario is determined by ensuring the effectiveness and quality of LD as well as by effectively integrating LA language into the design process. The dialectical relationship between the two can be mediated by the principles of BWD, to facilitate the alignment between critical elements of LD and LA. In empirical terms, BWD semantic mediation guides the identification of potential risk indicators that are pedagogically meaningful and closely tied to student performance toward the desired learning outcomes. These indicators are further reinforced by complementary data derived from student engagement and interaction with learning resources that are directly linked to the designed learning activities. LA language (semantic elements) integration in the process of designing technology-enhanced learning scenarios can significantly contribute to the assessment of the risk of failure. The metrics and risk indicators that the teacher identifies while designing the learning experience can serve as checkpoints and are important for providing valuable feedback on the student’s progress within their ZPD. In a broader context, the teacher gains an overall oversight of the design process, as well as feedback on their own ZPD. Addressing the risk of failure can become a design opportunity that focuses on cognitive development as well as on the management and improvement of the design itself. The HMPI methodology can be used to create a collaborative teacher-technology design environment based on BWD and LA semantic context, in which AI tools can assist teachers in operating within their zone of proximal-professional development.”

[lines 799-860, p. 20-21]

Comment 7: The article will benefit from a rewrite that follows a traditional outline - introduction, theoretical framework, literature review, methods / conceptual framework, results, discussion, and conclusion.

Response 7: Thank you for your valuable feedback regarding the structure of the article. We have revised the article to incorporate this structure and ensure it aligns more closely with your suggestion. Your input is much appreciated, and we believe these changes will strengthen the paper.

4. Response to Comments on the Quality of English Language

Point 1: The English is fine and does not require any improvement.

Response 1: Thank you!

5. Additional clarifications

We have incorporated all the suggestions made by the reviewers. Those changes are highlighted within the revised manuscript file with tracked changes.

Thank you again for the clear review and suggestions for corrections to improve our article.

Response to Reviewer 2's Comments

1. Summary

Dear Editor,

Thank you for allowing us to submit a revised draft of our manuscript titled “Vygotsky's Systemic Perspectives on Managing the Risk of Student Failure in Technology-Enhanced Learning Design” to the MDPI on Applied Sciences. We sincerely appreciate the time and effort you and the reviewers have dedicated to evaluating our manuscript. We are grateful for the insightful comments and constructive feedback, which have helped us improve the quality of our work. This manuscript has addressed all the issues raised during the review process. To facilitate the review, we have highlighted the revisions in red. Our response letter includes:

(1)        A point-by-point explanation of the revisions made in the manuscript,

(2)        Our responses to the reviewers' comments.

We appreciate your consideration of our revised manuscript and await your feedback. Please find the detailed responses below and the corresponding revisions/corrections highlighted/in track changes in the re-submitted files.

2. Questions for General Evaluation

Reviewer’s Evaluation

Response and Revisions

Does the introduction provide sufficient background and include all relevant references?

Can be improved

It is improved. Revisions have been made to enhance the background.

Is the research design appropriate?

Must be improved

It is improved. We have made revisions regarding the RQs and highlighted the research scope.

Are the methods adequately described?

Must be improved

We have added a “Methods” section as suggested.

Are the results presented?

Can be improved

It is improved. We have revised the relevant section to enhance clarity.

Are the conclusions supported by the results?

Must be improved

It is improved. We have made revisions to ensure that conclusions are supported by the results.

3. Point-by-point response to Comments and Suggestions for Authors

Comments 1: The Introduction section expounds on three dialectical relations by analyzing relevant literature. However, the related research on these three dialectical relationships is insufficient, and then the key content of this study is proposed to highlight the innovation of this study.

Response 1: Thank you for your thoughtful feedback. We appreciate your point regarding the need for a more comprehensive exploration of the related research on the three dialectical relationships discussed in the Introduction. We have revised the section to include a more thorough review of the relevant research and to provide better context before highlighting the key content and innovations of this study. Your feedback is invaluable, and we believe these revisions will enhance the clarity and depth of the Introduction.

“These techniques are based on the use of several algorithms to create predictive models of at-risk students and to generate early warning systems allowing for early intervention and support [3].” [lines 48-50, p. 2]

“Several studies have linked LD activity type data to data from student learning behavior [9]. As an example, time spent on communication activities in blended and online environments is a major predictor of academic retention. Moreover, LD activities have contributed to predicting a significant proportion of LMS behavior. This means that a large proportion of student behavior in a technology-enhanced learning environment can be determined by how teachers design their courses” [lines 81-86, p. 2]

“Although risk management processes emphasize data-driven technologies to support evidence-based decisions, research to demonstrate their connection with pedagogy is needed. This paper focuses on exploring design decisions that can benefit the learning process and help teachers manage the potential risk of student learning in technology-enhanced learning environments. The risk of student failure is determined by the level of acquiring desired learning outcomes. The likelihood of such risk may be influenced by several factors, some of which may be psychological or demographic and related to the individual’s broader socio-cultural environment. The purpose of this research is to suggest ways in which teachers might ensure the conditions for students to learn within the educational (socio-cultural) environment.” [lines 103-112, p. 3]

References added:

3. Bañeres, D.; Rodríguez, M.E.; Guerrero-Roldán, A.E.; Karadeniz, A. An Early Warning System to Detect At-Risk Students in Online Higher Education. Appl. Sci. 2020, 10, 4427, doi:10.3390/app10134427.

9.          Toetenel, L.; Rienties, B. The Virtuous Circle of Learning Design and Learning Analytics to Develop Student-Centered Online Education. In The Routledge International Handbook of Student-Centered Learning and Teaching in Higher Education; Routledge, 2020 ISBN 978-0-429-25937-1.

Comments 2: On page 5, lines 188-189, “This perspective introduces a dynamic dimension to the role of the teacher and the socio-cultural educational environment in managing the risk of failure. What does “This perspective” mean? Not clear enough.

Response 2: Thank you for your valuable feedback. We appreciate your comment regarding the clarity of this phrase. We have reorganized the text and removed the sentence to ensure greater clarity and coherence in the overall argument. We believe these changes will improve the readability and flow of the paper. Your input is greatly appreciated.

[lines 202-204, p. 5]

Comments 3: According to Literature 19, please explain the interaction mode among the three in Figure 2 and mark it in the figure, making Figure 2 more clear and complete. And comment on Figure 2.

Response 3: Thank you for your helpful feedback. We appreciate your suggestion to clarify the interaction mode among the three elements in Figure 2. We have revised the figure to more clearly illustrate these interactions and have marked the relevant relationships in the figure to make it more complete and easier to understand. Additionally, we have included a comment on Figure 2 in the text to explain the dynamics more thoroughly. Your input has been valuable in enhancing the clarity of the figure and its contribution to the paper.

“Learner-learner interaction refers to bidirectional communication among learners, such as discussing with each other and providing or receiving feedback from peers. Educator-learner interaction relates to bidirectional communication between learners and educators, such as asking questions, submitting assignments or responses, and providing feedback and guidance to learners. Learner-content interaction involves the flow of information from the course content to learners, such as instructional materials that students can view or read. Students’ activity in technology-enhanced learning environments, such as an LMS, produces digital traces, which are digitally recorded and stored. LA applications record and analyze data generated from these interactions (such as time spent, number of views, posts, or activities completed, grades/scores) for prediction (e.g., students at risk of failure), action (e.g., early intervention or support), and refinement (e.g., improving future learning designs). The teachers must construct high-level interactive mode to promote engagement and facilitate learning in modern learning settings.”

Revised: Figure 2. The triangle of socio-cultural interactions in TEL environments.

[lines 241-251, p. 6]

Comment 4: In Section 4, Learning Analytics as a Language to Mediate "Signs" of Risk, the relevant analysis of Learning Analytics is not seen. What is the significance of “Learning Analytics” here?

Response 4: Thank you for your insightful feedback. We appreciate your comments regarding Learning Analytics (LA). We have revised the text as well as relevant figures to provide a clearer explanation of how LA is currently used in the context of assessing student risk of failure and how this can be improved based on the article's contribution to the field. We believe these revisions will help clarify the connection between Learning Analytics and the broader discussion in the paper.

“LA techniques and methods have been used to analyze learning behavioral data generated from these interactions (such as time spent, number of views, posts, or activities completed, grades/scores) to investigate factors that influence the risk of student failure in technology-enhanced courses. Several algorithms have been used to develop predictive models. Predictive modeling is the most used practice in assessing student performance and detecting students at risk for early intervention and support. The results of a literature review on the use of LA in predicting at-risk students indicate that student behavioral and academic data are the main predictors of student performance and at-risk students [24].” [lines 303-311, p. 8]

“However, more detailed and practical research that illustrates which LD decisions can best guide LA techniques is needed [31]. Most studies tend to investigate LA during or after the implementation of the design and without the teacher’s proactive participation at the design time.” [lines 349-352, p. 9]

 “The integration of LA elements in the design process could serve as a language for the teacher to analyze possible signs of student risk in object-oriented learning activities. A proposal for this, following, is illustrated in Figure 5.” [lines 468-470, p. 11]

 “For instance, choosing a page link for a video-watching activity can give basic metrics, such as the number of views, while a quiz for a self-assessment activity can offer more metrics, such as number of views, scores, number of attempts, number of submissions, number of feedback viewed. If the teacher has planned a forum activity, one concrete metric is the number of posts. The low number of posts may indicate low participation or engagement.” [lines 482-487, p. 12]

Figures in the revised manuscript: Figure 2, Figure 5, Figure 6

Comment 5: On page 10, the source of "the risk of student failure in technology-enhanced learning scenarios" is not clear. At the same time, Figure 5 is confusing. It is suggested to redraw the figure to make the logic of the figure clearer.

Response 5: Thank you for your valuable feedback. We have revised the text to provide more clarity. Additionally, we have redrawn the figure to improve its logic and make the relationships it illustrates more intuitive. Your input has been very helpful.

 “In the context of designing quality and effective learning experiences, the concept of ZPD can be applied to ground higher-order, student-centered learning goals and highlight the role of assessment in supporting students to attain these goals. Managing the risk of student failure in technology-enhanced learning scenarios is tightly linked to creating conditions that can facilitate learning and engagement” [lines 381-385, p. 9]

 “How assessment can be operationalized within a design process that focuses on students’ ZPD in technology-enhanced learning environments are as follows:” [lines 394-395, p. 10]

 “As a tool for feedback and guidance. Defining various forms of formative assessment can reinforce student learning and engagement [37]. Teachers can understand students' needs and provide constructive feedback and guidance towards desired learning outcomes.” [lines 402-405, p. 10]

 “Engagement and feedback assessment activities also have the potential to identify students at risk of failure, predict academic performance and support early interventions to enhance student’s learning [38].” [lines 409-412, p. 10]

  “Figure 4. Assessment in the Zone of Proximal Development.” [lines 413-414, p. 10]

References added:

37.        Wafubwa, R. Role of Formative Assessment in Improving Students’ Motivation, Engagement, and Achievement: A Systematic Review of Literature. Int. J. Assess. Eval. 2020, 28, 17–31, doi:10.18848/2327-7920/CGP/v28i01/17-31.

38.        Kemp, P.R.; Bradshaw, J.M.; Pandya, B.; Davies, D.; Morrell, M.J.; Sam, A.H. The Validity of Engagement and Feedback Assessments (EFAs): Identifying Students at Risk of Failing. BMC Med. Educ. 2023, 23, 866, doi:10.1186/s12909-023-04828-7.

Figure in the revised manuscript: Figure 4

Comment 6: There are 6 steps in Figure 7. It is suggested to label the 6 steps in the relevant analysis of Figure 7 to make the structure clearer, instead of "italics".

Response 6: We sincerely thank you for pointing this out. We have updated the relevant analysis. We believe these changes will enhance the overall readability and understanding of the figure.

[lines 518-626, p. 13-16]

Comment 7: It is suggested to simplify the Conclusions section, which should summarize the study and present the main conclusions of the paper.

Response 7: Thank you for your valuable feedback. We appreciate your suggestion to simplify the Conclusions section. We have revised the section to provide a more concise summary of the study and highlight the main conclusions of the paper based on the research questions. We believe that these changes will make the conclusion more focused and easier to follow, aligning it more closely with the core findings of the research.

[lines 799-860, p. 20-21]

Comments 8: The format of references should be sorted out according to the requirements of journals.

Response 8: Thank you for your feedback. We appreciate your comment regarding the format of the references. We have already used Zotero to organize the references according to the journal's guidelines and applied the MDPI style as required. We have checked the format again in the revised manuscript. If there are any specific aspects of the formatting that still need adjustment, we will be happy to make further revisions.

4. Response to Comments on the Quality of English Language

Point 1: English grammar and structure need to be further improved.

Response 1:  Thank you for your feedback. We appreciate your suggestion regarding the improvement of English grammar and structure. We have carefully reviewed the paper and made revisions to enhance clarity, grammar, and overall readability. We believe these changes will improve the flow and presentation of the content. Your input is much appreciated, and we hope these revisions meet the journal’s expectations.

5. Additional clarifications

We have incorporated all the suggestions made by the reviewers. Those changes are highlighted within the revised manuscript file with tracked changes.

Thank you again for the clear review and suggestions for corrections to improve our article.

Response to Reviewer 3's Comments

1. Summary

Dear Editor,

Thank you for allowing us to submit a revised draft of our manuscript titled “Vygotsky's Systemic Perspectives on Managing the Risk of Student Failure in Technology-Enhanced Learning Design” to the MDPI on Applied Sciences. We sincerely appreciate the time and effort you and the reviewers have dedicated to evaluating our manuscript. We are grateful for the insightful comments and constructive feedback, which have helped us improve the quality of our work. This manuscript has addressed all the issues raised during the review process. To facilitate the review, we have highlighted the revisions in red. Our response letter includes:

(1)        A point-by-point explanation of the revisions made in the manuscript,

(2)        Our responses to the reviewers' comments.

We appreciate your consideration of our revised manuscript and look forward to your feedback. Please find the detailed responses below and the corresponding revisions/corrections highlighted/in track changes in the re-submitted files.

2. Questions for General Evaluation

Reviewer’s Evaluation

Response and Revisions

Does the introduction provide sufficient background and include all relevant references?

Yes/Can be improved/Must be improved/Not applicable

Are all the cited references relevant to the research?

Yes/Can be improved/Must be improved/Not applicable

Is the research design appropriate?

Yes/Can be improved/Must be improved/Not applicable

Are the methods adequately described?

Yes/Can be improved/Must be improved/Not applicable

Are the results presented?

Yes/Can be improved/Must be improved/Not applicable

Are the conclusions supported by the results?

Yes/Can be improved/Must be improved/Not applicable

3. Point-by-point Response to Comments and Suggestions for Authors

Comment 1: POINT 1: Clarify the original contribution
The article presents numerous connections between Vygotsky, Learning Analytics (LA), the Zone of Proximal Development (ZPD), and Backward Design (BWD). Still, the author's original proposal is not sufficiently demarcated from existing frameworks. i) Delineate what is new in your approach in a specific section or the conclusion: is it a new pedagogical model? A design framework? An application of HMPI in education. ii) Figure 6 is useful for visualizing the link between objectives, activities, LA, and assessment. However, it would benefit from being reworded to visually emphasize the added value of the proposed model and not just the interconnection of concepts.

Response 1: Thank you for your thoughtful feedback.

In response to your suggestions: i) We have revised the manuscript to more clearly delineate the novelty of the proposed approach and to highlight its originality and significance within the broader context of existing frameworks. ii) Regarding Figure 6, we have refined it to better emphasize the added value of the proposed model.

 “This point is emphasized by the Technological Pedagogical Content Knowledge and Pedagogical Knowledge (TPACK) framework [23]. It is a theoretical framework that has been widely adopted in teacher education programs [23]. The framework proposes three interacting bodies of knowledge that teachers need to create effective learning experiences with technology. Content knowledge includes the teacher’s understanding of the subject matter. Pedagogical knowledge includes teaching methods and practices. Technological knowledge is about understanding how to use technological tools and resources. However, the need to develop teachers' skills in these areas is still present.”

[lines 283-290, p. 7-8]

“Based on our previous work [71], we propose a design approach that guides the teacher design decisions on key LD elements under the principles of BWD to promote the conditions for quality and effective learning experiences. In this approach, LA semantic elements are incorporated to support teachers’ monitoring of potential risks when implementing technology-enhanced learning scenarios. Most risk management processes emphasize data-driven technologies to support evidence-based decisions. The risk of student failure in a learning scenario is a result of predictive models. However, the effectiveness of learning depends on decisions made by the teacher during the learning design process. This paper focuses on investigating design decisions that can benefit the learning experience and help teachers manage the potential risk of student learning in technology-enhanced learning environments. The risk of student failure refers to the failure of students to learn and is determined by the level of acquisition of desired learning outcomes.” [lines 799-810, p. 20-21]

References added:

23.        Mishra, P.; Warr, Melissa; and Islam, R. TPACK in the Age of ChatGPT and Generative AI. J. Digit. Learn. Teach. Educ. 2023, 39, 235–251, doi:10.1080/21532974.2023.2247480.

 71. Themeli, A.; Kotsifakos, D.; Psaromiligkos, Y. Framework for Designing Learning Activities to Manage the Risk of Failure in E-Learning Courses. In Proceedings of the 2024 19th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP); IEEE: Athens, Greece, November 21, 2024; pp. 172–177.

 Figure in the revised manuscript: Figure 6

Comment 2: POINT 2: Strengthen empirical foundations
The approach remains essentially theoretical. No applied examples, case studies, or feedback are offered. i) Include an illustrative scenario, even if fictional, showing how a teacher can use your model to design an e-learning environment integrating ZPD, LA, and BWD.

Response 2: Thank you for your thoughtful feedback. We appreciate your observation. In response to your suggestion, we have included a practical illustration of the proposed approach and Figure 8 to strengthen the paper and make the concepts more tangible.

 Α practical example of how the proposed design approach can be operationalized is shown in Figure 8, in which the design of a learning scenario, part of the undergraduate course “Management Information Systems”, is depicted. This course is delivered at the University of West Attica in Greece and for the e-learning part the Moodle platform is used. The SMART learning objective for this learning scenario was for students to be able to design in groups a new information system they will propose for an organization or business by analyzing at least five functional and non-functional requirements by the end of the second month of the semester. A set of formative assessment activities was designed to provide guidance and targeted feedback on the desired outcome before final exams and final project deliverables. To promote authentic and active learning, we leveraged problem-based learning principles in a collaborative context. Thus, the sequence of the planned learning activities was based on exploration, discussion, creation, and self-reflection/ co-reflection. The vertical lines show the connection of each learning activity to the corresponding Moodle resources, as well as the metrics and indicators that we considered that may be of value for risk identification. The metrics presented in this figure emerged after reflecting on the interaction possibilities of the Moodle resources we selected to identify important signs about students' performance and learning behavior. Based on these metrics, we concluded on two main risk indicators: low participation and low performance.”

 [lines 627-645, p. 16]

 Added Figure: Figure 8. Α practical illustration of the proposed approach for the design of a technology-enhanced learning scenario.

Comment 3: POINT 3: Reorganize certain sections to avoid redundancy
Several notions (e.g., ZPD, social mediation, the role of the teacher) are mentioned several times with similar wording. i) Suggestion: Merge or reorganize certain sections (e.g., sections 3, 4, and 5) to give more structure to the argumentative progression and avoid repetition.

Response 3: Thank you for your insightful feedback. We appreciate your observation. In response to your suggestion, we have merged and reorganized sections 3, 4, and 5 to provide more structure to the argumentative progression.

We believe these revisions will enhance the clarity and flow of the paper.

Comment 4: POINT 4: Triangle of Socio-Cultural Interactions in TEL
Figure 2 is classic and useful but offers nothing new in terms of the existing literature. i) Review this figure to illustrate how these interactions are captured in analytical data (LA contribution), to enrich the argument.

Response 4: Excellent suggestion! We have revised Figure 2 and the relevant text to emphasize this point. We appreciate your recognition of Figure 2 as a useful visual representation. We have revised Figure 2 to better illustrate how the interactions between the concepts are captured in analytical data. In addition, we added relevant text. We believe this revision will enrich the argument by showing how LA can be used to track and assess the dynamics within the technology-enhanced environment. Thank you again for your helpful suggestion.

“Learner-learner interaction refers to bidirectional communication among learners, such as discussing with each other and providing or receiving feedback from peers. Educator-learner interaction relates to bidirectional communication between learners and educators, such as asking questions, submitting assignments or responses, and providing feedback and guidance to learners. Learner-content interaction involves the flow of information from the course content to learners, such as instructional materials that students can view or read. Students’ activity in technology-enhanced learning environments, such as an LMS, produces digital traces, which are digitally recorded and stored. LA applications record and analyze data generated from these interactions (such as time spent, number of views, posts, or activities completed, grades/scores) for prediction (e.g., students at risk of failure), action (e.g., early intervention or support), and refinement (e.g., improving future learning designs). The teachers must construct high-level interactive mode to promote engagement and facilitate learning in modern learning settings.”

 Revised: Figure 2. The triangle of socio-cultural interactions in TEL environments.

[lines 241-251, p. 6]

“LA techniques and methods have been used to analyze learning behavioral data generated from these interactions (such as time spent, number of views, posts, or activities completed, grades/scores) to investigate factors that influence the risk of student failure in technology-enhanced courses. Several algorithms have been used to develop predictive models. Predictive modeling is the most used practice in assessing student performance and detecting students at risk for early intervention and support. The results of a literature review on the use of LA in predicting at-risk students indicate that student behavioral and academic data are the main predictors of student performance and at-risk students [24].” [lines 303-311, p. 8]

Comments 5: POINT 5: LA Language as a Mediating Tool (FIGURE 4) 

This is one of the most interesting figures. i) It could be strengthened by adding a concrete example of a “metric” and its possible interpretation (e.g.: number of posts in a forum → level of engagement). ii)Add a legend or explanatory speech bubbles to illustrate each link.

Response 5: Thank you for your positive feedback on the figure. We sincerely appreciate your suggestions for strengthening the figure.

In response to your suggestions: i) We have added text to explain a concrete example of a 'metric' and its possible interpretation. We believe this will provide a clearer and more tangible understanding of how the metrics are applied and interpreted.

ii) We have revised the figure, by adding explanatory speech bubbles to illustrate each link and to enhance the clarity and accessibility of the figure.

Thank you once again for your valuable input. We believe these revisions will make the figure even more informative and engaging.

“The integration of LA elements in the design process could serve as a language for the teacher to analyze possible signs of student risk in object-oriented learning activities. A proposal for this, following, is illustrated in Figure 5.” [lines 468-470, p. 11]

 “For instance, choosing a page link for a video-watching activity can give basic metrics, such as the number of views, while a quiz for a self-assessment activity can offer more metrics, such as number of views, scores, number of attempts, number of submissions, number of feedback viewed. If the teacher has planned a forum activity, one concrete metric is the number of posts. The low number of posts may indicate low participation or engagement.” [lines 482-487, p. 12]

 Figure in the revised manuscript: Figure 5

Comment 6: Point 6: Clarify the model's scope and limitations: i) Although the article mentions in the conclusion that factors such as socio-economic context are not taken into account, this should appear earlier in the text to frame the model's scope. ii) Add a separate “Limitations” section before the conclusion to make explicit the model's assumptions and conditions of validity.

Response 6: Thank you for your thoughtful feedback. In response,

i) We have revised the paper to include this information earlier, ensuring that the limitations of the model and the scope are clearly outlined within the body of the paper.

ii) Regarding the suggestion to add a separate 'Limitations' section, we believe this is an excellent idea. We have introduced a new section before the conclusion to explicitly state the model's assumptions and the conditions of its validity.

Thank you again for your valuable suggestions.

“Although risk management processes emphasize data-driven technologies to support evidence-based decisions, research to demonstrate their connection with pedagogy is needed. This paper focuses on exploring design decisions that can benefit the learning process and help teachers manage the potential risk of student learning in technology-enhanced learning environments. The risk of student failure is determined by the level of acquiring desired learning outcomes. The likelihood of such risk may be influenced by several factors, some of which may be psychological or demographic and related to the individual’s broader socio-cultural environment. The purpose of this research is to suggest ways in which teachers might ensure the conditions for students to learn within the educational (socio-cultural) environment.” [lines 103-112, p. 3].

 “Limitations

This paper presents a theoretical analysis and conceptual synthesis of Vygotsky's work and existing knowledge from the fields of LD and LA with a focus on managing the risk of student failure in TEL environments. The research is limited to exploring elements of the design process that may contribute to managing the risk of student failure and improving the effectiveness of technology-enhanced learning scenarios; it does not consider other factors, such as demographic or psychological, that may influence the risk of student failure, nor does it consider broader socio-cultural/economic factors that may influence risk management processes in different educational systems. “

[lines 790-797, p. 20]

Comment 7: The manuscript has undeniable scientific potential, with an ambitious attempt to articulate Vygotsky, instructional design, and modern technological tools. However, a major revision is needed to 1) clarify the original contribution, 2) empirically illustrate the theoretical concepts, 3) enhance the readability of the figures, and 4) better contextualize the proposed technological tools.

Response 7: Thank you for your thoughtful and constructive feedback. We deeply appreciate your recognition of the scientific potential of the manuscript. Your comments have been invaluable in helping us to identify areas for improvement. We believe the revisions we have made based on your suggestions will significantly improve the manuscript.

4. Response to Comments on the Quality of English Language

Point 1: The English could be improved to express the research.

Response 1:  Thank you for your feedback. We appreciate your suggestion regarding the improvement of English. We have carefully reviewed the paper and made revisions to enhance clarity, grammar, and overall readability. We believe these changes will improve the flow and presentation of the content. Your input is much appreciated, and we hope these revisions meet the journal’s expectations.

5. Additional clarifications

We have incorporated all the suggestions made by the reviewers. Those changes are highlighted within the revised manuscript file with tracked changes.

Thank you again for the clear review and suggestions for corrections to improve our article.

Response to Reviewer 1's Comments

1. Summary

Dear Editor,

Thank you for allowing us to submit a revised draft of our manuscript titled “Vygotsky's Systemic Perspectives on Managing the Risk of Student Failure in Technology-Enhanced Learning Design” to the MDPI on Applied Sciences. We sincerely appreciate the time and effort you and the reviewers have dedicated to evaluating our manuscript. We are grateful for the insightful comments and constructive feedback, which have helped us improve the quality of our work. In this manuscript, we have addressed all the issues raised during the review process. To facilitate the review, we have highlighted the revisions in red. Our response letter includes:

(1)        A point-by-point explanation of the revisions made in the manuscript,

(2)        Our responses to the reviewers' comments.

We appreciate your consideration of our revised manuscript and look forward to your feedback. Please find the detailed responses below and the corresponding revisions/corrections highlighted/in track changes in the re-submitted files.

2. Questions for General Evaluation

Reviewer’s Evaluation

Response and Revisions

Does the introduction provide sufficient background and include all relevant references?

Not applicable

Is the research design appropriate?

Must be improved

Are the methods adequately described?

Must be improved

Are the results presented?

Must be improved

Are the conclusions supported by the results?

Must be improved

3. Point-by-point Response to Comments and Suggestions for Authors

Comment 1: The article lacks a thorough review of learning analytics and activity theory and fails to establish a link.

Response 1: We sincerely thank you for this insightful and important comment. We have revised the background and made the link between them clearer. We are grateful for the opportunity to improve this part of the work, and we appreciate the valuable feedback.

“The second dialectical relationship identified in the literature is between the fields of Learning Design (LD) and LA with a link to technology-enhanced learning environments. The field of LD studies the process of designing learning activities that are pedagogically grounded and supported by technology. This process can provide the context for more meaningful analysis and interpretation of data captured by the field of LA. At the same time, LA can provide useful information about the effectiveness of LD as well as valuable feedback on teachers’ technological, pedagogical, and content knowledge [9]. It has been shown that LD has a strong impact on student engagement, performance, and learning outcomes. Several studies have linked LD activity type data to data from student learning behavior [10]. As an example, time spent on communication activities in blended and online environments is a major predictor of academic retention. Moreover, LD activities have contributed to predicting a significant proportion of LMS behavior. This means that a large proportion of student behavior in a technology-enhanced learning environment can be determined by how teachers design their courses. In addition, LDs that promote socio-collaborative and independent learning skills are found to have large positive effects on student outcomes [11]. Formative assessment can facilitate student learning and collaborative activities can substantially determine student engagement and achievement. This observation suggests the need for more extensive integration of these elements in LD [12]. Such findings suggest a close connection between LD activities and LA.”

[p.2-3, lines 76-94]

Added reference:

12.        Albuquerque, J.; Rienties, B.; Divjak, B. Decoding Learning Design Decisions: A Cluster Analysis of 12,749 Teaching and Learning Activities. In Proceedings of the Proceedings of the 15th International Learning Analytics and Knowledge Conference; Association for Computing Machinery: New York, NY, USA, November 3, 2025; pp. 407–417.

Comment 2: The revisions are minor, not major, and do not correct the original flaws in the manuscript.

Response 2: Thank you for your feedback. We appreciate your observations regarding the revisions. We have carefully considered your comments and worked to further improve the manuscript.

Comment 3: It is unclear how learning design is linked to outcomes and why or what type of learning design would be preferable based on this work. Learning Design, Backwards Design, and Learning Analytics are used almost synonymously.

Response 3:

Thank you for your feedback. We appreciate your observations regarding the revisions. In our option (conclusions), Learning Design (LD) focuses on planning and structuring effective learning experiences to achieve specific educational goals. On the other hand, Backwards Design (BD) is a method within LD that starts with identifying desired learning outcomes before planning instructional activities and assessments. Learning Analytics (LA) involves collecting and analyzing data on learners’ behaviors and performance to improve learning and teaching. These three connect as BD provides the framework for defining clear goals, LD structures the path to achieve them, and LA offers feedback on whether the goals are being met. Together, they form a cycle of informed planning, execution, and continuous improvement in managing the risk of student failure in technology-enhanced LD and education generally.

Comment 4: In terms of practical applications, Fig. 8 shows what activities in Moodle might be tracked with Moodle learning analytics, but it is again unclear how this works in Moodle, how it would translate to other contexts, and why the specific parameters were selected over others. It is likewise unclear how the risk of failure is operationalized.

Response 4: We sincerely thank you for this thoughtful observation and for highlighting areas that merit further clarification. Figure 8 was presented as an illustrative example of the implementation of the proposed approach utilizing Moodle features. In this environment, monitoring possibilities are available through course reports and individual student activity reports, and it is also possible to integrate tools for more advanced analysis. We aim to highlight the importance of teachers’ proactive action through the setting of checkpoints on their learning designs and before the implementation of their learning scenarios. In this way, early signs of risk can be identified, and targeted support for the learning process can be facilitated. Similar or other available resources could be used in different learning environments (e.g., other LMSs). The parameters selected for this learning scenario were considered by us to be of value for identifying the potential risk of failure in the context and educational environment. We understand that the selection of such parameters may vary depending on the pedagogical context, the resources available in the learning environment in question, and the accessibility of the data. Indicators of participation and performance were chosen as they are widely accepted in the literature in terms of student engagement and predicting students at risk of failure. We sincerely hope that our explanation helps to clarify our intent, and we are always open to further thoughts or suggestions.

“LA techniques and methods have been used to analyze learning behavioral data generated from these interactions (such as time spent, number of views, posts, activities completed, grades/scores) to investigate factors that influence the risk of student failure in technology-enhanced courses and to predict performance and at-risk students. Several algorithms have been used to develop predictive models. Predictive modeling is the most used practice in assessing student performance and detecting students at risk for early intervention and support. The results of a literature review on the use of LA in predicting at-risk students indicate that student behavioral and academic data are the main predictors of student performance and at-risk students [27]. These findings are consistent with a study on student engagement and educational technology, which showed that the most studied indicators of student engagement were participation/ interaction and achievement [28].”

[p. 8, lines 306-318]

“Α practical example of how the proposed design approach can be operationalized is shown in Figure 8, in which the design of a learning scenario, part of the undergraduate course 'Management Information Systems, is depicted. This course is delivered at the University of West Attica in Greece and for the e-learning part the Moodle platform is used. The SMART learning objective for this learning scenario was for students to be able to design in groups a new information system they will propose for an organization or business by analyzing at least five functional and non-functional requirements by the end of the second month of the semester. A set of formative assessment activities was designed to provide guidance and targeted feedback on the desired outcome before final exams and final project deliverables. To promote authentic and active learning, we leveraged problem-based learning principles in a collaborative context. Thus, the sequence of the planned learning activities was based on exploration, discussion, creation, and self-reflection/ co-reflection. The vertical lines show the connection of each learning activity to the corresponding Moodle resources, as well as the metrics and indicators that we considered that may be of value for risk identification. The metrics presented in this figure emerged after reflecting on the interaction possibilities of the Moodle resources we selected to identify important signs about students' performance and learning behavior. Student behavioral and achievement metrics have emerged in the literature as the main variables for risk identification. Based on these metrics, we concluded on two main risk indicators: low participation and low performance.”

[p. 17, lines 641-660]

Reference added:

27.        Bond, M.; Buntins, K.; Bedenlier, S.; Zawacki-Richter, O.; Kerres, M. Mapping Research in Student Engagement and Educational Technology in Higher Education: A Systematic Evidence Map. Int. J. Educ. Technol. High. Educ. 2020, 17, 2, doi:10.1186/s41239-019-0176-8.

4. Response to Comments on the Quality of English Language

Point 1: The English is fine and does not require any improvement.

Response 1: Thank you!