Skip to Content
MDPIMDPI

Advanced

Education SciencesEducation Sciences
PDF
  • Article
  • Open Access

2 November 2024

15 Pages

Monitoring Metacognitive Strategies Use During Interaction Collaborative Groups

,
,
,
,
,
,
and
1
Department of Health Sciences, GIR DATAHES, UIC JCYL Nº 348, Universidad de Burgos, 09001 Burgos, Spain
2
Department of Philology, Universidad de Burgos, 09001 Burgos, Spain
3
Department of Health Sciences, GIR DATAHES, Universidad de Burgos, 09001 Burgos, Spain
4
Department of Education, Universidad de Burgos, 09001 Burgos, Spain
Educ. Sci.2024, 14(11), 1205;https://doi.org/10.3390/educsci14111205
Version Notes
Order Reprints

Abstract

This study of the use of metacognitive strategies during learning interaction in collaborative groups is considered referential for understanding the development of such strategies. The use of metacognitive strategies is related to better learning outcomes. We monitored the interaction in five collaborative groups at three points in time (initial, intermediate and final). These groups consisted of undergraduate health science (n = 9) and biomedical engineering students (n = 10). The aims were the following: (1) To check whether there were significant differences in the use of metacognitive strategies between the groups of students monitored depending on the type of degree or the point in time measurement. (2) To test whether there were significant differences in students’ learning outcomes depending on whether or not they had participated in the interaction monitoring during collaborative interaction. (3) To test clustering without a pre-assignment variable against learning outcomes in collaborative groups. Significant differences were detected in the use of some of the metacognitive strategies of orientation, planning and elaboration in favor of the collaborative groups of the biomedical engineering students. Also, students who participated in monitoring collaborative groups in both clusters obtained better learning outcomes. Lastly, the assignment groups were exactly related to the collaboration groups with respect to the use of metacognitive strategies in the final measurement.

1. Introduction

The use of collaborative methodologies in teaching is being promoted by academic leaders at an international level, especially in higher education [1]. One of the reasons is the importance of collaborative and interdisciplinary interaction in 21st-century society, marked by high digitalization [2]. This fact is especially significant for future graduates in health sciences and engineering degrees to effectively put problem-solving skills into practice [3]. In response to this challenge, the use of project-based learning (PBL) methodology is proving its effectiveness [4] along with Educational Data Mining (EDM) visualization techniques for results analysis [5]. The use of these resources will provide the teacher with information on the effectiveness of the methodology and will guide them to apply a personalized educational response [6]. The results observed will allow the teacher to make a personalized improvement of the didactic materials [7].

1.1. Analyzing Protocols During Problem-Solving or Task Resolution Aloud

The analysis of think-aloud protocols is a technique with some tradition in cognitive psychology to identify cognitive processes and functions. Moreover, this technique has also been used in the field of instructional psychology, like the research of Meichenbaum and Goodman [8], developing their studies with children to an instructional intervention, like the research of Bash and Camp [9]. These authors included the metacognitive analysis proposed by Flavell [10], explaining the differences between declarative and procedural knowledge. Then, Brown and DeLoache [11] emphasized the idea of self-regulation by the teacher (modeling and shaping): the task execution or problem solving. From this proposal, instructional strategies focused on “what I have to do”, “how I am going to do it”, “how I am doing it” and “how I have done it”. Likewise, the work of Nelson [12] proposed three levels of development, the Object-Level, in which cognitive (attention, memory, etc.) and motivational processes would be developed, and the Meta-Level, in which metacognitive processes would be developed (reflection on the development of cognitive processes). Between these two levels is the Flow of Information, which includes control and monitoring, for example. In addition, Zimmerman and Schunk [13] proposed a model of three different phases: (1) The planning phase focuses on task analysis. In this phase, the most representative strategies are goal setting, strategy planning, self-efficacy, intrinsic motivation and goal orientation. (2) The execution phase including self-control. In this phase, the most representative strategies are task-specific strategies, self-instructions, time management, environmental control, help-seeking and motivation (intrinsic and extrinsic). (3) The self-reflection phase includes self-judgment. In this phase, the most representative strategies are self-evaluation, causal attribution and adaptation. Another relevant model in this context was proposed by Sternberg [14]. This author understood that meta-components (metacognitive skills) are part of cognition, namely higher-order executive components to control and manage information processing in problem-solving tasks. On the other hand, the mixed model provided a predictive weight (40% of the variance) to metacognitive skills on the development of learning outcomes [15]. The study of Veenman et al. [16] advocated for mixed models of explanation. As per this approach, metacognitive skills are part of the executive function and are directly related to the procedural knowledge required for the regulation and the control of learning activities [10,17]. According to this orientation, metacognitive strategies are differentiated as follows: (1) Orientation: strategies used to concretize the demands of the task in cognitive terms. These strategies guide the resolution process and activate the prior knowledge necessary for the execution of a task. (2) Planning: these strategies allow the process of sequencing by steps to solve a problem or a task. (3) Evaluation: supervised evaluation strategies using the problem-solving process. This group of metacognitive strategies allows the assessment of the steps performed during the resolution of a task, analyzing their effectiveness to reorient the planning. (4) Elaboration: these strategies are the highest level of metacognition including reflection on one’s own practice and modifications, if necessary, of the process for solving problems or tasks [15].
All of these strategies can be implemented with a series of internalized self-instructions, which can be prescribed to the learner (“what to do”, “when”, “why” and “how”) during task performance or problem solving [17]. Likewise, Meijer et al. [18] proposed a detailed problem-solving taxonomy based on think-aloud protocols elicited while students were problem-solving. In summary, the use of metacognitive skills is important [19] because it explains 40% of the variance in student learning outcomes [15]. Along these lines, different studies indicated that metacognitive training results in both improved metacognitive behavior and learning outcomes [20,21].

1.2. Measurement of the Use of Cognitive and Metacognitive Strategies

In this context, one limitation was measuring the use of cognitive and metacognitive strategies. Traditionally, off-line methods (i.e., scales or surveys) have been used. In this case, the work of Román and collaborators stand out [22] in the development of the Scale of Learning strategies (ACRAr). Initially, ACRAr was based on the studies of Nisbet and Shucksmith [23]. These authors differentiated processes of “acquisition”, “encoding” and “retrieval” of the information during the task or problem-solving. In turn, this hypothesis on cognitive functioning was based on the processing model of Atkinson and Shiffrin [24] and Craik and Tulving’s [25] theory of processing levels, the theories about the mental representation of knowledge in memory of Rumelhart and Ortony [26] and in the “instructional” approach [27]. However, recent theories of cognitive and metacognitive performance have been included in the latest ACRAr [22]. These include advances in the “mixed model” which considers cognitive and metacognitive processes as correlated but independent in predicting learning outcomes [15]. This model differentiates, following Flavell [10] during the learning process between declarative (knowing what) and procedural (knowing how) knowledge [28]. Also, Brown and DeLoache [11] considered the role of self-regulation in the development of metacognition. This component is directly related to the development of motivation towards learning [13] and procedural knowledge as a relevant component for the achievement of effective learning [29,30]. From the mixed model, metacognition is understood as a higher-order agent that supervises and directs the cognitive system, as well as being part of it [16]. In summary, the “mixed model” explanation of information processing combines the interaction between cognitive and metacognitive processes, in an interactive resolution process [31,32,33]. On the other hand, online methods are an alternative to assess metacognitive strategies, i.e., recording think-aloud protocols while a subject is solving a problem or task, following Schellings et al. [28] p. 965 in line with the studies of Veenman and Beishuizen [34]. These authors indicated that there are no significant correlations between the results found with both methods. Furthermore, it seems that on-line methods are more reliable than off-line methods in terms of the results of information processing analysis. The reason could be that in on-line methods the learner gives answers about the perception they have of their cognitive processing during the resolution of a task or problem. This self-perception can lead to errors of misinterpretation because long-term memory plays an important role in the answers and can generate adjustment problems with respect to the actual form of processing executed. Another problem with the use of off-line methods is that they require a long time to analyze the collected results. This fact makes it difficult to obtain a large sample and generalize the results. This aspect is easier to address in off-line methods, since the scale for assessing the use of metacognitive strategies can be applied to a larger number of learners, although the control over their responses is much lower [28]. Figure 1 presents, following the WWWH (what, when, why and how) model of Veenman et al. [16], a relationship between the moments of processing task resolution and the type of cognitive and metacognitive strategies that are implemented.
Figure 1. Relationship between information processing moments and the use of metacognitive strategies [15].

1.3. Measurement of the Use of Cognitive and Metacognitive Strategies in Collaborative Group Interaction

From the theoretical framework presented above, another limitation handicap is was assessing the use of cognitive and metacognitive strategies involved in solving tasks or problems during collaborative interaction. This research focuses on the context of co-regulation and social–emotional interactions [35] and the analysis of collaborative learning processes, including different interactions at various levels [36]. Co-regulation requires social interactions between members of the collaborative group focused on addressing the challenges of the task or problem to be solved and applying monitoring and evaluation metacognitive strategies [37]. According to Vygotsky’s theory [38], co-regulation starts from the development of thought and therefore of behavior that occurs in the interaction with others through language. Language regulates behavior, first from the language of others (monitoring and overt modeling and shaping) and then from one’s own language, first overt and then covert. Therefore, co-regulation develops from social self-regulation [39]. Each group member has their own regulation process but in turn participates in the learning regulation of the other group members through questions or findings [40]. It should be considered that each member of a collaborative group may have different levels of prior knowledge and different ways of self-regulating their thinking. This fact implies that negotiation strategies to solve a problem or a task must be implemented for correct and productive functioning of the group in a collaborative learning environment [41]. In addition, emotional strategies also play a role [42]. Positive or negative emotional manifestations can be differentiated and be manifested verbally and/or behaviorally. This whole process is a challenge for the teacher because if they implement a good student orientation program, they can facilitate the self-regulation processes and achieve effective learning [17,43,44]. In summary, it must be considered that during the problem-solving process, it is complex to differentiate between the use of cognitive and metacognitive strategies [17,30]. This challenge is increased in the process of collaborative interaction, since the group must converge the resolution processes to reach a successful resolution [35].
In addition, as if the above-described framework were not complex enough, currently teaching, especially in higher education, is increasingly using the problem-based learning (PBL) methodology [45]. This methodology applied to the b-Learning context is known as online project-based learning (OPBL) [3]. In this framework, there are few works that develop protocols for observing metacognitive strategies in collaborative interactions during problem solving [46]. This is the challenge of the present research.
Thus, high-level metacognitive actions are fundamental to learning. However, the use of these actions does not usually occur naturally and must be promoted through appropriate scaffolding strategies [47]. Furthermore, the use of metacognitive instruction has been shown to be very effective in engineering students applying computer science [48] and in health science students [2].
Specifically, there is little research on the use of metacognitive strategies during collaborative work in situ. However, recent studies indicate that collaborative group learning can facilitate the development of self- and co-regulatory processes and significantly improve students’ written argumentation skills [49]. This type of collaboration based on the use of metacognitive strategies seems to improve learning outcomes [50]. However, it is not generalizable to all outcomes in all disciplines [51].
Based on the aforementioned state of the art, the research questions (RQs) of this study were as follows:
  • RQ1. Will there be significant differences in the use of metacognitive strategies in the intermediate measurement as a function of the type of degree (degree in occupational therapy vs. degree in biomedical engineering)?
  • RQ2. Will there be significant differences in the use of metacognitive strategies in the final measurement as a function of the type of degree (degree in occupational therapy vs. degree in biomedical engineering)?
  • RQ3. Will students who have participated in the monitoring of the use of metacognitive strategies during collaborative work have better learning outcomes than those who have not participated?
  • RQ4. What clustering with respect to the use of metacognitive strategies will be found by applying unsupervised learning clustering techniques?

2. Materials and Methods

2.1. Participants

Participants were a group of 19 students, in their final years, from two different programs: 9 occupational therapy degree (Group 1) and 10 biomedical engineering degree (Group 2) (see Table 1). Within Group 1, 2 collaborative groups were formed and within Group 2, 3 collaborative groups were formed. Throughout the research, an experimental death was detected in the group of students in biomedical engineering, specifically. Both groups will be considered as experimental groups; the rest of the students in the class group who did not participate in the monitoring process of the collaborative activity in the Gesell chamber will be considered the control group.
Table 1. Characteristics of the sample.

2.2. Instruments

(a) The protocol for observing the use of metacognitive strategies during collaborative interaction (POMSCI) was used [52]. The POMSCI included 4 dimensions: orientation, planning, evaluation and elaboration. These dimensions were further divided into sub-dimensions. The structure of the POMSCI is presented in Table 2. In total, the POMSCI included 36 items which were evaluated on a Likert-type scale from 1 to 5, where 1 indicates never and 5 always. The POMSCI had an overall reliability index of α = 0.91, and for each part, orientation of α = 0.89, planning of α = 0.90, evaluation of α = 0.91 and elaboration of α = 0.89. The POMSCI can also be found in Appendix A Table A1.
Table 2. The structure of the POMSCI.
(b) A one-way mirror Gesell camera was used to monitor collaborative interaction. The use of the camera allowed for non-participant observation. Specifically, the camera used belongs to the high-fidelity simulation center of the Faculty of Health Sciences of the University of Burgos. More information about it is available on https://bit.ly/3UqEmrb (accessed on 30 October 2024).

2.3. Procedure

This study obtained a positive report from the Bioethics Committee of the University of Burgos, no. IO 03/2022. Likewise, written informed consent to participate was obtained. Informed consent was also obtained from some of the participating students for research dissemination of the images. We worked with students from two courses, one specific to health sciences and the other to biomedical engineering. In both, we applied the instructional methodology of PBL, which consisted of the resolution of practical cases. The students of each grade were grouped in small collaborative interaction groups consisting of 2–5 students. The teacher was the same in both groups. The measurement of the collaborative strategies through the application of the POMSCI was carried out by another teacher, previously trained in observation techniques, through non-participant observation in both groups. This observation was carried out through a one-way mirror camera where the teacher could observe without being seen by the students. Three measurements were taken over nine weeks, including an initial measurement (in the first week), an intermediate measurement (in the fourth week) and a final measurement (ninth week). The phases of the study are shown in Figure 2.
Figure 2. Protocol for observing the use of metacognitive strategies during collaborative interactions.
Also, Table S2 in the Supplementary Materials document shows the steps following in the empirical research.
A summary of the working structure for observing the use of metacognitive strategies in the dynamics of collaborative group interaction can be found in Figure 3.
Figure 3. Procedure for monitoring the use of metacognitive strategies during collaborative interactions.

2.4. Data Analysis and Design Research

As we were working with a sample size of less than 30, non-parametric tests were used. Firstly, before starting the testing of the research questions, the homogeneity of the students in the two groups with respect to the use of the metacognitive strategies in the POMSCI in the initial assessment phase were tested. For this purpose, the Mann–Whitney U test was used. Also, to contrast RQ1 and RQ2, the non-parametric Mann–Whitney U test was used. To test RQ3, as in this case the sample was N = 72, ANCOVA of one fixed effects factor (participation vs. non-participation in collaborative group monitoring) and one covariate (grade type) with eta squared effect value was used. The statistical package SPSS v.28 was used to carry out these tests [53].
Likewise, to contrast RQ4, unsupervised machine learning was used. The data mining program Orange v. 3.37.0 was used to perform the analyses [54].
A quasi-experimental test with the control group was applied in RQ1, RQ2 and RQ3. Also, descriptive correlational design was applied in RQ3.

3. Results

3.1. Previous Analyses

Before the contrast of the research questions, it was tested whether there were significant differences in the use of metacognitive strategies during collaborative group interactions between Group 1 (students of health sciences) and Group 2 (students of the biomedical engineering). As can be seen in Table S1 in the Supplementary Materials document, no significant differences were found in the POMSCI.

3.2. Contrast RQ1

Regarding RQ1 (“Will there be significant differences in the use of metacognitive strategies in the intermediate measurement as a function of the type of degree (degree in occupational therapy vs. degree in biomedical engineering)?”), no significant differences were found in the use of metacognitive strategies during interaction in the collaborative groups in the intermediate measurement between Group 1 (students of health sciences, specifically the degree in occupational therapy) and Group 2 (students of the biomedical engineering). However, a mean effect value was detected in some of the items of the POMSCI. (For an analysis of the results and the Mann–Whitney U contrast, please see Table S3 in Supplementary Materials document.)

3.3. Contrast RQ2

With regard to RQ2, (“Will there be significant differences in the use of metacognitive strategies in the final measurement as a function of the type of degree (degree in occupational therapy vs. degree in biomedical engineering)?”), significant differences were found in the final measurement. In particular, these differences were found in the use of the metacognitive strategies of orientation, specifically in item 9 (‘students in the group differentiate the procedural knowledge part’) and item 10 (‘members of the collaborative group participate cooperatively in the orientation phase’). Also, significant differences were found in the use of the metacognitive strategies of planning, specifically in item 13 (‘the group distributes the resolution tasks among the members’). Also, significant differences were found in the use of the metacognitive strategies of elaboration in item 31 (‘each of the members of the group explains what the major difficulties have been encountered’). All cases in favor of Group 2 were made up of groups of biomedical engineering students. It is also noteworthy that the effect value (r) was high for these items and for those items where a trend towards difference was detected. (For an analysis of the results and the Mann–Whitney U contrast, please see Table S4 in the Supplementary Materials document.)

3.4. Contrast RQ3

Regarding RQ3, (“Will students who have participated in the monitoring of the use of metacognitive strategies during collaborative work have better learning outcomes than those who have not participated?”) significant differences in learning outcomes, specifically in the multiple choice test, were found between students who participated in the follow-up groups versus students who did not participate, in favor of students who participated in the follow-up process. However, the effect value was low (η2 = 0.01). Also, no effect of the covariate type of qualification was detected (see Table 3).
Table 3. ANCOVA of one fixed effects factor (participation in monitoring vs. non-participation) and one covariate (grade type).

3.5. Contrast RQ4

In order to analyze RQ4 (“What clustering with respect to the use of metacognitive strategies will be found by applying unsupervised learning clustering techniques?”), a cluster analysis was carried out for the two measures in the orientation, planning, evaluation and elaboration strategies. Two clusters were found. The best scores on the strategies described above were in cluster 1 (see Table 4). Subsequently, a cross-tabulation was performed to check in which degree type. The results indicate a complete of the assignment to the cluster with the type of degree. Thus, the collaborative groups of the degree in biomedical engineering were the ones that obtained the best results in the use of metacognitive strategies, with significant differences being found in the use of the metacognitive strategies of planning and evaluation (see Table 5). Likewise, an exact correspondence was found between the relationship between the collaborative groups by type of degree and the cluster of membership, and the Contingency Coefficient was of C = 0.71 p < 0.03 (see Table S5 in the Supplementary Materials document).
Table 4. Centers of final clusters.
Table 5. ANOVA test of final cluster.
Figure S1 in the Supplementary Materials document shows the distribution of the collaborative groups with respect to the dimensions of the POMSCI: orientation, planning, evaluation and elaboration. Also, Figure S2 in the Supplementary Materials document shows the representation of the clusters by degree type.

4. Discussion

The use of metacognitive strategies during tasks or problem solving is an important predictor of learning success. The use of these strategies can explain 40% of the variance in learning outcomes [15]. However, assessing the use of such strategies in face-to-face mode (online method following Veenman et al. [16,17,30]) while the user(s) performs an activity is complex. This scenario becomes even more complex when it comes to monitoring the use of metacognitive strategies during collaborative group interaction. In this study, the instructional methodology used was PBL in all students, both those who participated in the monitoring process and those who did not. However, in the monitored groups, group members were provided with more personalized feedback on the use of metacognitive strategies following the WWWH proposal by Veenman et al. [16,17,18,19] based on the questions: what, when, why and how [17,18]. The results indicate that this type of monitoring with feedback on the use of metacognitive strategies was effective, as the students in the monitored groups obtained better learning results, specifically in the conceptual content tests [8,9,10,11]. However, no significant differences were found in the PBL tests, such as the PBL elaboration and defense tests, since all the students (both those who were monitored and those who were not) worked with the PBL methodology, which facilitates interaction [35,36,38,39] and self-regulation [17,43,44] and problem solving [37,40,41,42,46]. It seems that working with the PBL methodology is not sufficient to encourage the use of metacognitive strategies. The use of these must be regulated through personalized interventions guided by the teacher [47]. On the other hand, no significant differences were found regarding the use of metacognitive strategies during collaborative group interactions depending on the type of degree in the initial and intermediate measurements. However, they were found in the final measurement, specifically in the orientation dimension, namely in the sub-dimensions of “Cognitive knowledge” and “Cooperative learning”. Similarly, significant differences were also found in the Planning dimension, specifically in the sub-dimension of “Group organization”. Finally, significant differences were also found in the elaboration dimension, in particular, in the sub-dimension of “Self-efficacy” [2,48]. These differences may be due to the development of a different approach to activities resolution that developed over the course of the semester. Future research will address the longitudinal study of resolution patterns in students from different degree programs.

5. Conclusions

The use of non-participant monitoring resources is important to gain insight into the interaction processes in collaborative groups. The reasons are that this procedure is hardly invasive and allows a rather objective analysis. However, the use of non-participant monitoring via a one-way mirror is a laborious and resource-intensive procedure. Specifically, this procedure has a series of limitations that are related to the availability of material resources (one-way mirror camera and instruments for recording) and personnel (sufficient teachers to carry out non-participant monitoring, as at least two teachers are required, one interacting with the group and the other observing). All this makes this type of monitoring quite costly in terms of time and resources. These aspects reflect the strengths and weaknesses of this approach. The former refers to the personal and material resources described above. The second lies in the fact that this type of monitoring can provide the teacher with a lot of information which can help them improve their instructional process. In this sense, it was observed that the collaborative groups subject to non-participant monitoring with individualized feedback obtained better learning results with respect to conceptual knowledge. Furthermore, the results suggest differences in the use of metacognitive strategies in the final monitoring phase according to the branch of knowledge. In summary, the limitations of this study lie in the difficulty in generalizing the results. Specifically, the limitations focus on the choice of the sample (convenience sampling was applied) and the number of the sample (only a limited number of students accepted to be monitored and recorded during the group collaborative interaction). However, the difficulty of having large and random samples in this type of research must be considered. In this line, as indicated by the studies of Çini et al. [46], work in this field is essential to advance metacognitive research during collaborative learning. Furthermore, as future lines of research, in addition to those mentioned above, future studies will delve into the analysis of intra-group and inter-group interaction maps, taking a large number of participants to better explicit eventual differences in statistics analysis. Also, studies will be carried out to predict the use of metacognitive strategies in collaborative groups. For this purpose, supervised (prediction) and unsupervised (clustering) EDM techniques will be used. It is expected that the conclusions of these studies will guide teachers on the most appropriate types of support for each type of collaborative group in the use of metacognitive strategies (guidance, planning, evaluation and elaboration). All of this is expected to facilitate the development of personalized learning. Finally, we would like to highlight that this study has provided an instrument for monitoring the use of metacognitive strategies during collaborative group interactions. It has also provided an example of monitoring collaborative group interactions through a one-way mirror camera.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/educsci14111205/s1, Table S1: Mann–Whitney U contrast in the POMSCI in the initial measurement of the collaborative groups in two types of degrees (health sciences vs. biomedical engineering); Table S2: Summary of the protocol to analyze the use of metacognitive strategies in performing collaborative problem-solving tasks; Table S3: Mann–Whitney U contrast in the POMSCI in the initial measurement of the collaborative groups in two types of degrees (health sciences vs. biomedical engineering); Table S4: Mann–Whitney U contrast in the POMSCI in the final measurement of the collaborative groups in two types of degrees (health sciences vs. biomedical engineering); Table S5: Cross-table between cluster type and type of measurement; Figure S1: Radviz data of the dimensions in the two collaborative groups; Figure S2: Clustering with ordered leaves in the two collaboratives groups.

Author Contributions

Conceptualization, M.C.S.-M., I.G.-D., A.J.E., C.V.V., L.S.A. and R.V.S.; methodology, M.C.S.-M., I.G.-D., A.J.E., C.V.V., L.S.A., C.M. and R.V.S.; software, M.C.S.-M. and I.G.-D. and C.V.V. and A.C.R.; validation, M.C.S.-M., I.G.-D., A.J.E., C.V.V., L.S.A. and R.V.S.; formal analysis, M.C.S.-M.; investigation, M.C.S.-M., I.G.-D. and C.V.V.; resources, M.C.S.-M., I.G.-D., and C.V.V.; data curation, M.C.S.-M., I.G.-D. and C.V.V.; writing—original draft preparation, M.C.S.-M.; writing—review and editing, all authors; visualization, M.C.S.-M.; supervision, all authors; project administration, M.C.S.-M.; funding acquisition, M.C.S.-M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was part of the project “Voice assistants and artificial intelligence in Moodle: a path to a smart university” -SmartLearnUni-. Project number: PID2020-117111RB-I00, funded by Agencia Estatal de Investigación. Ministerio de Ciencia e Innovación y Universidades. Gobierno de España.

Institutional Review Board Statement

Details omitted for double blind reviewing.This study has been carried out under the umbrella of the Ethics Committee of the University of Burgos report No. IO 03/2022 and report No. IO 04/2022.

Data Availability Statement

Data will be made available upon request of the researchers and signature of responsible use and privacy due to ethical reasons.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

Table A1. Protocol for monitoring the use of metacognitive strategies during the group collaborative interaction (POMSCI) [52].

References

  1. Adedoyin, O.B.; Soykan, E. Covid-19 pandemic and online learning: The challenges and opportunities. Interact. Learn. Environ. 2023, 31, 863–875. [Google Scholar] [CrossRef] [Scilit]
  2. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Muñoz-Rujas, N.; Rodríguez-Arribas, S.; Escolar-Llamazares, M.C.; Alonso-Santander, N.; Martínez-Martín, M.Á.; Mercado-Val, E.I. Teaching and learning styles on Moodle: An analysis of the effectiveness of using stem and non-stem qualifications from a gender perspective. Sustainability 2021, 13, 1166. [Google Scholar] [CrossRef] [Scilit]
  3. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Rodríguez-Díez, J.J.; Rodríguez-Arribas, S.; Díez-Pastor, J.F.; Ji, Y.P. Improve teaching with modalities and collaborative groups in an LMS: An analysis of monitoring using visualisation techniques. J. Comput. High. Educ. 2021, 33, 747–778. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  4. Liu, S.; Liu, M. The impact of learner metacognition and goal orientation on problem-solving in a serious game environment. Comput. Hum. Behav. 2020, 102, 151–165. [Google Scholar] [CrossRef] [Scilit]
  5. Moos, D.C.; Azevedo, R. Self-efficacy and prior domain knowledge: To what extent does monitoring mediate their relationship with hypermedia learning? Metacogn. Learn. 2009, 4, 197–216. [Google Scholar] [CrossRef] [Scilit]
  6. Sáiz-Manzanares, M.C.; García Osorio, C.I.; Díez-Pastor, J.F.; Martín Antón, L.J. Will personalized e-Learning increase deep learning in higher education? Inf. Discov. Deliv. 2019, 47, 53–63. [Google Scholar] [CrossRef] [Scilit]
  7. Xie, H.; Chu, H.C.; Hwang, G.J.; Wang, C.C. Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017. Comput. Educ. 2019, 140, 103599. [Google Scholar] [CrossRef] [Scilit]
  8. Meichenbaum, D.; Goodman, J. Reflection-Impulsivity and Verbal Control of Motor Behavior. Child Dev. 1969, 40, 785–797. [Google Scholar] [CrossRef] [Scilit]
  9. Bash, M.A.; Camp, B.W. Think Aloud: Increasing Social and Cognitive Skills: A Problem Solving Approach for Children. In Chrome Program (Grades 1 and 2); Research Press: Champaign, IL, USA, 1985. [Google Scholar]
  10. Flavell, J.H. Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. Am. Psychol. 1979, 34, 906–911. [Google Scholar] [CrossRef]
  11. Brown, A.L.; DeLoache, J.S. Skills. Plans. and Self-Regulation. In Children’s Thinking: What Develops? Siegler, R.S., Ed.; Lawrence Erlbaum Associates. Inc.: Hillsdale, NJ, USA, 1978; pp. 3–35. [Google Scholar]
  12. Nelson, T.O. Metamemory: A Theoretical Framework and New Findings. Psychol. Learn. Motiv. 1990, 26, 125–173. [Google Scholar] [CrossRef] [Scilit]
  13. Zimmerman, B.J.; Schunk, D.H. Self-Regulated Learning and Performance: An Introduction and an Overview. In Handbook of Self-Regulation of Learning and Performance; Zimmerman, B.J., Schunk, D.H., Eds.; Routledge/Taylor & Francis Group: New York, NY, USA, 2011; pp. 1–12. [Google Scholar]
  14. Sternberg, R.J. Beyond IQ: A Triarchic Theory of Intelligence; Cambridge University Press: New York, NY, USA; Cambridge, UK, 1985. [Google Scholar]
  15. van der Stel, M.; Veenman, M.V.J. Metacognitive skills and intellectual ability of young adolescents: A longitudinal study from a developmental perspective. Eur. J. Psychol. Educ. 2014, 29, 117–137. [Google Scholar] [CrossRef] [Scilit]
  16. Veenman, M.V.J.; Van Hout-Wolters, B.H.A.M.; Afflerbach, P. Metacognition and learning: Conceptual and methodological considerations. Metacogn. Learn. 2006, 1, 3–14. [Google Scholar] [CrossRef] [Scilit]
  17. Veenman, M.V.J. Assessing metacognitive deficiencies and effectively instructing metacognitive skills. Teach. Coll. Rec. 2017, 119, 1–20. [Google Scholar] [CrossRef] [Scilit]
  18. Meijer, J.; Veenman, M.V.J.; van Hout-Wolters, B.H.A.M. Metacognitive activities in text-studying and problem-solving: Development of a taxonomy. Educ. Res. Eval. 2006, 12, 209–237. [Google Scholar] [CrossRef] [Scilit]
  19. Veenman, M.V.J.; van Cleef, D. Measuring metacognitive skills for mathematics: Students’ self-reports versus on-line assessment methods. ZDM—Math. Educ. 2019, 51, 691–701. [Google Scholar] [CrossRef] [Scilit]
  20. Azevedo, R.; Greene, J.A.; Moos, D.C. The effect of a human agent’s external regulation upon college students’ hypermedia learning. Metacognition Learn. 2007, 2, 67–87. [Google Scholar] [CrossRef] [Scilit]
  21. Veenman, M.V.J.; Elshout, J.J.; Busato, V.V. Metacognitive mediation in learning with computer-based simulations. Comput. Hum. Behav. 1994, 10, 93–106. [Google Scholar] [CrossRef] [Scilit]
  22. Román, J.M.; Gallego, S. ACRA (r): Escalas de Estrategias de Aprendizaje [Learning Strategies Scales]; TEA: Madrid, Spain, 2008. [Google Scholar]
  23. Nisbet, J.; Shucksmith, J. Learning Strategies; Routledge Education Books: London, UK, 1986. [Google Scholar]
  24. Atkinson, R.C.; Shiffrin, R.M. Human Memory: A Proposed System and Its Control Processes. In The Psychology of Learning and Motivation: II; Spence, K.W., Spence, J.T., Eds.; Academic Press: Cambridge, MA, USA, 1968. [Google Scholar] [CrossRef] [Scilit]
  25. Craik, F.I.M.; Tulving, E. Depth of processing and the retention of words in episodic memory. J. Exp. Psychol. Gen. 1975, 104, 268–294. [Google Scholar] [CrossRef]
  26. Rumelhart, D.E.; Ortony, A. The representation of knowledge in memory. J. Study Educ. Dev. 1982, 5, 115–158. [Google Scholar] [CrossRef] [Scilit]
  27. Genovard, C.; Gotzens, C.; Castelló, A.; González, C.; González, J.P. Assessment of cognitive processes and instructional optimization. Infanc. Aprendiz. 1992, 15, 143–152. [Google Scholar] [CrossRef] [Scilit]
  28. Schellings, G.L.L.M.; Van Hout-Wolters, B.H.A.M.; Veenman, M.V.J.; Meijer, J. Assessing metacognitive activities: The in-depth comparison of a task-specific questionnaire with think-aloud protocols. Eur. J. Psychol. Educ. 2013, 28, 963–990. [Google Scholar] [CrossRef] [Scilit]
  29. Cerezo, R.; Núñez, J.C.; Rosário, P.; Valle, A.; Rodríguez, S.; Bernardo, A.B. New media for the promotion of self-regulated learning in higher education. Psicothema 2010, 22, 306–315. [Google Scholar] [PubMed]
  30. Dignath, C.; Veenman, M.V.J. The Role of Direct Strategy Instruction and Indirect Activation of Self-Regulated Learning-Evidence from Classroom Observation Studies. Educ. Psychol. Rev. 2021, 33, 489–533. [Google Scholar] [CrossRef] [Scilit]
  31. Harks, B.; Rakoczy, K.; Hattie, J.; Besser, M.; Klieme, E. The effects of feedback on achievement. interest and self-evaluation: The role of feedback’s perceived usefulness. Educ. Psychol. 2014, 34, 269–290. [Google Scholar] [CrossRef] [Scilit]
  32. Reoyo, N.; Carbonero, M.Á.; Martín, L.J. Teaching efficacy characteristics from the perspectives of secondary school teachers and future teachers. Rev. Educ. 2017, 376, 62–84. [Google Scholar] [CrossRef] [Scilit]
  33. Veenman, M.V.J. Learning to Self-Monitor and Self-Regulate. In Handbook of Research on Learning and Instruction; Mayer, R., Alexander, P., Eds.; Routledge: New York, NY, USA, 2011; pp. 197–218. [Google Scholar]
  34. Veenman, M.V.J.; Beishuizen, J.J. Intellectual and metacognitive skills of novices while studying texts under conditions of text difficulty and time constraint. Learn. Instr. 2004, 14, 621–640. [Google Scholar] [CrossRef] [Scilit]
  35. Lajoie, S.P.; Lee, L.; Poitras, E.; Bassiri, M.; Kazemitabar, M.; Cruz-Panesso, I.; Hmelo-Silver, C.; Wiseman, J.; Chan, L.K.; Lu, J. The role of regulation in medical student learning in small groups: Regulating oneself and others’ learning and emotions. Comput. Hum. Behav. 2015, 52, 601–616. [Google Scholar] [CrossRef] [Scilit]
  36. Saab, N. Team regulation. regulation of social activities or co-regulation: Different labels for effective regulation of learning in CSCL. Metacogn. Learn. 2012, 7, 1–6. [Google Scholar] [CrossRef] [Scilit]
  37. Lajoie, S.P.; Lu, J. Supporting collaboration with technology: Does shared cognition lead to co-regulation in medicine? Metacogn. Learn. 2012, 7, 45–62. [Google Scholar] [CrossRef] [Scilit]
  38. Vygotsky, L.S. Mind in Society: Development of Higher Psychological Processes; Harvard University Press: Cambridge, MA, USA, 1978. [Google Scholar] [CrossRef] [Scilit]
  39. Hadwin, A.F. Commentary and future directions: What can multi-modal data reveal about temporal and adaptive processes in self-regulated learning? Learn. Instr. 2021, 72, 101287. [Google Scholar] [CrossRef] [Scilit]
  40. Järvelä, S.; Hadwin, A.F. New Frontiers: Regulating Learning in CSCL. Educ. Psychol. 2013, 48, 25–39. [Google Scholar] [CrossRef] [Scilit]
  41. Hadwin, A.; Oshige, M. Self-regulation. coregulation. and socially shared regulation: Exploring perspectives of social in self-regulated learning theory. Teach. Coll. Rec. 2011, 113, 240–264. [Google Scholar] [CrossRef] [Scilit]
  42. Lajoie, S.P.; Pekrun, R.; Azevedo, R.; Leighton, J.P. Understanding and measuring emotions in technology-rich learning environments. Learn. Instr. 2020, 70, 101272. [Google Scholar] [CrossRef] [Scilit]
  43. Dignath, C.; Büttner, G. Teachers’ direct and indirect promotion of self-regulated learning in primary and secondary school mathematics classes—Insights from video-based classroom observations and teacher interviews. Metacognition Learn. 2018, 13, 127–157. [Google Scholar] [CrossRef] [Scilit]
  44. Martín-Antón, L.J.; Almeida, L.; Sáiz-Manzanares, M.C.; Álvarez-Cañizo, M.; Carbonero, M.Á. Academic Procrastination in Higher Education: Psychometric Properties of the Academic Procrastination Scale in Spanish University Students. Assess. Eval. High. Educ. 2022. [Google Scholar] [CrossRef] [Scilit]
  45. Sáiz-Manzanares, M.C.; Alonso-Martínez, L.; Calvo-Rodríguez, A.; Martin, C. Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques. J. Vis. Exp. 2022, e63601. [Google Scholar] [CrossRef] [Scilit]
  46. Çini, A.; Järvelä, S.; Dindar, M.; Malmberg, J. How multiple levels of metacognitive awareness operate in collaborative problem solving. Metacogn. Learn. 2023, 18. [Google Scholar] [CrossRef] [Scilit]
  47. Yang, Y.; Feng, X.; Zhu, G.; Xie, K. Effects and mechanisms of analytics-assisted reflective assessment in fostering undergraduates’ collective epistemic agency in computer-supported collaborative inquiry. J. Comput. Assist. Learn. 2024, 40, 1098–1122. [Google Scholar] [CrossRef] [Scilit]
  48. Zheng, L.; Zhen, Y.; Niu, J.; Zhong, L. An exploratory study on fade-in versus fade-out scaffolding for novice programmers in online collaborative programming settings. J. Comput. High. Educ. 2022, 34, 489–516. [Google Scholar] [CrossRef] [Scilit]
  49. Zheng, X.-L.; Huang, J.; Xia, X.-H.; Hwang, G.-J.; Tu, Y.-F.; Huang, Y.-P.; Wang, F. Effects of online whiteboard-based collaborative argumentation scaffolds on group-level cognitive regulations, written argument skills and regulation patterns. Comput. Educ. 2023, 207, 104920. [Google Scholar] [CrossRef] [Scilit]
  50. Klar, M.; Buchner, J.; Kerres, M. Limits of Metacognitive Prompts for Confidence Judgments in an Interactive Learning Environment. Open Educ. Stud. 2024, 6, 20220209. [Google Scholar] [CrossRef] [Scilit]
  51. Li, W.; Liu, C.-Y.; Tseng, J.C.R. Effects of the interaction between metacognition teaching and students’ learning achievement on students’ computational thinking, critical thinking, and metacognition in collaborative programming learning. Educ. Inf. Technol. 2023, 28, 12919–12943. [Google Scholar] [CrossRef] [Scilit]
  52. Sáiz-Manzanares, M.C. Protocol for Observing the Use of Metacognitive Strategies During Collaborative Inter-Action (POMSCI); Repositorio Institutional Universidad de Burgos: Burgos, Spain, 2024. [Google Scholar]
  53. IBM Corp. SPSS Statistical Package for the Social Sciences (SPSS), 28th ed.; IBM Corp: Armonk, NY, USA, 2023. [Google Scholar]
  54. Demšar, J.; Curk, T.; Erjavec, A.; Gorup, Č.; Hočevar, T.; Milutinovič, M.; Možina, M.; Polajnar, M.; Toplak, M.; Starič, A.; et al. Orange: Data Mining Toolbox in Python. J. Mach. Learn. Res. 2013, 14, 2349–2353. Available online: http://jmlr.org/papers/v14/demsar13a.html (accessed on 27 July 2024).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Effects of Observing Urban and Natural Scenes on Working Memory Depletion and Restoration: An EEG Study
Previous
Principals in Preschool Management and Leadership: A Necessary Combination to Ensure That Preschool Teachers Are Given Conditions to Be Responsible for Teaching in Preschool
Next