Next Article in Journal
China’s Outward Foreign Direct Investment and the Environmental Performance of the “Belt and Road Initiative” Countries
Next Article in Special Issue
Fostering AI Literacy in Elementary Science, Technology, Engineering, Art, and Mathematics (STEAM) Education in the Age of Generative AI
Previous Article in Journal
From the Sea to Mosquito Control: The Potential of Halymenia dilatata Marine Alga as an Eco-Friendly Mosquitocidal Agent
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 15
Issue 15
10.3390/su151511901
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Research on the Quality of Collaboration in Project-Based Learning Based on Group Awareness

by
Rui Zhang
1,
Ji Shi
2 and
Jianwei Zhang
1,*
1
School of Physics Science and Engineering, Tongji University, Shanghai 200092, China
2
Institute of Vocational Education, Tongji University, Shanghai 200092, China
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(15), 11901; https://doi.org/10.3390/su151511901
Submission received: 4 July 2023 / Revised: 28 July 2023 / Accepted: 31 July 2023 / Published: 2 August 2023
(This article belongs to the Special Issue Technology Enhanced Science Education: Research Innovations in Pedagogy and Practices)
Browse Figures
Versions Notes

Abstract

:
Project-based learning (PBL) is an important form of collaborative learning that has a significant positive impact on student capacity development. However, data generated during long periods of PBL are difficult to record in real time, and there is a dearth of specific empirical research on the relationship between the quality of collaboration and the effectiveness of collaboration. In this study, we employ text mining to measure and analyze process data from a college physics course that utilizes project-based learning at a university. Under the group awareness theory, we construct a project-based learning system and extract six multi-dimensional awareness data (including numbers of text, cognitive awareness, emotional awareness, behavioral awareness, social awareness of group members, and group leaders’ cognitive awareness of the project). These data are then utilized to build a multiple regression model, which enables the analysis of collaboration quality within collaborative groups. The results show that the group awareness information automatically processed by text mining can reflect the quality of collaboration, and the quality of collaboration can determine about 50% of the effectiveness of collaboration. The important factors affecting the collaboration quality of project-based learning were group leaders’ cognitive awareness of the project and the cognitive awareness of group members. Therefore, in order to improve the quality of collaboration in PBL, teachers should select responsible group leaders and encourage meaningful communication between group members, thereby fostering high-quality and sustainable collaboration development.

1. Introduction

In the 21st century, the development of complex talents has become a major direction in the training of human resources. Collaborative learning enriches the overall quality of learners and plays an important role in developing higher-order learning skills. Project-based learning is a student-centered instructional approach that prioritizes the development of collaboration, inquiry, and problem-solving skills among students [1]. By engaging them in small-group work on real-life projects, this approach fosters the cultivation of interpersonal skills and, more specifically, generates collaborative skills, thereby enhancing their interpersonal skills, collaboration, and communication skills [2,3]. In higher education, project-based learning empowers students to acquire a diverse range of knowledge and innovative skills essential for tackling future challenges and achieving success [4]. According to the consensus among scholars, the development of enhanced basic collaboration skills, such as effective communication of ideas, respect for others, and teamwork through social learning, is regarded as a crucial component of project-based learning [5]. In recent years, project-based collaborative learning is receiving attention from educational research institutes and educational administrators. The latest Educause Horizon Report (2022, 2023) highlighted the emerging significance of hybrid and collaborative learning as a crucial area of research within the realm of artificial intelligence [6,7,8]. The Chinese Government has put forward the “China’s Education Modernization 2035” initiative, aiming to foster first-class talents, with collaborative learning emerging as a vital approach to cultivate students’ multi-dimensional abilities [9].
Project-based learning (PBL) is an inquiry-based, holistic instructional approach rooted in authentic contexts. It represents a distinctive form of collaborative learning that places greater emphasis on student-centered engagement with tangible, real-world artifacts [10,11]. In this context, PBL has been extensively adopted in higher education, particularly in the field of engineering education for its authentic problem-solving skills training [11,12,13]. Physics, being a fundamental course within the engineering disciplines, plays a critical role in nurturing highly skilled professionals. Consequently, the development of students’ core proficiency in physics through PBL holds significant importance [14,15]. The effectiveness of project-based learning is commonly evaluated through a comprehensive assessment encompassing various dimensions, including cognitive, affective, behavioral, and artifact performance [16]. Instruments frequently employed to assess the effectiveness of project-based learning encompass a range of methodologies, such as self-report questionnaires, tests, interviews, observations, self-reporting, and artifact performance evaluations [17]. It is notable that the majority of project-based learning evaluations heavily depend on teacher judgment, even in the context of long-term activities. Evaluation procedures typically involve the utilization of appropriate scales, along with self-reports, reflective journals, and other relevant components [18]. There are, of course, studies that report on the use of computer-mediated construction of collaborative environments for project-based learning, but mainly as a mediating tool, such as an online peer assessment environment for conducting project-based learning [19], as a knowledge forum or as a blackboard tool to facilitate communication [20]. Rather, it is not designed to conduct real-life scenario-based project activities for intelligent assessment through computer-supported collaborative learning (CSCL). Therefore, computer-supported text mining of mutual assessment texts to assess the quality of collaboration provides a new perspective on the assessment of PBL.

2. Literature Review

2.1. Quality of Collaboration

Collaboration quality greatly influences the success of the collaboration, and various methods have been employed to examine collaboration quality. One such approach involves the utilization of a collaboration quality assessment tool that focuses on behavioral communication at both the individual and group levels [21]. The quality of team collaboration was assessed using the collaboration maturity model developed by Boughzala and De Vreede [22]. Furthermore, Jiang and Lou conducted a study on the quality of collaboration by incorporating participatory design into PBL activities [23]. Additionally, specific group moderation mechanisms have been identified as facilitators of student performance in PBL [24]. The assessment of collaboration quality often relies on the utilization of scales. Previous scholarly work has successfully employed such scales to measure collaboration quality in various domains, including medical teaching [25], mathematics [26], and cultural preservation participation [27]. However, it is important to note that this approach to measurement tends to be more subjective in nature.
Multimodal technology has emerged as a valuable approach for assessing the quality of collaboration. In particular, it enables the collection of various types of data, such as audio, logs, and eye movements, during face-to-face collaborative learning activities. These data, obtained through multimodal technology, facilitate effective evaluation of collaboration quality [28,29]. Som et al. employed a machine learning approach to evaluate the video and audio data of the collaboration process. They utilized the mixup data augmentation method as part of their analysis [30]. Chounta and Avouris conducted real-time assessments of short online collaboration activities by evaluating six dimensions of collaboration quality [31]. The primary focus of research has predominantly revolved around the utilization of technology for monitoring human collaboration. Nevertheless, it has been observed that this approach entails significant expenses, costs, and challenges in achieving widespread implementation.
Building upon this foundation, text mining techniques for collaborative learning have emerged as important tools for assessing the quality of collaboration, particularly due to the widespread adoption of CSCL and the availability of large-scale interaction data recording, which have provided the necessary conditions for their development [32]. Research on teaching and learning through text data mining conducted by Yang and An revealed that four methods, namely information extraction, text clustering, text classification, and topic modeling, are widely employed to address various educational problems [33]. Sentiment classification of texts by conditional random fields has also been used in the field of education [34]. According to Rosé et al., the integration of text classification in CSCL allows for more cost-effective analysis of the collaborative process. They further argue that monitoring tools can be utilized to some extent to assess the quality of manual coding [35].
Overall, the existing body of literature on PBL has primarily concentrated on higher-order thinking, self-regulated learning, and metacognition among students. However, comparatively less emphasis has been placed on examining the quality of collaboration within PBL contexts. To address this gap, leveraging online platforms to collect collaborative texts and employing text mining techniques for measuring collaboration quality emerges as a promising approach. This methodology offers several advantages, including enhanced objectivity, reduced operational costs, and suitability for large-scale implementation in evaluating collaborative quality within the context of PBL.

2.2. Group Awareness

Group awareness encompasses the process through which individuals form a perception of the collective team and its overall situation. This concept entails an understanding or perception of the characteristics of a learning partner or collaborative group [36]. It has found extensive utilization in research, particularly in the domain of CSCL, with the aim of enhancing collaborative effectiveness [37,38]. Most scholars consider group awareness as a measure of individuals’ understanding of various aspects of the collaborative group and their perception of the information related to it in CSCL [39]. Group awareness encompasses a range of types and definitions. However, it can be broadly classified based on the generally accepted categorization into three main types: behavioral awareness, cognitive awareness, and social awareness [40,41]. Su et al. highlight the significance of emotional involvement as a dimension [42].
Behavioral awareness centers on the role, participation, and contribution of the group or peers in the ongoing project or activity. It involves perceiving and being aware of the tasks and work being carried out within the group. A significant body of research has been dedicated to analyzing learner data collected from online learning platforms. These data include metrics such as the number of comments, responses, likes, and other indicators that aim to capture learners’ collaborative engagement [43]. Additionally, visual graphs have been employed to explicitly label student contributions [44], fostering increased group collaboration efficiency.
Cognitive awareness pertains to the level of awareness among the group or peers regarding the knowledge acquired and constructed within the group, including knowledge relevant to the completed project. It is also regarded as a metacognitive process [45] that forms the foundation for self-regulated behavior [46]. Cognitive load theory has been employed to investigate cognitive awareness as well [47]. The measures used to assess cognitive awareness encompass various approaches. These measures include self-evaluation [48], gathering opinions and evaluations from group members [49,50], as well as examining the knowledge performance and information of collaborative members [51]. The dimensions of cognitive awareness involve perceptions of the current knowledge level within the collaborative group, understanding the relationships within the knowledge concept structure, and recognizing different viewpoints. Visualization tools utilized in cognitive awareness assessment include graphical representations, shared knowledge situations [52], textual visualizations such as word clouds and tags, as well as network concept maps that visualize associative relationships.
Social awareness encompasses the perception and understanding of how the group functions, including an understanding of the dynamics of interaction, the movements within the group, and the level of communication [53]. On the other hand, affective awareness involves sensing the emotional states of peers during interactions and is often derived from peer assessment data [54]. Successful collaboration relies on awareness of social and emotional awareness, as it significantly impacts the group climate and individuals’ willingness to participate [39]. The primary measures employed to assess social and emotional awareness include systematic interaction data, inter-rater data, and scale data [55]. Some scholars have delved into characterizing group perception data by comparing dimensions such as the frequency of interaction and the extent of engaged relationships [56].
Multiple scholars have confirmed the influence of group awareness information on collaborative learning, emphasizing its significance in the collaborative process [46,57]. Group awareness provides members with valuable insights into their cognitive, social, and other behaviors, enabling them to make positive behavioral adjustments [58]. Group awareness information has the potential to enhance learners’ self-regulation and promote increased individual contributions and peer interactions in collaborative learning [46]. Furthermore, group awareness can facilitate the establishment of connections among collaborative partners, improve knowledge sharing among group members [59], and promote more positive affective interactions [55].
Most scholars believe that the group awareness tool has a positive impact on students’ collaborative processes, group performance, and individual performance [60]. The utilization of group perception and peer assessment as a means to enhance self-perceived efficacy has been found to effectively facilitate collaborative activities [19]. On the contrary, the absence of perceived information about peers may impact the success of collaboration [61].
Recently, there has been a growing research interest in the integration of group awareness and peer assessment, with an increasing number of empirical studies focusing on collaborative learning through peer feedback and peer assessment [19,42,46]. However, there is less research on the relationship between group awareness data and collaborative quality, particularly in the context of PBL, and existing research is largely focused on online collaboration. In reality, PBL is a hands-on, face-to-face collaborative learning approach that places significant emphasis on offline practical skills development among students. Neglecting research on the group awareness quality of collaboration within PBL would be a disadvantage, and existing research has primarily concentrated on exploring group awareness through group awareness tools in the CSCL domain. At the same time, there is a scarcity of direct measurement of group awareness as it unfolds during collaboration [62]. Moreover, the conceptualization of group awareness as a theory has been studied with inconclusive evidence, lacking empirical study [63].
According to the literature review, the measurement of collaboration quality has primarily focused on the field of online collaboration, while there has been limited attention given to the measurement of collaboration quality in PBL using group awareness theory. Moreover, the existing body of research lacks sufficient quantitative evidence in this area. Therefore, the present study aims to address the following research questions:
(1)
How to design the project-based learning system we have built to collect and measure group awareness data through log text on project-based learning?
(2)
How to use the log text data generated during project-based learning to determine the quality of collaboration under the framework of group awareness theory?

3. Design

3.1. Research Content and Participants

The participants in this study were 197 students (52 females) attending a university in China who were enrolled in three distinct classes. Notably, all participants were first-year students pursuing experimental engineering programs, encompassing disciplines such as intelligent manufacturing, architectural planning landscape and design, civil engineering, and information technology. They had no prior exposure to project-based learning and possessed comparable backgrounds in terms of their mastery of the physics curriculum. It is noteworthy that the class in question is selective, adhering to the Chinese Ministry of Education’s top-notch program, which means students enrolled in this class demonstrated superior academic performance upon admission compared to their counterparts in other classes.
Forty-seven groups undertook project-based learning in this study, with each group comprising 5 to 6 individuals. The process of grouping was based on the participants’ performance on the entrance exam and the physics knowledge test at the beginning of the course. The physics knowledge test was designed by the instructors. We employed the following method for grouping the students: Initially, we sorted them into five nearly equal size clusters based on their entrance exam and physics knowledge test scores, with the original plan to create groups of five students each. We then proceeded to randomly choose one student from each cluster to form collaborative groups. Subsequently, the teaching assistant made minor adjustments to ensure optimal group compositions due to some missing data. The groupings aimed to establish homogeneity between groups and heterogeneity within groups. To facilitate the project-based learning process, a leadership model incorporating both vertical and shared approaches was implemented [64]. The group members were involved in the selection of their own leader, who played a crucial role in task assignment and coordination within the group. Importantly, the group leader also actively engaged as a regular member during individual project-based learning, thus fostering a comprehensive and integrated learning experience.

3.2. Construction of PBL System

To collect student process data, the study constructs a PBL system based on group awareness theory. This system served as a platform for recording the PBL outcomes achieved by group members throughout various stages of the process. Moreover, it incorporated a feature that facilitated mutual evaluation among group members. As shown in Figure 1, the collaborative group’s phases can be clearly seen. Once a phased task is finalized, the system will update its status as “Finished” and visually represent the completion by displaying a green button.
Students are expected to document their own work and individual contributions in each phased task and actively engage in peer evaluation. Upon completion of a task, it will be visually denoted with a green marker, which is visible to all students. When conducting peer evaluation, the system will draw attention to incomplete task points, while completed tasks will be visually highlighted with a green marker. This functionality facilitates students in monitoring the progress of their group work, reminding them of their individual achievements and current tasks. Each task point consists of a task leader, task status, task time, and task description. The task leader could submit the work context, and the task member could evaluate the work. The task description is assigned by the task leader, and an ongoing task is represented by a yellow task point, indicating its active status. The task status and task time are automatically generated by the system. Upon the completion of a task by all group members, it will be visually indicated as green and will be accessible solely for viewing purposes. The process is shown in Figure 2.

3.3. Content and Process of PBL

The objective of this PBL initiative is to foster the development of students’ comprehensive skills, such as critical thinking, innovation, and collaboration. In accordance with the physics curriculum for the semester, a total of 18 topics have been thoughtfully designed to align with the teaching schedule. These topics are based on the content covered in university-level physics education, encompassing areas such as mechanics and thermodynamics, as shown in Table 1.
To ensure diversity and encourage collaboration, each project topic will be allocated to a minimum of two groups, which will be paired together. At the conclusion of the designated period, the paired groups will have the opportunity to showcase and exchange their projects. Each project has a duration of approximately two months, allowing for the completion of two projects within two terms. During the project timeline, group members are expected to fulfill their individual tasks in accordance with the project requirements. Meanwhile, the group leader, in addition to finishing their own work, is responsible for coordinating the assignment of specific tasks to each group member (including themselves) through group discussions and consensus-building. Moreover, it is mandatory for each group to maintain a meticulous record of their project-based learning process. The culmination of the project entails the submission of completed artifacts, the lab report book, and the debriefing materials. The project production process is shown in Figure 3.
Each project comprises three essential components: theoretical derivation, numerical simulation, and practical fabrication. The team leader assumes the pivotal role of assigning project tasks, while each task point designates a team member as the task leader responsible for executing the assigned work. The specific process of project-based learning is outlined in Figure 4.

3.4. Evaluation of PBL

The effectiveness of students’ engagement in project-based learning was assessed through the evaluation of group project outcomes conducted by two professors and two teaching assistants from the university. The professors have taught physics courses for more than ten years at Tongji University. The teaching assistants are new teachers with a theoretical physics doctorate degree. The evaluation process entailed four levels of assessment, encompassing diverse dimensions such as reporting materials and experimental reports, which are described in detail in Table 2. In the evaluation scheme, the resulting quality and innovative ideas have a larger weight for encouraging a student to solve a problem and think creatively. Scoring was based on the assessments provided by the two professors, and upon completion, Cronbach’s coefficient was calculated to be 0.764, indicating a high level of consistency in evaluating students’ project completion.

3.5. Data Collection

Data collection was conducted using a project-based learning system developed by the research group. Additionally, in some cases where groups were affected by the outbreak of COVID-19, data were collected through separate submissions. The data collection process involved the following steps: Prior to starting each subtask, the team leader was responsible for task allocation and assignment of weekly phased tasks and specific subtasks through the project-based learning system. This ensured that each team member had assigned responsibilities corresponding to their respective tasks. Each group member is the task leader for his or her own subtask. The division of labor arrangements was recorded in the project-based learning system in textual form, and these records were accessible to all members of the group. This ensured transparency and allowed all group members to view and understand their assigned tasks and responsibilities. The evaluation process is shown in Figure 5. After completing their respective tasks, each group member is required to submit their work in the system. The submitted work can be viewed by all group members and is subject to peer evaluation, which involves assessing the content and providing feedback on the text. It is important to note that the students’ discussions and specific work take place offline, and this process is represented by a dotted line in the diagram.
The peer assessment data adheres to the following guidelines, which were communicated by the teaching assistants prior to commencing the PBL [1]. When evaluating the content of other group members’ work, the feedback text should cover three areas: strengths, weaknesses, and suggestions for improvement. Additionally, the mutual evaluation scores should fall within the range of 1 to 5. To emphasize the importance of meticulous completion of the project documentation, the students were informed at the beginning of the course that the quality of this work would contribute to their final grade.
There are forty-seven groups in total. Data submissions were carefully screened to remove groups with significant missing data. In the final analysis, a total of thirty-nine groups were included in this study because some groups had minimal participation, with only one or two students actively engaging in the project-based learning process and completing the inter-assessment.
The data sources for this study are presented in Figure 6 and encompass various sources of information pertaining to PBL. These sources include documentation of the work undertaken to fulfill the project requirements, such as experimental reports, work records, and other related materials. In addition, data were collected from the system itself, which involved capturing information related to group leadership, task schedules of the leaders, and peer inter-assessment. Furthermore, additional data were obtained through interviews, debriefing sessions, and examination of the project products generated during the course of the study.

4. Method

The data processing methodology employed in this study is depicted in Figure 7. Initially, the diverse PBL data were extracted utilizing natural language processing (NLP) techniques. Subsequently, the extracted data were transformed into six dimensions of features based on the principles of group awareness theory. Finally, correlation and regression analyses were conducted to fulfill the objectives of the study.

4.1. Text Mining

To intelligently identify group awareness data during PBL, we employed NLP techniques to automate the detection of collaborative process data. This process involved the following three steps. Firstly, we extract the textual content of each project-based group by employing word separation techniques to obtain text lexicality. Secondly, we construct a split-word corpus, which includes a collaborative corpus, an inquiry corpus, and a lexicon of the physical knowledge texts involved. Thirdly, we calculate the feature texts corresponding to each aspect of participation from the student collaboration data based on the principles of group awareness theory.
To assess the cognitive information within group awareness, we created a lexicon comprising the physical vocabulary necessary for the semester class and relevant to the project. Following that, we utilized NLP techniques to extract the project-related physical vocabulary from the student inter-assessment texts. This enabled us to calculate the group members’ cognitive awareness regarding the project work.
During the extraction of behavioral awareness data, we assigned codes to vocabulary within the categories of inquiry. Additionally, vocabulary unrelated to the completion of PBL was filtered out. The weighting assigned to these categories of vocabulary reflects the underlying classification of the type of work performed by the collaborative group in project completion.
For the social awareness data, we extracted them through the text of the group members’ mutual evaluations. This includes data on their interactions, the content of their evaluations, and collaborative texts.
The task assignment work of the group leader was also utilized as a data indicator. By analyzing the textual data related to the group leader’s task assignments, we employed NLP techniques to extract the leader’s cognitive awareness of individual tasks, awareness of individual subtasks, and awareness of the tasks in the weekly task schedule.

4.2. Data Transformation and Processing

After processing the data through text mining, we obtained the following data dimensions:
Text number (TN): In our study, we employed discussion texts as an overall factor to quantify the frequency of student interactions.
Cognitive awareness (CA): We employed the frequency of project-related physical words used by students in their interactive text to assess their cognitive awareness of the completed task. For example, “Taylor’s formula”, “shoot range formula”, and so on.
Emotional awareness (EA): We assessed the emotional dimension of the group work by analyzing the emotional tendencies expressed in the evaluations found in the students’ interactive texts. It includes some emotional words, including “excellent”, “good”, “Wow”, and so on.
Social awareness (SA): Collaboration words and interactions were extracted from the established collaborative corpus to assess and quantify collaborative interactions. For example, the words of SA include “organization”, “team”, and so on.
Behavior awareness (BA): Inquiry words were extracted from the established inquiry corpus to assess and quantify inquiry thinking. We record and extract students’ exploratory behavior related to the project in real time. For example, “discuss”, “work”, and so on.
Group leaders’ cognitive awareness on the project (GL): We employ the frequency of relevant physical word usage by the group leader and task leader to assess their perception of the task’s demands and objectives. That is, the group leader’s cognitive awareness of the project. For example, “water bottle pendulum”, “curve”, and so on.
The characterization of these data formed the foundation for the data used in our study.

4.3. Interview

In order to enhance the comprehensiveness of our study, we conducted random online chat-based interviews with students engaged in PBL. These interviews aimed to gather insights into their perceptions and opinions regarding the group leader’s contribution, collaborative task organization, and work dynamics during the collaboration process. It is important to note that these interviews were used solely as a supplementary approach to complement the quantitative data.

5. Results

5.1. Overview of PBL Data with NLP

The system employed NLP to analyze the task schedule of the group leader, along with the task completion, student self-assessment, and mutual assessment texts associated with each task node. Following data pre-processing, groups with a low number of collaborative recorded texts were excluded from the analysis. The final dataset included data from 39 groups, comprising 3060 comment text data and 239 leader working arrangement data. This encompassed a total of 33,713 group mutual Chinese characters and 14,367 group leaders’ Chinese characters. Descriptive statistics of the text data from these 39 groups are presented in Table 3.

5.2. Results of the Correlation Analysis of the PBL Data

Through the normality test, it was observed that the data exhibited a predominantly normal distribution in the SA and BA dimensions. However, for other variables, the normality assumption was not significant, as shown in Figure 8. Consequently, we opted to explore the relationship between the variables of interest using Spearman correlation analysis. Unlike the Pearson correlation coefficient, Spearman correlation analysis is suitable for examining the relationship between variables that do not follow a normal distribution.
Correlation analysis was conducted on the variables of interest, and the results are presented in Table 4.
Based on the correlation analysis, a significant positive correlation was observed between the effectiveness of groups and the cognitive awareness of the group leader on the project, as indicated by the GL (p = 0.667, sig < 0.01). The overall PBL-generated TN for interactions exhibited a significant positive correlation (p = 0.549, sig < 0.01) with the outcomes of PBL. Similarly, the extracted text data CA for group interactions showed a positive correlation with the outcomes of PBL (p = 0.558, sig < 0.01). On the other hand, the extracted emotional awareness (EA), behavioral awareness (BA), and social awareness (SA) did not demonstrate significant correlations.

5.3. Differentiation of PBL Dimensions

We conducted a t-test to compare the high and low-effectiveness groups, and the results revealed significant differences in most variables between these groups, as shown in Table 5.

5.4. Multiple Regression Results of Quantitative Analysis

A regression analysis was performed with the quality of collaboration as the dependent variable, revealing that the cognitive awareness represented by CA exhibited a variance inflation factor (VIF) value exceeding 10 when considering the total word count, TN, as an independent variable. This suggests a potential issue of multicollinearity [65,66]. As a result, the TN dimension was removed from the analysis.
During the investigation of factors influencing collaboration quality, we obtained regression results, which are presented in Table 6.
Based on the multiple regression analysis, it was observed that the VIF values for all variables were below 5, indicating their acceptability in the model. Upon comparison, it was found that the two dimensions, namely GL and CA, had the most substantial impact on the regression model, resulting in the biggest change in the R-square value. Consequently, CA and GL were selected for the regression analysis.
The results of the regression model are presented in Table 7. The adjusted R-squared value, which is close to 0.5, suggests that the model can account for approximately 50% of the variance in the outcomes. Notably, the factor of group leader demonstrates a substantial positive effect (B = 0.62) on the effectiveness of project-based learning, indicating its significant contribution. Following this, the frequency of physical vocabulary occurring in group interactions also exhibits a positive effect (B = 0.24) on the outcomes.

6. Discussion

This study aimed to design a PBL activity for science and technology within a college physics course. The activity involved addressing 18 specific questions that were directly relevant to physics knowledge in a real-life context. Throughout the implementation of the PBL activity, peer assessment data were collected from the participating groups using a project-based learning system. In addition, this study presents an evaluation framework for analyzing and assessing the effectiveness of collaboration in project-based learning. The framework extends previous research by integrating the evaluation system of PBL with the content of CSCL [21]. Furthermore, the framework incorporates text mining to enable automated evaluation. It focuses on evaluating the quality of collaboration through two dimensions: the group leader level and the collaboration level. This study offers empirical evidence by examining the group leader’s and the task leader’s task allocation in the PBL process and the peer assessment generated by group members during project completion (as presented in Table 3). Through this analysis, the study aims to uncover the interconnected factors and relationships that influence the quality of collaboration between the group leader and group members in PBL.
The multiple regression analysis results (refer to Table 7) indicate an R-squared value of 0.491, suggesting that the dimensions derived from the data using NLP techniques can account for approximately 50% of the variance in the quality of collaboration within PBL. This finding implies that the leader’s awareness of the project and the effectiveness of collaboration among group members play significant roles in determining the effectiveness of PBL.

6.1. Leaders’ Cognitive Awareness of Project

Based on the correlation analysis presented in Table 4, we observed a positive relationship between the group leader’s awareness of the work situation and the group’s performance in PBL. These findings suggest that a vertical approach to collaborative leadership, where the group leader takes charge and provides guidance, has a positive impact on collaborative learning outcomes [67]. Research has also provided evidence supporting the significance of a leader’s awareness of their own capabilities [68,69]. The effective performance of a leader is an essential aspect that reflects their leadership capabilities. To effectively address PBL tasks, the team must divide the problem into individual projects to facilitate a division of labor. This division of labor encompasses various aspects, including organizing discussions, coordinating task assignments, calculating workloads, ensuring fairness, monitoring individual task progress, and making necessary adjustments to the PBL schedule and processes. The leadership of the group leader plays a crucial role in guiding and facilitating these processes. These processes involve the leadership style of the group leader.
The interviews also revealed that some group leaders had achieved high academic grades (A or above) but were not meticulous about the division of tasks in the project-based learning, i.e., they carried out the project work through a simple and rough work schedule. For example, in the collaborative group of Steepest Descent Curve, there was a group where the group leader received an A, but the tasks were arranged in teams of two to complete the experiments and the report materials so that the group did not have a holistic view of the project content, which made the group’s performance in PBL unsatisfactory. On the contrary, there was a group where although the individual academic grades of the team leader and team members did not reach the highest (below B), the team leader fully understood the project objectives and gave a detailed description of each person’s task nodes, so that each member of the group could clearly understand the work content and work progress of the other members, which was an important contribution to the overall project advancement, as the group leader could provide better team orientation and ensure that the group activities were carried out as planned [70]. The results of this study revealed a significant positive correlation between the group leader’s cognitive awareness of the project (GL) and the effectiveness of PBL (p = 0.667, sig < 0.01). This finding suggests that when the group leader possesses a strong understanding of the PBL task, it positively contributes to the overall completion of the group’s project. In our research, the model was able to reflect and support this relationship.

6.2. Group Members’ Cognitive Awareness

Based on our data analysis, we found that the group’s cognitive awareness data (CA) had a significant positive effect on PBL (p = 0.558, sig < 0.01). This finding was consistent with the results obtained from the regression analysis (refer to Table 7) as well as the insights gained from the interviews. Both sources indicated that cognitive awareness within group interactions, specifically the frequency of content related to the PBL topic appearing in the text, contributed to the overall effectiveness of PBL. When the interactive text topics within the group primarily focused on the ongoing PBL tasks, there was a decrease in extraneous information, such as off-topic discussions [71]. This heightened focus enabled the group’s thinking to be highly concentrated, providing favorable conditions for generating higher-order thinking and enhancing collaboration effectiveness. Furthermore, when the collaborative group’s awareness of the PBL task was slightly more advanced in cognitive development than their own, it facilitated their active exploration of the question and construction of knowledge [6,72].
In the context of group interaction, we found that strong intergroup collaboration would significantly contribute to the overall effectiveness of PBL. These findings are consistent with numerous scholarly studies that have also emphasized the importance of effective collaboration in achieving positive outcomes in PBL [55]. When collaborative interactions among group members establish a virtuous cycle, it can create conditions that foster higher-order cognition, thereby enhancing the overall effectiveness of the group. In our PBL process, highly effective collaborative groups tend to place greater emphasis on the collaborative level during peer assessments of each task point. This emphasis involves prioritizing interworking, enhancing communication, and engaging in in-depth discussions. Such practices provide better support and facilitation throughout the nearly four-month duration of PBL implementation. They also align with student-centered teaching philosophy, allowing for increased opportunities for collaboration and learning among students.
The dimensions of behavioral awareness (BA), social awareness (SA), and emotional awareness (EA) did not show significant correlations. This lack of correlation could be attributed to the disparity between the hands-on practice and the verbal recording and expression abilities of students. It implies that students might not have been able to articulate and present their project work in verbal form accurately [72]. This observation highlights a potential mismatch between understanding and representation within the PBL process.
The findings from the interviews also revealed that group members were positively influenced when they observed the completed work of their peers during the task. This observation served as a motivational signal, encouraging them to overcome procrastination and continue working on their individual PBL tasks. When students are able to witness the progress and achievements of the group through recording their collaborative work [73], it instills a sense of motivation and enhances their commitment to the project. Additionally, when peer evaluations are conducted, it fosters a deeper understanding of each other’s work and promotes the acquisition of disciplinary knowledge and competence. In summary, our empirical results demonstrate that PBL, facilitated by task scheduling, recording, and peer evaluation, has a positive impact on students’ completion of project work. Given this finding, we conclude that the quality of group collaboration can be assessed by focusing on two dimensions: the leader’s cognitive awareness and the effectiveness of group collaboration. These dimensions provide valuable insights into the evaluation of the PBL process, as they capture essential aspects of group dynamics and outcomes.
In light of these findings, it is crucial for teachers to consider not only academic performance but also the sense of responsibility when selecting leaders for collaborative learning. Responsible leaders play a pivotal role in ensuring that group members obtain a comprehensive understanding of the project, thereby motivating them to excel and enhancing the quality of their collaboration.
Furthermore, during the monitoring of group interactions, teachers should intervene when they observe a lack of project-relevant content in the communication between group members. This is because interactions that do not revolve around the task hinder effective collaboration. The quality of collaboration experiences a significant improvement only when the interactions primarily focus on the project’s objectives and requirements.
However, there are several limitations to this study. Firstly, the inquiry factors, which represent the latter 50% of the factors in the regression analysis, were not measured using alternative methods. Secondly, due to the offline nature of the project, the students’ offline project work hours were not fully recorded in real-time through the system. Consequently, system data such as login time, number of clicks, and discussion time were not utilized for relevant studies. Finally, the primary focus has been on analyzing the collaborative group as an integrated entity, overlooking the potential impact of specific group member compositions on group awareness so that we did not pay attention to how gender differences among group members could influence group awareness and information processing.
In the future, consideration could be given to building a knowledge graph based on student collaboration to enable better evaluation. Meanwhile, research on whether there are different effects on group awareness information in collaborative groups composed of different genders is one of the directions of our subsequent research.

7. Conclusions

This study presents an empirical investigation that employs natural language processing techniques to extract features. Specifically, we conducted data collection and analysis within the context of a college physics course that utilized text mining. The results reveal that collaboration quality in PBL is influenced by two key factors: the leader’s cognitive awareness and the group members’ cognitive awareness. Notably, both dimensions exhibit a positive impact on the overall quality of collaboration. Our study highlights the significant contribution of the group leaders’ cognitive awareness and sense of responsibility to the success of PBL. Furthermore, we observe a positive effect of intergroup collaboration on PBL outcomes. It is worth noting that our methodology allows for the measurement of approximately 50% of the collaborative effects, which represents a hybrid approach when compared to traditional methods employed in the field of CSCL for assessing group awareness.
Overall, we assess the quality of collaboration in PBL by text mining. This methodology proves particularly valuable in the current context of increased collaboration, especially in response to the challenges posed by the COVID-19 pandemic. By leveraging NLP techniques, we are able to analyze substantial volumes of textual data, effectively reducing the burden of manual assessment and offering valuable support to educators. In short, it contributes to the ongoing investigation of collaboration quality within the realm of higher education.

Author Contributions

R.Z., conceptualization, designed the research programs and methodology, conducted data analysis, and reviewed and edited the manuscript. J.S., software, visualization, and original draft preparation. J.Z., project administration, funding acquisition, supervision. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China (Grant No. 12174287).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Zhang, D.; Hwang, G.-J. Effects of Interaction between Peer Assessment and Problem-Solving Tendencies on Students’ Learning Achievements and Collaboration in Mobile Technology-Supported Project-Based Learning. J. Educ. Comput. Res. 2022, 61, 208–234. [Google Scholar] [CrossRef]
  2. Konrad, T.; Wiek, A.; Barth, M. Learning to Collaborate from Diverse Interactions in Project-Based Sustainability Courses. Sustainability 2021, 13, 9884. [Google Scholar] [CrossRef]
  3. Setyowidodo, I.; Jatmiko, B.; Susantini, E.; Handayani, A.D.; Pramesti, Y.S. The role of science project-based peer interaction on improving collaborative skills and physical problem solving: A mini review. J. Phys. Conf. Ser. 2020, 1521, 022032. [Google Scholar] [CrossRef]
  4. Zen, Z.; Reflianto; Syamsuar; Ariani, F. Academic achievement: The effect of project-based online learning method and student engagement. Heliyon 2022, 8, e11509. [Google Scholar] [CrossRef] [PubMed]
  5. Bell, S. Project-based learning for the 21st century: Skills for the future. Clear. House J. Educ. Strateg. Issues Ideas 2010, 83, 39–43. [Google Scholar] [CrossRef]
  6. O’Dea, X.; O’Dea, M. Is Artificial Intelligence Really the Next Big Thing in Learning and Teaching in Higher Education? A Conceptual Paper. J. Univ. Teach. Learn. Pract. 2023, 20, 5. [Google Scholar] [CrossRef]
  7. EDUCAUSE. EDUCAUSE Horizon Report (Teaching and Learning Edition). 2023. Available online: https://library.educause.edu/resources/2023/5/2023-educause-horizon-report-teaching-and-learning-edition (accessed on 1 July 2023).
  8. EDUCAUSE. EDUCAUSE Horizon Report (Teaching and Learning Edition). 2022. Available online: https://library.educause.edu/-/media/files/library/2022/4/2022hrteachinglearning.pdf (accessed on 1 July 2023).
  9. Kokotsaki, D.; Menzies, V.; Wiggins, A. Project-based learning: A review of the literature. Improv. Sch. 2016, 19, 267–277. [Google Scholar] [CrossRef] [Green Version]
  10. Davidson, N.; Major, C.H. Boundary crossings: Cooperative learning, collaborative learning, and problem-based learning. J. Excell. Coll. Teach. 2014, 25, 7–55. [Google Scholar]
  11. Mora, H.; Signes-Pont, M.T.; Fuster-Guilló, A.; Pertegal-Felices, M.L. A collaborative working model for enhancing the learning process of science & engineering students. Comput. Hum. Behav. 2020, 103, 140–150. [Google Scholar]
  12. Dym, C.L.; Agogino, A.M.; Eris, O.; Frey, D.D.; Leifer, L.J. Engineering design thinking, teaching, and learning. J. Eng. Educ. 2005, 94, 103–120. [Google Scholar] [CrossRef]
  13. Chen, J.; Kolmos, A.; Du, X. Forms of implementation and challenges of PBL in engineering education: A review of literature. Eur. J. Eng. Educ. 2021, 46, 90–115. [Google Scholar] [CrossRef]
  14. Raine, D.; Symons, S. Problem-based learning: Undergraduate physics by research. Contemp. Phys. 2012, 53, 39–51. [Google Scholar] [CrossRef]
  15. Hidaayatullaah, H.N.; Dwikoranto; Suprapto, N.; Mubarok, H.; Wulandari, D. Implementation of problem-based learning to train physics students’ problem-solving skills. J. Phys. Conf. Ser. 2020, 1491, 012053. [Google Scholar] [CrossRef]
  16. Goyal, M.; Gupta, C.; Gupta, V. A meta-analysis approach to measure the impact of project-based learning outcome with program attainment on student learning using fuzzy inference systems. Heliyon 2022, 8, e10248. [Google Scholar] [CrossRef]
  17. Guo, P.; Saab, N.; Post, L.S.; Admiraal, W. A review of project-based learning in higher education: Student outcomes and measures. Int. J. Educ. Res. 2020, 102, 101586. [Google Scholar] [CrossRef]
  18. Vogler, J.S.; Thompson, P.; Davis, D.W.; Mayfield, B.E.; Finley, P.M.; Yasseri, D. The hard work of soft skills: Augmenting the project-based learning experience with interdisciplinary teamwork. Instr. Sci. 2018, 46, 457–488. [Google Scholar] [CrossRef]
  19. Lin, J.W. Effects of an online team project-based learning environment with group awareness and peer evaluation on socially shared regulation of learning and self-regulated learning. Behav. Inf. Technol. 2018, 37, 445–461. [Google Scholar] [CrossRef]
  20. Splichal, J.M.; Oshima, J.; Oshima, R. Regulation of collaboration in project-based learning mediated by CSCL scripting reflection. Comput. Educ. 2018, 125, 132–145. [Google Scholar] [CrossRef]
  21. Alozie, N.M.; Dhamija, S. Automated collaboration assessment using behavioral analytics. Int. Soc. Learn. Sci. 2020, 2, 1071–1078. [Google Scholar]
  22. Boughzala, I.; De Vreede, G.-J. Evaluating team collaboration quality: The development and field application of a collaboration maturity model. J. Manag. Inf. Syst. 2015, 32, 129–157. [Google Scholar] [CrossRef]
  23. Jiang, C.; Lou, Y. Integrated Strategies of Participatory Design and PBL Towards Collaboration Quality. In Cross-Cultural Design. Applications in Cultural Heritage, Creativity and Social Development, Proceedings of the 10th International Conference, CCD 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, 15–20 July 2018; Springer: Berlin/Heidelberg, Germany, 2018; pp. 46–59. [Google Scholar]
  24. Lai, C.L. Effects of the group-regulation promotion approach on students’ individual and collaborative learning performance, perceptions of regulation and regulation behaviours in project-based tasks. Br. J. Educ. Technol. 2021, 52, 2278–2298. [Google Scholar] [CrossRef]
  25. Parsell, G.; Bligh, J. The development of a questionnaire to assess the readiness of health care students for interprofessional learning (RIPLS). Med. Educ. 1999, 33, 95–100. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Sa’diyah, K.; Muchyidin, A.; Izzati, N. Application of Collaborative Teamwork Learning Model and Guided Note Taking Model and Their Influence on Students’ Ability to Understand Mathematical Concepts. J. Math. Instr. Soc. Res. Opin. 2022, 1, 14–26. [Google Scholar] [CrossRef]
  27. Xia, H.; Li, Y.; Chen, F.; Xu, B. The Influence of Participant Subject Factors on Collaboration Effects in the Protection of China’s ICH: The Mediating Role of Relationship Quality. Sustainability 2022, 14, 1223. [Google Scholar] [CrossRef]
  28. Chejara, P.; Prieto, L.P.; Rodriguez-Triana, M.J.; Kasepalu, R.; Ruiz-Calleja, A.; Shankar, S.K. How to build more generalizable models for collaboration quality? lessons learned from exploring multi-context audio-log datasets using multimodal learning analytics. In Proceedings of the LAK23: 13th International Learning Analytics and Knowledge Conference, Arlington, TX, USA, 13–17 March 2023; pp. 111–121. [Google Scholar]
  29. Schneider, B.; Pea, R. Real-time mutual gaze perception enhances collaborative learning and collaboration quality. Int. J. Comput. Support. Collab. Learn. 2013, 8, 375–397. [Google Scholar] [CrossRef]
  30. Som, A.; Kim, S.; Lopez-Prado, B.; Dhamija, S.; Alozie, N.; Tamrakar, A. A machine learning approach to assess student group collaboration using individual level behavioral cues. In Proceedings of the Computer Vision–ECCV 2020 Workshops, Glasgow, UK, 23–28 August 2020; pp. 79–94. [Google Scholar]
  31. Chounta, I.A.; Avouris, N. Towards the real-time evaluation of collaborative activities: Integration of an automatic rater of collaboration quality in the classroom from the teacher’s perspective. Educ. Inf. Technol. 2016, 21, 815–835. [Google Scholar] [CrossRef]
  32. Dascalu, M.; Rebedea, T.; Trausan-Matu, S. A deep insight in chat analysis: Collaboration, evolution and evaluation, summarization and search. In Artificial Intelligence: Methodology, Systems, and Applications, Proceedings of the 14th International Conference, AIMSA 2010, Varna, Bulgaria, 8–10 September 2010; Springer: Berlin/Heidelberg, Germany, 2010; pp. 191–200. [Google Scholar]
  33. Yang, J.; Kinshuk; An, Y. A survey of the literature: How scholars use text mining in Educational Studies? Educ. Inf. Technol. 2023, 28, 2071–2090. [Google Scholar] [CrossRef]
  34. Song, D.; Lin, H.; Yang, Z. Opinion mining in e-learning system. In Proceedings of the 2007 IFIP International Conference on Network and Parallel Computing Workshops (NPC 2007), Dalian, China, 18–21 September 2007; pp. 788–792. [Google Scholar] [CrossRef]
  35. Rosé, C.; Wang, Y.-C.; Cui, Y.; Arguello, J.; Stegmann, K.; Weinberger, A.; Fischer, F. Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. Int. J. Comput. Support. Collab. Learn. 2008, 3, 237–271. [Google Scholar] [CrossRef] [Green Version]
  36. Schnaubert, L.; Bodemer, D. Group awareness and regulation in computer-supported collaborative learning. Int. J. Comput. Support. Collab. Learn. 2022, 17, 11–38. [Google Scholar] [CrossRef]
  37. Buder, J.; Bodemer, D.; Ogata, H. Group awareness. In International Handbook of Computer-Supported Collaborative Learning; Cress, U., Rose, C., Wise, A.F., Oshima, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2021; pp. 295–313. [Google Scholar] [CrossRef]
  38. Rojas, M.; Nussbaum, M.; Guerrero, O.; Chiuminatto, P.; Greiff, S.; Del Rio, R.; Alvares, D. Integrating a collaboration script and group awareness to support group regulation and emotions towards collaborative problem solving. Int. J. Comput. Support. Collab. Learn. 2022, 17, 135–168. [Google Scholar] [CrossRef]
  39. Janssen, J.; Bodemer, D. Coordinated computer-supported collaborative learning: Awareness and awareness tools. Educ. Psychol. 2013, 48, 40–55. [Google Scholar] [CrossRef]
  40. Bodemer, D.; Dehler, J. Group awareness in CSCL environments. Comput. Hum. Behav. 2011, 27, 1043–1045. [Google Scholar] [CrossRef]
  41. Li, Y.; Li, X.; Zhang, Y.; Li, X. The effects of a group awareness tool on knowledge construction in computer-supported collaborative learning. Br. J. Educ. Technol. 2021, 52, 1178–1196. [Google Scholar] [CrossRef]
  42. Su, Y.; Ren, J.; Song, X. The effects of group awareness tools on student engagement with peer feedback in online collaborative writing environments. Interact. Learn. Environ. 2022, 1–17. [Google Scholar] [CrossRef]
  43. Lin, J.-W.; Tsai, C.-W. The impact of an online project-based learning environment with group awareness support on students with different self-regulation levels: An extended-period experiment. Comput. Educ. 2016, 99, 28–38. [Google Scholar] [CrossRef]
  44. Strauß, S.; Rummel, N. Promoting regulation of equal participation in online collaboration by combining a group awareness tool and adaptive prompts. But does it even matter? Int. J. Comput. Support. Collab. Learn. 2021, 16, 67–104. [Google Scholar] [CrossRef]
  45. Xing, W.; Zhu, G.; Arslan, O.; Shim, J.; Popov, V. Using learning analytics to explore the multifaceted engagement in collaborative learning. J. Comput. High. Educ. 2022, 1–30. [Google Scholar] [CrossRef]
  46. Schnaubert, L.; Bodemer, D. Providing different types of group awareness information to guide collaborative learning. Int. J. Comput. Support. Collab. Learn. 2019, 14, 7–51. [Google Scholar] [CrossRef]
  47. Bodemer, D. Tacit guidance for collaborative multimedia learning. Comput. Hum. Behav. 2011, 27, 1079–1086. [Google Scholar] [CrossRef]
  48. Dehler, J.; Bodemer, D.; Buder, J.; Hesse, F.W. Guiding knowledge communication in CSCL via group knowledge awareness. Comput. Hum. Behav. 2011, 27, 1068–1078. [Google Scholar] [CrossRef]
  49. Gijlers, H.; de Jong, T. Sharing and confronting propositions in collaborative inquiry learning. Cogn. Instr. 2009, 27, 239–268. [Google Scholar] [CrossRef]
  50. Buder, J.; Bodemer, D. Supporting controversial CSCL discussions with augmented group awareness tools. Int. J. Comput. Support. Collab. Learn. 2008, 3, 123–139. [Google Scholar] [CrossRef]
  51. Sangin, M.; Molinari, G.; Nüssli, M.-A.; Dillenbourg, P. Facilitating peer knowledge modeling: Effects of a knowledge awareness tool on collaborative learning outcomes and processes. Comput. Hum. Behav. 2011, 27, 1059–1067. [Google Scholar] [CrossRef]
  52. Yilmaz, F.G.K.; Yilmaz, R. Impact of pedagogic agent-mediated metacognitive support towards increasing task and group awareness in CSCL. Comput. Educ. 2019, 134, 1–14. [Google Scholar] [CrossRef]
  53. Bødker, S.; Christiansen, E. Computer support for social awareness in flexible work. Comput. Support. Coop. Work CSCW 2006, 15, 1–28. [Google Scholar] [CrossRef]
  54. Foo, S.Y. Analysing peer feedback in asynchronous online discussions: A case study. Educ. Inf. Technol. 2021, 26, 4553–4572. [Google Scholar] [CrossRef]
  55. Peng, Y.; Li, Y.; Su, Y.; Chen, K.; Jiang, S. Effects of group awareness tools on students’ engagement, performance, and perceptions in online collaborative writing: Intergroup information matters. Internet High. Educ. 2022, 53, 100845. [Google Scholar] [CrossRef]
  56. Kimmerle, J.; Cress, U. Visualization of group members’ participation: How information-presentation formats support information exchange. Soc. Sci. Comput. Rev. 2009, 27, 243–261. [Google Scholar] [CrossRef]
  57. Kimmerle, J.; Cress, U. Group awareness and self-presentation in computer-supported information exchange. Int. J. Comput. Support. Collab. Learn. 2008, 3, 85–97. [Google Scholar] [CrossRef]
  58. Phielix, C.; Prins, F.J.; Kirschner, P.A.; Erkens, G.; Jaspers, J. Group awareness of social and cognitive performance in a CSCL environment: Effects of a peer feedback and reflection tool. Comput. Hum. Behav. 2011, 27, 1087–1102. [Google Scholar]
  59. Ma, X.; Liu, J.; Liang, J.; Fan, C. An empirical study on the effect of group awareness in CSCL environments. Interact. Learn. Environ. 2020, 31, 38–53. [Google Scholar] [CrossRef]
  60. Chen, D.; Zhang, Y.; Lin, Y. Group Awareness and Group Awareness Tools in Computer-Supported Collaborative Learning: A Literature Review. In Proceedings of the 2022 International Symposium on Educational Technology (ISET), Hong Kong, 19–22 July 2022; pp. 18–22. [Google Scholar] [CrossRef]
  61. Pifarré, M.; Cobos, R.; Argelagós, E. Incidence of group awareness information on students’ collaborative learning processes. J. Comput. Assist. Learn. 2014, 30, 300–317. [Google Scholar] [CrossRef] [Green Version]
  62. Bodemer, D.; Janssen, J.; Schnaubert, L. Group awareness tools for computer-supported collaborative learning. In International Handbook of the Learning Sciences; Routledge: London, UK, 2018; pp. 351–358. [Google Scholar]
  63. Schnaubert, L.; Vogel, F. Integrating collaboration scripts, group awareness, and self-regulation in computer-supported collaborative learning. Int. J. Comput. Support. Collab. Learn. 2022, 17, 1–10. [Google Scholar] [CrossRef]
  64. Rouvrais, S.; Ormrod, J.; Landrac, G.; Mallet, J.; Gilliot, J.-M.; Thépaut, A.; Tremenbert, P. A mixed project-based learning framework: Preparing and developing student competencies in a French Grande Ecole. Eur. J. Eng. Educ. 2006, 31, 83–93. [Google Scholar] [CrossRef]
  65. Paul, R.K. Multicollinearity: Causes, effects and remedies. IASRI New Delhi 2006, 1, 58–65. [Google Scholar]
  66. Daoud, J.I. Multicollinearity and regression analysis. J. Phys. Conf. Ser. 2017, 949, 012009. [Google Scholar] [CrossRef]
  67. Nicolaides, V.C.; LaPort, K.A.; Chen, T.R.; Tomassetti, A.J.; Weis, E.J.; Zaccaro, S.J.; Cortina, J.M. The shared leadership of teams: A meta-analysis of proximal, distal, and moderating relationships. Leadersh. Q. 2014, 25, 923–942. [Google Scholar] [CrossRef]
  68. Frank, M.; Lavy, I.; Elata, D. Implementing the project-based learning approach in an academic engineering course. Int. J. Technol. Des. Educ. 2003, 13, 273–288. [Google Scholar] [CrossRef]
  69. Chen, L.P.; Gregory, J.K.; Camp, C.L.; Juskewitch, J.E.; Pawlina, W.; Lachman, N. Learning to lead: Self-and peer evaluation of team leaders in the human structure didactic block. Anat. Sci. Educ. 2009, 2, 210–217. [Google Scholar] [CrossRef]
  70. Yilmaz, R.; Karaoglan Yilmaz, F.G.; Keser, H. Vertical versus shared e-leadership approach in online project-based learning: A comparison of self-regulated learning skills, motivation and group collaboration processes. J. Comput. High. Educ. 2020, 32, 628–654. [Google Scholar] [CrossRef]
  71. Hou, H.T.; Wang, S.M.; Lin, P.C.; Chang, K.E. Exploring the learner’s knowledge construction and cognitive patterns of different asynchronous platforms: Comparison of an online discussion forum and Facebook. Innov. Educ. Teach. Int. 2015, 52, 610–620. [Google Scholar] [CrossRef]
  72. Eun, B. The zone of proximal development as an overarching concept: A framework for synthesizing Vygotsky’s theories. Educ. Philos. Theory 2019, 51, 18–30. [Google Scholar] [CrossRef]
  73. Carrasco, G.A.; Behling, K.C.; Lopez, O. Weekly team-based learning scores and participation are better predictors of successful course performance than case-based learning performance: Role of assessment incentive structure. BMC Med. Educ. 2021, 21, 521. [Google Scholar] [CrossRef] [PubMed]
Sustainability 15 11901 g001
Figure 1. Phase tasks of PBL system.
Figure 1. Phase tasks of PBL system.
Sustainability 15 11901 g001
Sustainability 15 11901 g002
Figure 2. Task point visualization in system.
Figure 2. Task point visualization in system.
Sustainability 15 11901 g002
Sustainability 15 11901 g003
Figure 3. Images of undergoing physical project.
Figure 3. Images of undergoing physical project.
Sustainability 15 11901 g003
Sustainability 15 11901 g004
Figure 4. The overall PBL process.
Figure 4. The overall PBL process.
Sustainability 15 11901 g004
Sustainability 15 11901 g005
Figure 5. The evaluation process for each node of the task leader after the group leader has been assigned.
Figure 5. The evaluation process for each node of the task leader after the group leader has been assigned.
Sustainability 15 11901 g005
Sustainability 15 11901 g006
Figure 6. PBL data source.
Figure 6. PBL data source.
Sustainability 15 11901 g006
Sustainability 15 11901 g007
Figure 7. Research method in PBL.
Figure 7. Research method in PBL.
Sustainability 15 11901 g007
Sustainability 15 11901 g008
Figure 8. Normality test of PBL data.
Figure 8. Normality test of PBL data.
Sustainability 15 11901 g008
Table 1. PBL topics and target content.
Table 1. PBL topics and target content.
TopicBrief Description of Content
Investigation of oblique throwing motionExamines the height of the throwing point and the effect of wind speed on the range of the throw.
Measurement of gravitational accelerationMeasuring the acceleration of gravity through a sensing device on a mobile phone, combined with a single pendulum experiment.
Bouncing of a small ballExamining the process of inelastic collision of a small ball with the ground and measuring the coefficient of recovery.
Paper springsExamining the relationship between the distance a paper spring stretches under its own gravity and the parameters of the spring.
Water rocketsExploring ways to make water rockets go the furthest.
Steepest descent curveStudy of a small ball descending along the steepest descending curve.
Rolling of a ball on an inclined planeStudy of a ball rolling on an inclined plane with different coefficients of friction and inclination angles.
Transfer orbit of a spaceshipStudy of the transfer process of a spacecraft between two orbits.
Study of the twin phenomenonExamining the causes of the baryogenesis phenomenon in the framework of special relativity.
DominoesThe speed of dominoes as they fall to the ground.
Design of stone-throwing machineDesigning a stone thrower with a long range.
Design of a paper bridgeDesigning a paper bridge that spans between two stacks of books, requiring it to be able to carry as much weight as possible.
Exploring the tennis racket effectInvestigate why the tennis racket effect (Janibekov effect) occurs.
Velocity of sound measurementDesigning an experiment to measure the speed of sound.
Water wave velocity investigationInvestigate the factors that affect the wave speed of water waves.
Acoustic investigation of the guitarInvestigating how the guitar occurs.
Diffusion of ink in waterStudying the rate of diffusion of ink in water.
Drinking water birdsMaking a drinking bird.
Table 2. PBL evaluation scheme.
Table 2. PBL evaluation scheme.
Evaluation DimensionsEvaluation ScalesEvaluation Details and Scores
First Class
(90–100)		Second Class
(80–90)		Third Class
(70–80)		Fourth Class
(60–70)
Reporting materials
(60%)		Whether completed
(10%)		Successful experiments.Experiment largely successful.Experiment fails after attempt.Experiment failed.
Quality
(70%)		Experiment achieves true value and is within a 5% error.The experiment was close to the true value.Experiment barely reaches true value.Experiment does not match the true value.
Inter-group ranking
(10%)		The best quality completed among the groups selected for the same topic.The next highest quality completed in the group that chose the same topic.Third highest quality completed in the group that chose the same topic.Worst quality of work in the group that chose the same topic.
Experimental reports
(40%)		Whether the study process is standardized
(10%)		Experiment report reflects background, methods, experimental procedure, results, discussion, etc.Part of the experiment report is missing or written concisely.Two parts of the experiment report were missing or written concisely.Experiment report has a lot of missing parts and is concise.
Workload meets PBL requirements
(20%)		The workload is high.The amount of work was appropriate.Average amount of work.The workload was low.
Whether innovative experimental protocols are proposed
(40%)		Creative thinking about the project and commitment to practice.There was innovative thinking about the project.Little innovation shown for the project.The project was not innovative.
Whether multiple protocols are proposed
(40%)		The same project can be solved by multiple ideas and there is a comparison of options.There is a comparison of different options.The project is solved by one idea, and there are no options to compare.The project was not solved, and there were no options to compare.
Table 3. Descriptive statistics of PBL data texts in collaborative group, including group mutual Chinese characters and group leaders’ Chinese characters.
Table 3. Descriptive statistics of PBL data texts in collaborative group, including group mutual Chinese characters and group leaders’ Chinese characters.
NMinMaxMeanSD
Comment texts391414487864.44798.911
Leader texts3969816368.38230.692
Table 4. Results of correlation analysis of PBL grade and student awareness.
Table 4. Results of correlation analysis of PBL grade and student awareness.
PBL GradeTNCAEASABAGL
PBL Grade1
TN0.549 **1
CA0.558 **0.819 **1
EA0.017−0.085−0.0911
SA0.1230.0550.02−0.2521
BA−0.0940.028−0.033−0.1680.0021
GL0.667 **0.336 *0.399 *0.0150.0420.1071
Note. * p < 0.05, ** p < 0.01.
Table 5. Results of t-test of the high and low-effectiveness groups.
Table 5. Results of t-test of the high and low-effectiveness groups.
DataSig.
TN0.042 *
CA0.04 *
EA0.113
SA0.031 *
BA0.136
GL0.005 **
Note. * p < 0.05, ** p < 0.01.
Table 6. Comparison of regression results.
Table 6. Comparison of regression results.
R SquareVariables Incorporated
0.435GL
0.117CA
0.006EA
0.006BA
0.000SA
Table 7. Results of multiple regression analysis.
Table 7. Results of multiple regression analysis.
VariableBtSig.
GL0.6205.1440.000
CA0.2401.9870.055
R square = 0.491, Adjusted R square = 0.463.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhang, R.; Shi, J.; Zhang, J. Research on the Quality of Collaboration in Project-Based Learning Based on Group Awareness. Sustainability 2023, 15, 11901. https://doi.org/10.3390/su151511901

AMA Style

Zhang R, Shi J, Zhang J. Research on the Quality of Collaboration in Project-Based Learning Based on Group Awareness. Sustainability. 2023; 15(15):11901. https://doi.org/10.3390/su151511901

Chicago/Turabian Style

Zhang, Rui, Ji Shi, and Jianwei Zhang. 2023. "Research on the Quality of Collaboration in Project-Based Learning Based on Group Awareness" Sustainability 15, no. 15: 11901. https://doi.org/10.3390/su151511901

APA Style

Zhang, R., Shi, J., & Zhang, J. (2023). Research on the Quality of Collaboration in Project-Based Learning Based on Group Awareness. Sustainability, 15(15), 11901. https://doi.org/10.3390/su151511901

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop