Opening the ‘Black Box’ of Cooperative Learning in Face-to-Face versus Computer-Supported Learning in the Time of COVID-19

Abstract

This paper aims to identify the changes in student behaviors that resulted from the switch from face-to-face (F2F) learning to computer-supported cooperative learning (CSCL) due to the coronavirus (COVID-19) pandemic. We constructed a triple-dimensional index with “thinking ability improvement”, “horizontal knowledge construction”, and “vertical social relationship evolution” to make comparisons. According to majors, we selected 23 students who registered for entrepreneurship courses from March to June 2019 in F2F and 23 students from March to June 2020 in CSCL formats. We utilized mixed methods, including experimental, content-based, and social network methods, to conduct evaluations. The results show the following: (1) Cooperative learning is beneficial in cultivating creative thinking for both F2F and CSCL groups. (2) The level of knowledge construction was slightly higher in F2F than that in CSCL in general. The effect of F2F learning in the early stage of the course was better, and in the later stage of the class CSCL attained a higher value. (3) For social abilities, the interactions in CSCL were closer than those in the F2F group. F2F cooperative learning was more prone to “fake cooperation” and free-riding behavior, whereas CSCL led to “pan-cooperation” and lacked the in-depth exploration of knowledge. Therefore, this pandemic provides opportunities for cooperative learning with in-depth exploration. CSCL offers sustainable and more hybrid learning activities that allow for the combination of online and offline learning to be experienced according to course contents.

1. Introduction

Cooperative learning is a teaching technique that organizes students into small groups for learning activities, with rewards or recognition being provided based on their overall group performance in achieving educational goals [1,2]. Students participate in cooperative interaction through consultation or engaging in a real learning situation, and this is specifically achieved by incorporating new knowledge into the collective knowledge structure and promoting the innovation and growth of collective knowledge [3]. Cooperative learning also introduces changes in classroom technology, such as computer-supported collaborative learning (CSCL). CSCL is an emerging paradigm of educational technology [4] that has been introduced into classrooms and has attracted the attention of many scholars [5,6,7,8]. It has been proposed that information communication technologies (ICT) can promote the interaction between teachers, so it can be used to promote cooperative learning, debate, and collaboration-based knowledge construction [9]. CSCL can improve the quality of student interaction by providing a virtual learning environment [10].

However, some educators are still skeptical that discipline can effectively be taught at an online level. Some scholars propose that the quality of online courses can be improved, including teaching and learning processes, which benefit from non-verbal cues presented in face-to-face (F2F) situations [11,12]. The lack of non-verbal cues and physical settings online will fail to convey important information on participants’ characteristics, emotions, and attitudes, which will cause communication barriers and invalid communication [13]. Through a meta-analysis, Kyndt et al. [14] proposed that F2F cooperative learning positively affects achievement and attitudes. CSCL is generally considered to be synonymous with asynchronous teaching. It exists separately from or supplements the face-to-face teaching in traditional physical classrooms [15].

The COVID-19 pandemic has forced the shutdown of face-to-face educational activities. It has resulted in tremendous online learning using educational web platforms as a crisis-response method. The extreme circumstances may heighten the awareness of the differences between CSCL and F2F cooperative learning. Thus, are there challenges or opportunities for CSCL during these unprecedented times? Are there differences in the contribution of F2F and CSCL to the performance of undergraduate education? Can students benefit from CSCL like F2F? Scholars have compared CSCL and F2F performance from the perspectives of fostering thinking [16], behavior categories [17], and behavior paths and patterns [18]. Additionally, cooperative learning strategies have various applications. In language disciplines, cooperative learning mainly encourages group members to exchange information and engage in cooperative thinking [19,20]. In the natural sciences, most of the team members work together to complete assignments [21]. Different learning stages should correspond to different cooperative learning strategies [22]. Hence, the contributions and effectiveness of F2F and CSCL to training goals need to be evaluated based on specific disciplines in order to obtain more reasonable results.

Entrepreneurship is mainly regarded as a way to promote economic development and employment growth. Many countries seek to develop entrepreneurial skills among young people and foster entrepreneurship education. Entrepreneurship education may be one of the few examples of integration and combination of the three aims to improve knowledge, skills, and attitudes [23] (p. 4). The main goals of most entrepreneurship education or training programs are as follows:

• To acquire, apply, and use knowledge in relation to entrepreneurship;
• To develop and apply skills in the use of techniques, in the analysis of business situations, and the synthesis of action plans;
• To develop empathy and support for all unique aspects of entrepreneurship;
• To develop attitudes towards change.

In sum, entrepreneurship education can help students to understand entrepreneurship, and, simultaneously, it can integrate knowledge into innovations [24,25]. It entails creative thinking [26], a positive attitude, and a sense of self-efficacy [27]. Most entrepreneurial activities depend on the entrepreneur’s capabilities. Still, they are embedded in the network structure [28], and social capital can help entrepreneurs obtain resources that they do not yet own [29]. The acquisition of innovative advantages, applying knowledge, and social ability require support from diversified comprehensive learning capabilities and effects. Thus, in entrepreneurship, cooperative learning research that integrates different actors is particularly important [30]. Most works have discussed entrepreneurship education in the STEM (Science, Technology, Engineering, Mathematics) field [31], teaching models [32], etc. These studies are mainly derived from individual empirical learning, but few investigations have focused on college students outside the seminars and psychology laboratory in entrepreneurship education.

Existing pedagogical approaches focus on face-to-face instruction in traditional entrepreneurship education classrooms. The global spread of COVID-19 poses a particular challenge and a potential opportunity for entrepreneurship education. “The integration of online learning in higher education over the last 20 years remains slow to gain widespread traction, especially in entrepreneurship education” [33] (p. 347). The effectiveness of virtual, online entrepreneurship education is relatively unknown. Therefore, in this particular time, from the perspective of “teaching,” treating cooperative learning as a “black box” and comparing the differences in student behaviors between these two types of cooperative learning styles in the context of entrepreneurship courses are significant avenues of inquiry. Evaluations can proceed from three dimensions for entrepreneurial course purposes: “thinking ability improvement,” “horizontal knowledge construction,” and “vertical social relationship evolution.” The inputs are tasks, groups, and teacher interventions, and the outputs are group presentations, evaluations, etc. As for what kind of interactions occur within F2F cooperative learning and CSCL, how this knowledge is constructed in the process of cooperation, and whether the association occurs and fosters students’ thinking ability—these are all in this “black box”, which needs to be opened for further exploration.

This remainder of this article is as follows. In Section 2, we review the literature related to thinking, knowledge construction, and the construction of social abilities. Then, the mixed and integrated methods are described, including content and social network analyses. The final section discusses the results of this paper and suggests directions for future research.

This study’s three main contributions are as follows. First, we built a triple-dimensional evaluation index focusing on horizontal, vertical, and creative spatial aspects, including creative thinking, knowledge construction, and social ability, in order to perform this evaluation. Second, we used a mixed-methods approach to compare these indices and correlated each index’s performance, thereby improving the assessment’s accuracy. Third, our research is based on course objectives, and through the comparison and analysis of the different stages of the entire course, we aim to enrich the research field of entrepreneurship education and teaching and provide some references for after the COVID-19 pandemic with regard to how the efficiency of teaching via F2F and CSCL can be improved.

2. Literature Review

2.1. Cooperative Learning and High-Order Thinking

Critical thinking, creative thinking, communication, and cooperation are the core means via which college students can be stimulated. Cooperative learning can enhance higher-order thinking, for example, by improving critical thinking [34]. It is a means of developing critical and creative thinking skills [35]. Knowledge sharing accelerates the transformation of new knowledge and the formation of thoughts. It demonstrates the importance of cooperative learning in the learning–innovation relationship. Devi et al. [36] used cooperative learning to train critical thinking in reading. Catarino et al. [37] proved that cooperative learning could improve creative thinking in a linear algebra class through experiments. Hasan et al. [38] designed a variation model to enhance students’ creative thinking and motivation for learning. Creative thinking can be improved in children in a cooperative learning classroom [39]. Expanding access to computer-mediated communication technologies has now made new models possible. Sharing ideas facilitates thinking in the online community because the participants’ posts amplify their views [40]. Integrating computer-supported cooperative learning and creative problem solving into a single teaching strategy can generate high-level creative thinking [7]. The literature shows that cooperative learning plays an essential role in developing students’ high-order thinking in both F2F and CSCL settings.

2.2. Cooperative Learning and Knowledge Construction

A goal of cooperative learning is the construction of shared knowledge [41]. Vygotsky [42] proposed the constructivism theory of knowledge and learning, which emphasizes learners’ initiative. Education is the construction of knowledge, and collaboration runs through the entire process of knowledge construction. It also highlights the importance of cooperative situations for knowledge construction. Collaborative activities promote the improvement of participants’ interaction levels, learning satisfaction, knowledge levels, etc. Students can work together to maximize their own learning and that of others [43]. The theoretical and empirical research on cooperative learning activities, such as the cooperative learning model [6,44,45], the learning environment [46], the technique of cooperative learning (e.g., the script and scaffolding structure), the learning process [47,48], and interactive behaviors and patterns in cooperative learning [49,50,51], guided by knowledge construction theory, are essential components of current and future discussions. For both CSCL and F2F, knowledge construction mainly focuses on how students conduct practical discussions, refine information in cooperation, exchange opinions, deal with conflicts, integrate thoughts, etc. The main goal is to guide students to participate in cooperative learning and always engage in cooperation.

2.3. Cooperative Learning and Social Ability Construction

Cooperative learning is a psychological process of individual knowledge construction and a function of social participation. Social interdependence theory provides a foundation for cooperative learning [52]. Dillenbourg and Jermann [53] pointed out that in the broad study of “cooperation,” we should study “interaction” in depth, because interaction is the key to understanding the nature of cooperative learning. Every community member shares learning interests, seeks knowledge together, and interacts with a common goal. Moller [54] proposed that the learning community primarily includes two functions—social reinforcement and information exchange. These can enable learners to create knowledge and share experiences together while communicating with others. Information communication technologies (ICTs) can promote the interaction of teachers to encourage collaborative learning, debate, and cooperation-based knowledge construction [9]. Social ability is the core element of a cooperative learning environment [55]. Knowledge is not a static object acquired by individuals, but an active cooperative construction achieved through continuous social interaction and the cooperation of multiple social network learners [56]. Derived from social networks, learning means achieving social and collective results through seamless communication, everyday practice, and social network relationships [57]. As the main channel for resources and knowledge exchange, social networks also play a role [58]. Therefore, social network analysis is more suitable for studying cooperative learning interaction, which helps to understand the learner’s cooperative interaction process.

3. Methods

3.1. Evaluation Model Construction

Cooperative learning cultivates high-level thinking that is suitable for future work and life through in-depth reflection and gradual adjustment of cognitive processes and results, including observation and exploration, criticism and solving complex problems, creativity, and innovation. Therefore, in this study, we constructed a three-dimensional evaluation model, with “horizontal knowledge construction,” “vertical social relationship evolution,” and “spatial thinking improvement” based on the goals of courses (Figure 1). The process of knowledge construction includes knowledge sharing, cognitive conflicts, a collaborative structure, and evaluation of reflective formation, which expresses the depth of knowledge construction. During this process, there are three kinds of dialogue behaviors intertwined, namely, teacher–student, student–student, and self-dialogue, to demonstrate the interactive social behaviors in cooperative learning. The thinking mode is the goal of the course. Since this article takes the entrepreneurial curriculum as an example, it aims to promote creative thinking. Therefore, we selected creative thinking as an index, along with knowledge construction and social interaction, to measure the differences between F2F learning and CSCL.

Considering the dimension of space, we paid attention to the degree of achievement in entrepreneurship courses, regarding whether a process impacts creative thinking in terms of the horizontal dimension. In this article, we intend to compare the social interactions of participants in F2F and CSCL and the development process in terms of the frequency and closeness of the exchange in these groups. From a vertical perspective, we focused on knowledge construction during the cooperative learning process and considered the degree of realization of knowledge sharing and cognitive conflict in F2F and CSCL groups. Therefore, we explored the differences in the contribution of F2F learning and CSCL to the performance of entrepreneurship education based on these three dimensions and made the following assumptions:

Hypothesis 1 (H1).

Both face-to-face cooperative learning and CSCL are conducive to the cultivation of creative thinking;

Hypothesis 2 (H2).

CSCL is more helpful in the construction of students’ knowledge due to the support of information communication technology;

Hypothesis 3 (H3).

CSCL is more beneficial to student social interaction, and in terms of emotional establishment, F2F learning is better than CSCL.

3.2. Research Design

The evaluation process had two stages, and one was conducted in 2019. We first collected F2F data, and then CSCL data were collected in the same way. We followed four principles in each stage: (1) it must be a comparative study between cooperative learning groups with the same materials; (2) there must be evidence that the initial conditions are equal; (3) the duration of the study is at least four weeks (20 h); (4) the performance measurement must be based on the teaching goals. The course was divided into four steps to carry out experiments, collect the data, and develop social network analysis methods for creative thinking, knowledge construction, and social abilities. The corresponding ideas are as follows (

Table 1. Design of the research steps.

Items	Objects	Task	Methods
Creative thinking	(1) How to find a good business idea (2) How to identify and evaluate entrepreneurial opportunities	T1: Each group is asked to choose a word (such as “apple”) to brainstorm and generate as many business ideas as possible, and to report the results. T2: Group members choose objects from the surrounding environment to generate ideas for a start-up and classify and summarize the answers.	Experimental analysis (Pre-test)
Knowledge construction	(1) Business model (2) Marketing strategies (3) Financial plan (4) Risk management	T3: A firm is chosen and its business model is analyzed; a business model is designed for their enterprises. T4: The groups are asked to design the corresponding marketing strategy for their products or services and to sell them to other groups.	Content analysis
Social ability	(1) Team cohesion (2) Interactive behavior	T5: The group displays the basic situation of the establishment of the enterprise, and other group members ask questions and engage in discussion. T6: The cooperative learning process is summarized and the benefits and suggestions are discussed.	Social network analysis
Creative thinking	(1) Thinking ability	T7: The groups’ projects are combined with AI technology to generate more Internet-based entrepreneurial ideas.	Experimental analysis (Post-test)

).

(1) Scripting ideas for reference, structured discussion sheets are used to guide the process of group cooperation. For creative thinking, a pre-test and post-test are conducted for comparative analysis based on entrepreneurial ideas. We designed the topic as follows—“Each group is asked to choose a word (such as ‘apple’) to brainstorm, and generate as many business ideas as possible.” The knowledge of business models, market, and finance in entrepreneurship courses play a vital role in improving entrepreneurship’s success rate. Therefore, in terms of knowledge construction, content analysis methods were used to cultivate students’ knowledge on business model design, marketing strategy, financial planning, and risk management. Taking the business model and marketing strategy design in entrepreneurial projects as the topics, we recorded the contents of group discussions and processed and analyzed the knowledge construction level in F2F learning and CSCL. For social interaction ability, we recorded the exchanges and interactions of group members in the discussion process. Simultaneously, in the work display stage, we made records on the interaction between the group members in F2F and CSCL settings. Then, through social network analysis, we made comparative judgments on the situations.

(2) Utilizing the scaffolding technique, we produced conversation guidance prompts, self-evaluation forms, and other forms to guide students to talk more effectively and reflect on their behavior and performance in the cooperative learning process. For example, we added a small check item to each step of the structured discussion list: “Have the opinions of our group been integrated or summarized?”

(3) Each group completes an assignment and displays typical group work. Then, each group completes their materials, allowing for the products to be used as the intermediary to make the group more focused on discussion and sharing and seeking solutions through the interdependence of materials. Finally, the three evaluative performances for each group were correlated.

3.3. Participants

Participants in face-to-face cooperative learning included 113 students who selected entrepreneurship courses from March to June 2019. According to their majors, we selected 23 students to form 4 separate groups, including polytechnic, biology, food, clothing design, and art design, in the F2F group. From each major, we selected 4–5 students who were assigned into different groups to maintain the heterogeneity in order to improve the interaction effects. The entrepreneurship courses were conducted online at our university from March to June 2020 due to the COVID-19 pandemic. Thus, the members of the CSCL group came from the 120 students who chose the entrepreneurship course. We randomly assigned the students with the group activities in the massive open online course (MOOC) platform named Superstar (http://mooc.chaoxing.com/ (accessed on 5 March 2020), a digital platform to support online courses, by which students could form their teams through forum postings or emails. In the case of a major or gender imbalance in one group, we performed manual deployment to diversify the members’ majors and genders. Finally, we chose 23 students, forming four groups similar to the F2F groups in terms of participants. The F2F groups were named Group A, from A1 to A4, and CSCL groups were called group B, from B1 to B4. Each group consisted of five to six students. Most of the students were in the first or second year of college, aged between 18 and 22, of which 46% were male and 54% were female.

3.4. Evaluation Procedure

The research team included two teachers and two teaching assistants who were master’s students. We conducted evaluations at the end of June 2019 and 2020. The group’s cooperative learning situations in F2F learning in 2019 were recorded and evaluated by carrying out individual conversations with the group members, and the assistants transcribed the recordings. We collected discussion information online from CSCL groups based on the Superstar platform after the class in June 2020. Simultaneously, team members reported discussions outside of the platform, such as on the phone or on social media. After the debate, the group reported the corresponding results. Each task strictly adhered to the turn-in time, and the discussion time during the online platform and the face-to-face teaching process coincided.

There were 46 students in this study in total. Group A was the F2F cooperative learning group, and group B was the CSCL group. We processed each group task’s original cooperative communication data into an N × N symmetric relation matrix. The “row” represented the relationship’s sender, whereas the “column” represented the receiver in the matrix. The number “1” indicated that there was an interaction between two actors and “0” indicated otherwise [59]. Each node in the row and column represented a student. To respect the participants’ privacy, the researchers assigned codes to the corresponding members, such as A11, A12, B11, and B12. We recorded A11’s conversation with A12 as ‘A11 interacted with A12 once’ and A12’s reply to A11 as ‘A12 interacted with A11 once’. We carefully studied the content of discussions based on the following provisions: irrelevant content was considered invalid information and excluded; when the information did not have a clear target or reply from the learner, it was not included in the content’s experimental data and social network analysis; if A11 replied to A12 three times with valid information, it was recorded as three interactions between A11 and A12. We judged the strength of the business based on the number of interactions.

3.5. Measures

3.5.1. Creative Thinking

Most of the research used the Verbal Torrance Test of Creative Thinking (TTCT-V) to test creative thinking, and was scored on four scales: fluency, flexibility, originality, and elaboration [60,61]. Then, we designed a test (

Table 2. Creative thinking evaluation form.

Group:	Members
Elements	Evaluation Description	Scores
Numbers	7–10 There are many creative ideas; 4–6 There are 3–5 creative ideas; 1–3 There are 1–2 creative ideas.
Flexibility	7–10 There are many types of ideas; 4–6 There are 4-5–types of ideas; 1–3 There are only 2–3 types of ideas.
Fluency	7–10 There are many ideas, and thinking is carried out without interruption; 4–6 There are many ideas and 1–2 interruptions in the thinking process; 1–3 There are some ideas and too many interruptions in the thinking process.

) according to TTCT-V and scored it based on the number of creative ideas in teams and cognitive thinking styles used, including flexibility and fluency. The two topics we set as the pre-test were “asking each group to choose a word of their own such as ‘apple’ to generate as many business ideas as possible, and report the result”, and “choosing from the surrounding environment in which things can be used to generate ideas for the company”. The topic set as the post-test was “combining the group’s projects with AI technology to generate more Internet-based entrepreneurial ideas”. The results were scored to compare groups A and B, and average higher points indicated more creative thinking. Moreover, since the participants were seniors, we focused on their mindset rather than their professional ability.

3.5.2. Knowledge Construction

Knowledge sharing belongs to the lower level of knowledge construction, argumentation and consulting constitute the middle level, and framing and reflection represent the high level. Drawing on social constructivism theory, Stahl and Gerry [62] used a process model to evaluate the knowledge construction level and learning effect in an online cooperative learning environment. Beasley and Smyth [63] studied the level of knowledge construction through content analysis. According to the classification of collaborative knowledge construction proposed by Gunawardena et al. [64], it was divided into two stages of knowledge sharing and knowledge construction through topic cues. In knowledge-sharing dialogue, knowledge views were stacked, and there were no apparent problem-solving features. However, in knowledge building dialogue, students would explain and argue about a topic and refine and integrate different knowledge and views. The data was encoded for these two stages (

Table 3. Content analysis coding table for knowledge construction.

Stage	Items	Elaboration	Examples
Knowledge Sharing (low-level)	Asking (SQ)	Pointing out some opinions and questions	What is market positioning?
	Description (SD)	Answering the described questions	Position is ……
	Corresponding (SC)	A description of agreement with others’ opinions	I agree with you. The environment can affect people’s buying habits.
	Proposal (SP)	The proposal of establishing collaboration and interaction	When shall we discuss?
Knowledge Construction (mid-level)	Argumentation (CA)	Using experiences and information to support or oppose views	But your ideas about creating a mini travel program do not necessarily have a very suitable promotion platform. Most of Mafengwo is still based on travel strategies, and the profit model is not particularly clear.
	Consulting (CC)	Finding common ground in opinions and supplementary views and suggestions	What you said is very similar with mine, channels and business models are very important.
Knowledge Construction (high-level)	Framing (CF)	According to the new understanding, revising the viewpoint and evaluating the plan	That can be done in a mini program in WeChat.
	Reflection (CR)	Perceiving and evaluating the learning process, methods, and results	The classification of views is very clear, and the reasons are very good. I am very happy to talk with you.

) in order to compare the dynamic processes and differences in F2F and CSCL knowledge construction.

Two researchers conducted data analysis. First, the data were classified and encoded. When the analyzed interactive data reached 40% of the sample, the kappa coefficient was used for reliability testing. If the coefficient was <0.7, a clear coding category was re-started. After adjustment, the rest of the data were coded according to the newly determined system until the coefficient became >0.7, indicating that the analysis results were consistent. The final kappa value was 0.728. Thus, the content analysis framework’s reliability was credible, and the above table was able to be used for content analysis.

3.5.3. Social Ability

Based on the social network and interactive behavior theory, there were two objectives in the mathematical measurement method of social ability. The first one was calculating the number of actors in a particular type of relationship to express each actor’s strength in this type of association [65]. The second was defined by the length of time in a specific relationship between two existing actors [66]. Scholars also determined the relationship’s relative strength according to the type of cooperative relationship between the actors. If more communication and mutual understanding were needed, the power of the harmonious relationship would be higher. Evaluating social ability from the interactive breadth and intensity, we chose network density and network centralization through the number of connections among actors in the group (

Table 4. Social analysis index table for social interaction.

Level	Index	Index Expression	Elaboration
Relationship level	Interaction structure	$C_i,i\in \left\{1,2,\cdots ,\mathrm{N}\right\}=\left\{\mathrm{The}\mathrm{collection}\mathrm{of}\mathrm{groups}\mathrm{ties}\right\}$	Frequency of interaction between two nodes
Network level	Network destiny	$D_t$ = $\frac{l_t}{n_t\left(n_t-1\right)/2}$ $t\in \left\{\mathrm{A},\mathrm{B}\right\}$	The stability of the network structure; the closeness of cooperation.
	Network proximity centrality	${\mathrm{c}}_{\mathrm{pit}}^{-1}$ = (${\sum }_1^{n_t}{\mathrm{d}}_{ij}$)/($n_t-1)$, ${\mathrm{d}}_{ij}$ means the shortcut distance between points i and j	Measures the proximity of an actor to other actors in the network. The higher the proximity to other actors, the easier it is to transfer knowledge.

).

4. Results

4.1. Creative Thinking

First, we compared the differences between each group at the beginning and the end of cooperative learning by using paired sample statistics with bootstrap analysis. As

Table 5. Creative thinking in face-to-face (F2F) and computer-supported cooperative learning (CSCL).

Items	Group A(F2F) (Average ± Standard Deviation)		t	p	Group B(CSCL) (Average ± Standard Deviation)		t	p
	Before (n = 23)	After (n = 23)			Before (n = 23)	After (n = 23)
Numbers	5.67 ± 0.333	5.89 ± 0.423	−1.000	0.047	5.62 ± 0.290	5.85 ± 0.274	−1.897	0.082
Flexibility	4.56 ± 0.333	6.11 ± 0.484	−5.292	0.001	5.23 ± 0.323	5.92 ± 0.760	−2.250	0.044
Fluency	5.67 ± 0.527	6.78 ± 0.494	−5.547	0.001	5.85 ± 0.373	6.54 ± 0.291	−2.920	0.013

shows, the differences in creative thinking before and after the course in both groups A and B were almost significant at a 5% level (p_A and p_B-value < 0.05). The students in groups A and B made clear progress. The results show that cooperative learning played a role in creative thinking. Thus, H1 is supported.

The significant differences between group A and B were compared in terms of numbers, flexibility, and fluency. Since the data for group A and B were generally distributed across those three factors, we could carry out an independent sample t-test separately. The p-value was over 0.05 for all three elements, and the differences were not statistically significant.

4.2. Knowledge Construction

4.2.1. Comparing Two Stages of Knowledge Construction

For a more specific analysis of each level, we examined the content of the discussion in F2F and CSCL in order to record the collaborative learning cognitive interaction behaviors. The results showed that students had 513 conversations in F2F and 671 conversations in CSCL. Figure 2 showed the proportion of knowledge sharing, argumentation, consulting, framing, and reflection during knowledge construction. The stage of knowledge sharing accounted for 47% of CSCL and 42% of F2F. For mid-level knowledge construction, including argumentation and consulting, collaborative learning accounted for 37% of F2F, which was 4% higher than that of CSCL. In the high-level stage, the value for F2F was 21%, whereas for CSCL it was 20%. Then, the significant differences between group A and B were compared in terms of knowledge sharing and knowledge construction by means of separate t-tests. The p-values (p = 0.03 < 0.05) for two factors and the differences were statistically significant. Therefore, the knowledge construction level was slightly higher in the F2F group than in the CSCL group.

4.2.2. Comparing Three Stages of the Whole Course

Figure 3 reveals the trend during cooperative learning. Entrepreneurship education courses and students’ cooperative learning dialogue activities in 12 weeks were equally divided into three time periods (four weeks for each period). During the first stage, both F2F and CSCL displayed low-level construction, with basic knowledge representing more than 60%. The second stage of entrepreneurship education reflected key differences. In F2F, the amount of knowledge sharing significantly decreased, and the amount of consulting increased. The amount of framing and mirroring growth was higher in CSCL, and its percentage of high-level flaming and reflection was slightly higher than that of F2F, indicating that CSCL’s knowledge construction efficiency was better than that of F2F. In the third stage, the high-level development tendency was more obvious in CSCL than in F2F. Consequently, from the second stage of the course, the level of knowledge construction for CSCL was better than that of F2F, which indicates that CSCL’s knowledge construction level became higher than that of F2F as the course time increased.

4.2.3. Comparing Cognitive Level

To compare the two groups’ knowledge construction levels more intuitively, we analyzed the discussion contents statistically according to cognitive interaction theory. Cognitive levels were represented by numbers one to five from low to high, with one representing a lower level of knowledge sharing and five a higher one. We utilized SPSS software to calculate the weighted average of learners’ knowledge construction levels. The S-W significance p-value was 0.831 (>0.05), which indicates that the data obeyed a normal distribution. We then performed an independent sample t-test on the F2F and CSCL groups’ cognition levels, and the p-value was 0.489 (>0.05). Thus, there were no significant differences in cognition between the F2F and CSCL groups. However, based on the analysis of knowledge construction level and personal performance, a significant correlation was observed between the F2F and CSCL groups in concert. The correlation coefficients were β_A = 0.854 and β_B = 0.923, indicating that knowledge construction had a more substantial impact on performance. Simultaneously, it showed that the CSCL team’s performance had more relevant interaction than F2F cooperative learning (

Table 6. Effect on group scores of F2F and CSCL groups.

Pearson coefficients			F2F	CSCL
	Group Scores	Coefficients	0.854 **	0.923 **
		p	0.000	0.000
		N	23	23

** Correlation is significant (double-tailed test) at the 0.01 level.

). In general, after the second stage, the business in the CSCL group was better than that in the F2F group; thus, H2 is partially supported.

4.2.4. Comparing Group Performance

“False cooperation” is prone to appear in cooperative learning. Therefore, to ensure the participants’ representativeness, we conducted a discrete analysis of the group scores, which were the average of the individual scores. The dispersion coefficient is a statistical method that tests whether the mean value is representative. If the dispersion coefficient was greater than 15%, it was considered to contain deviant data. The standard deviations of the F2F and CSCL groups were calculated using the STDEVP function, and then the discrete coefficients were obtained (

Table 7. The coefficients of dispersion (CDs) in F2F and CSCL groups.

Group	SD (F2F)	CD (F2F)	SD (CSCL)	CD (CSCL)
1	10.78	0.15	03.06	0.04
2	08.06	0.10	04.14	0.05
3	11.08	0.15	10.61	0.15
4	07.69	0.09	02.14	0.03

). The coefficients of dispersion in A1, A3, and B3 were 0.146, 0.148, and 0.147, respectively, close to 15%, indicating deviant data in these groups. This result means that the free-riding phenomenon was present in A1, A3, and B3, suggesting that active participation in the discussion did not occur. Thus, the F2F groups engaged in interaction less often than the CSCL groups.

4.3. Social Ability

4.3.1. Basic Network Structure

To discuss the comparative analysis of F2F and CSCL in terms of social capabilities, we analyzed the network structures using Ucinet 6.1 through each group’s interaction matrix. Figure 4 shows the interactions among groups A and B. The strength of the line represents the frequency of exchange, and the thicker the line, the more frequent the interaction between nodes. From Figure 4a,b, we can see that in group A, group A3 (thickest line = 8) and A4 (thickest line = 7) had frequent interactions, and the interaction effect was better. The interaction frequency of group A1 was the lowest, indicating that in the process of cooperative learning, communication was relatively inactive in group A1. The performance difference of the interaction between the sub-groups in group A was more prominent. In group B, the group’s interaction behaviors were similar (thickest line = 9) and more frequent than those in group A, and the interaction performance between sub-groups was not significantly different.

4.3.2. Network Density

Network density is one of the most commonly used social network analysis measures; for a fixed-scale group network, the more frequent the contacts between actors, the greater the group network’s density. With the possibility of the group network affecting the actors’ attitudes and increasing behaviors, the degree of interaction between members is high. We dichotomized the data and calculated the density of each group. The density of group A was 0.2905 and that of group B was 0.2911, indicating that the cohesive force of F2F cooperative learning was slightly lower than the analysis result of the network density of CSCL activities (see

Table 8. Network density in F2F and CSCL groups.

Number	Group A (F2F)	Group B (CSCL)
Destiny	0.2905	0.2911
S.D.	0.454	0.4797

). There was not an apparent gap between the two groups; the reason may be that the dichotomized data show the existing relationship and cannot express such a relationship’s strength. Thus, we further compared and analyzed the individual ego network to test the inner connections.

4.3.3. Centrality and Power

Actors have more connections than others and may occupy an advantageous position. People with more links are generally considered more “important” or more prestigious. Actors with high degrees are deemed to have “influence”. This can give a reasonable explanation for whether the formation of the group leader or power center in group interaction can affect cooperative learning performance. Network in-centralization was 21.58% and 22.79% in group A and group B, respectively. This shows that members in the CSCL group were more willing to connect with each other, and the interaction between group members was better than that of F2F. Opportunities for students to participate in learning increased with the platform’s data records. The out-centralization in group A was 21.43%, whereas that of group B was 19.44%. This shows that on the whole, the influence of the members of the F2F group was more significant than that of the CSCL group, indicating that the face-to-face group members were more likely to form power centers. The results also demonstrate that students’ motivation was more intense in F2F groups. If there was a lack of enthusiasm, free-riding phenomena would occur and reduce the team’s learning performance. Overall, the interaction level of CSCL collaborative learning was higher than that of F2F, and the emotional energy in F2F was more cohesive than that of CSCL. Thus, H3 is supported.

4.4. Triple-Dimensional Integrated Analysis

We took knowledge construction as the x-axis and social ability as the y-axis, and the bubble’s size represented the level of creative thinking. We then constructed a three-dimensional evaluation system. The full score for each item was 10 points. The scores of creative thinking were calculated based on each group’s average across the three evaluation elements. The level of knowledge construction was calculated by multiplying the average score of each group member by the coefficient of reaching the high-level knowledge construction stage (Equation (1)):

$$Know\_so{c}_i=\overline{X_i}\times HK{p}_i/\overline{HKp}\times 0.1$$

(1)

where Know_soc_i represents knowledge construction scores, ${\overline{x}}_i$ means the average performance of group i, HKp_i is the percentage of high-level construction in group i, and HKp is the average percentage of the group. To make the values of the dimensions equal, we multiplied the result, making the data balance by 0.1. Social abilities were sorted based on the number of degree centers of each group. The results are shown in the graph in Figure 5. The overall performance of group A3 was the best, with good knowledge construction and social ability and high creative thinking. At the same time, the overall performance of group A1 was low. The CSCL group’s equivalent was relatively average. B3 had the highest knowledge construction but lacked innovation. Group B4 had more increased social capabilities, but the level of knowledge construction was not high.

Combining the previous analysis results, we observed that it was easier to formalize the “false cooperation” in the F2F group. Taking A3 as an example, a student in the group was particularly competent, making the entire group achieve a higher level of achievement and knowledge construction. The average score of group A3 was 75 points, but one student scored only 60. There was apparent free-riding behavior, but this also shows that group leaders or excellent group performance members were more evident in cooperative learning. In the CSCL group, it was easy to form “pan-cooperation” and to lack the potential for in-depth exploration. Taking the B4 group as an example, although it had high social ability, it had no direct impact on improving the knowledge construction level. This is because in online learning, teachers can quickly grasp students’ login statuses. At the same time, online results are directly related to messages, discussions, etc. Therefore, these discussions or speeches for the sake of achievement have limited influence on inner self-efficacy.

5. Discussion and Conclusions

The outbreak of COVID-19 provided us with an excellent opportunity to consider the learning outcomes of CSCL and F2F. Therefore, a primary driving force of this study was to understand the differences in student behaviors in regard to F2F cooperative learning with CSCL in entrepreneurship education through a three-dimensional index with a “horizontal knowledge construction,” vertical social relationship evolution,” and “thinking ability improvement” framework in the context of the COVID-19 pandemic. We utilized mixed methods, including experimental, content-based, and social network analysis methods, to evaluate 46 students who registered for this course separately in 2019 and 2020. As the experimental results reveal, cooperative learning helped to improve students’ creative thinking. The results support Devi’s [35] and Caldwell’s [39] conjecture that the amplified effects of participants’ posts on sharing ideas could improve thinking ability. In the shift from F2F to CSCL in the context of the COVID-19 pandemic, there were not many differences between the traditional face-to-face classroom and cultivating creative thinking.

Some teachers have previously rejected online learning due to the challenges or difficulties of technology [67]. There is no other option for teachers to adopt a pedagogical approach during COVID-19. Through content analysis of knowledge construction, we observed that F2F’s knowledge construction capabilities were slightly higher than those of CSCL. The present research results are in agreement with those of the studies of Molinari’s [9] and Kirschner’s [10] groups. These findings demonstrate that each group could reach the desired level of reflection, indicating that both the F2F and CSCL groups were able to achieve knowledge construction, as proposed by Legrain [44], in which cooperative learning is an instructional model for improving teaching efficiency. However, in the early cooperative learning process, students’ collective responsibility was relatively low in all groups. The collaboration was somewhat scattered, and each topic was sparse, mostly forming a dialogue based on knowledge sharing. In the later cooperative dialogue process, students’ collective responsibility was improved, collaboration was relatively close, and students were more involved in knowledge construction dialogues. The advantage of CSCL gradually became evident. Learning environments that use information communication technology (ICT) play a more crucial role in the later stage of knowledge construction.

The comparative analysis of the two forms of social abilities through social network analysis suggested that F2F’s network destiny was significantly lower than that of CSCL. This finding is consistent with that of Kyndt [14]. As Lin [47] proposed, scripting ideas and scaffolding techniques are essential for CSCL. In both the pre-COVID and COVID pandemic context, scholars have unanimously agreed on the role of CSCL in developing social competence [9,10]. The medium of the computer provides a more convenient communication platform for teachers and students. It is also an excellent way to record students’ dialogue processes, which promotes the improvement of social ability. In the context of the COVID-19 pandemic, instructors, and students have to concisely adapt their teaching and learning activities without training and with little preparation. The results do not show that the learning outcomes of F2F were much higher than those of CSCL. The findings indicate that CSCL is sustainable, which is not entirely consistent with the opinion that CSCL supplements face-to-face teaching in traditional physical classrooms [15].

The results of our analysis also show that for students in F2F groups, motivation is more important. If enthusiasm is lacking, free-riding phenomena can easily occur and reduce the team’s learning performance. Recording the login and discussion data on the platform had a particularly restrictive effect on students’ behaviors; thus, CSCL could appropriately reduce free-riding behavior since it focused more on students’ interactive behavior. In the correlation analysis of scores, free-riding phenomena were more common in F2F than in CSCL. The superficial “pan-cooperation” of CSCL was more prominent. Although it showed an adequate knowledge construction level in the second stage of the course, the high-level knowledge construction stage’s performance was not apparent, and it lacked the potential for in-depth exploration.

Teachers and students have learned more about online education in three months than in the last ten years. Some teachers who previously rejected CSCL changed their attitudes [67]. Many learning systems for learning management, such as Zoom, DingDing, Tencent class, and others put forward synchronous online courses and learning applications, leading teachers to think about computer-supported pedagogical approaches. Therefore, there are opportunities for teachers to reflect on their provision of education under or after the COVID-19 pandemic. Learning motivation and team management play essential roles in the F2F cooperative learning process, as pointed out by Ehsan [20]. Face-to-face cooperative learning has a noticeable effect on a team’s cohesion and the argumentation and consulting stages of knowledge construction. However, some problems remain, such as the lack of enthusiasm and less participation in the discussion. Therefore, in future teaching processes, for F2F cooperative learning, teachers should pay attention to how students conduct practical conversations, how they cooperatively refine information, how they exchange opinions, how they deal with conflicts, how they integrate views, etc., drawing on the perspectives and strategies of CSCL. They can guide students to carry out cooperative learning methods and help students to continuously conduct cooperative reflection.

Based on cognitive interaction, scaffolding theory, and metacognitive theory, CSCL puts forward various active teaching intervention strategies, which completely open up new ideas for the strategy design of face-to-face cooperative learning in the classroom. We should pay attention to the content of interactive quality and emotional communication during the learning process, since it lacked some F2F nonverbal communication information in this study. Despite the large number of replies, the quality was relatively low. The content was more relative to low-level knowledge construction. Simultaneously, although interactions between the groups were frequent and the network distance was relatively short, the participants’ emotions were limited and not communicated. Thus, teachers should guide students to analyze the similarities and differences between their own and others’ opinions and question and refute others’ views in discussion. To increase communication abilities, teachers can focus on leading students to consider listening, expressing, etc.

In conclusion, this COVID-19 pandemic has forced us to develop online learning, especially in entrepreneurship education. Our findings show that extreme circumstances can be transformed into opportunities for cooperative learning through in-depth exploration. F2F and CSCL have their advantages and disadvantages and need to be targeted according to the discipline’s characteristics for designing cooperative learning. CSCL is sustainable, and more hybrid learning activities combining online and offline settings can be applied, according to course contents. In the first stage, we can apply face-to-face cooperative learning to conduct emotional compatibility analysis and team formation. CSCL is more suitable for financial risks and other related content at a later stage, compared to F2F learning. At the end of the course, F2F cooperative learning can be used to reflect on and develop thinking ability.

6. Research Limitation

This article is subject to some limitations. One limitation is that we focused on the performance of the teaching process and ignored related elements, such as teachers, scripts, and team management in cooperative learning. The current study only looked into cooperative learning design for entrepreneurship education courses, and it did not integrate the professional learning background of students into the cooperative learning process. Moreover, we arranged a pretest adapted from the Verbal Torrance Test of Creative Thinking (TTCT-V) for creative thinking, not including the Figural Torrance Test of Creative Thinking (TTCT-F). We should consider adding this aspect to measure and compare these two visions’ validity in future work. Facing the COVID-19 outbreak, we conducted online learning activities at individual homes separately, not in a classroom. This lack of a learning atmosphere may affect the performance of CSCL. Other possible limitations are that we collected the data amid the COVID-19 outbreak and that all the courses that the students registered to shifted to online learning. This was the first time instructors and students experienced this type of education provision, which may have resulted in increased stress levels. Therefore, emotion and suitability also may affect the results of CSCL. The COVID-19 pandemic nevertheless provides a unique challenge and an opportunity to understand online and offline cooperative learning. The subsequent effects of COVID-19 force us to continue to focus on online learning and to explore the relationship between online and offline learning and teaching activities.