An Action Research in Critical Thinking Concept Designed Curriculum Based on Collaborative Learning for Engineering Ethics Course

Abstract

(1) Background: Critical thinking, CT, contributes to success in both career and higher education, and may be more important than professional content knowledge. Nevertheless, it is challenging to cultivate CT in a standalone course, especially for the engineering students who think less critically than those in other colleges. (2) Methods: This research incorporated CT concept into 18 weeks curriculum of Engineering·ethics and Society course, with the assistance of collaborative learning process for formative assessment and problem-based learning for summative assessment, in addition to 3 questionnaires to evaluate the progress in CT and collaboration. (3) Results: Both measurements in CT and collaboration improved significantly. In general, the participants enjoyed the course materials and thought these CT and values infused course activities were helpful to the learning. On the other hand, CT was also the most noticeable problem. About one over every five participants lacked the habit to think, while 17% of participants were afraid of complex questions to think. In addition, 10% doubted their CT skills. It concludes that total 46% participants thought CT is their most crucial shortage. (4) Conclusions: the pretest demonstrated the CT of the participants was below the college norm; fortunately, the assistance of the social interaction, including team work practices, peer evaluation, and pressure to push individuals work harder and think deeper, did promote their CT cognitive development.

1. Introduction

Critical thinking (CT) contributes to career success and also the success of higher education. In research conducted for the Bill and Melinda Gates Foundation, Professor David T. Conley of the University of Oregon found that “it is necessary to determine the success of university courses, analysis, interpretation, precision and accuracy, problem-solving and reasoning.” Habits may be more important than professional content knowledge [1].

This action research arose when the author noticed a serious problem that CT is significant for the ethics course while the majority of the engineering students are quite poor at it. It is essentially meaningful to infuse CT content and investigate the progress in the action of teaching. The origins of action research are unclear within the literature [2]. According to Stipe, however, action research in education is defined as a study conducted by colleagues to improve instruction and it starts with identifying a problem or dilemma in a classroom or with particular students [3]. In addition, Gummesson [4] listed ten major characteristics of action research: (1) taking action; (2) always involving two goals: solve a problem and contribute to science; (3) interactive; (4) aiming at developing holistic understanding; (5) fundamentally about change; (6) requiring an understanding of the ethical framework; (7) including all types of data gathering methods; (8) pre-understanding of the corporate environment; (9) in real time; (10) the paradigm with its own quality criteria.

Accordingly, this research first examined the concept of CT and related course designs for engineering students and then figured some limitations and suggestions, including that team work learning is recommended. Next, the differentiation among three common learning paradigms of teamwork in college was addressed, which are PBL (project based learning), collaborative and collaborative learning. Finally, a research framework based on the literatures is proposed in the end of this section.

1.1. CT and Engineering Students

First of all, CT is a broad concept and the experts among the fields of education, psychology, and philosophy define CT differently. According to the literatures, CT has three aspects in general. The first is knowledge, such as Eggan, Lsegold [5], and Gambrill [6] believe that there are four types of knowledge that good critical thinkers must possess, content knowledge, procedure knowledge, self-knowledge, and situation knowledge; the second is dispositions, such as Ennis’s 13 intentions of critical thinking, including rational integrity, rational empathy, and rational humility, rational courage, rational perseverance, etc. [7]; the third is skills or abilities, such as Dewey’s 5 journeys or Ennis’s 12 critical thinking abilities [8,9,10,11].

Next, how to apply CT into an engineering course? The Faculty of Applied Sciences at the University of Cincinnati conducted a four-person inter-departmental faculty team to develop an upper-level general education course called Professional Ethics in Technology. It used case studies to build critical thinking, teamwork and communication skills [12].

IUPUI (Indiana University-Purdue University-Indianapolis) defines critical thinking as the ability to analyze information and ideas carefully and logically from multiple angles. This ability is proven in the following aspects: (1) Analyze complex problems and make wise decisions; (2) Synthesize information to reach reasonable conclusions; (3) Evaluate the logic, validity and relevance of data; (4) Solve challenging problems; (5) Use knowledge and understanding to generate and explore new issues [13].

In addition, they used the National Survey of Student Engagement (NSSE) to describe the degree of critical and analytical thinking for their students in the different colleges. The results show that the students of the College of Engineering are far below the average of the entire campus, and it is the only college on campus that is statistically significantly lower than the average of the entire campus. The goal of the research is to find teaching tools and assignments that can be adjusted and used in one’s own existing courses, and cultivate critical thinking in one class (or at most the interaction between two classes).

They suggested two different ways to develop CT: (1) Through writing, (2) Problem-oriented learning (PBL). The research provides a writing method and problem-oriented method to students of an engineering school in the teaching process, evaluation methods and suggestions of critical thinking. It concluded that multiple choice questions and short answer questions does not give students the opportunity to practice various solutions or reflections. Students’ CT skills will mature in the education process, but educators must provide opportunities. When students learn and use the writing or PBL steps in this article, they will discover the characteristics of critical thinking and use these characteristics as standards to assess students’ abilities.

In addition, in order to meet the requirements of ABET, the Department of Civil and Environmental Engineering of Northern Arizona University integrated professional ethics into multiple professional courses. Among them, the engineering design course of the junior year used the ethical decision-making process, which match the step-by-step engineering design process. Therefore, it can evaluate a specific problem more comprehensively, produce a feasible solution and a reasonable response more efficiently [14].

However, Adair and others mentioned that the logical thinking ability of CT cannot be learned using traditional step-by-step teaching methods. This article analyzed the differences between critical thinking and traditional teaching, including the vagueness (no standard answers/steps) that traditional engineering education is less good at. Solutions often need to come from a broader knowledge background and context (contrary to the usual simple thinking mode of engineering) and so on. Under these limitations, they incorporated CT teaching into the course of Fluid Mechanics. From the beginning, Paul’s critical thinking model was used from Intellectual Traits/Values, including intellectual humility, intellectual autonomy, intellectual integrity, intellectual courage and so on. The elements of thought were applied in the second stage, using engineering design materials to discuss or practice to help students evaluate their own ideas or the ideas of others, including goals, assumptions, and consequences. Finally, the knowledge standards were applied into the review of standards such as clarity, accuracy, relevance, logical validity, precision, importance, completeness, fairness, and depth [15].

Afterwards, Adair had three suggestions for curriculum implementation: (1) Additional shorter courses should be used for CT training and evaluation, instead of occupying the teaching time of professional courses; (2) In the CT training system, each process contains different evaluation items individually. When different professors score differently, the problems of consistency of evaluation should be concerned; (3) It is easy to infuse CT into homework or plans, but it is difficult to integrate CT into a short regular teaching format.

1.2. CT and Collaborative Learning

Many researchers agree that collaborative learning is effective to foster thinking critically. The results were proved by the achievement of the participants in mathematics [16], CT test [17], and higher levels of thought [18,19].

In addition, team works are capable of improve problem-solving strategies because the students are confronted with different interpretations of the given situation [20]. Thus, the learners have to analyze, synthesize, and evaluate ideas cooperatively and therefore achieve a better chance to solve complex problems [21].

Gokhale explained the reasons to make collaborative learning enhance CT abilities including group diversity, and then the interaction to learn from each other. Because the participants have to offer the statements of opinion by giving reasons for their judgments and reflecting upon the criteria employed in making these judgments. Each opinion was subject to careful scrutiny and the ability to admit that one’s initial opinion may have been incorrect was still valued [21].

However, team work does not always work. There are causing factors and it is important to seek conditions under which collaborative learning occurs efficiently [22,23]. Le created a collaborative learning environment with the use of the Web and found that knowledge is given from teachers to learners and collaborative learning does not play an important role. Interacting between educators and learners was proposed a key to make a classroom successful [24]. In addition, the students have to engage in discussion, take responsibility for their own learning, and thus can become critical thinkers [25].

1.3. PBL, Collaborative and Collaborative Learning

The concept of the grouping of students for the purpose of achieving an academic goal, has been widely researched and advocated throughout the professional literature, including PBL, collaborative and collaborative learning. Their meanings and differences are explained below.

PBL was originally designed for medical schools, but has been applied by many different disciplines as a student-centered, active-learning paradigm. Students work in collaborative groups to identify what they need to learn in order to study an authentic case that includes an open ended problem to be solved. In the team meetings, they discuss the information collected, suggest solutions, evaluate solutions, and present the team’s conclusion finally [13,26,27].

Some paper used collaboration and cooperation alternatively [21,28,29]. On the other hands, Matthews addressed that confusion about their similarities and differences leads not only to misconceptions, but to different opinions sometimes. However, their significant differences are among adherents, while a great deal of overlap between their boundaries [30].

According to Cohen [31], cooperative learning was defined as “students working together in a group small enough that everyone can participate on a collective task that has been clearly assigned”. Johnson defined cooperative learning as “the instructional use of small groups so that students work together to maximize their own and each other’s learning” [18]. It requires that each of the members of the team uses their knowledge and skills to help the other members of the team understand the content. It focuses on working together, or interdependence.

Collaborative is an adjective that implies working in a group of two or more to achieve a common goal, while respecting each individual’s contribution to the whole [32]. Panitz defined collaborative learning “a personal philosophy, not just a classroom technique. In all situations where people come together in groups, it suggests a way of dealing with people which respects and highlights individual group members’ abilities and contributions [33]”. It means that the students come together to work on a project as a team. Each student is responsible for their own individual work separately, but also in charge of the team work as a whole.

Based on the meanings of collaborative and cooperative learning, the team works designed in the course are mostly collaborative learning. For example, this study applied collaboration-based PBL for the final project as a summative assessment. Nevertheless, the class time activities for formative assessment are between the two types of the collaborative and cooperative learning. Moreover, many papers did not distinguish the two and just simply applied collaborative learning in their studies; for example, “collaborative learning” was straightforwardly defined as “an educational method in which students work groups towards a common academic goal” and then applied through the whole process [16,17,24,34,35,36]. As a result, the learning style of the team work in the course will still be termed collaborative learning.

1.4. Purpose of Study

The comprehensive literature reveals the following difficulties of CT teaching in the engineering colleges: (1) The CT definition is not unified; as a result, it is also difficult to have a universal and reasonable evaluation standard; (2) The traditional step-by-step engineering education style is not used to rearrange the course materials based on the real-time, irregular and unexpected students’ responses; (3) Engineering students are also less trained to deal with a thinking model without a single right or wrong conclusion; (4) Engineering students’ critical and analytical thinking is much lower than the college’s students average. As a result, some employers believe that the ability to think critically and independently is one of the biggest shortcomings of the recent engineering graduates. Moreover, even if teachers want to improve, it was concluded that the complexity to incorporate CT into traditional step-by-step professional courses is beyond the ability of the most engineering professional teachers.

Therefore, it is quite meaningful to design an unconventional course to help engineering students think more critically and the following literature recommendations were noticed and applied in this study. First, IUPUI suggested Problem-oriented learning (PBL) to develop CT [13]. Next, it is difficult to integrate CT into a short period of regular lecture [15]. Therefore, a long term training strategy should be implemented. Moreover, collaborative or cooperative teaching methods were recommended by using the value of social interaction to promote cognitive development [12,37,38]. Lastly and most importantly, CT was recognized with 3 aspects of knowledge, disposition, and skills [5,6,7,8,9,10,11].

Notably, the research questions examined in this study were:

• Will students improve collaboration quality by collaborative learning?
• Will students improve CT disposition by collaborative learning?
• How will students cope with collaborative learning in the course progress of CT concept based curriculum design?

Overall, fifty-eight 58 senior mechanical engineering students, 50 males and 8 females, participated in this study. Three questionnaires used in this study: Inventory of Critical-thinking Dispositions, ICTD [39], collaboration questionnaire [29,40,41], and the learning experience opinions investment (LEOI) [42]. The quantitative data are analyzed by means, standard deviation, and paired sample T test by the statistical software SPSS 23.0.

This research contributes to consider the factors which literatures recommended into practice into one course and then observe the consequences. Specifically, the concept of the 3 CT aspects was infused into 18 weeks’ curriculum of an engineering ethics course. Then collaborative learning process for formative assessment and finally PBL-based final project for summative assessment were integrated. In addition, how the students coped with this course design and their improvement both on CT and collaboration were monitored by multi-tools of questionnaires.

2. Materials and Methods

2.1. Participants

The participants were 58 senior mechanical engineering students who enrolled in the course “Engineering·ethics and society” at Southern Taiwan University of Science and Technology in Taiwan. There were 50 males and 8 females participated in this study.

2.2. Course Design

According to the three aspects of CT, the course was design with the three parts sequentially: knowledge, disposition and skills. In the beginning of the course, preliminary exploration of ethics was the main topic for the CT knowledge and gradually moved to the second part which is critical thinking definition and conundrum discussion as the training for CT disposition. Finally, three classic cases of engineering ethics (BP Deepwater Horizon and so on) which are like a mini-PBL process, were introduced and the students collected data and organized the information for a final report, as listed in

Table 1. The course design in 18 weeks.

Weeks and Topics	Course Content	Assignment and Evaluation
1–9 Preliminary Exploration of Ethics (CT knowledge)	Definition, requirements, origin/purpose Ethical problems Ethical dilemma	Questionnaire: ICTD (1) 1 Quiz: ethical concept (3, 7) HW: ethical 1 min report and peer assessment (8)
10–12 Critical thinking (CT disposition)	CT Definition Conundrum discussion: Lake pollution Technology, society and ethical issue discussion: South Korea “Room N” incidents	Quiz: CT concept (11) Zuvio QA response
13–18 Engineering ethics case study (CT skill)	2010 BP Deepwater Horizon 2011 Fukushima nuclear disaster 2020 Taoyuan algae reef and third natural gas receiving harbor	Learning experience Survey (16) Questionnaire: ICTD (17) Final oral presentation in group (18)
Weeks and Topics	Course Content	Assignment and Evaluation
1–9 Preliminary Exploration of Ethics (CT knowledge)	Definition, requirements, origin/purpose Ethical problems Ethical dilemma	Questionnaire: ICTD (1) 1 Quiz: ethical concept (3, 7) HW: ethical 1 min report and peer assessment (8)
10–12 Critical thinking (CT disposition)	CT Definition Conundrum discussion: Lake pollution Technology, society and ethical issue discussion: South Korea “Room N” incidents	Quiz: CT concept (11) Zuvio QA response
13–18 Engineering ethics case study (CT skill)	2010 BP Deepwater Horizon 2011 Fukushima nuclear disaster 2020 Taoyuan algae reef and third natural gas receiving harbor	Learning experience Survey (16) Questionnaire: ICTD (17) Final oral presentation in group (18)

¹ The brackets indicate the week of the activity.

.

Each topic had multiple “contents” including definition, content/procedure/self/situated knowledge, topic discussion, and PBL-based case study; “assignment practices” including homework (HW), Zuvio QA response and oral presentation; and finally “evaluation tools” including quiz, questionnaires, survey, peer assessment, and final project presentation. These course activities were held by team based collaborative learning process.

For a successful collaborative learning, lots of course management are necessary. Particularly, the following three efforts were made to enhance the collaboration.

• Change team members twice by regrouping to enhance their abilities to coordinate to new fellows.
• Peer evaluation offer every participant the power to watch others and behave themselves.
• Practice teamwork by abundant team activities including white board competition, team/class discussion, debating, team brain-storming/reporting and real-world case study.

2.3. Research Tools

There are three questionnaires used in this study: Inventory of Critical-thinking Dispositions, ICTD [39], collaboration questionnaire [29,40,41] and the learning experience opinions investment (LEOI) [42]. The learning aspects and the qualitative and quantitative evaluation tools designed to cultivate the CT and collaboration competencies were listed in

Table 2. List of course evaluation tools and operation week.

Aspects	Qualitative	Quantitative
Knowledge	Zuvio 1 QA responses (2–12) 2 HW-ORID ethical 1 min report 3 (8)	Quiz: ethics concept (3, 7) Quiz: CT concept (11)
CT	Questionnaire: LEOI (16) Final Project: Engineering ethic case study (18)	Questionnaire: LEOI (16) Questionnaire: ICTD (1, 17)
Collaboration	Questionnaire: LEOI (16) Final Project: Engineering ethic case study (18)	Questionnaire: LEOI (16) Questionnaire: collaboration (2, 18)

¹ Zuvio is a learning app to enhance classroom communication. ² The brackets indicate the week of the activity. ³ It is a homework that the students recorded their commands on the ethics knowledge.

.

The ICTD, a 6-point Likert type scale, was employed to measure the participants’ CT-dispositions. The response options were “never” to “always”, represented by “1” point through “6” points. The ICTD included four factors: systematicity and analyticity, open-mindedness, inquisitiveness, and reflective thinking. The Cronbach’s α for the whole inventory and the four factors were .88, .83, .58, .70, and .63, respectively.

The collaboration questionnaire was divided into four parts including demographic variables, constructed of coordination of intra-group team work, constructed of intra-group communication, and constructed of intra-group trust. The items of constructs were measured by seven-point Likert-type scales from strongly disagreement (1 point) to strongly agree (7 points).

The LEOI investigated the participants learning experience with 35 quantitative and 4 qualitative questions. The three questionnaires were filled out online, with the pre-test in the first three weeks and the post-test in the final three weeks of the semester. The use of registered questionnaires facilitates the statistical verification of paired samples. Most of them are closed questions, and some are open questions, collecting students’ opinions and suggestions on the course learning. The quantitative data are analyzed by means, standard deviation, and paired sample T test by the statistical software SPSS 23.0.

3. Results

3.1. Improvements in Critical Thinking Dispositions

This study employed the ICTD to measure participants’ improvements in CT dispositions. 35 participants completed both the pretest and posttest with 5 females and 30 males among the 59 class students. The questions of the ICTD and entirety results of the participants in the four constructions are shown in

Table 3. Statistic results of the participants in the ICTD (n = 35).

Construction	Item	M (Pretest)	M (Posttest)	SD (Pretest)	SD (Posttest)	Sig. (2-Tailed)
Systematicity and analyticity	4. Even when facing complex problems, I still try to maintain rational and logical thinking.	4.29	4.60	.85	.835	.09
	5. Before using a message, I will first consider whether the message is reliable.	4.51	4.83	.73	.736	.08
	6. I try to verify the value and reliability of the new ideas.	4.57	4.74	.73	.805	.26
	8. When dealing with problems, I try to define the problem clearly.	4.40	4.69	.96	.747	.11
	10. When solving the problem, I try to keep myself the latest and most complete relevant information.	4.37	4.74	.76	.805	.04 *
	11. During the discussion and observation, I can quickly understand the feelings and thoughts of others.	4.37	4.57	.68	.767	.18
	14. When there is sufficient evidence to show that my opinion is biased, I will immediately revise my opinion.	4.37	4.60	.76	.835	.10
	15. Before proceeding to solve a problem, I first try to find out the cause of this problem.	4.46	4.77	.69	.759	.04 *
	16. Regarding the recent controversial issues, I try to understand the ins and outs.	4.69	4.74	.71	.731	.73
Open-mindedness	3. In the context of the discussion, I try to respect the opinions of others.	5.14	5.06	.54	.674	.54
	12. When the evidence is insufficient, I will postpone my judgment.	4.49	4.60	.77	.725	.52
	13. When solving problems, I try to consider various possible solutions.	4.49	4.80	.84	.786	.08
	19. In the context of the discussion, I will listen carefully to what others are saying.	4.94	5.11	.67	.747	.30
Inquisitiveness	1. I try to think about a problem from a different perspective.	4.46	4.49	.81	1.025	.89
	2. I try to apply some new ideas or concepts.	4.51	4.66	.77	.924	.50
	18. I try to further explore novel things or viewpoints.	4.57	4.86	.77	.639	.07
Reflective thinking	7. When making a decision, I will take the influencing factors of the situation into consideration.	4.63	4.86	.68	.761	.12
	9. I try to determine whether my views are sufficiently convincing by self-questioning.	4.26	4.66	1.05	.893	.07
	17. When someone else puts forward an argument, I try to find out the main hypothesis implicit in the argument.	4.43	4.49	.84	.967	.75
	20. Before making a decision, I try to predict the possible outcomes of all alternatives.	4.54	4.73	.81	.817	.74
The 20 items of the column	max	5.14	5.11	1.05	1.025	-
	min	4.26	4.49	.54	.64	-
	mean	4.52	4.73	.77	.80	-
	SD	.20	.16	.11	.09	-

* p < .05.

.

Mostly, the means in the posttest are higher than those in the pretest. 18 items increase while 2 items (#3 and 20) decrease. #9 (I try to determine whether my views are sufficiently convincing by self-questioning.) made a greatest improvement of the means, .4, with lowest pretest mean, 4.26, among all items; even though the posttest mean, 4.66, is still lower than the average of the all posttest items, 4.73. Moreover, the SDs of the #9 pretest, 1.05, is highest among the 20 items. It shows more divergent opinions of the participants in this question. On the opposite, #3 (In the context of the discussion, I try to respect the opinions of others.) regressed the most. It had the highest pretest mean of 5.14 and then decreased to the posttest mean of 5.06 which is still higher than the average of the all posttest items, 4.73 though. Finally, neither #9 or #3 reached statistical significance.

According to the means of the 20 items, their maximum, minimum, mean and SD are shown in the bottom of Table 3 for further comparison. First of all, the SDs in the pretest and posttest were .2 and .16, respectively; while the means in the pretest and posttest were 4.52 and 4.73, respectively. Moreover, the maximum mean of the 20 items decreases from 5.14 to 5.11; while the minimum mean increases from 4.26 to 4.49. It demonstrates the participants’ responses were more discrete in the pretest and became more convergent to general higher posttest values.

Finally, there are two items (#10 and 15) increasing statistically significantly. Based on this ICTD evaluation, the participants demonstrated the improvements among the most items of the critical thinking dispositions.

The 20 questions of ICDT were divided into 4 constructions as shown in Table 3 and their statistic data of the participants were summarized in

Table 4. Statistic results for the ICTD in the four constructions (n = 35).

Construction (Number of Questions)	Systematicity and Analyticity (9)			Open-Mindedness (4)			Inquisitiveness (3)			Reflective Thinking (4)			Entirety (20)
	Pre	Post	norm *	Pre	Post	norm	Pre	Post	norm	Pre	Post	norm	Pre	Post	norm
max	6	6	6	6	6	6	6	6	6	6	6	6	6	6	6
min	1	2	3	3	3	3	3	1	2	1	3	2	1	1	2
mean	4.45	4.70	4.72	4.76	4.89	4.97	4.51	4.67	4.62	4.54	4.69	4.63	4.54	4.73	4.74
SD	.78	.79	.48	.77	.76	.49	.78	.89	.63	.89	.86	.61	.81	.82	.43
Cronbach α	.88	.86	.76	.78	.63	.53	.77	.44	.61	.71	.75	.66	.924	.903	.86

* It is the college students’ norm from Yeh [39], n = 1343.

. All questions got 6 as the maximum number from our participants and other study for the college students [39]. However, our participants reached 1 or 2 of the minimum responses, lower than those in the college norm, 2 or 3. In addition, the four means in the pretest were all lower than the norms. Fortunately, the posttest means of “inquisitiveness” and “Reflective thinking” are higher than the norms, while the other two constructions of “systematicity and analyticity” and “open-minded” are still lower than the norms, and though they are better than their own pretest.

Next, the SDs in this study are higher than those in the norm. It is expected owing to limited case number of 35, in comparison of case number 1343 in the Yeh’s study. Finally, the Reliability of Cronbach α in the pre and posttest of the four constructions are all above .7, the acceptable level, with one exception of the posttest “Inquisitiveness”, .44. However, the overall results in the both pre and posttest are more than .9 with excellent internal consistency.

For a clear view of comparison, the data of Table 4 was drawn in Figure 1. Simple main effect analyses revealed that the participants performed better in “open-mindedness” than in “systematicity and analyticity”, “reflective thinking”, and “reflective thinking”. On the other hand, the participants performed better in the posttest than in the pretest on the all four factors.

The statistic results of the difference between the pretest and posttest of the constructions and entirety are shown in

Table 5. The paired T test results of the constructions in the ICTD (n = 35).

Posttest-Pretest	Mean	SD	95% Confidence Interval of the Difference		t	Sig. (2-Tailed)
			Lower	Upper
Systematicity and analyticity	.25	.60	.04	.46	2.48	.02 *
Open-mindedness	.13	.70	−.11	.37	1.08	.29
Inquisitiveness	.15	.83	−.13	.44	1.09	.28
Reflective thinking	.16	.71	−.09	.40	1.31	.20
Entirety	.19	.54	.01	.38	2.10	.04 *

* p < .05.

. It shows that the intra-group “systematicity and analyticity” improved most dramatically while the other three constructions and entirety increased too. In addition, the intra-group “systematicity and analyticity” and the entirety also increased with statistical significance. It proves the participants were improved in the critical thinking disposition overall and in the “systematicity and analyticity” and entirety particularly.

3.2. Improvements in Collaboration

The questionnaire has three constructions with the statistic results of the pretest and posttest shown in

Table 6. Pretest and posttest of the collaboration questionnaire in the three constructions (n = 44).

	Pretest			Posttest
Construction	Coordination	Communication	Trust	Coordination	Communication	Trust
max	7	7	7	7	7	7
min	1	1	1	1	1	1
mean	4.56	4.84	5.18	5.05	5.60	5.62
SD	1.33	1.27	1.14	1.48	1.28	1.18
Cronbach α	.81	.88	.82	.71	.83	.78

and Figure 2. Forty-four participants completed both the pretest and posttest with 6 females and 38 males among the 59 class students. The coordination was weakest among the three constructions, while trust was best both in the pretest and posttest. On the opposite side, the SDs of the trust were lowest and those of the coordination were best both in the pretest and posttest. It shows the participants believed the coordination was weakest with more divergent opinions; while the trust was best with highly agreement. In addition, all the means in the posttest increased.

The statistic results of the difference between the pretest and posttest are shown in

Table 7. The paired T test results of the three constructions of the collaboration questionnaire (n = 44).

Posttest-Pretest	Mean	SD	95% Confidence Interval of the Difference		t	Sig. (2-Tailed)
			Lower	Upper
Coordination	.49	.70	−.70	−.28	−4.67	.000 ***
Communication	.76	.84	−1.01	−.50	−5.98	.000 ***
Trust	.48	.69	−.69	−.27	−4.56	.000 ***

*** p < .001.

. It shows that the communication improved most dramatically. In addition, all the three means increased with statistical significance. It proves the participants were improved in the collaboration in the all three constructions significantly.

3.3. Learning Experience Opinion Investment- Quantitative

The LEOI investigated the participants learning experience with 35 quantitative and 4 qualitative questions. The quantitative questions and results are shown in

Table 8. The quantitative results of learning experience opinions investment (LEOI).

Item	Max	Min	Mean	Level	SD
2. I learned actively in the class.	5	2	3.92	m. h.	.84
3. I could concentrate in the class.	5	3	3.96	m. h.	.76
4. I was willing to participate in the activities the teacher wants us to carry out.	5	3	4.02	m. h.	.80
5. I think this course is very interesting.	5	1	3.81	m. h.	.83
6. I think my learning effect is good.	5	2	3.92	m. h.	.76
7. I think the overall learning effect in the class is good.	5	2	3.85	m. h.	.71
8. I am confident in my ability to learn in this course.	5	2	3.83	m. h.	.80
9. I often thought in class.	5	3	3.92	m. h.	.70
10. I listened carefully to the others.	5	3	4.29	h.	.61
11. I dared to express my opinion.	5	1	3.81	m. h.	.93
12. I could accept different opinions from my classmates.	5	3	4.31	h.	.62
13. I was willing to cooperate with others.	5	3	4.35	h.	.69
14. I was happy to share my thoughts or collected information.	5	2	4.15	m. h.	.82
15. When students encounter problems in their studies, I helped them solve them.	5	3	4.17	m. h.	.69
16. When I encountered problems, I actively sought help from my classmates.	5	3	4.15	m. h.	.74
17. I could concentrate on participating in group learning activities and do nothing else.	5	3	4.08	m. h.	.76
18. I often felt the support or encouragement from my classmates.	5	3	3.94	m. h.	.80
19. When encountering controversial issues, I could discuss matters and express my opinions without personal attacks.	5	3	4.23	h.	.71
20. When the group students had different opinions, I could coordinate everyone to reach a consensus.	5	2	3.85	m. h.	.79
21. When encountering controversial issues, I remained silent.	5	1	3.10	m.	.90
22. I was close to my classmates.	5	2	4.19	m. h.	.75
23. My interaction with the teacher was good.	5	2	3.73	m. h.	.78
24. I often felt the teacher’s concern for classmates.	5	3	4.17	m. h.	.72
25. I could understand the learning objectives of this course	5	2	4.15	m. h.	.68
26. I could understand the teacher’s assessment method.	5	2	3.98	m. h.	.85
27. I could understand what the teacher teaches.	5	2	4.13	m. h.	.70
28. I could grasp the unit progress and key points of this course.	5	2	3.98	m. h.	.72
29. I think the classroom environment and equipment were helpful for my study.	5	2	3.83	m. h.	.82
30. I think the discussion with the group members was helpful to my study in this course.	5	2	4.13	m. h.	.83
31. I think the sharing and feedback between different groups helped me in this course.	5	3	4.08	m. h.	.70
32. I think the ORID focus discussion method in the final homework of the ethics exploration was helpful to my study in this course.	5	2	4.08	m. h.	.79
33. I think that the critical thinking conundrum discussions was helpful to my learning in this course.	5	2	4.06	m. h.	.77
34. I think the activities that explain values helped me in this course.	5	3	4.25	h.	.60
35. I think the final project of the case study was helpful to my study in this course.	5	3	4.15	m. h.	.74
39. Overall, I feel this course improved my knowledge and understanding to cultivate positive, practical and in-depth moral qualities.	5	3	4.15	m. h.	.76

. The items were measured by five-point Likert-type scales from strongly disagreement (1 point) to strongly agree (5 points). The means were divided into five levels with four equal range of .8 as shown in

Table 9. The data of the five performance levels in the quantitative LEOI.

Performance Level	Symbol	Range	Number of Items	Percentage%
high	h.	X ≥ 4.2	5	14
medium high	m. h.	3.4 < X ≤ 4.2	29	83
medium	m.	2.6 < X ≤ 3.4	1	3
medium low	m. l.	1.8 < X ≤ 2.6	0	0
low	l.	X ≤ 1.8	0	0

. Forty-eight participants completed the survey in the last 3 weeks of the semester with 6 females and 42 males among the 59 registered students.

The numbers and percentages of the 35 items in medium, medium high, and high are 1 (3%), 29 (83%), and 5 (14%), respectively. However, the item with the lowest value (item 21, ranked as medium level) is not necessary to be negative when the value is low. This question was designed to complete the scenario to question 20 originally and became the only question with opposite score scale, the lower possibly the better. As a result, it could be claimed that all items are above medium. In addition, medium high is majority while level high is minority; meanwhile, the items below and equal to medium are trivial. Overall, the students’ quantitative learning opinions to this course could be leveled between medium high and high.

Meanwhile, the question 32, 33, 34, and 35 correlate the course materials in week 7–8, 10–12, 13–16, and 17–18, respectively, while question 39 correlates the whole course. In other words, the question 32 and 33 responded to the learning of critical thinking, 34 related to the values, and 35 was about final project (a CT and values oriented case study). It shows the participants highest respected the course materials of values, 4.25, with lowest appreciation about CT, 4.06–4.08. The mixing part of the final project was in the middle, 4.15, which is the same as the overall course. Overall, the five values are all higher than the average of the 35 quantitative questions, 4.02. It implies that the participants enjoyed the course materials and thought these CT and values infused course activities are helpful to the learning generally.

Next, the top and bottom 5 items of the quantitative LEOI are listed in

Table 10. The highest and lowest rank items of the quantitative LEOI.

Rank	Item	Category	Mean
Top 1	13. I was willing to cooperate with others.	Coordination	4.35
Top 2	12. I could accept different opinions from my classmates.	Open minded	4.31
Top 3	10. I listened carefully to the others.	Communication	4.29
Top 4	34. I think the activities that explain values (such as money, failure) helped me in this course.	Values	4.25
Top 5	19. When encountering controversial issues, I could discuss matters and express my opinions without personal attacks.	Communication	4.23
B * 5	29. I think the classroom environment and equipment were helpful for my study in this course.	Hardware	3.83
B 5	8. I am confident in my ability to learn in this course.	Learning	3.83
B 3	5. I think this course is very interesting.	Learning	3.81
B 3	11. I dared to express my opinion.	Communication	3.81
B 2	23. My interaction with the teacher was good.	Trust	3.73
B 1	21. When encountering controversial issues, I remained silent. **	communication	3.10

* B represents “bottom”. Here are 2 items with the same mean of 3.83 and ranked both the same, B5. ** This lower value of this question indicates the participants would participate into communication more actively and it is positive to communication.

. Based on the category, coordination is best, while communication is also higher than trust. It indicates that the quantitative LEOI responded the same ranks as the three intra-group constructions of the collaboration questionnaire.

3.4. Learning Experience Opinion Investment- Qualitative

The four LEOI qualitative questions are listed below.

• 1. Your gender.
• 36. Please write down a thing that brings difficulties to your study.
• 37. Write down a thing that you particularly enjoy in class.
• 38. So far, the overall reflections and suggestions to the course.

Based on the purpose of the study, these responses were analyzed based on the course purpose into the following subjects: collaboration (mainly including coordination, communication, trust) and critical thinking with some sub-categories, shown in

Table 11. Categorized responses in the qualitative question 36, difficulty to the course.

Category	Sub-Category	Number of Responses	Ratio	Sample Response
Coordination	Team work	2	.04	23 * Most of the team members did not participate when I was leading a table discussion.
	Timing	3	.06	5 There is less time to find everyone to discuss together, so cherish the time you can discuss in class!
Communication	Express myself	3	.06	30 It’s hard to express myself.
	Opinion integration	5	.10	96 It’s difficult to unify different opinions and sort them together.
Trust		2	.04	12 When there is a problem, I am embarrassed to ask my classmates and teachers around me.
CT	Too complex to think	8	.17	57 I can’t accommodate too many complicated thinking questions at once.
	Lack of thinking habit	9	.19	19 I am easily distracted while thinking.
	Unfamiliar to CT skills	5	.10	94 This course includes complex topics, and we need to find a lot of information with in-depth discussion.
Other issues		6	.13	9 Being lazy
No opinions		5	.10
Total		48	1

* The initial numbers indicate the respondents.

,

Table 12. Categorized responses in the qualitative question 37, enjoyment to the course.

Category	Sub-Category	Number of Response	Ratio	Sample Response
Coordination	Team work	4	.08	30 * We can come up with solutions to problems. It shows the teamwork is important.
	Team competition	1	.02	42 The whiteboard group competition is very interesting.
Communication	Reporting	2	.04	90 I enjoy the moment when I was reporting on stage.
	Debating	4	.08	43 Owing to debating, people attentively discuss the issues increasingly.
	Discussion	18	.38	52 I like to discuss issues with small groups and interact with people through democratic investigations.
	Listen to others	7	.15	96 When the teacher talks about topics I am interested in, it is my favorite time in class
CT	Thinking	1	.02	47 Enjoy thinking problem.
	Learning	4	.08	69 Learn different perspectives.
No opinions		7	.15
Total		48	1.0

* The initial numbers indicate the respondents.

,

Table 13. Categorized responses in the qualitative question 38, reflections and suggests to the course.

	Suggestion				Reflection
C 1	S 2	N 3	R 4	Sample Response	N	R	Sample Response
Coordination	T 5	5	.10	801 * It’s too frequent to change the groups	6	.13	73 Because of grouping, we have cohesion and I like it this way.
	Competition	-	-	-	3	.06	66 I think the whiteboard group competition is fun and learn in happiness, so we can learn things quickly.
Trust		1	.02	69 I think it’s actually good, but I’m still not good to integrate into the group.	-	-	-
CT	-	-	-		5	.10	5 In life, I don’t reflect on these matters, and I don’t care too much about these events, because they have nothing to do with us. When we really understand, they are related to the grievances in our lives!
Other s		7	.15	90 The classroom is too crowded for group discussion. 30 Maybe the final report is a bit rushed.	10	.21	96 Thank teacher for the great instruction. **
None		11	.23		-	-	-
Total		-	-	-	48	1

¹ Category; ² Sub−category; ³ Number of the responses; ⁴ Ratio; ⁵ Team work. * The initial numbers indicate the respondents. ** These 10 responses are all for appreciation.

and

Table 14. Summation of the LEOI qualitative questions.

	Difficulty and Suggestion			Enjoyment and Reflection			R 1
Category	Sub-category	N 2	Ratio	Sub-category	N	Ratio
Coordination	Team work	7	.13	Team work	10	.15	.14
	Timing	3	.05	Team competition	4	.06	.06
Communication	Express myself	3	.05	Reporting	2	.03	.04
	Opinion integration	5	.09	Debating	4	.06	.08
	-	-	-	Discussion	18	.28	.14
	-	-	-	Listen to others	7	.11	.05
Trust	-	3	.05	-	-	-	.03
CT	Too complex to think	8	.14	Thinking	1	.02	.08
	Lack of thinking habit	9	.16	Learning	4	.06	.11
	Unfamiliar to CT skills	5	.09	Reflective thinking	5	.08	.08
Other issues	-	13	.23	Appreciation	10	.15	.20
Total	-	56	1.00	-	65	1.00	1.00

¹ Ratio average of the ratios of “Difficulty and Suggestion” and “Enjoyment and Reflection” ² Number of responses.

.

3.4.1. Difficulties, Item 36

For the coordination, Table 11 shows there were 5 responses and 3 of them mentioned they were limited by the timing; therefore, it is important to discuss in class effectively. For the communication, there are two kinds of difficulties: hard to express themselves and then assemble respective thoughts. There are three difficulties about critical thinking for the participants: too complex, lack of thinking habit, and unfamiliar to CT skills.

For the collaboration, 10%, 16%, and 4% mentioned the difficulties to coordination, communication, and trust, respectively. It shows the participants most sensible to their communication problems in the course learning process for the collaboration. The most significant problem is for the critical thinking. 46% (.19, .1, and .13 for too complex to think, lack of thinking habit, and unfamiliar to CT skills, respectively) is referring to the difficulties to critical thinking. About one over five participants were lack the habit to think, and therefore easily out of focus or distracted while studying. Seventeen percent of participants were afraid of complex questions to think. Additionally, 10% doubted if their CT skills, like finding data, thinking or discussion in depth, could deal with complex engineering ethics issues. Finally, 10% had no opinions while 13% relates to other less focused problems.

3.4.2. Enjoyment, Item 37

For the coordination, Table 12 shows that four responses are referring to team work while one is about team competition. Meanwhile, there were four kinds of activities to cause the communication the most enjoyable in the class: discussion, debating, reporting, and listening to others. For critical thinking, there was one response relating to the enjoyment of thinking and 4 replies that they were glad to learn different perspectives.

Overall, 10% participants replied that they prefer coordinative activities like either the teamwork or group competition. Then the majority of 65% participants enjoyed the communication process in the collaboration including reporting, debating, discussion, and listening to others. Only 10% addressed that they enjoyed the critical thinking like thinking problems and learning different perspectives. This value, 10%, is less than 15% with no opinions in this question. It indicates only a few people enjoyed CT.

3.4.3. Reflections and Suggests, Item 38

Participants could freely respond either personal reflections or suggests to the course in the question 38. Table 13 categorized the opinions according to coordination, trust, CT and other issues. No one referred to communication which had drew much attention in the previous two questions.

For coordination, 5 participants suggested less re-grouping while 6 people reflected their agreement to the team cohesion. In addition, 3 people thought that the team competition was enjoyable and effective to learning. For trust, one participant replied he was not used to integrate into the group. For CT, five people reflected that this course made them to think in-depth and associate some ethical issues with their daily life, which had not been cared before the class.

3.4.4. Summation of the Qualitative Questions

All of the IEOL qualitative questions, except for the responses of no opinions, were summarized in Table 14.

First of all, the collaboration was most significant to draw totally 53% attention with the ratios of .2, .41, and .03 for coordination, communication, and trust, respectively. In addition, it was quite balanced referring to coordination in the two sides, the summation ratios of .18 and .21 for difficulty and suggestion and enjoyment and refection, respectively. Nevertheless, the category of communication was more leaning to positive side, .14 to .45. On the opposite hand, the issue about trust was only mentioned in the negative side with a tiny ratio, .05.

The second category of CT drew totally 27% attention while the rest of the other issues were 20%. Table 14 shows the learning experience for CT was quite unbalanced also. Thirty-nine percent of participants considered CT difficult with different reasons while 16% enjoyed the moment of CT learning.

4. Findings

This study demonstrated an action research using critical thinking concept to design the curriculum and collaborative learning process to implement the class activities for the engineering ethics course. In general, the participants made progress both in CT and collaboration. The overall results are summarized in Section 4.1, Section 4.2 and Section 4.3.

4.1. Research Questions I

Will students improve collaboration quality by collaborative learning?

The major findings by the comparisons of the 3 constructions: coordination, communication and trust, in the collaboration questionnaire are the following:

• All the 3 means in the posttest increased significant statistically.
• Both in the pretest and posttest, the intra-group coordination was weakest with more divergent opinions; while the trust was best with highly agreement.
• The intra-group communication had been improved most.

4.2. Research Questions II

Will students improve CT disposition by collaborative learning?

The major findings by the comparisons of the 20 questions in the ICTD are the following:

• The participants made a greatest improvement in the item of “I try to determine whether my views are sufficiently convincing by self-questioning,” in the construction of Reflective thinking. However, the posttest mean was still lower than the average of the all posttest items though
• On the opposite, the item of “In the context of the discussion, I try to respect the opinions of others,” in the construction of Open-mindedness, regressed the most. The posttest mean was still higher than the average of the all posttest items though
• The responses were mostly lower and more discrete in the pretest and became more convergent to generally higher posttest values.
• The participants demonstrated the improvements among the most items of the critical thinking dispositions. 18 items increased while 2 decreased. Two items (#10 and 15) among the 18 items improved statistically significantly.

The major findings by the comparisons of the 4 constructions, systematicity and analyticity, open-mindedness, inquisitiveness, and reflective thinking, in the ICTD are the following:

• “Systematicity and analyticity” improved most dramatically while the other three constructions and entirety increased too.
• The intra-group “systematicity and analyticity” and the entirety also increased with statistical significance.
• The four means in the pretest were all lower than the college norms.
• The posttest means of “inquisitiveness” and “Reflective thinking” are higher than the norms, while the other two constructions of “systematicity and analyticity” and “open-minded” are still lower than the norms, and though they are better than their own pretest.

4.3. Research Questions III

How will students cope with collaborative learning in the course progress of CT based curriculum design?

The major findings in the LEOI quantitative part are the following:

• There are no items below medium level among the 35 questions. It indicates that the participants coped well with the designed curriculum.
• The level of medium high (83%) is majority while level high (14%) is minority. Others are trivial. It indicates medium high to high level satisfaction to the course.
• In general, the participants enjoyed the course materials and thought these CT and values infused course activities are helpful to the learning.
• The participants ranked coordination highest, communication next, and trust last. It is the same ranking result to the responses in the collaboration questionnaire.

The major findings in the LEOI qualitative part about the learning difficulties are the following:

• The participants were most sensible to their communication problems in the course learning process during collaborative activities.
• CT is the most noticeable problem for the participants. About one out of every five participants lacked the habit to think, while 17% of participants were afraid of complex questions to think. In addition, 10% doubted their CT skills. It concludes that a total of 46% participants thought CT is their most crucial shortage.

The major findings in the LEOI qualitative part about the learning enjoyments are the following:

• Ten percent of participants preferred coordinative activities, while the majority of 65% participants enjoyed the communication process. Again, communication skills are most sensible to the participant’s drawbacks or advantages in the collaboration.
• Compared with that the 46% participants thought CT is their most crucial shortage, only 10% addressed that they enjoyed the critical thinking. It again demonstrated that CT was quite desperate for a lot of the participants.

The major findings in the overall LEOI qualitative part are analyzed below:

• The collaboration was most significant to draw totally 54% attention with the ratios of .2, .31, and .03 for coordination, communication, and trust, respectively.
• The coordination was quite balanced on the positive and negative sides, 9% and 10.5%, respectively.
• The most noticeable category of the communication was more leaning to positive side, 7% to 24%.
• The trust was only mentioned in the negative side with a tiny ratio, 3%.
• CT drew totally 27% attention and leaning to negative learning experience.
• Nineteen percent of participants considered CT difficult, while 8% enjoyed the moment of CT learning.

5. Discussion of the Findings

Collaborative learning has been proved effectively to enhance the achievement of the participants in mathematics [16], or CT [17,21]; however, the quality of the collaboration after collaborative learning was less concerned directly. This study demonstrated a proper designed curriculum based on collaborative learning is capable to enhance the learners’ collaboration levels in all the three constructions of coordination, communication and trust. Moreover, the means of intra-group coordination and trust are lowest and highest, respectively, in both the pretest and posttest; while the means of the intra-group communication makes the greatest improvement between the pretest and posttest. These are in agreement with the previous research results [29].

Many studies have statistically verified that collaborative learning is more effective to improve CT than individual learning [17,21]. This study also verified the CT disposition of the four constructions, systematicity and analyticity, open-mindedness, inquisitiveness, and reflective thinking, were all improved; while the intra-group “systematicity and analyticity” and the entirety also increased with statistical significance. This agrees with the previous learning theories proposed by the promoters of collaborative learning that the development of critical thinking improved through discussion, clarification of ideas, and evaluation of others’ ideas [21].

However, collaborative learning is not always working to enhance thinking critically. Le proved that knowledge is given from teachers to learners and collaborative learning does not play an important role [24]. It indicates the quality of the interactions among the learners and educators would be a key issue. Moreover, the instructor’s role is not to transmit information, but to serve as a facilitator for learning [21].

Finally, to compare the statistical results of our participants in the performance of CT disposition, the means of the four constructions in the pretest were all lower than the college norms evaluated by Yeh [39]. Fortunately, the posttest means of “inquisitiveness” and “Reflective thinking” are higher than the college norms, while the other two constructions of “systematicity and analyticity” and “open-minded” are still lower than the norms, and though they are better than their own pretest. In addition, the means of “open-minded” in both the pretest and posttest are highest among the four constructors; while the mean of “Inquisitiveness” in the posttest is lowest. These trends are consistent with the previous research too, with a tiny difference that “Systematicity and analyticity” was lowest and “Inquisitiveness” was the second lowest. Overall, it indicates that the participants are more confident on the attitude of “open-minded” while less capable to handle thinking “systematically and analytically” or “Inquisitively.” Meanwhile, it agrees with the LEOI qualitative findings that thinking critically was the most noticeable issue in the progress of the course, with 46% participants treating CT as the most crucial shortage while 10% enjoyed it.

6. Conclusions and Suggestions

Literature shows that CT is important but challenging to teach [1]. It is even worse that some employers believe that the recent engineering graduates are less able to think critically and independently. Some engineering professors tried very hard to implant CT into their standalone course and figured lots of difficulties as addressed in the introduction [13,15]. Based on their experience, 4 strategies were implemented into the engineering ethics course: 18 weeks of CT infusion, PBL in the final summative assessment, collaborative learning, and the 3 CT aspects curriculum design [5,6,7,8,9,10,11].

For successful collaborative learning, lots of course management is necessary. Just like that Le has proved that the quality of the interactions among the learners and educators would be significant [24]. This study particularly made some efforts to assist the collaboration including regrouping twice, peer evaluation to their team members in each grouping and finally abundant collaborative practice by team activities. The quantitative results show that the collaboration was improved by collaborative learning, including all the three constructions of collaboration in the intra-group coordination, communication and trust. In addition, the means of intra-group coordination and trust were lowest and highest, respectively; while the means of the intra-group communication makes the greatest improvement between the pretest and posttest. These are in agreement with the previous research results [29]. Next, the qualitative results indicate that communication were more sensible for the students while trust drew least attention. Therefore, more efforts put in the communication related activities are recommended for the future course design for immediate effects. Meanwhile, trust should not be ignored for a long term collaborative benefits. Finally, the participant’s statistics showed that coordination is controversial to either cause teamwork problems or achievements.

The 3 CT aspects of knowledge, disposition, and skills were integrated and evaluated in the learning process. Most participants considered CT complicated and less easily observed in their behavior, owing to lack of the habit to think, easily distracted, and unfamiliar to CT skills. As a result, almost one in every five students reported they were easily distracted while studying. Forty-seven percent of the learning difficulties were about CT skills. This quite coincides with Cooney’s research results and some employers’ opinions [13]. Fortunately, the ICTD in the pretest, which was below the college students’ norm, increased significantly and was higher than the norm in the posttest.

The major goal of this study is to improve CT by mainly the support of the collaborative learning process. It should be aware that the participants were below the college norm [39] in the pretest and it indicates that it is necessary to improve their CT abilities. Their resistance was also shown in the qualitative responses; however, the assistance of the social interaction, including of plenty teamwork practices, peer evaluation, and pressure to push individuals to work harder and think deeper and broader, did promote their CT cognitive development. Finally, their CT quiz, final projects, and ICDT also demonstrated the positive progress after 18 weeks’ participation in the engineering ethics course.