Developing Critical Thinking in Technical and Vocational Education and Training

Abstract

Critical thinking has been difficult to develop in technical and vocational education and training, where acquiring practical skills is often the priority. This study looks at whether tried-and-tested methods for developing critical thinking in higher education are also effective in this educational context. To test this, an intervention was carried out as part of a compulsory, semester-long “Basic Communication Skills” class for 149 first year engineering students. This involved linking the expected learning outcomes for the course to a series of sub-skills comprising an updated definition of critical thinking. Furthermore, a set of strategies promoting active participation among students was also implemented. The proposed methodology led to improved levels of critical thinking when compared to traditional teaching methods. It was found that lessons characterized as interactive, dynamic, and encouraging active student participation facilitate the teacher’s job in the classroom and improve the development of critical thinking in a specific technical and vocational education context.

1. Introduction

Technical and vocational education and training (TVET) refers to the education, training, and skills development required for specific occupational fields [1]. The automation of many manufacturing and production processes has put TVET in the spotlight [2]. In response to this issue, a recent UNESCO report has acknowledged the need to develop digital, environmental, entrepreneurial, and soft skills among workers to prepare them for the future of work [3]. In this sense, 21st Century Skills are seen as one possible solution for TVET [4].

21st Century Skills can be defined as a wide-ranging set of abilities, including creativity, communication, collaboration, and critical thinking, among others [5]. Critical thinking is acknowledged as being an important goal for many professionals within higher education [6]. This is because it is an important tool for facing different challenges that may arise in the workplace. This includes the ability to identify which knowledge may be suitable for a certain problem and how to apply it as part of the solution [7]. However, critical thinking has been difficult to develop in technical and vocational education and training, where acquiring practical skills is often the priority [8]. This is mainly because of a focus on developing competences and skills that directly help solve the problems that workers will face when they join the workforce. This, in turn, has created a skills gap which must be addressed [9]. Developing critical thinking in TVET is one way of reducing such gaps.

Critical thinking development in TVET is novel. Among other 21st century skills, critical thinking has been recognized as being one of the most desirable attributes within industry (Din Nugraha et al., 2020). There is some evidence regarding activities that can develop higher-order thinking skills in general [10], as well as general guidelines on how to teach critical thinking and integrate it into pedagogical practices in TVET [11,12]. However, there is still a need to design interventions that can develop this skill [9].

To do so, critical thinking must first be defined. There are many approaches to defining this skill in the literature. One such approach claims that critical thinking can be seen as the process of analyzing and evaluating the thinking process in order to improve it [13]. Others claim it is a skill related to solving problems in different contexts [14]. Through the Delphi Report, Facione defined critical thinking as a set of sub-skills, namely interpretation, analysis, evaluation, inference, explanation, and self-regulation [15]. This is the definition that has been adopted for our study. This is because in critical thinking, as in any problem-solving situation, you first need to have a deep understanding of a problem before thinking about the solution. Failing to do so would lead to a surface-level solution that does not require critical thinking [16]. Facione’s definition therefore proposes a taxonomy of sub-skills, requiring a person to first understand a problem (interpretation and analysis) before assessing solutions (evaluation). They must then search for non-explicit causes or consequences (inference) and communicate this process as a whole (explanation) before reflecting on the process and results (self-regulation). Such a process leaves no room for a superficial solution, which is precisely the problem facing today’s TVET workforce [9]. In this sense, the other approaches to critical thinking do not seem as appropriate as they may lead to superficial thinking. In doing so, subjects may never develop the sort of deeper thinking and reasoning that would allow them to become more integral workers [17].

Facione’s approach to defining critical thinking not only provides benefits for TVET students in their work-related activities; it also benefits other aspects of their life. It can help improve their comprehension of media information and critical reasoning [18]. It can also help facilitate authentic dialogues and mediation between different worldviews [19]. Consequently, critical thinking can help people better understand and evaluate data so as to make better real-life decisions [17]. In consequence, critical thinking can also be key in developing citizens for the 21st century [20].

The above definition provides us with a clear understanding of critical thinking and how this approach can lead to deeper thinking. This definition will be operationalized in Section 2.1. Now, it must be understood how critical thinking can be developed. Therefore, there needs to be an analysis on how to develop critical thinking in a TVET context and understand which activities may foster the development of this skill. Dialogue, exposing students to real-life situations, and mentoring have all proven to be effective strategies for developing critical thinking [6]. Setting out a problem, activating prior knowledge, demonstrating, applying, and integrating have also been shown to promote this skill [21]. Furthermore, active learning methodologies, teacher training, and student support have all been shown to be essential for developing critical thinking [22,23,24]. However, all of these activities have been proven in a general higher education context, but not in TVET.

Some more recent activities for developing critical thinking include promoting critical dialogue between students, facilitating metacognitive processes, using guided and practical models, and giving ongoing formative feedback, among others [25]. Some other effective activities include providing oral and written reflections; argumentation; reading, analyzing, and summarizing texts; and case studies [26].

These references, as well as a review of the literature, suggest how to develop critical thinking at every level of education, except for TVET [23]. This is particularly relevant given that the nature of TVET is different to traditional higher education, as was shown before [27]. TVET tends to attract students from lower-income families and with lower levels of motivation [28]. Furthermore, TVET is often seen as a fallback option [29] and can result in low employability [30]. This therefore raises the question of whether the experiences, activities, and methodologies that have been proven to develop critical thinking in higher education are also applicable to TVET. Consequently, our research question asks: “How can critical thinking be developed within technical and vocational education and training?”.

2. Materials and Methods

2.1. Operationalization of Definition of Critical Thinking

The chosen definition of critical thinking proposes it as a taxonomy of the following sub-skills: interpretation, analysis, evaluation, inference, explanation, and self-regulation.

Interpretation is the ability to extract information from a wide range of situations. Analysis is the ability to identify implicit and explicit relationships between written and audiovisual resources. Evaluation is the ability to evaluate judgements and develop rubrics in order to do so. Inference is the ability to draw conclusions. Explanation is the ability to justify one’s reasoning. Finally, self-regulation is the ability to consciously self-monitor one’s cognitive processes.

These sub-skills fail to consider a person’s argumentation skills [26], i.e., the ability to establish a position and provide arguments [31]. Evidence, in the form of an argument, can be used to support an explanation [31]. Therefore, explanation was replaced with argumentation in our definition of critical thinking.

Furthermore, metacognition is defined as the ability to conceptualize the cognitive process of oneself or of others [32]. This includes the planning, monitoring, and evaluation of said process [25]. Furthermore, self-regulation is considered an act of cognitive self-monitoring [33]. Therefore, self-monitoring was replaced with metacognition in our definition of critical thinking.

Consequently, the definition of critical thinking used in this study comprises the sub-skills interpretation, analysis, evaluation, inference, argumentation, and metacognition. It is worth noting that this definition has been used consistently for critical thinking development experiences [16,23,25,34]. This therefore suggests that the definition is also suitable for a development context. Now, the difference between sub-skills and dispositions will be analyzed. The Delphi Report from Facione not only proposes critical thinking as a composition of sub-skills but also a set of dispositions [15]. Truth-seeking, open-mindedness, analyticity, systematicity, self-confidence, inquisitiveness, and cognitive maturity are mentioned as cognitive and motivational dispositions that may promote the development of critical thinking [35]. Critical thinking sub-skills and dispositions are needed to become a critical thinker, with the link suggesting which attitudes are required for a person to develop critical thinking [16]. In this sense, these dispositions were addressed in our study by including active engagement as a basis for the critical thinking development methodology. Guidelines were included for areas such as peer discussion, higher-order thinking, and working with real-world topics, among others.

2.2. Experimental Design

The study was conducted at Chile’s second largest technical and vocational education and training college. The intervention was carried out as part of a compulsory, semester-long “Basic Communication Skills” class for first year students enrolled at the college. This course aims to develop the students’ writing and speaking skills, as well as their reading comprehension, taking a pragmatic and functional approach to the study of language. The study took place over the course of one semester (18 weeks) and involved ten different groups of students from the Electrical and Industrial Automation degree (149 students in total). The students were randomly assigned to experimental or control groups (see

Table 1. Sample Characteristics.

Group	Number of Students
Control	70
Experimental	79

). All groups followed the same course plan and were taught by the same teacher. However, the teacher used different methodologies for the experimental and control groups.

The course “Basic Communication Skills” was chosen mainly because communication is considered a 21st Century Skill. Furthermore, TVET is characterized by the teaching of content and techniques that are specific to a particular domain [1]. Therefore, students in TVET must be trained in the adequate use of language [36]. More importantly, and on a more general level, communication is key to transmitting information. This is done by ensuring that the message is effectively expressed by taking the audience into consideration. Consequently, this allows people to regulate their own needs and goals and align them with those of society [37]. Communication is, therefore, how TVET students will interact with the rest of the world. Furthermore, if critical thinking is to be a tool to help people understand the world better, then language is key [38].

2.3. Creating a Methodology for Developing Critical Thinking

Following an immersive approach, students can acquire critical thinking skills while developing subject content knowledge. In this case, there is no need for explicit instruction of critical thinking [39]. As part of this approach, learning is fostered when students work on solving real-world problems, when prior knowledge is activated as the basis for acquiring new knowledge, when new knowledge is demonstrated to students, when students can apply new knowledge, and when new knowledge is integrated into the students’ own world [40]. Therefore, the challenge was to determine how to build a methodology for developing critical thinking while also achieving the expected learning outcomes of the “Basic Communication Skills” course. To do so, the learning outcomes for the course were associated with the critical thinking sub-skills.

Table 2. Relationships between learning objectives and the sub-skills of critical thinking.

Sub-Skill of Critical Thinking	Expected Learning Outcome
Interpretation	Objectives associated with the extraction of literal information from written and spoken texts.
Analysis	Objectives associated with summarizing ideas and organizing information from written and spoken texts hierarchically.
Inference	Objectives associated with extracting non-literal information from written and spoken discourse.
Evaluation	Objectives associated with grammar (accent marks, punctuation, and connectors, among others) and the structure of a text.
Argumentation	Objectives associated with identifying an author or speaker’s point of view and their reasoning.
Metacognition	Objectives associated with self-regulation and monitoring during the process of speaking, writing, and listening/reading comprehension.

shows the links between the expected learning outcomes for the course and the six sub-skills comprising the definition of critical thinking.

Several studies have suggested that student engagement is a key factor in the learning process, associated with a higher probability of academic success and lower levels of student attrition [41]. It has been shown to positively affect critical thinking, self-efficacy, and communication skills [42] as it requires autonomy and active participation from the students [43].

Motivation is understood as “the mental state in which students find themselves while learning, representing the intersection between thoughts and feelings” [44]. Furthermore, student engagement is directly correlated with student motivation and active learning [44]. The concept of student engagement will therefore be taken as a student’s active participation in their own learning process. Furthermore, the use of technology was also included, as it can have a direct impact on student motivation [45]. This was accomplished using Plickers, which will be explained later.

Strategies to demonstrate critical judgement, examine statements, and determine the robustness of an argument [46] were looked at to foster active student engagement, the development of critical thinking, and the learning of subject content knowledge. Therefore, the following guidelines for student-centered learning were set out:

• Use of multimedia in class: Allow students to work with different types of discourse, whether text, videos, images, audio, animations, or others. These are tools that help students develop concepts, analytical reasoning, creative thinking, problem solving and critical thinking [47]. Such resources should be brief and concrete so that the students do not become distracted.
• Working with controversial or real-world topics: To boost student interest and engagement, there must be strong social relationships between the teacher and students, as well as suitable rules for interaction and the facilitation of debate [48]. This relationship can be strengthened in class by addressing topics that are controversial or from real-life situations, as there is evidence suggesting that a real-world connection can be an effective methodology for developing critical thinking [26]. Controversial topics were proposed because exposing students to opportunities for dialogue is also an effective way of developing critical thinking [34]. Even though it can polarize students, it is also an opportunity for them to consider alternative perspectives [49], which may also be directly related to the evaluation sub-skill in or definition of critical thinking.
• Introducing peer discussion routines: Peer discussion promotes the development of critical thinking [50].
• Reflective questions and higher-order thinking: There is a need for reflective questions, which go beyond basic recall and instead promote metacognition on current topics [51].
• Using low-cost technology: Simple, open-access technology was used to promote student participation. The class poll system Plickers was used [52], where students answer using a preprinted QR code that is scanned by the teacher’s cellphone.

Based on these general guidelines, a methodology was proposed with the following objectives in mind:

(1)

To be a dynamic class, in which the student frequently gets to “do” things.

(2)

To have clearly defined and distinct phases: presentation of content, practice, and a conclusion involving a metacognitive activity to reflect on the learning process.

(3)

To regularly ask questions about one of the sub-skills of critical thinking using Plickers.

These guidelines and objectives provided the following general structure for the 20 classes:

• Icebreaker or “Do it now!”: A short activity to introduce the main topic of the class in no more than 5 min. Icebreakers are an effective way of boosting student engagement [53] as they improve enthusiasm and help get the attention of the class [54].
• Sharing the lesson objective: A short activity of no more than 2 min to explain the expected learning outcomes for the lesson.
• Presentation of content: In no more than 5 min, the content is presented to the class.
• Practice: Practical exercises to consolidate the newly-acquired knowledge and relate it to one of the sub-skills of critical thinking based on different texts, images, or videos, in groups or individually. After each practice, a Plickers activity is performed based on the content and the aforementioned sub-skill.
• Focus on grammar: A short section of no longer than 8 min focusing on grammar. The main aim is to measure the sub-skill of evaluation and provide space for teaching grammar, a core element of the course.
• Turn and discuss: A section for talking about a controversial point from the class. The students are asked an open-ended question and must then reflect on it both individually and with a peer.
• Metacognitive routine: A concluding section in which the students go through a metacognitive routine based on an activity or item of content from the class.

While the experimental groups were taught following the methodology described above, the control groups were taught from the coursebook provided by the institution and without any additional materials. The approach for the control group was based heavily on reading texts and drilling reading comprehension, listening comprehension, writing, and speaking. The classes were complemented by the use of videos and infographics, with a particular focus on levelling out the students’ reading comprehension skills.

An example of a class following the proposed methodology can be found in Supplementary Material S1.

2.4. Instruments for Measuring Critical Thinking and Learning

A wide range of critical thinking tests is described in the literature. A list of 18 instruments that can be used to assess critical thinking can be found in Appendix A. All of these instruments have different types of questions, are of different lengths, and are based on different constructs. The instruments include information about the context in which they were validated and general characteristics of the subjects that answered them. This is particularly relevant, as each time an instrument is validated with a different sample population, it should be revalidated [55]. Moreover, everything within the context in which an instrument is validated can be defined as being culturally relevant in an instrument for assessing critical thinking [56]. However, culture is often ignored as being a relevant factor in such tests. This is because, as a skill, it cannot really be measured independently of its cultural context [57]. Therefore, when existing critical thinking assessments were analyzed for suitability for our subjects, it was decided that the cultural elements of each test would interfere in its comprehension and probably in the results. This is why we decided to create our own instruments instead.

Consequently, pre- and post-tests were developed to measure critical thinking. These were based on the theorical definition of critical thinking presented in Section 1 and its operationalization (Section 2.1). The difference in the scores on both tests was used to assess improvements in the students’ critical thinking skills. Although equivalent, the two tests were not identical. This was to avoid students learning from the test and/or remembered elements that may then affect the results [58].

Both tests used the definition of critical thinking established above as a theoretical construct. In this sense, each question on the two tests focused on one of the sub-skills included in the definition of critical thinking: interpretation, analysis, inference, evaluation, argumentation, and metacognition.

Furthermore, the tests included both multiple-choice and open-ended questions. The questions related to higher-order thinking skills (from a taxonomical point of view), i.e., argumentation and metacognition, were open-ended. This is because open-ended questions are better at measuring these kinds of skills than multiple-choice [59]. The responses to these questions were corrected by an expert and given a dichotomous score. The questions related to lower-order thinking (i.e., interpretation, analysis, inference, and evaluation) were expressed as multiple-choice [60] and also given dichotomous scores. Some questions, especially those for higher-order thinking skills, were separated into multiple items. The aim of doing so was to convey their complexity while keeping item scores between 0 and 1. Appendix B includes a detailed breakdown of the items on the test, including the sub-skill they relate to as well as the item type. As an example, question 15, which was related to the sub-skill of argumentation, consisted of asking subjects to write a short essay on a specific topic, including two arguments and one counterargument. This question was separated into five items, which were scored dichotomously (0 or 1). The first item checked whether subjects referred to the topic stated in the question. The second item checked whether they included a thesis statement. The third item checked whether they included an argument that supported this thesis. The fourth item checked whether they included another argument that supported their thesis (different from the last one). The final item checked whether they included a counterargument correctly. All questions (regardless of their type) were therefore given a score of 0 or 1.

The questions were based on a range of resources (such as advertisements, news articles, micro stories, opinion columns, and infographics, among others). These resources were based on real-life situations and problems, which provided a suitable context for evaluating the students’ level of critical thinking [61]. For example, for the sub-skill of metacognition, all of the questions on the pre- and post-tests were based on a 30-s advertisement. Specifically, the questions addressed culturally relevant issues for the corresponding students. In Supplementary Material S2, we find, for example, question IC16 (from the pre-test) and question IC17 (from the post-test), which cope with a story that connects directly to the students’ reality.

Details of each test, item, critical thinking sub-skill related to each item, question type, and type of resource used can also be found in Appendix B. To compare the scores on the pre- and post-tests, the total score for each test are expressed as a percentage.

The pre- and post-tests were validated on a total sample of 774 students. The pre-test was taken by a total of 502 students, while the post-test was taken by 274 students. These 774 students differ from the 149 specified in Section 2.2 because the validation of the critical thinking assessment required a larger sample [62]. These 774 students are all part of the same TVET institution where the study took place and have similar characteristics as the subjects from the sample specified in Section 2.1.

Additionally, and throughout the course, students had to complete several assessments, including essay questions, multiple-choice quizzes, and roleplays, among others. The aim of these assessments was to measure learning of the initial learning outcomes from the course. The final grade for the course, called “course score”, was the only one of these assessments that was taken into consideration for this study. It used a scale from 1 to 7, commonly used in the Chilean education system. It was included to assess a possible relationship between critical thinking and academic achievement, a common relationship in other educational contexts [63]. This is relevant for the research question as the existence of this relationship may be a direct consequence of the development of critical thinking and, more importantly, assesses whether the methodology also allows the achievement of the learning outcomes. If not, and considering the context in which it was created (TVET education), it may fail as a methodology if it does not allow the obtention of the learning outcomes, regardless of whether critical thinking is developed or not.

2.5. Teacher Surveys

As a course requirement and as requested by the college, all of the students had to fill out an end-of-course survey. This survey featured 16 questions and a space for students to leave their comments. The first 15 questions were based on a four-point Likert scale, ranging from “Strongly disagree” to “Strongly agree”. Question 16 required students to grade their teacher on a scale of 1 to 7. Finally, after these 16 questions, the students were provided with a space to leave their comments. The full survey can be found in Supplementary Material S3. This survey was applied as part of a teacher improvement process that is conducted every six months across all courses. While it is a requirement for every course, not all students were required to respond. In this case, the survey was built by the department in charge of all Basic Communication courses within the college. The results from the survey were included in order to have a quantitative and qualitative appraisal of the proposed methodology for developing critical thinking. Quantitatively, to analyze whether there are significant differences in any of the first 16 questions between the experimental and control group. Qualitatively, and by an appraisal analysis, to assess which elements of the methodology are more important for developing critical thinking.

2.6. Data Analysis

2.6.1. Quantitative Analysis

The reliability of the tests was analyzed using Cronbach’s alpha. Item difficulty was measured using the p-value for each question. Item discrimination was measured using item-total correlation, specifically an uncorrected point-biserial, following a Classical Test Theory Approach [64].

The internal structure of the pre- and post-tests was analyzed to demonstrate that they are equivalent. Both tests have the same construct (i.e., they are built around the same definition of critical thinking). Therefore, if they both show a unidimensional model (critical thinking), they can be considered equivalent [65]. Information-based model fit statistics [66] were used to show that a unidimensional model explains the internal structure. This model was based on Item Response Theory, where the dimension in question is critical thinking. The tests can therefore be considered equivalent. This was achieved by first analyzing whether the data can be used to identify factors using the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy [67], as well as Bartlett’s test of sphericity to analyze the factorability of the correlation matrix [68]. The internal structure is then validated using the root mean square error of approximation (RMSEA) [62].

This analysis was performed using the results from the 774 students who participated in the validation of pre- and post-tests.

Bayesian linear regression modeling [69] was proposed for identifying the association between the response variable (post-test) and the available independent variables (pre-test, group, and course score). This analysis was performed using the results from the 149 subjects enrolled in the “Basic Communication Course”. As the sample size is relatively small (149 subjects in total, with 70 belonging to the control group and 79 to the experimental group), the Bayesian approach works more robustly than the frequentist approach [70]. Another advantage of the Bayesian framework is that uncertainty is measured probabilistically without resorting to a test statistic (or p-value) or asymptotic approximations that depend on the sample size [71]. Mathematically, the proposed model is written as follows (see Equation (1)):

$$Y_i=X_i^{\top }{\beta }_i+{ϵ}_i,$$

(1)

where $Y_i$ represents the post-test score of the ith student and ${\mathit{X}}_i$ is their covariate vector with coefficients $\mathit{\beta }$. The error term is denoted by ${ϵ}_i$ and follows a Normal (0, ${\sigma }^2$) distribution. The prior distributions are non-informatively assigned [69] according to the default specification of the brms R-package [72]. The coefficients for the independent variables used in this regression can be found in

Table 3. Bayesian linear regression coefficients.

Coefficient	Reference	Description
${\beta }_0$	Intercept	Attributable to the regression model
${\beta }_1$	Pre-Test	Decimal number between 0 and 100, representing the student’s score on the pre-test
${\beta }_2$	Group	Student’s group. 0 for control and 1 for experimental
${\beta }_3$	Course score	Student’s final grade for the course, expressed as a decimal number between 0 and 100

.

All analyses were performed in R [73].

The data from the teacher survey were analyzed as aggregated data. Welch’s t-test [74] was used to analyze any potentially significant differences in the mean scores between the control group and experimental group. The effect size of any significant differences was also calculated. Any significant differences between the groups can shed light on which elements from the methodology may have been more successful in developing critical thinking.

2.6.2. Qualitative Analysis

The results from the survey were included in order to have a qualitative appraisal of the proposed methodology for developing critical thinking. More specifically, there was an interest in assessing which elements of the methodology are more important for developing critical thinking.

In order to assess which elements of the methodology are more important for developing critical thinking, only 32 comments (17 experimental and 15 control group) were received on the teacher survey, considering that the completion of it was voluntary. This data was studied using Discourse Analysis, specifically Appraisal Analysis, which falls under the umbrella of Systemic Functional Linguistics (SFL). This theory allows construction and interpretation of meaning within its social context [75], thus allowing the discovery of relevant elements from the data. Three aspects were considered, Attitude, Engagement, and Graduation, as explained in Appendix C.

3. Results

3.1. Instrument Validation

Item difficulty and discrimination were used to validate the pre- and post-tests. These values and the items that were eliminated from each test can be found in Appendix D. Any questions with a difficulty index outside the range of 0.1–0.9 were removed [62]; three from the pre-test (leaving 29 questions) and two from the post-test (leaving 32 questions). Then, the internal structure of the instruments was analyzed to determine their equivalence. Since the RMSEA threshold of 0.08 was met with a 90% confidence interval (

Table 4. Analysis of the Internal Structure of the Instruments.

Instrument	KMO	Barttlet’s Test of Sphericity	Log-Likelihood	M2	p-Value	df	RMSEA 5% *	RMSEA	RMSEA 95% **
Pre-Test	0.65	p < 0.001	−8.507	810	<0.01	348	0.047	0.051	0.056
Post-Test	0.67	p < 0.001	−5.038	888	<0.01	432	0.056	0.062	0.068

* Lower limit for the RMSEA with a 90% confidence interval ** Upper limit for the RMSEA with a 90% confidence interval.

), we can state that a unidimensional model based on our proposed definition of critical thinking explains the structure and that, consequently, both instruments are equivalent [55,76].

Finally, Cronbach’s alpha can be used to analyze the reliability of both instruments in order to validate them. Cronbach’s alpha for the pre-test is greater than 0.6 (

Table 5. Reliability of the instruments.

Test	Cronbach’s Alpha	Marginal Reliability
Pre-Test	0.67	0.76
Post-Test	0.72	0.87

). Having conducted an IRT analysis, this validation can be complemented using marginal reliability based on an Expected a Posteriori (EAP) estimate [77], which is greater than 0.7 (Table 5). Both of these values are acceptable for a low-stakes learning outcome [62]. In the case of the post-test, Cronbach’s alpha is greater than 0.7, which is the threshold usually found in the literature [76,78], with a marginal reliability also greater than 0.7, thus demonstrating the validity of both instruments.

3.2. Differences between Pre- and Post-Tests

The results from the pre- and post-tests, as well as the course score for each group (control and experimental), can be found in

Table 6. Descriptive statistics by group.

Group	Type	N	Mean	Std. Dev.	Median	Min	Max	Skew	Kurtosis	Std. Error
Control	Pre-Test	70	60.25	14.55	59	28	93	0.1	−0.57	1.74
	Post-Test	70	55.19	15.43	55	18	88	−0.12	−0.38	1.84
	Course Score	70	71.74	7.71	73	55	88	−0.14	−0.54	0.92
Experimental	Pre-Test	79	55.91	13.23	55	17	90	0.1	0.19	1.49
	Post-Test	79	57.04	14.1	58	27	91	0	−0.29	1.59
	Course Score	79	72.90	7.31	74	55	90	−0.31	−0.06	0.82
Total	Pre-Test	149	57.95	13.99	55	17	93	0.14	−0.16	1.15
	Post-Test	149	56.17	14.72	55	18	91	−0.08	−0.26	1.21
	Course Score	149	72.36	7.49	73	55	90	−0.23	−0.28	0.61

. To compare the scores on the pre- and post-tests, the total scores on each test are expressed as a percentage. This was calculated by adding up the scores from each item and dividing by the total.

The information available for each student (i.e., pre-test, group, and course score) was analyzed using the Bayesian linear regression model [69]. The running configuration to achieve convergence was set at 3 chains with 20,000 iterations, where the first 10,000 are discarded (warm-up).

Table 7. Posterior summary of the model.

Parameter	Reference	Mean	Std. Dev.	2.5%	97.5%	p (>0|Data)
${\beta }_0$	Intercept	−11.12	10.26	−31.17	8.98	0.14
${\beta }_1$	Pre-Test	0.22	0.09	0.05	0.39	0.99
${\beta }_2$	Group	1.97	2.17	−2.30	6.19	0.82
${\beta }_3$	Course score	0.74	0.16	0.42	1.06	1.00

shows a posterior summary for the regression parameters of the model (Table 3).

3.3. Teacher Survey Results

The results from the teacher survey can be found in

Table 8. Teacher survey results.

	Control Group		Experimental Group		Total		Welch t-Test	Cohen’s d (Effect Size)
Item	Mean	Std. Dev	Mean	Std. Dev	Mean	Std. Dev		d	5% CI	95% CI
1	93.38	6.3	95.55	3.52	94.47	4.94	t(6.27) = −0.67, p = 0.52
2	95.14	4.7	95.55	3.52	95.35	3.92	t(7.41) = −0.15, p = 0.87
3	92.63	5.9	95.59	3.39	94.11	4.79	t(6.38) = −0.97, p = 0.37
4	93.36	6.53	94.88	3.93	94.12	5.14	t(6.56) = −0.45, p = 0.67
5	92.22	6.4	94.51	3.62	93.37	5.05	t(6.32) = −0.7, p = 0.51
6	91.16	6.77	94.88	4.3	93.02	5.7	t(6.78) = −1.04, p = 0.33
7	92.13	7.14	95.62	3.86	93.87	5.72	t(6.16) = −0.96, p = 0.37
8	90.86	7.06	94.88	3.93	92.87	5.79	t(6.26) = −1.11, p = 0.31
9	86.19	6.08	94.86	3.52	90.53	6.55	t(6.41) = −2.76, p = 0.03	2.18	0.19	4.07
10	91.17	5.28	94.92	4.17	93.05	4.9	t(7.6) = −1.26, p = 0.25
11	92.18	5.93	95.62	3.86	93.9	5.05	t(6.88) = −1.09, p = 0.31
12	92.63	5.9	95.59	3.39	94.11	4.79	t(6.38) = −0.98, p = 0.37
13	80.38	20.57	83.22	3.11	81.8	13.95	t(4.18) = −0.31, p = 0.77
14	66.73	17.04	74.96	4.66	70.85	12.55	t(4.59) = −1.04, p = 0.35
15	68.71	18.44	73.99	4.22	71.35	12.92	t(4.42) = −0.62, p = 0.56
16	89.93	4.32	96.25	3.09	93.09	4.86	t(7.25) = −2.66, p = 0.03	1.98	0.17	3.7

.

The results of the qualitative analysis (i.e., the appraisal analysis) can be found in Supplementary Material S4. These results suggest that the respondents adopt a monoglossic approach [79] as they do not acknowledge any other voices in their discourse. This is expected given the nature of the teacher survey, which is both personal and targeted. In terms of Graduation, the concept of force was only present in the form of intensifiers. This is due to the frequent use of linguistic resources that broaden and intensify meaning, such as very, really, and amazing, among others. More complex structures can be found in the comments submitted by students in the experimental group. These generally look to broaden the meaning given by the author, e.g., “she changed my mind” or “a lovely person”.

The comments that were analyzed can be largely grouped into two types: comments on the teacher and comments on the class or methodology.

With regards to the teacher, the students in both the control and experimental groups made positive comments such as excellent, good, empowered, and very clear. One negative comment about the teacher had to do with failing to learn (“I didn’t learn how to write an email.”). Another negative comment referred to the learning environment (“Some students don’t let the class run smoothly.”).

It is worth noting that the positive comments about the teacher have a significant impact on the second area that was evaluated (i.e., the class), as she is acknowledged as being the one who managed or enabled this learning environment. There are positive comments about the teacher from both groups of students. However, the students in the experimental group describe more of her attributes, e.g., “She also tries to show you your individual weaknesses”, “She was motivated to teach and help get the best out of her students”, and “Her sense of vocation is amazing”.

The second area that was evaluated was the class, methodology, or learning system. In this sense, the comments are overwhelmingly positive, especially among students from the experimental group: “Her classes are really educational and practical”, “It’s a really interactive class, which allows for a pleasant environment and this helps when it comes to the assessments”, “Really good learning system”, “The way the teacher teaches is really educational, while she also tries to show you your individual weaknesses. I hope they show us how get even more out of how we use language”. In quantitative terms, only two of the students’ comments refer to the class, labelling it as “practical” and highlighting the “pleasant environment”. However, none of the other comments referred to this aspect of the course. Instead, seven of the comments made by students in the experimental group focused on the methodology, highlighting it as being educational, interactive, dynamic, attention-grabbing, and as lending itself to generating a positive learning environment.

4. Discussion

A unidimensional model based on our proposed definition of critical thinking explains the structure of the two instruments. Consequently, the pre- and post-tests can be considered equivalent [55,76]. This means that there is one common factor among all items, in this case, critical thinking. The instruments are also validated by having acceptable levels of reliability [62,78] and suitable scores on the item analysis [55,64]. Both instruments are characterized by addressing cultural issues related to the students. This has shown to be a relevant element when designing instruments [35].

The Bayesian linear regression used the pre-test, group, and course scores as variables explaining the post-test scores (see Equation (1)) (Table 7). The results reveal that ${\beta }_1$ (pre-test), ${\beta }_2$ (group), and ${\beta }_3$ (course score) are positive with a very high probability, while ${\beta }_0$ is surely negative. Interpretatively, and taking ${\beta }_1$ as the example, this means that a one-unit increase in the pre-test score implies an average increase of 0.22 in the post-test.

The difficulty index for the items was calculated as the proportion of students who answered the question correctly [64]. As the score for each question was dichotomous, the average score on the test also represents its average difficulty level. In this sense, the post-test can therefore be considered more difficult than the pre-test (Table 6). For the control and experimental groups, this explains the smaller average increase in the post-test scores when compared to the pre-test. When analyzing by group, there is an increase from pre-test to post-test in the experimental group, which is consistent with other studies that show different interventions that were successful in developing critical thinking in community colleges [80]. It is also consistent with specific activities that were included in the proposed methodology, such as open-ended questions and small group discussions, that have also been proved to be successful in developing critical thinking skills in TVET. Therefore, this would suggest that our proposed methodology for developing critical thinking was successful. In the case of the control group, the post-test score was lower than the pre-test score. This may be explained by the increase in difficulty between the two tests.

Furthermore, on average, students in the experimental group scored 1.97 points more on the post-test than students in the control group (Table 7). Therefore, there is a significant difference in favor of the experimental group. This demonstrates the effectiveness of the proposed methodology for developing critical thinking.

Similarly, it can be seen how a one-unit increase in the course score implies an average increase of 0.74 in the post-test score (Table 7). This is in line with the literature, which suggests that there is a positive relationship between academic performance and critical thinking [63], and would suggest that the methodology not only develops critical thinking but also helps to achieve the expected learning outcomes, which is important, considering the context in which it was developed. A methodology that promotes the development of critical thinking but does not allow the achievement of learning outcomes would have failed nevertheless.

To evaluate which aspects of the methodology were more important when it came to developing critical thinking, the effect sizes based on the students’ responses to the teacher survey and the subsequent appraisal analysis can be observed. In this case, there are only significant differences for two of the questions: question 9 (“The teacher shows a willingness to clarify doubts and/or answer questions during class.”) and question 16 (“How would you rate your overall experience with the teacher who taught this class?”). The effect size for both of these questions is large [81]. This suggests that the experimental group’s perception of the teacher is, in general, better than the control group’s perception. There are two possible explanations for this. The first is that having been exposed to this new methodology, the teacher was more willing to help her students in class. The second is that the methodology itself facilitates the teacher’s job in the classroom. Any methodology for developing critical thinking must be accompanied by well-planned teaching experiences and sequences that lead to significant learning [26]. This relationship is, therefore, synergistic and cannot/must not be separated. Furthermore, the students’ comments show how this relationship comes about, highlighting how aspects such as the teacher’s motivation, commitment, and sense of vocation have a positive impact on the learning environment.

Similarly, the appraisal analysis shows how essential aspects of the methodology were highlighted more frequently by students in the experimental group than in the control group when answering the teacher survey. The statement “Her classes are really educational and practical” may relate to the fact that one of the guidelines established for the proposed methodology is that students should work with controversial or real-world topics. This has been highlighted by teachers as being a good way of developing critical thinking [26]. Furthermore, it is especially important for students in technical and vocational education [82].

Furthermore, students in the experimental group highlighted aspects such as the interactive and dynamic nature of the lessons, as well as the positive learning environment. These elements are not highlighted to a similar degree by the students in the control group. This is something that is covered by the main objectives of the methodology and is also intimately linked to engagement. This is because the fact that lessons that manage to grab the students’ attention is key to learning [44]. Indeed, this is further supported by one of the students, who suggested that their opinion of the subject changed thanks to the course: “The truth is, in the beginning I wasn’t thrilled about taking this class, but your enthusiasm changed my mind”. Furthermore, there is also evidence to suggest that the general principles of active learning are related to the development of critical thinking [83]. In this case, active learning is included in the methodology thanks to the objectives relating to active student participation and dynamism, among others. There is also evidence of the positive impact of active learning on employability for students in technical and vocational education [84]. Finally, the importance students place on interactive and dynamic classes can also be explained when considering that they and their families usually have lower levels of motivation [28].

5. Conclusions

The research question asked whether it is possible to develop a methodology for developing critical thinking in technical and vocational education. Considering that the critical thinking tests used in this study are both valid and equivalent, the results of the Bayesian linear regression model show that our proposed methodology develops students’ critical thinking skills more than traditional methods. This is consistent with the proposed methodology for developing critical thinking, given that it was developed based on general recommendations, proven experiences, and activities that have been shown to promote critical thinking in other contexts within higher education [17,85]. The guidelines, methodology, and general structure of a class are outlined in Section 2.3, while a full lesson developed using this methodology can be found in Supplementary Material S1.

Lessons characterized by being interactive, dynamic, and boosting active participation facilitate the teacher’s job in the classroom while improving the development of critical thinking. This is particularly true when working with controversial or real-world topics. Several of the activities, methodologies, and principles for developing critical thinking that have been validated in other contexts may also be applicable to technical and vocational education.

This finding also shows that the methodology proposed in this study is a concrete example of how to develop critical thinking in a technical and vocational education and training context. This methodology may also provide a starting point for developing critical thinking for more general purposes, such as developing more productive or critical citizens [20]. This is particularly true considering it contains elements such as the use of real-world examples, discussions, and other activities that can be expanded from specific work skills to more general aspects of life.

The main limitations of our study relate to the experimental design of the sample. All of the participants were students enrolled at the same technical and vocational college in Chile. Furthermore, the students all belonged to the same faculty within the college. Therefore, future work should look to the study of this methodology in different contexts (within the field of technical and vocational education and training) so that the results can be studied in a broader context within this type of education. For example, the study of the relationship between critical thinking and academic achievement in the broader TVET context would provide new reasons to adopt practices for the development of this skill.

Another important factor to bear in mind is the teacher effect. All of the participants in this study (both control and experimental) were taught by the same teacher so as not to let this affect the results. However, this may also mean that the teacher is a factor in the students’ development of critical thinking. Consequently, future work should replicate our study with different teachers of different profiles to control for this factor.

Future work should also analyze how to develop critical thinking among TVET students who have undergone this type of intervention. This means considering how these skills may be further developed among TVET students who have already improved their level of critical thinking. In such cases, the context of our study (i.e., TVET as a fallback option, families with low motivation, etc.) may no longer be true [28]. Therefore, there should be research on how the activities for developing critical thinking can adapt and evolve if the characteristics of the subjects change. It should also look at how the sample class designed for our study can be used to plan for and implement the proposed methodology in other courses, contexts, and domains.