Development and Validation of a Critical Thinking Assessment-Scale Short Form
Abstract
:1. Introduction
2. Materials and Methods
2.1. Shorthening of the CTSAS
2.2. Participants
2.3. Instruments and Procedures
2.3.1. Translation of the CTSAS Short Form into Different Languages
2.3.2. Data Collection
2.4. Statistical Analysis
3. Results
3.1. Descriptive Analysis of Items
3.2. Confirmatory Factor Analysis (CFA) and Reliability
- Model 1: One-factor model. This model tests the existence of one global factor on critical thinking skills, which explains the variances of the 60 variables.
- Model 2: Six-factor (non-correlated) model. This model tests the existence of six non-correlated factors that explain the variance of the set of items.
- Model 3: Six-factor (correlated) model. This model tests the existence of six correlated latent factors, each one explaining the variance of a set of items.
- Model 4: Second-order factor model. This model represents the original model proposed by Nair [36], in which a global critical-thinking-skills construct explains the six latent-skills variance, which, in turn, each explain a set of items.
- Model 5: Bi-factor model. This model tests the possibility that the 60 scale-items variances are being explained by a global critical-thinking-skills construct, and by the six latent skills, independently.
3.3. Multigroup Invariance
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Mean | Sd. | Skew. | Kurt. | K-S Test | p | |
---|---|---|---|---|---|---|
1. I try to figure out the content of the problem. | 5.04 | 0.958 | −0.744 | −0.232 | 0.152 | 1.000 |
2. I classify data using a framework. | 3.89 | 1.319 | −0.452 | −0.140 | 0.994 | 0.276 |
3. I break the complex ideas into manageable sub-ideas. | 3.96 | 1.357 | −0.467 | −0.049 | 0.718 | 0.682 |
4. I observe the facial expression people use in a given situation. | 4.63 | 1.380 | −1.071 | 0.715 | 0.914 | 0.374 |
5. I examine the values rooted in the information presented. | 4.12 | 1.284 | −0.532 | −0.172 | 0.754 | 0.620 |
6. I restate another person’s statements to clarify the meaning. | 3.63 | 1.515 | −0.359 | −0.545 | 0.762 | 0.607 |
7. I figure out an example which explains the concept/opinion. | 4.53 | 1.097 | −0.785 | 0.550 | 0.601 | 0.863 |
8. I clarify my thoughts by explaining to someone else. | 4.29 | 1.348 | −0.803 | 0.203 | 0.864 | 0.445 |
9. I seek clarification of the meanings of another’s opinion or points of view. | 4.23 | 1.185 | −0.483 | −0.196 | 0.718 | 0.682 |
10. I examine the similarities and differences among the opinions posed for a given problem. | 4.23 | 1.166 | −0.742 | 0.765 | 0.518 | 0.951 |
11. I examine the interrelationships among concepts or opinions posed. | 3.84 | 1.222 | −0.364 | 0.101 | 0.629 | 0.823 |
12. I look for supporting reasons when examining opinions. | 4.44 | 1.174 | −0.692 | 0.436 | 0.640 | 0.808 |
13. I look for relevant information to answer the question at issue. | 4.62 | 1.147 | −0.855 | 0.657 | 0.651 | 0.790 |
14. I examine the proposals for solving a given problem. | 4.65 | 1.089 | −0.626 | −0.100 | 0.260 | 1.000 |
15. I ask questions in order to seek evidence to support or refute the author’s claim. | 4.09 | 1.341 | −0.566 | −0.084 | 1.041 | 0.229 |
16. I figure out if author’s arguments include both for and against the claim. | 3.97 | 1.316 | −0.433 | −0.229 | 1.044 | 0.226 |
17. I figure out unstated assumptions in one’s reasoning for a claim. | 3.63 | 1.289 | −0.287 | −0.190 | 0.723 | 0.673 |
18. I look for the overall structure of the argument. | 3.99 | 1.332 | −0.580 | 0.136 | 0.864 | 0.444 |
19. I figure out the process of reasoning for an argument. | 4.02 | 1.306 | −0.578 | 0.253 | 0.381 | 0.999 |
20. I figure out the assumptions implicit in the author’s reasoning. | 3.73 | 1.275 | −0.436 | −0.032 | 0.828 | 0.500 |
21. I assess the contextual relevance of an opinion or claim posed. | 4.00 | 1.192 | −0.493 | 0.387 | 0.810 | 0.528 |
22. I seek the accuracy of the evidence supporting a given judgment. | 4.18 | 1.283 | −0.693 | 0.306 | 0.858 | 0.453 |
23. I assess the chances of success or failure in using a premise to conclude an argument. | 4.08 | 1.344 | −0.599 | −0.007 | 1.120 | 0.163 |
24. I examine the logical strength of the underlying reason in an argument. | 4.06 | 1.295 | −0.464 | −0.030 | 0.919 | 0.367 |
25. I search for new data to confirm or refute a given claim | 4.15 | 1.288 | −0.644 | 0.142 | 0.708 | 0.698 |
26. I search for additional information that might support or weaken an argument. | 4.34 | 1.195 | −0.520 | −0.206 | 0.435 | 0.992 |
27. I examine the logical reasoning of an objection to a claim. | 4.17 | 1.310 | −0.552 | 0.025 | 0.883 | 0.417 |
28. I seek useful information to refute an argument when supported by unsure reasons. | 4.37 | 1.186 | −0.655 | 0.478 | 0.314 | 1.000 |
29. I collect evidence supporting the availability of information to back up opinions. | 4.21 | 1.317 | −0.771 | 0.585 | 0.794 | 0.554 |
30. I seek for evidence/information before accepting a solution. | 4.49 | 1.241 | −0.729 | 0.176 | 0.355 | 1.000 |
31. I figure out alternate hypotheses/questions, when I need to solve a problem. | 4.21 | 1.311 | −0.645 | 0.166 | 1.042 | 0.228 |
32. Given a problem to solve, I develop a set of options for solving the problem. | 4.33 | 1.255 | −0.685 | 0.234 | 0.683 | 0.739 |
33. I systematically analyse the problem using multiple sources of information to draw inferences. | 4.11 | 1.381 | −0.596 | −0.103 | 0.325 | 1.000 |
34. I figure out the merits and demerits of a solution while prioritizing from alternatives for making decisions. | 4.01 | 1.320 | −0.455 | −0.130 | 0.812 | 0.525 |
35. I identify the consequences of various options to solving a problem. | 4.36 | 1.208 | −0.558 | −0.009 | 0.625 | 0.830 |
36. I arrive at conclusions that are supported with strong evidence. | 4.30 | 1.164 | −0.328 | −0.484 | 0.490 | 0.970 |
37. I use both deductive and inductive reasoning to interpret information. | 4.00 | 1.330 | −0.419 | −0.259 | 0.766 | 0.600 |
38. I analyse my thinking before jumping to conclusions. | 4.39 | 1.335 | −0.710 | 0.065 | 0.437 | 0.991 |
39. I confidently reject an alternative solution when it lacks evidence. | 3.89 | 1.417 | −0.312 | −0.587 | 0.541 | 0.932 |
40. I figure out the pros and cons of a solution before accepting it. | 4.64 | 1.175 | −0.721 | 0.216 | 0.710 | 0.695 |
41. I can describe the results of a problem using inferential evidence. | 3.78 | 1.206 | −0.269 | 0.068 | 0.701 | 0.709 |
42. I can logically present results to address a given problem. | 4.18 | 1.138 | −0.425 | 0.111 | 1.533 | 0.018 |
43. I state my choice of using a particular method to solve the problem. | 4.03 | 1.277 | −0.530 | 0.164 | 0.305 | 1.000 |
44. I can explain a key concept to clarify my thinking. | 4.10 | 1.246 | −0.408 | −0.141 | 0.585 | 0.883 |
45. I write essays with adequate arguments supported with reasons for a given policy or situation. | 3.13 | 1.734 | −0.208 | −0.966 | 0.833 | 0.492 |
46. I anticipate reasonable criticisms one might raise against one’s viewpoints. | 3.92 | 1.319 | −0.438 | −0.340 | 0.730 | 0.661 |
47. I respond to reasonable criticisms one might raise against one’s viewpoints. | 3.82 | 1.292 | −0.456 | −0.055 | 1.772 | 0.004 |
48. I clearly articulate evidence for my own viewpoints. | 4.22 | 1.159 | −0.353 | −0.283 | 0.195 | 1.000 |
49. I present more evidence or counter evidence for another’s points of view. | 3.61 | 1.338 | −0.258 | −0.540 | 0.664 | 0.770 |
50. I provide reasons for rejecting another’s claim. | 4.04 | 1.400 | −0.535 | −0.309 | 1.255 | 0.086 |
51. I reflect on my opinions and reasons to ensure my premises are correct. | 4.43 | 1.136 | −0.442 | −0.421 | 0.540 | 0.932 |
52. I review sources of information to ensure important information is not overlooked. | 4.26 | 1.317 | −0.628 | −0.074 | 1.009 | 0.260 |
53. I examine and consider ideas and viewpoints even when others do not agree. | 4.20 | 1.156 | −0.380 | −0.235 | 0.174 | 1.000 |
54. I examine my values, thoughts/beliefs based on reasons and evidence. | 4.41 | 1.159 | −0.455 | −0.151 | 0.143 | 1.000 |
55. I continuously assess my targets and work towards achieving them. | 4.46 | 1.182 | −0.472 | −0.367 | 0.354 | 1.000 |
56. I review my reasons and reasoning process in coming to a given conclusion. | 4.18 | 1.187 | −0.349 | −0.236 | 0.415 | 0.995 |
57. I analyze areas of consistencies and inconsistencies in my thinking. | 4.01 | 1.294 | −0.448 | −0.192 | 0.926 | 0.358 |
58. I willingly revise my work to correct my opinions and beliefs. | 4.27 | 1.263 | −0.457 | −0.172 | 0.663 | 0.772 |
59. I continually revise and rethink strategies to improve my thinking. | 4.34 | 1.280 | −0.601 | −0.073 | 0.683 | 0.739 |
60. I reflect on my thinking to improve the quality of my judgment. | 4.53 | 1.187 | −0.805 | 0.752 | 0.235 | 1.000 |
Item | Interpretation | Analysis | Evaluation | Inference | Explanation | Self-Regulation |
---|---|---|---|---|---|---|
1. I try to figure out the content of the problem. | 0.662 | |||||
2. I classify data using a framework. | 0.661 | |||||
3. I break the complex ideas into manageable sub-ideas. | 0.633 | |||||
4. I observe the facial expression people use in a given situation | 0.386 | |||||
5. I examine the values rooted in the information presented. | 0.654 | |||||
6. I restate another person’s statements to clarify the meaning. | 0.499 | |||||
7. I figure out an example which explains the concept/opinion. | 0.594 | |||||
8. I clarify my thoughts by explaining to someone else. | 0.422 | |||||
9. I seek clarification of the meanings of another’s opinion or points of view. | 0.536 | |||||
10. I examine the similarities and differences among the opinions posed for a given problem. | 0.614 | |||||
11. I examine the interrelationships among concepts or opinions posed. | 0.734 | |||||
12. I look for supporting reasons when examining opinions. | 0.671 | |||||
13. I look for relevant information to answer the question at issue. | 0.650 | |||||
14. I examine the proposals for solving a given problem. | 0.701 | |||||
15. I ask questions in order to seek evidence to support or refute the author’s claim. | 0.666 | |||||
16. I figure out if author’s arguments include both for and against the claim. | 0.670 | |||||
17. I figure out unstated assumptions in one’s reasoning for a claim. | 0.619 | |||||
18. I look for the overall structure of the argument. | 0.707 | |||||
19. I figure out the process of reasoning for an argument. | 0.772 | |||||
20. I figure out the assumptions implicit in the author’s reasoning. | 0.745 | |||||
21. I assess the contextual relevance of an opinion or claim posed. | 0.723 | |||||
22. I seek the accuracy of the evidence supporting a given judgment. | 0.735 | |||||
23. I assess the chances of success or failure in using a premise to conclude an argument. | 0.702 | |||||
24. I examine the logical strength of the underlying reason in an argument. | 0.725 | |||||
25. I search for new data to confirm or refute a given claim | 0.674 | |||||
26. I search for additional information that might support or weaken an argument. | 0.732 | |||||
27. I examine the logical reasoning of an objection to a claim. | 0.761 | |||||
28. I seek useful information to refute an argument when supported by unsure reasons. | 0.717 | |||||
29. I collect evidence supporting the availability of information to back up opinions. | 0.740 | |||||
30. I seek for evidence/information before accepting a solution. | 0.691 | |||||
31. I figure out alternate hypotheses/questions, when I need to solve a problem. | 0.734 | |||||
32. Given a problem to solve, I develop a set of options for solving the problem. | 0.710 | |||||
33. I systematically analyse the problem using multiple sources of information to draw inferences. | 0.738 | |||||
34. I figure out the merits and demerits of a solution while prioritizing from alternatives for making decisions. | 0.742 | |||||
35. I identify the consequences of various options to solving a problem. | 0.704 | |||||
36. I arrive at conclusions that are supported with strong evidence. | 0.756 | |||||
37. I use both deductive and inductive reasoning to interpret information. | 0.696 | |||||
38. I analyse my thinking before jumping to conclusions. | 0.636 | |||||
39. I confidently reject an alternative solution when it lacks evidence. | 0.470 | |||||
40. I figure out the pros and cons of a solution before accepting it. | 0.656 | |||||
41. I can describe the results of a problem using inferential evidence. | 0.745 | |||||
42. I can logically present results to address a given problem. | 0.749 | |||||
43. I state my choice of using a particular method to solve the problem. | 0.672 | |||||
44. I can explain a key concept to clarify my thinking. | 0.740 | |||||
45. I write essays with adequate arguments supported with reasons for a given policy or situation. | 0.511 | |||||
46. I anticipate reasonable criticisms one might raise against one’s viewpoints | 0.606 | |||||
47. I respond to reasonable criticisms one might raise against one’s viewpoints. | 0.650 | |||||
48. I clearly articulate evidence for my own viewpoints. | 0.720 | |||||
49. I present more evidence or counter evidence for another’s points of view. | 0.573 | |||||
50. I provide reasons for rejecting another’s claim. | 0.536 | |||||
51. I reflect on my opinions and reasons to ensure my premises are correct. | 0.719 | |||||
52. I review sources of information to ensure important information is not overlooked. | 0.785 | |||||
53. I examine and consider ideas and viewpoints even when others do not agree. | 0.705 | |||||
54. I examine my values, thoughts/beliefs based on reasons and evidence. | 0.756 | |||||
55. I continuously assess my targets and work towards achieving them. | 0.673 | |||||
56. I review my reasons and reasoning process in coming to a given conclusion. | 0.728 | |||||
57. I analyze areas of consistencies and inconsistencies in my thinking. | 0.737 | |||||
58. I willingly revise my work to correct my opinions and beliefs. | 0.750 | |||||
59. I continually revise and rethink strategies to improve my thinking. | 0.786 | |||||
60. I reflect on my thinking to improve the quality of my judgment. | 0.763 |
Skills | Alpha’s Cronbach | Sub-Skills | Std Alpha’s Cronbach |
---|---|---|---|
Interpretation | 0.772 | Categorization | 0.670 |
Clarifying meaning | 0.673 | ||
Decoding significance | 0.473 | ||
Analysis | 0.888 | Detecting arguments | 0.632 |
Analyzing arguments | 0.812 | ||
Examining ideas | 0.799 | ||
Evaluation | 0.858 | Assessing claim | 0.723 |
Assessing arguments | 0.821 | ||
Inference | 0.905 | Drawing conclusions | 0.743 |
Conjecturing alternatives | 0.843 | ||
Querying evidence | 0.752 | ||
Explanation | 0.853 | Stating results | 0.688 |
Justifying procedures | 0.681 | ||
Presenting arguments | 0.778 | ||
Self-regulation | 0.905 | Self-examining | 0.860 |
Self-correction | 0.834 |
References
- Dumitru, D.; Bigu, D.; Elen, J.; Jiang, L.; Railienè, A.; Penkauskienè, D.; Papathanasiou, I.V.; Tsaras, K.; Fradelos, E.C.; Ahern, A.; et al. A European Collection of the Critical Thinking Skills and Dispositions Needed in Different Professional Fields for the 21st Century; UTAD: Vila Real, Portugal, 2018. [Google Scholar]
- Cruz, G.; Payan-Carreira, R.; Dominguez, C.; Silva, H.; Morais, F. What critical thinking skills and dispositions do new graduates need for professional life? Views from Portuguese employers in different fields. High. Educ. Res. Dev. 2021, 40, 721–737. [Google Scholar] [CrossRef]
- Braun, H.I.; Shavelson, R.J.; Zlatkin-Troitschanskaia, O.; Borowiec, K. Performance Assessment of Critical Thinking: Conceptualization, Design, and Implementation. Front. Educ. 2020, 5, 156. [Google Scholar] [CrossRef]
- Cinque, M.; Carretero, S.; Napierala, J. Non-Cognitive Skills and Other Related Concepts: Towards a Better Understanding of Similarities and Differences; Joint Research Centre, European Commission: Brussels, Belgium, 2021; 31p. [Google Scholar]
- Pnevmatikos, D.; Christodoulou, P.; Georgiadou, T.; Lithoxoidou, A.; Dimitriadou, A.; Payan Carreira, R.; Simões, M.; Ferreira, D.; Rebelo, H.; Sebastião, L.; et al. THINK4JOBS TRAINING: Critical Thinking Training Packages for Higher Education Instructors and Labour Market Tutors; University of Western Macedonia: Kozani, Greece, 2021. [Google Scholar]
- Facione, P. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction (The Delphi Report); California Academic Press: Millbrae, CA, USA; Newark, DE, USA, 1990; 112p. [Google Scholar]
- Payan-Carreira, R.; Sebastião, L.; Cristóvão, A.; Rebelo, H. How to Enhance Students’ Self-Regulation. In The Psychology of Self-Regulation; Dutton, J., Ed.; Psychology of Emotions, Motivations and Actions; Nova Science Publishers, Inc.: Hauppauge, NY, USA, 2022; p. 22. (in press) [Google Scholar]
- Rear, D. One size fits all? The limitations of standardised assessment in critical thinking. Assess. Eval. High. Educ. 2019, 44, 664–675. [Google Scholar] [CrossRef]
- Thaiposri, P.; Wannapiroon, P. Enhancing Students’ Critical Thinking Skills through Teaching and Learning by Inquiry-based Learning Activities Using Social Network and Cloud Computing. Procedia-Soc. Behav. Sci. 2015, 174, 2137–2144. [Google Scholar] [CrossRef] [Green Version]
- Lai, R.E. Critical Thinking: A Literature Review. Pearson Res. Rep. 2011, 6, 40–41. [Google Scholar]
- Shavelson, R.J.; Zlatkin-Troitschanskaia, O.; Beck, K.; Schmidt, S.; Marino, J.P. Assessment of University Students’ Critical Thinking: Next Generation Performance Assessment. Int. J. Test. 2019, 19, 337–362. [Google Scholar] [CrossRef]
- Pnevmatikos, D.; Christodoulou, P.; Georgiadou, T. Promoting critical thinking in higher education through the values and knowledge education (VaKE) method. Stud. High. Educ. 2019, 44, 892–901. [Google Scholar] [CrossRef]
- Facione, P.A. The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill. Informal Log. 2000, 20, 61–84. [Google Scholar] [CrossRef] [Green Version]
- Ennis, R.H. The Nature of Critical Thinking: Outlines of General Critical Thinking Dispositions and Abilities. 2013. Available online: https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf (accessed on 17 November 2022).
- Halpern, D.F. Teaching critical thinking for transfer across domains. Dispositions, skills, structure training, and metacognitive monitoring. Am. Psychol. 1998, 53, 449–455. [Google Scholar] [CrossRef]
- Nair, G.G.; Stamler, L. A Conceptual Framework for Developing a Critical Thinking Self-Assessment Scale. J. Nurs. Educ. 2013, 52, 131–138. [Google Scholar] [CrossRef]
- Rapps, A.M. Let the Seuss loose. In Rutgers; The State University of New Jersey: Camden, NJ, USA, 2017. [Google Scholar]
- Tight, M. Twenty-first century skills: Meaning, usage and value. Eur. J. High. Educ. 2021, 11, 160–174. [Google Scholar] [CrossRef]
- Ryan, C.; Tatum, K. Objective Measurement of Critical-Thinking Ability in Registered Nurse Applicants. JONA J. Nurs. Adm. 2012, 42, 89–94. [Google Scholar] [CrossRef] [PubMed]
- Patrício, M.F.; Julião, M.; Fareleira, F.; Carneiro, A.V. Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Med. Teach. 2013, 35, 503–514. [Google Scholar] [CrossRef] [PubMed]
- Hyytinen, H.; Ursin, J.; Silvennoinen, K.; Kleemola, K.; Toom, A. The dynamic relationship between response processes and self-regulation in critical thinking assessments. Stud. Educ. Eval. 2021, 71, 101090. [Google Scholar] [CrossRef]
- Simper, N.; Frank, B.; Kaupp, J.; Mulligan, N.; Scott, J. Comparison of standardized assessment methods: Logistics, costs, incentives and use of data. Assess. Eval. High. Educ. 2019, 44, 821–834. [Google Scholar] [CrossRef]
- Verburgh, A.; François, S.; Elen, J.; Janssen, R. The Assessment of Critical Thinking Critically Assessed in Higher Education: A Validation Study of the CCTT and the HCTA. Educ. Res. Int. 2013, 2013, 198920. [Google Scholar] [CrossRef]
- Hart, C.; Da Costa, C.; D’Souza, D.; Kimpton, A.; Ljbusic, J. Exploring higher education students’ critical thinking skills through content analysis. Think. Ski. Creat. 2021, 41, 100877. [Google Scholar] [CrossRef]
- Williamson, D.M.; Xi, X.; Breyer, F.J. A Framework for Evaluation and Use of Automated Scoring. Educ. Meas. Issues Pract. 2012, 31, 2–13. [Google Scholar] [CrossRef]
- Haromi, F.; Sadeghi, K.; Modirkhameneh, S.; Alavinia, P.; Khonbi, Z. Teaching through Appraisal: Developing Critical Reading in Iranian EFL Learners. Proc. Int. Conf. Current Trends Elt 2014, 98, 127–136. [Google Scholar] [CrossRef] [Green Version]
- Ku, K.Y.L. Assessing students’ critical thinking performance: Urging for measurements using multi-response format. Think. Ski. Creat. 2009, 4, 70–76. [Google Scholar] [CrossRef]
- de Bie, H.; Wilhelm, P.; van der Meij, H. The Halpern Critical Thinking Assessment: Toward a Dutch appraisal of critical thinking. Think. Ski. Creat. 2015, 17, 33–44. [Google Scholar] [CrossRef]
- Liu, O.L.; Frankel, L.; Roohr, K.C. Assessing Critical Thinking in Higher Education: Current State and Directions for Next-Generation Assessment. ETS Res. Rep. Ser. 2014, 2014, 1–23. [Google Scholar] [CrossRef]
- Hatcher, D.L. Which test? Whose scores? Comparing standardized critical thinking tests. New Dir. Inst. Res. 2011, 2011, 29–39. [Google Scholar] [CrossRef]
- Cole, J.S.; Gonyea, R.M. Accuracy of Self-reported SAT and ACT Test Scores: Implications for Research. Res. High. Educ. 2010, 51, 305–319. [Google Scholar] [CrossRef]
- Althubaiti, A. Information bias in health research: Definition, pitfalls, and adjustment methods. J. Multidiscip Healthc. 2016, 9, 211–217. [Google Scholar] [CrossRef] [Green Version]
- Payan-Carreira, R.; Cruz, G.; Papathanasiou, I.V.; Fradelos, E.; Jiang, L. The effectiveness of critical thinking instructional strategies in health professions education: A systematic review. Stud. High. Educ. 2019, 44, 829–843. [Google Scholar] [CrossRef]
- Kreitchmann, R.S.; Abad, F.J.; Ponsoda, V.; Nieto, M.D.; Morillo, D. Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items. Front. Psychol. 2019, 10, 2309. [Google Scholar] [CrossRef] [Green Version]
- Nair, G. Preliminary Psychometric Characteristics of the Critical Thinking Self-Assessment Scale; University of Saskatchewan: Saskatoon, SK, Canada, 2011. [Google Scholar]
- Nair, G.G.; Hellsten, L.M.; Stamler, L.L. Accumulation of Content Validation Evidence for the Critical Thinking Self-Assessment Scale. J. Nurs. Meas. 2017, 25, 156–170. [Google Scholar] [CrossRef]
- Gudmundsson, E. Guidelines for translating and adapting psychological instruments. Nord. Psychol. 2009, 61, 29–45. [Google Scholar] [CrossRef]
- Tsang, S.; Royse, C.F.; Terkawi, A.S. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J. Anaesth. 2017, 11, S80–S89. [Google Scholar] [CrossRef]
- Gerdts-Andresen, T.; Hansen, M.T.; Grøndahl, V.A. Educational effectiveness: Validation of an instrument to measure students’ critical thinking and disposition. Int. J. Instr. 2022, 25, 685–700. [Google Scholar] [CrossRef]
- Flora, D.B.; Curran, P.J. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychol. Methods 2004, 9, 466–491. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hu, L.t.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. A Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
- Hair, J.F.; Page, M.; Brunsveld, N. Essentials of Business Research Methods, 4th ed.; Routledge: New York, NY, USA, 2019. [Google Scholar]
- Cheung, G.W.; Rensvold, R.B. Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Struct. Equ. Model. A Multidiscip. J. 2002, 9, 233–255. [Google Scholar] [CrossRef]
- Chen, F.F. Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Struct. Equ. Model. A Multidiscip. J. 2007, 14, 464–504. [Google Scholar] [CrossRef]
- Muthén, L.K.; Muthén, B.O. Mplus User’s Guide; Muthén & Muthén: Los Angeles, CA, USA, 2012. [Google Scholar]
- Brown, T.A. Confirmatory Factor Analysis for Applied Research, 2nd ed.; Guiford Press: New York, NJ, USA, 2015; 462p. [Google Scholar]
- MacCallum, R.C.; Widaman, K.F.; Zhang, S.; Hong, S. Sample size in factor analysis. Psychol. Methods 1999, 4, 84–99. [Google Scholar] [CrossRef]
- Commission, E. Tertiary Education Statistics; Eurostat: Luxembourg, 2022. [Google Scholar]
- Feinian, C.; Curran, P.J.; Bollen, K.A.; Kirby, J.; Paxton, P. An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models. Sociol. Methods Res. 2008, 36, 462–494. [Google Scholar] [CrossRef] [Green Version]
- Rosenman, R.; Tennekoon, V.; Hill, L.G. Measuring bias in self-reported data. Int. J. Behav. Healthc. Res. 2011, 2, 320–332. [Google Scholar] [CrossRef] [Green Version]
- Taber, K.S. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Res. Sci. Educ. 2018, 48, 1273–1296. [Google Scholar] [CrossRef] [Green Version]
- Marôco, J. Análise de Equações Estruturais—Fundamentos Teóricos, Software & Aplicações, 2nd ed.; ReportNumber, Análise e Gestão de Informação, Ltd.: Pero Pinheiro, Portugal, 2014; p. 390. [Google Scholar]
- Maroco, J. Análise Estatística com o SPSS Statistics, 7th ed.; ReportNumber-Análise e gestão de Informação, Ltd.: Pero Pinheiro, Portugal, 2018; 1013p. [Google Scholar]
- Clark, L.A.; Watson, D. Constructing validity: New developments in creating objective measuring instruments. Psychol. Assess. 2019, 31, 1412–1427. [Google Scholar] [CrossRef]
CTSAS Dimensions (Skills/Sub-Skills) | Items in the Original CTSAS | Eliminated Items | Items in the CTSAS Short-Form | |
---|---|---|---|---|
Interpretation | Categorization | 1–9 | 2, 4, 6–8 | 1–3 |
Clarifying meaning | 15–21 | 18–20 | 6–9 | |
Decoding significance | 10–14 | 10, 12, 14 | 4, 5 | |
Analysis | Detecting arguments | 28–33 | 32, 33 | 15, 16 |
Analyzing arguments | 34–49 | 34, 39 | 17–20 | |
Examining ideas | 22–27 | 27–29 | 10–14 | |
Evaluation | Assessing claims | 40–44 | 40–42 | 21, 22 |
Assessing arguments | 45–52 | 46, 50, 52 | 23–27 | |
Inference | Drawing conclusions | 67–74 | 67, 68, 73 | 36–40 |
Conjecturing alternatives | 60–66 | 62, 65 | 31–35 | |
Querying evidence | 53–59 | 53, 54, 58, 59 | 28–30 | |
Explanation | Stating results | 75–79 | 76, 77, 79 | 41, 42 |
Justifying procedures | 80–88 | 81, 83–88 | 43, 44 | |
Presenting arguments | 89–96 | 95, 96 | 45–50 | |
Self-regulation | Self-examination | 97–105 | 98, 104 | 51–57 |
Self-correction | 106–115 | 107, 109–111, 113–115 | 58–60 |
Models | χ2 (df) | p | RMSEA [90%IC] | CFI | TLI |
---|---|---|---|---|---|
Model 1: 1-factor model | 5159.412 (1710) | <0.0001 | 0.061 [0.059–0.063] | 0.893 | 0.890 |
Model 2: 6-factor model (non-correlated) | 29275.338 (1710) | <0.0001 | 0.174 [0.172–0176] | 0.148 | 0.118 |
Model 3: 6-factor model (correlated) | 3871.243 (1695) | <0.0001 | 0.049 [0.047–0.051] | 0.933 | 0.930 |
Model 4: second-order factor model | 3975.885 (1704) | <0.0001 | 0.051 [0.049–0.053] | 0.927 | 0.924 |
Model 5: bi-factor model | 18,656.904 (1657) | <0.0001 | 0.139 [0.137–0.141] | 0.474 | 0.439 |
Skills | α | CrT-Skills | 1 | 2 | 3 | 4 | 5 |
---|---|---|---|---|---|---|---|
1. Interpretation | 0.772 | 0.881 | |||||
2. Analysis | 0.888 | 0.925 | 0.905 | ||||
3. Evaluation | 0.858 | 0.965 | 0.810 | 0.934 | |||
4. Inference | 0.905 | 0.956 | 0.806 | 0.858 | 0.937 | ||
5. Explanation | 0.853 | 0.907 | 0.765 | 0.825 | 0.864 | 0.868 | |
6. Self-regulation | 0.905 | 0.851 | 0.750 | 0.750 | 0.781 | 0.841 | 0.805 |
Baseline Models | χ2 (df) | p | RMSEA [90%IC] | CFI | TLI |
Female | 3488.157 (1704) | <0.0001 | 0.052 [0.049–0.054] | 0.929 | 0.926 |
Male | 2314.349 (1704) | <0.0001 | 0.050 [0.045–0.055] | 0.948 | 0.946 |
Invariance | χ2 (df) | p | RMSEA [90%IC] | CFI | TLI |
Configural invariance | 5521.460 (3390) | <0.0001 | 0.049 [0.046–0.051] | 0.939 | 0.936 |
Metric invariance | 5490.717 (3444) | <0.0001 | 0.047 [0.045–0.050] | 0.941 | 0.940 |
Scalar invariance | 5613.987 (3732) | <0.0001 | 0.044 [0.041–0.046] | 0.946 | 0.949 |
Model comparison | χ2 (df) | p | ΔRMSEA | ΔCFI | |
Metric vs. Configural | 45.988 (54) | 0.773 | 0.002 | 0.002 | |
Scalar vs. Configural | 370.658 (342) | 0.137 | 0.005 | 0.007 | |
Scalar vs. Metric | 328.786 (288) | 0.049 | 0.003 | 0.005 |
Skills | Interpretation | Analysis | Evaluation | Inference | Explanation | |||||
---|---|---|---|---|---|---|---|---|---|---|
F | M | F | M | F | M | F | M | F | M | |
Analysis | 0.888 | 0.941 | ||||||||
Evaluation | 0.760 | 0.900 | 0.922 | 0.955 | ||||||
Inference | 0.759 | 0.890 | 0.838 | 0.902 | 0.924 | 0.956 | ||||
Explanation | 0.739 | 0.849 | 0.816 | 0.877 | 0.850 | 0.907 | 0.856 | 0.925 | ||
Self-regulation | 0.720 | 0.808 | 0.738 | 0.780 | 0.759 | 0.825 | 0.805 | 0.907 | 0.782 | 0.885 |
Skills | ΔMeans | SE | Est/SE | p |
---|---|---|---|---|
Interpretation | −0.014 | 0.106 | −0.129 | 0.897 |
Analysis | 0.023 | 0.096 | 0.244 | 0.807 |
Evaluation | 0.071 | 0.096 | 0.736 | 0.462 |
Inference | −0.051 | 0.099 | −0.512 | 0.608 |
Explanation | 0.177 | 0.097 | 1.832 | 0.067 |
Self-regulation | −0.005 | 0.098 | −0.046 | 0.963 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Payan-Carreira, R.; Sacau-Fontenla, A.; Rebelo, H.; Sebastião, L.; Pnevmatikos, D. Development and Validation of a Critical Thinking Assessment-Scale Short Form. Educ. Sci. 2022, 12, 938. https://doi.org/10.3390/educsci12120938
Payan-Carreira R, Sacau-Fontenla A, Rebelo H, Sebastião L, Pnevmatikos D. Development and Validation of a Critical Thinking Assessment-Scale Short Form. Education Sciences. 2022; 12(12):938. https://doi.org/10.3390/educsci12120938
Chicago/Turabian StylePayan-Carreira, Rita, Ana Sacau-Fontenla, Hugo Rebelo, Luis Sebastião, and Dimitris Pnevmatikos. 2022. "Development and Validation of a Critical Thinking Assessment-Scale Short Form" Education Sciences 12, no. 12: 938. https://doi.org/10.3390/educsci12120938