A Comprehensive Analysis on Computational Thinking in Education: Open Issues and Challenges ^†

Abstract

Computational thinking (CT) is a cognitive approach for solving problems using the concepts of algorithmic thinking, decomposition of a problem into components, identifying patterns among commonly occurring activities, and abstraction. CT promotes interdisciplinary learning and enhances problem-solving and logical reasoning abilities. In this study, a comprehensive analysis of the current issues and challenges of applying CT in the educational landscape is presented with a focus on the various assessment tools and their implementation in teaching methods. The study identifies the various techniques that can be used by educators to evaluate the skills of students based on their ability to solve problems that require CT. A systematic review of the available literature and related works was conducted to analyze their importance in CT, as well as their issues and challenges. This study finds that there is a need for a unified definition and implementation guidelines on CT. The available assessment tools mainly focus on programming constructs, leaving little room for evaluating abstract concepts as challenges in the field; hence, designing and developing assessment mechanisms are also required for effective implementation of CT in an academic context.

1. Introduction

Computational thinking (CT) is a cognitive framework that promotes problem-solving, analytical, and logical reasoning by obtaining concepts from computer science. The term computational thinking (CT) was first coined by Jeanette Wing in 2006 [1] and defined as

“CT is a process of solving problems, designing systems, and understanding human behavior by drawing on the concepts fundamental to computer science.”

Ever since then, significant research has been happening in various aspects, especially cognitive processes and computer science education. Researchers have delved into various terms associated with CT and how educational institutions can assess CT skills in students.

CT and its application in education have attracted significant interest among researchers and educators to explore more approaches in the domain. The main advantage of introducing CT into curricula is that it helps students to gain higher-order cognitive skills and algorithmic thinking approaches. The conceptualization, teaching, and assessment of CT remain a challenge with varying interpretations of its scope and implementation strategies. Programming and programming approaches are integral parts of CT and its application. Apart from programming, the ability to form an appropriate approach that helps analyze and solve problems reflects CT.

This research reviews various works on CT and its role in education to understand the various limitations and issues in evaluating CT, as well as its associated issues and challenges. This review paper is structured into the following sections: Section 2 reviews the recent literature, Section 3 summarizes the findings, open issues, and challenges we obtained from the literature, and Section 4 concludes the paper.

2. Literature Review

In this study, we conducted an extensive review of CT. Our aim was to identify and get a clear view of advancements made in CT concerning computer science education and cognitive processes. A systematic review of the literature was performed for this paper. The related works were searched for in online sources with the strings “computational thinking”, “computer science education”, and “problem-solving in CT” in the following online databases: Google Scholar, Education Resources Information Centre (ERIC), ResearchGate, and Elsevier. After obtaining papers from these databases, relevant papers were chosen for review based on the context in which they relate to CT processes and computer science education. The papers were then reviewed by taking note of their objectives, the findings, and the methodology used. After the literature review, the review was analyzed, extracting the common themes, trends, and emerging perspectives in CT in computer science education. Selby and Wollard [2] reviewed several related works, searching for a definition of CT to aid in curriculum design and assessment in computer science education. The authors used an analysis of various literature containing the term CT, wherein they filtered multiple words that could contribute to the definition of the term. They propose that CT is the cognitive processes that reflect the ability of abstraction, which is the process by which complex problems are simplified, and there is a focus only on essential features, decomposition is the breaking down of a problem into sub-problems, algorithmic design refers to the process of solving a problem in a step-by-step process, evaluation is the process of assessing the solutions in terms of time and space complexity, and generalization in which is the realization of common patterns from specific problems so that they may be applied in broader contexts.

Antonella Nuzzaci [3] reviewed the conceptualization, implementation, and significance of CT in education. The integration of CT in educational settings relies heavily on the preparedness of teachers with the necessary skills to teach CT to enhance students’ ability to think and solve problems critically. The paper highlights the need for a proper definition of CT so that it can be integrated into curricula more efficiently. Joseph A. Lyon and Alejandra J. Magana [4] reviewed empirical studies on CT in higher education. The study finds that research in CT is growing but lacks clear definitions and implementations in higher education. Studies show that some curricula emphasize problem-solving and algorithmic thinking skills, and others focus on the applications of CT, so there is a diverse approach to incorporating CT in educational settings. The study finds benefits of incorporating CT in classrooms to improve critical thinking and problem-solving skills, but it is not always uniform and is at times context-dependent. The assessment of CT skills still remains inconsistent, with some researchers advocating for standardized assessments while others prefer formative assessments that capture student learning in real time. Their methodology is a systematic literature review identifying 13 relevant studies from four online databases: Education Resources Information Center (ERIC) and EBSCO: Education Source, along with Engineering Village: Inspec and Engineering Village: Compendex.

Vance Kite et al. [5] reviewed the trends and issues in CT education research. The study finds that there are varied conceptualizations of CT in existing curricula due to a lack of a proper definition, and most curricula focus on teaching programming as part of CT, ignoring the multidisciplinary approaches in teaching CT, such as through unplugged activities. The study also finds a gap in the professional development of teachers with reference to pedagogies for teaching CT. The methodology they used to conduct the review involved iterative searches in databases, specifically the ERIC database and Google Scholar, to locate relevant empirical studies on CT in education, with a time frame from 2006 to 2017.

Nor Hasbiah Ubaidullah et al. [6] contributed to developing a model teachers can use to enhance students’ CT skills. The methodology they used to create the model was the Delphi technique. This structured communication technique involves a panel of experts who are asked to opine on a particular field of research. Their opinions were then further analyzed, and a survey was carried out for three rounds to ensure the experts would finally reach a strong consensus or agreement on the elements and sub-elements of the proposed teaching and learning model. The Delphi technique in this study successfully identified and validated six elements and 25 sub-elements of the teaching and learning model. The six elements they considered are as follows:

• E1: Understanding and Defining Problems.
• E2: Planning Problem Solving.
• E3: Designing Problem Solving.
• E4: Writing Codes.
• E5: Testing.
• E6: Metacognitive Skill.

In their study, Nor Hasbiah Ubaidullah et al. identified that E2 and E3 are the most crucial for developing students’ CT skills.

Betul C. Czerkawski and Eugene W. Lyman [7] have explored CT issues in higher education. They remark that the integration of CT in K-12 has been systematic; however, it has been sporadic in higher education for graduate studies. The paper emphasizes that CT should also be taught to students in the humanities. They exemplify innovative courses designed to teach CT without requiring programming skills, such as using flowchart simulators that show data flow in a process, allowing students to think algorithmically. They also stress the importance of accommodating diverse socio-economic and cultural backgrounds in CT curricula.

Siu-Cheung Kong et al. [8] discuss the concept of CT Education and its integration into Science, Technology, Engineering, and Mathematics (STEM) education. They have explored designing programming environments to facilitate learning CT and also assessment tools, which include formative iterative tools – providing continuous feedback during the learning period, skill transfer tools – the ability to apply CT skills in different contexts, and summative tools – evaluating students’ overall performance at the end of the learning period. They have also examined how educational policies can support the integration of CT into existing curricula. Anna Akerfeldt et al. [9] reviewed empirical studies conducted between 2006 and 2018, focusing on K-12 education and highlighting key teaching and learning programming findings. The review finds that teaching CT skills and programming improves student academic performance. Video games, such as puzzle solving, also serve as a potential pedagogy for inculcating CT skills in students. The review included a comprehensive search for articles using the EBSCO Discovery Service and the Science Web. This resulted in 57 empirical studies meeting the criteria for focusing on formal education for students aged 0–18.

Shute et al. [10] proposed a competency model, which is a framework that outlines a collection of skills necessary for CT to aid in guiding assessments and supporting development in educational settings. The methodology they used to develop the model was a systematic literature review, in which they collected related work and divided it into conceptual and empirical papers related to CT. The authors have also analyzed models proposed by researchers to understand the cognitive processes underlying CT. The proposed model aims to integrate CT into K-12 educational settings through programming, robotics, and game design interventions, which shows promise in enhancing an individual’s problem-solving abilities. The paper emphasizes that CT is not merely knowledge of a programming language but the ability to think logically in abstractions and the decomposition of complex problems.

Peter J. Rich et al. [11] have reviewed the tools used for teaching CT, ranging from unplugged activities such as board games and software applications such as Scratch to physical devices such as robotics, which all contribute immensely to developing CT skills. The study finds that teaching methods with hands-on learning engage students more and improve the learning process. The skills taught in elementary schools include CT skills (algorithmic thinking, abstractions, decomposition, generalization), programming skills (iteration, selection, debugging), CT practices (collaboration, testing, communication), and data practices. The authors conducted a systematic review of 1000 articles, ultimately including 154 studies that met their criteria of the tools and trends used to teach CT in K-6 education.

Joseph Agbo et al. [12] have examined the use of CT in programming education at higher education institutions (HEIs). The study finds that educators have increasingly adopted CT approaches into their teaching methods through course design, project-based learning, visualization tools, puzzles and games, and smart learning environments, such as a student-centered approach, collaboration, and real-time feedback. The impact of such practices in programming education has significantly enhanced students’ problem-solving abilities. The findings suggest that this approach could foster critical thinking and adaptability among students. The methodology they used to conduct the research was a systematic literature review of 161 articles, with 33 meeting the inclusion criteria for the study.

Ting Ting Wu et al. [13] reviewed advancements in CT and its relation to problem-solving skills. The authors used a systematic literature review, following the systematic Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) review model developed by Moher et al. (2009). The paper concludes that the stages of decomposition, pattern recognition, abstraction, and algorithms in CT are closely linked to those in problem-solving. The review also shows that problem-solving is a part of CT, and there is a high correlation between the two concepts, such that more confident problem-solvers have higher CT skills. Min Jou et al. [14] studied the relationship between CT ability and motor skills in engineering students. CT is a mental process that involves solution thinking and algorithmic thinking, which is very close to motor skills, which, in turn, consist of specific movements of body muscles to perform a certain task. The teaching model adopted was Creative Problem Solving (CPS), which teaches basic programming in the first three weeks, followed by practical projects on creating a remote temperature and humidity monitoring system, focusing on concrete, hands-on learning, and application of CT in real-world scenarios. The result indicates that, compared with the usual teaching model, a creative problem-solving teaching model could improve CT and motor skills, and the effectiveness of improving the two competencies would be more significant in technology universities. It highlighted that integrating CT into subjects, especially those closely related to students’ daily lives, was necessary to enhance learning effectiveness.

Jian Liao et al. [15] conducted a study on scaffolding CT with ChatGPT. The term scaffolding refers to a supportive educational environment where students can get help while they navigate through complex problems to promote students’ critical thinking ability and creativity. Here, the process of scaffolding is achieved through ChatGPT. The study utilizes a tool, namely Intelligent Programming Scaffolding System with ChatGPT (IPSSC), with three modules: Solution Assessment (SA), Code Assessment (CA), and Free Interaction (FI). The SA module helped students break down a particular problem; the CA module was designed to help students write better code and algorithms. The FI module allowed the student to interact with ChatGPT to clarify doubts. The findings of the study show that students’ CT ability developed while following the scaffolding technique with ChatGPT, which was analyzed through a mixed method of evaluation utilizing both quantitative through a paired test using the Computational Thinking Scale (CTS) by Korkmaz et al. and qualitative, through interviews and questionnaire, data to assess the effectiveness of the Intelligent Programming Scaffolding System using ChatGPT (IPSSC). Tongxi Liu [16] introduces an innovative approach to exploring a student’s CT skills through gameplay, such as puzzle solving. The game that was used is Zoombinis, which is a logical puzzle-based game that enhances students’ CT skills. The findings conclude that researchers can assess students’ CT skills in this context by observing their gameplay actions.

Antonia Gonzalez Torres et al. [17] present a study wherein they have implemented a prototype for assessing the CT skills of students by utilizing a syntactic analysis of code methodology written by students. The assessment utilized in this paper evaluates the student performance based on programming structures, code clones analysis to identify similarities with reference solutions, and execution verification of the code. The findings suggest that this methodology can significantly aid educators in measuring the CT skills of students. Zhang Wei et al. [18] have implemented an assessment model of college students’ CT with text-based programming. Text-based programming is writing textual code instead of visual programming, where programming is performed through manipulating graphical elements. The study identifies five core elements of CT: Abstraction and Decomposition, Data Processing, Algorithm Design, Modeling, Perceptions, and Attitudes. The proposed model maps the mentioned core elements of CT with the core elements of text-based programming. The study was conducted on 52 college students whose skills were assessed using the model. The findings reveal a normal distribution of the students with CT levels with a significant correlation and consistency, implying that the assessment is consistent across different programming tasks. The assessment tools first analyze the code written by the student and then map it to the core elements of CT, after which they measure each element using normalization methods. The methodology used in the paper is a comprehensive and structured approach that combines theoretical analysis, practical assessment, and statistical validation to evaluate the CT skills of college students through text-based programming.

Soham Bhatt et al. [19] present a method for developing a process-based assessment for CT tasks. This methodology emphasizes a formative assessment technique through creating and problem-solving. The authors outline a mapping between programming tasks to specific CT subskills, defining measurable factors for CT use and using a Hidden Markov Model (HMM) for analyzing the choices made by students while solving a task. The HMM is a statistical model representing hidden states and their transition based on probabilistic rules. They are commonly used in natural language processing to infer hidden processes from observable data and create some patterns of behavior based on the same. The assessment tool highlights four phases a student goes through while solving a problem: start, experimental, logic, and action. This empirical study assessed 29 student solutions to a tic-tac-toe programming task using the tool. The assessment results were compared with a traditional-based assessment, and the analysis showed that the two results did not align. However, the assessment tool gave more insights into student engagement and strategies.

Ritzki Zakwandi et al. [20] implemented an assessment tool using a two-tier computerized adaptive test. The two tiers in the two-tier Computerized Adaptive Test (CAT) refer to the structure of the assessment items, which are designed to evaluate not only the students’ answers but also their reasoning behind those answers. This allows for a more thorough assessment of the student’s solution for a particular problem. This tool can accept varied responses, allow for measurement across difficulty levels, and provide insights into the student’s ability level. The study revealed that while students gave the reason for a problem in the second tier, 37.02% of the students were solving the problem incorrectly. This allowed the teachers to intervene and clear the misconceptions of the students. This assessment tool reported a high percentage of acceptance rate among both students and educators.

The literature review findings show that research in CT has grown tremendously, focusing on integrating CT into existing curricula in K-12 settings and higher educational institutions. The processes involved in CT, such as problem-solving, algorithmic thinking, and so on, are a recurring theme in the literature review. They have been incorporated into various teaching approaches in programming, such as through unplugged activities like puzzle solving and memory games. Many assessment tools have been developed, and there is a commonality among them: the mapping of CT skills to a programming construct, as most of the measurements are based on the program. Figure 1, shows the frequency count of papers selected based on year of publication.

3. Findings, Open Issues, and Challenges

The literature review has elaborated on the various cognitive processes and skill sets associated with CT and its incorporation into computer science education, with each paper broadening the scope of the various CT processes, such as abstractions, decomposition, and algorithmic thinking, with Planning Problem Solving and Designing Problem-Solving as two key aspects in the process of CT.

Though studies define CT differently, a convergence of the literature indicates that CT may be defined generally as a mental process that allows people to write down and resolve problems by applying computational ideas like abstraction, decomposition, pattern recognition, and algorithmic thinking. This definition is consistent with Jeanette Wing’s original work and incorporates lessons from subsequent studies that underscore its cross-disciplinary uses outside computer science. CT is not just coding capability but more of a systematic approach to solving difficult problems, so it is applicable across fields like mathematics, engineering, and even the humanities, although more research is needed in the humanities sector as to how we can apply concepts of computational thinking. As research continues to develop, a more integrated definition of CT will enable teachers to create systematic curricula and testing instruments that test a broader array of CT skills, creating a better understanding of its place in education.

The review has shown that educators need to emphasize developing the key cognitive processes of CT in students through various pedagogical approaches. Using modern teaching methods such as using software to demonstrate problem-solving techniques, digital devices to aid in learning, unplugged activities such as board games or puzzle games like Zoombinis, and most importantly, providing a hands-on approach, is most effective in ensuring that students enhance their CT skills.

Table 1. Open Issues in Computational Thinking.

Paper	Methodology	Achievement	Open Issues
Selby and Wollard (2010) [2]	Literature analysis, filtering terms associated with CT to refine its definition.	Defined five skills that can be mapped to CT	Lack of consensus on a universally accepted definition of CT.
Czerkawski and Lyman (2015) [7]	Systematic Literature Review	Highlighted the challenges and significance of integration of CT	Need for more systematic integration of CT in higher education.
Shute et al. (2017) [10]	Systematic Literature Review of conceptual and empirical papers on existing CT models	Development of a competency model which defines the outline for CT	Need for empirical studies to validate their proposed model and to refine CT pedagogy and assessment practices.
Kong et al. (2019) [8]	Design and analysis of tools and environments for CT education.	Providing a structured approach to understanding, assessing, and implementing CT in schools	Policy challenges in scaling CT across curricula. The development and use of assessment tools in CT education.
Agbo et al. (2019) [12]	Systematic literature review of 161 articles, with 33 meeting inclusion criteria.	Establishes how CT can be effectively integrated into programming education	Limited studies on long-term impacts of CT integration.
Ubaidullah et al. (2020) [6]	Delphi Technique	Developed a teaching model to enhance CT skills	Practical tools for CT model implementation are needed.
Lyon and Magana (2020) [4]	Systematic Literature Review	Understanding how CT is implemented in Higher Education settings	Variability in CT implementation across disciplines. Lack of standardized CT assessments.
Kite et al. (2021) [5]	Systematic Literature Review on Empirical Studies	Given recommendations on how to implement CT in curricula	Gaps in professional teacher training for CT.
Min Jou et al. (2023) [14]	Implementation of Creative Problem Solving (CPS) model in programming education	Enhanced the CT abilities of engineering students.	Challenges in generalizing CPS models to other settings.
Akerfeldt et al. (2023) [9]	A structured literature review of 57 empirical studies.	Understanding of how CT and programming are integrated into education	Scalable game-based CT interventions are needed.
Wu et al. (2024) [13]	Systematic literature review using the PRISMA framework	Established that CT and problem-solving go hand in hand	Lack of diverse CT applications beyond problem-solving.
Nuzzaci (2024) [3]	Systematic Literature Review	Established the challenges for teachers in integrating CT into curricula	Need for universal CT frameworks and teacher training.
Rich et al. (2024) [11]	Systematic Literature Review	Highlights effective strategies for teaching CT	Limited multidisciplinary applications for CT tools.
Liu (2024) [16]	Empirical data analysis with game-based learning	Finds Zoombinis as an effective game for enhancing students’ CT skills	The scalability of gameplay-based CT assessment is a challenge.
Jian Liao et al. (2024) [15]	A mixed-methods approach, combining quantitative and qualitative research methods	Improved students’ CT skills and enhanced engagement	Balancing scaffolding and independence. Overreliance on AI.
Antonia Gonzalez Torres et al. (2024) [17]	Analysis of syntactic code	Introduced a robust method for analyzing large amounts of student submission	The long-term effectiveness of the tool and poor feedback mechanism.
Zhang Wei et al. (2023) [18]	Text-based Programming Assessment tool	Mapping relationship between CT Skills and Programming Constructs	Generalizing the model for a higher sample size.
Soham Bhatt et al. (2024) [19]	Process-based assessments using Hidden Markov Models (HMMs)	Effective use of HMM	Dataset Limitations and interpretation of hidden states.
Ritzki Zakwandi et al. (2024) [20]	Design and Development Research (DDR) approach, following the Borg and Gall model	Accurate estimation of CT skills and clearing up misconceptions	Further research on the two-tier items and long-term impact is yet to be assessed.

summarizes the key open issues in CT as identified in various studies. These issues range from challenges in defining and integrating CT into educational curricula to the need for effective teacher training, scalable interventions, and standardized assessment tools. Each issue highlights areas requiring further research and development to strengthen the role of CT in education.

A more critical review of the methodologies adopted in the analyzed studies reveals a few significant weaknesses. Although numerous studies highlight the significance of CT education, their research approaches remain largely non-empirical in nature. For example, the 2015 paper by Czerkawski and Lyman indicates the difficulties involved in implementing CT in higher education, but it is heavily based on theoretical debates without meaningful empirical support. The 2017 papers by Shute et al. suggest a competency-based model for CT assessment, but its validity has not been tested on heterogeneous student populations and learning environments. Most CT assessment instruments are centered on programming constructs, such as the assessment tool proposed in the 2023 paper by Zhang Wei et al., which might not reflect the more general cognitive processes involved in CT. These gaps point toward the necessity for more rigorous, longitudinal work that evaluates the effect of CT interventions on diverse educational settings so that comparative conclusions can be drawn and translated to real instructional settings.

The CT assessment tools reviewed depend mainly on program-based assessment, like those in the works of Zhang Wei et al. in 2023 and Soham Bhatt et al. in 2024. As much as the tools efficiently identify students’ potential to implement CT in programming problems, they mostly cannot pick up on such abstractions in the form of decomposition of a problem, recognizing a pattern, or reasoning. Few models cover these cognitive aspects, as pointed out by Selby and Wollard in 2010, emphasizing the importance of tools that measure CT outside programming contexts. New AI-based models, like Hidden Markov Models (HMMs) and text-based programming analysis software, hold potential for measuring CT through process-oriented approaches. Limitations in datasets and the interpretation of hidden states deter their wider usage. As the assessment tools in this study follow a common trend of evaluating through programming constructs, with a few tools giving room for human reasoning factors; this brings us to a key finding of this review wherein there does not exist a tool that can evaluate the computational thinking skills without the involvement of programming constructs which brings the scope for identifying one without using programming concepts.

Lyon, Magana, and Ubaidullah et al., in 2020, point to the absence of universally agreed-upon CT assessment frameworks, resulting in differences in the measurement of CT skills within various learning environments. The scalability of game-based learning platforms, for example, Zoombinis, is not yet achievable because of resource limitations and limited large-scale use. The majority of CT assessment research is conducted on short-term evaluations, with few studies on long-term CT skill development and retention. These shortcomings suggest that there is a requirement for more comprehensive assessment methods that combine cognitive psychology-based tests, AI-powered analytics, and longitudinal studies to gain a thorough understanding of CT skill development over time.

The challenges identified through the comprehensive analysis are the lack of a universally accepted definition of CT. The assessment tools identified in this study mostly rely on programming constructs for evaluation, which poses a challenge in measuring the abstract aspects of CT skills, such as problem-solving and decomposition of the problem. There is also an implementation gap regarding teaching and learning CT in different socio-economic contexts. The curriculum integration across different levels is structured in the K-12 level, while the higher education level is not uniform. Most curricula also focus mainly on programming to teach CT, whereas they could also use unplugged activities to do the same, which, however, raises another challenge, which is the training of teachers to teach such techniques.

4. Conclusions

The cognitive processes of abstraction, pattern recognition, decomposition, algorithmic design, complexity, and generalization all play an important role in a student’s problem-solving capabilities and CT skills. The comprehensive analysis reveals that when students are exposed to solving problems that require them to use the mentioned cognitive processes, the higher the improvement in the students’ CT skills. With the cognitive processes in CT listed, incorporation of the same into computer science education has been rapidly developing, with various trends and tools being used to teach CT not only in the specific domain of computer science but also branching out to other disciplinary courses, such as the humanities.

In education, incorporating a combination of programming-based and unplugged CT activities so that students develop both computational thinking abilities and abstract problem-solving abilities is needed. Adding on to this, implementing game-based learning, project-based assignments, and interdisciplinary CT applications can also enhance engagement and comprehension.

For researchers, there is room for a focus on longitudinal studies assessing the long-term impact of CT education by subject and student group. There is scope for developing more holistic assessment tools that test CT competencies beyond programming, such as problem decomposition, abstraction, and creative problem-solving.

For policymakers, they can contribute to standard CT frameworks for K-12 and higher education to ensure uniformity in which CT is included within the curricula, as well as invest in teacher education courses that will best equip teachers with the necessary skills to teach CT effectively, both through the use of digital and unplugged approaches. Investment in scaling up CT interventions that can also be scaled up or down, depending on the education environment and student exposure, is also recommended. The future of CT education is to prepare educators with the competencies and resources needed to instill computational thinking above programming skills. CT is not only about coding but also includes abstraction, algorithmic thinking, and problem-solving in multiple domains. The creation of standardized, domain-independent assessment tools is still a priority area for research. Implementing standard assessment procedures would not only better measure student progress but also inform and continue to refine pedagogical approaches to teaching CT. Ongoing activity in these respects will make sure that CT training remains flexible, effective, and affordable across a range of educational environments.