Teaching and Learning to Program: Umbrella Review of Introductory Programming in Higher Education

: Introductory programming courses in science, technology, engineering, and mathematics (STEM) degrees are critical to student success throughout the students’ academic and professional route but have traditional failure and dropout rates. The number of publications and reviews on the subject is growing, so we realize the need to prepare an umbrella review, or review of reviews, to group previous studies and extract more comprehensive and ambitious results. Based on the databases ACM, Google Scholar, IEEE, SCOPUS and Web of Science libraries, a formal search was created that resulted in 21 reviews on programming fundamentals in higher education context. Results include bibliometric information on the CS1 reviews, in the context of higher education (namely annual evolution, number and information on authors, types and sources of information, countries of afﬁliation, languages, keywords and most cited articles), the purpose of the reviews and research questions, methods (namely search strategy, databases used, eligibility criteria) and results (number of records and tables or divisions were made to catalog the articles). We present a taxonomy with four different types of purpose: general, speciﬁc, student group and teacher directed. We found very interesting catalogs that can serve as a tool for future work, whether by researchers in the ﬁeld or by authors who intend to carry out reviews related to introduction to programming, especially in the context of higher education. geek genes, misconceptions, Predicting & measuring success, Programming process data, Other).


Introduction
Introduction to programming is the curricular unit that promotes the initial contact of science, technology, engineering and mathematics (STEM) undergraduate students with computational thinking and programming languages. Programming fundamentals are more than coding, more than translating an algorithm into a language that computer can understand. It is necessary to think and solve the problem to create an algorithm [1]. Much research has been done to explain the traditional failure and dropout rate of students in these early programming units, often responsible for the dropout of students in undergraduate courses [2,3] and also to improve retention in the major [4,5]. This problem concerns researchers and teachers who try in various ways to combat poor results and contribute to the success of the teaching-learning process, sometimes adapting the design of the units and learning objectives [6][7][8]. The Association for Computing Machinery (ACM) and Institute of Electrical and Electronics Engineers (IEEE) curriculum recommendations are a powerful tool for those who must design courses and more precisely those curricular units. In one of the first curricular recommendation documents [9], this curricular unit was named CS1, computer science 1, a name that has already been changed to introduction to programming or programming fundamentals [10,11]. There have been made distinctions between types of courses [12], departments that teach courses or even by continents [13]. While in many countries computer programming is included in early years, from kindergarten to 12th grade (K-12) [14], in other countries students enter university without having any knowledge related to computers [13].
Many of the studies are concerned with the paradigm to adopt [15] and the programming language [1,16], an old discussion [17]. Various techniques such as educational data mining have been used to predict failures in advance [18][19][20], also concerned with initial motivation [21] or promoting increased motivation [22]. Several experience reports have been made to improve the teaching-learning process and the respective success in introducing programming [23,24]. Many of them are related to pedagogical strategies with encouraging results, such as pair programming [25][26][27], flipped classrooms [28][29][30], peer instruction [31,32], cooperative learning [33] and collaborative learning [34]. The attempt to involve students by promoting "learning by doing" has been widely used with active learning approaches [35,36], like Project Based-Learning [37] and Problem-Based Learning [38]. There have been many approaches to the use of gamification [39,40], robots [41,42] and augmented reality [43]. To improve online learning [44,45], use of analogies [46], automatically providing feedback [47,48] or incorporate social media [49]. There are studies that direct to the instruments like Mobile Phones [50] and bring-your-own-device (BYOD) to the exam [51]. There is concern about not leaving anyone out like students with disabilities [52] and having gender concerns, especially regarding how to attract women to informatics [53,54].
As described, there are different perspectives and approaches. There has been an exponential growth of work in this field, as well as an exponential growth of literature reviews, to rigorously bring together several studies for informed decision-making. The following figure ( Figure 1) shows this exponential growth from 2001 to 2020, considering groups of 5 years and two axis scales (left for publications and right for revisions), using a search in the SCOPUS database carried out in April 2021 with the words "introductory programming", "introduction to programming", "novice programming", "novice programmers", "learn programming", "Learning to program", "teach programming", "teach* programming", "learn* programming", "programming education", "Learning to program" or "teach to program" to publications and also "review*" for reviews. We know that these numbers are only indicative, as publications appear in the results that are not exactly the intended ones, such as "code review" instead of the literature review perspective. For these reasons, we think that there is a need for an innovative study that groups these literature reviews, a kind of review of the reviews, so that they can serve as an instrument for work in the area. The research questions to be addressed by this study are: RQ1: What is the bibliometric information on the CS1 reviews, in the context of higher education (namely annual evolution, number and information on authors, types and sources of information, countries of affiliation, languages, keywords and most cited articles)? RQ2: What is the purpose of the reviews and what research questions do they set? RQ3: What were the methods used to find the studies each review was based on (namely search strategy, databases used, eligibility criteria)?
RQ4: What were the results found (number of records and which tables or divisions were made to catalog the articles)?
This document is divided into methods, results and discussion as suggested by PRISMA checklist [55].

Design
Thus, we identify reviews of introductory programming in the context of higher education, commonly known as CS1. This overall review was conducted following the Guidelines for performing Systematic Literature Reviews in Software Engineering, namely the Protocol for a Tertiary study of Systematic Literature Reviews and Evidence-based Guidelines in IT and Software Engineering [56]. This umbrella study is also in accordance with the Cochrane Manual principles for systematic reviews of interventions [57] and the preferred reporting items for systematic reviews and meta-analyses (PRISMA) [55]. In review of reviews, the unit of search, inclusion and analysis of data is the systematic review, not the primary study [57]. There are several different denominations for systematic reviews of reviews such as umbrella reviews, overviews of reviews, reviews of reviews, a summary of systematic reviews and synthesis of reviews [58]. Considering individual studies (or primary research) as the lowest level of the synthesis pyramid, the intermediate levels will be the systematic review, followed by the meta-analysis and finally, the top level will be the umbrella reviews [59]. A tertiary review uses almost the same methodology as a standard systematic literature review.

Search Strategy
A search string was constructed and the logical operators used to complement the execution results. For each database, it was necessary to build a specific query, as each one has a different syntax; and an example of a resulting query is shown below. TITLE (("introductory programming" OR "introduction to programming" OR "noviceprogramming" OR "novice programmers" OR "learn programming" OR "learning to program" OR "teach programming" OR "teach* programming" OR "learn* programming" OR "programming education" OR "Learning to program" OR "teach to program") AND (revie*)).

Information Sources
We considered five different databases to execute the search strings, shown in Table 1. The choice of the ACM, IEEE, SCOPUS and WoS databases was made because they are the most used databases in computer science with high quality and peer reviewed, among other reasons [60,61], while the GoogleS database seems to be more complete [62,63].

Eligibility Criteria
We did not consider language as an inclusion or exclusion criterion because we have access to a greater number of articles. The type of articles accepted were journals, conference proceedings and reviews, so we excluded editorials, for example. The size of the articles to be included must be at least three pages long. The time horizon was not used, considering all publications until May 2021. The study design must be of the review type to be included. Study settings is being in a higher education context, introductory programming, or similar focus. The identification of inclusion and exclusion criteria are listed in the following table (Table 2).

Selection Process
Research result records were found that were not related to the present study, such as related with "peer review" or "program review". First, the databases were searched, then the records were joined and the duplicates eliminated. Subsequently, the records that were not eligible for the inclusion criteria were deleted, first through the title, then through the abstract and then through the reading of the full article. There were only doubts in three records, so the opinion of two experts was asked to get the most correct results.

Results
The literature search identified a total of 42 unique records after removing the duplicate records retrieved by ACM (n = 11), IEEE (n = 10), GoogleS (n = 7), SCOPUS (n = 34) and WoS (n = 26). A total of 14 records were excluded by title and/or abstract, one record because we cannot access the entire text and six for other reasons (length three pages or less, not formal computer science 1 or similar and not higher education context). The result is a sample resulting in 21 reviews on programming fundamentals in the context of higher education. Figure 2 shows Study selection flow diagram (PRISMA diagram adapted) [55]. It was very interesting to see that all papers retrieved from the ACM and IEEE libraries were at SCOPUS too. It was found that only one GoogleS record was unique, but it was discarded as it was not a review in the literature sense but a sequence of words "selfreview". In the end, it turns out that there are only two articles that are not included in the SCOPUS database, but in WoS: they are two articles from the journal Tecnológicas and written in Spanish.

Bibliometrics
Bibliometrics is the statistical analysis of published works, a term used since 1969 by Pritchard [64] and has been used to analyze scientific literature in various fields [65]. Bibliometrics refers to the research methodology employed in library and information sciences, which utilizes quantitative analysis and statistics to describe articles' distribution patterns within a given topic, field, institution or country [66]. Bibliometric studies can build solid foundations for advancing a field, allowing scholars to obtain a complete overview and identify gaps in knowledge, as well as allowing for the derivation of new ideas for investigation [67], a way to provide a valuable reference for future research [68]. This process was originated in medical science due to the increasing amount of research in each area [69]. Consequently, it was necessary to identify and guide the research towards an uninvestigated subject [70]. The scientific community has proposed steps to apply these protocols, more specifically in the software engineering area. In this section we will make use of bibliometric analysis to obtain the information we propose, namely the annual evolution of the research, type of publications, information about authors and their affiliations, publication sources and keyword analysis.   There are 16 different publication sources: ACM International Conference Proceeding Series publishes three of the articles, ACM Technical Symposium on Computer Science Education, ACM Transactions on Computing Education and journal Tecnológicas publish two articles each. Seven articles are linked to the ACM and three to the IEEE.
Three articles were written in Spanish and the rest in English (86%; n = 18). Figure 5 shows the number of authors by article. There are articles with 1 to 5 coauthors and one article with 11 co-authors. The average is 3.29 authors per article, with a standard deviation of 1.93 and the mode is 3 (48%; n = 10). There are 61 different authors, eight of them having two articles. The following table (Table 3) presents these authors name with the indication of their affiliation and ORCID id. The articles have authors with affiliation in 15 countries. The authors with affiliations in Brazil are responsible for 14% of the articles and from Finland for 12% of the articles, as we can see in the following figure ( Figure 6). There are 72 different keywords, the most frequent being systematic review (seven times), introductory programming and programming (six each) and cs1 and programming education (five each). The following figure (Figure 7) represents these keywords in the cloud words format created from the website WordClouds.com [71]. Considering the number of citations presented in SCOPUS, which excludes the two articles published in the journal Tecnológicas, we verified that there are three articles that, at the time of the research, have about 100 citations (one with 96 [72] and two with 95 citations [73,74]).

Purpose
We have divided the articles into four different types of purpose: general, specific, student group and teacher directed. In the specific type, we find nine different goals: Blended learning [75], collaborative work [76], computational thinking [77], Collaborative learning with computer support [78], Intelligent programming tutors [79], MIT App Inventor [80], Robotics [81], Serious programming games [82] and Self-regulation of learning [83]. The papers aimed at students relate to a specific audience: blind [84] and novice [85,86], for the generality of students [72] or directed to the anxiety of students [87]. There are five articles that are general in scope [74,[88][89][90][91] and two that are aimed at teachers and teaching methodologies [73,92].
In PRISMA checklist [93], it is suggested that authors include a systematic review, meta-analysis, or both in the title of the report. Analyzing the title of the articles, we found that the most frequent (n = 8) is the use of the designations "A systematic literature review" and "A systematic review" (n = 5). Other authors use the longer "A Systematic review of the literature", "A literature review", "A Mapping Review" or the more synthetic "A review". Titles can be descriptive (containing the essential elements, namely patients/subjects, interventions, comparisons/control, outcome, and study design-PICOS-but not the conclusion), declarative (shows the main finding of the study) or interrogative (search question in the title) [94]. None of the articles have an interrogative or declarative title. Eight of the articles use the designation "Introductory Programming" [72][73][74][75]85,[88][89][90], five use "Programming Education" [72,77,78,86,91], two just the word "Programming" [81,86] while the rest use teaching programming [80,84,92], learning programming [82,87] or both [76]. The articles referred to that have a special purpose, have the designation of the purpose in the title. Others use words like "evolution" [88], "analysis" [85] or even "innovative approaches" [92].
Two of the articles do not explicitly have research questions; however, we have adapted the text to potential research questions. Almost all research questions start by identifying topics and measuring the methods-the most efficient or the most used instruments. There are articles that intend to know the programming languages used and others the methodology designs. We chose to list all research questions for each article. In Appendix A (Table A1) we present the reference (author and year), the title, purpose and research questions of each article, including the two adapted by us and mentioned above.

Methods
A total of 25 different sources of information were used. The mean is 4.57, the standard deviation is 2.79, the minimum is one and the maximum is 12 sources of information used by each review. ACM Library is the most used data base (n = 19), followed by the IEEE Digital library (n = 18), ScienceDirect (n = 9), ERIC and SpringerLink (n = 8), Google Scholar (n = 5), Web Of Science (n = 4), SCOPUS (n = 3) and the remaining 17 sources of information were used only in one or two studies.
We can consider that the keywords (search strategy) are divided into two parts: the first one related to CS1 and the second related to the specific objective of the literature review. The latter does not exist for reviews that have a general purpose. There are 45 different keywords to search for CS1 related articles, with CS1 (9 times), programming (7 times) and intruductory programming (6 times) being the most frequent keywords. In the second type, there are several words used, for example (robots OR robotics) [81], "App Inventor" [80], Anxiety [87] or cscl [78]. Interestingly, we found that only one of the records [74] uses asterisks for complementary search keywords.
Inclusion criteria are varied, for example language criteria (only articles in english [82,89,91], or in english or spanish [76]),: there are cases where it is limited to one or more publications soureces [73,88], a certain period of time [85,86], domain of computer science [85], directly answered one or more research question [87], beyond the obvious obligations of being related to higher education [75], for example. The exclusion criteria, in addition to those that contradict the inclusion criteria, are related to the minimum size of the article [85,88,89], work in progress [74] or did not address the research questions [89].
In Appendix B (Table A2) we present the reference (author and year), data bases, the search strategy (first part and second), as well the inclusion and exclusion criteria.

Reviews Results
The average of articles contained in each review is 166, with the minimum number being 10 [72] and the maximum number 2189 [74].
The way each systematic review structures its results is one of the most important points in this umbrella review. As described above, we found four different types of purpose: general, specific, student group and teacher directed. We found very interesting divisions. We will then list these divisions by category and article. We tried to make a model, but it seems to us that a model would lose the spectacularity of the different divisions found in each of the articles.
(a) Five articles that are general in scope [74,[88][89][90][91] Ref. [ Ref. [89] Overview of methods (Pedagogical interventions evaluated through analysis of student performance and/or feedback, survey, questionnaires, interviews and/or focus groups, observation of students' learning and strategies, analysis of students' learning styles and strategies, analysis of student performance with regard to type of assessment, pedagogical approach or contextual factors, analysis of student errors, comparison between teachers« predictions/perceptions and students' actual performance, theoretical reflection based on teaching experience, literature review), student skills (category + skill) (Programming-related skills [problem solving, mathematic ability, previous knowledge in programming, abstraction], General education skills [basic knowledge in English, critical thinking and discussion skills, creativity, time management]), student difficulties (category + difficulty) (problem formulation [problem solving, abstract nature of programming, algorithmic and logical reasoning], solution express [syntax of programming language, control structures, data structures, structure of the code, Others], solution execution and evaluation [debugging, tracing the execution], behavior [motivation and engagement, time management, study skills, confidence, perception of programming as a complete discipline], challenges [methods and tools for teaching programming, scale problem, keeping students' motivation, engagement and persistence, teacher-student communication and feedback, programming language, curriculum and instructional sequences, addressing students' inadequate mathematical background]).
Ref. [74] The student (student learning, underrepresented groups, student attitudes, student behavior, student engagement, student ability, the student experience, code reading, tracing, writing and debugging), Teaching (teaching tools, pedagogical approaches, theories of learning, infrastructure), Curriculum (competencies, programming languages, paradigm), Assessment (assessment tools, approaches to assessment, feedback on assessment, academic integrity).
Ref. [90] Teaching/learning problems of computer programming (Difficulties in learning the computer programming, Disadvantages in teaching the computer programming), Tools used in the teaching-learning programming of computers (Visualization or algorithm simulators, Automatic evaluation tools, educational games focused on teaching a specific learning unit, Educational games focused on the teaching of multiple learning units, collaborative environments), Strategies used in teaching-learning of Computer programming (Intervention strategies, Pair programming, Pair code evaluation, Intelligent collaborative virtual environment, Collaborative environments, MOOC Visualization systems, Multimedia tools, Intelligent tutoring systems, Visual learning tools, Serious games, Educational games, Pseudo-languages, Video game creation, Analogies, Metaphors, Robots, Problem Based Learning (PBL), Augmented Reality, Mixed Reality, Frameworks, Programming paradigm and programming language, Procedural programming, Current programming language, Peer tutoring, Languages with a simple syntax, Pre-programming course, Support tools, Projects, Environments, Development of mental models, Spiral approach, Finite state machines, Lectures and laboratory classes, Automatic qualification tools), Methodological considerations in a first programming course Computers (Semiotic ladder, Cognitive taxonomy objectives, Problem solving, Own didactics, Constructivist, Constructionist, Based on the methodology of the didactic process, Based on teaching, Based on the curriculum, Instructional development based on video games, Instructional development oriented towards paradigm, Instructional development focused on particular VARK topics, Based on coding and debugging, Implementation of basic algorithms, Focused on algorithmic complexity).
Ref. [91] year; publication source; number of pages; 25 Most Cited Publications; Research Methods (simple, complex, or mixed); Educational Levels and Participants (k12, high school . . . ), Course Contexts and Themes (CS1 . . . ), Pedagogical Approaches and Observations, Tools and Frameworks (Programming languages, Graphical User Interfaces/Computer Graphics, Robotics), Programming languages that were used in more than two publications.
(c) Five that are focus on students: blind [84] and novice [85,86], for the generality of students [72] or directed to the anxiety of students [87].
Ref. [85] year, publication title, Country, theoretical framework Ref. [86] Development Phase for CS1 (Problem abstraction and solution proposal, Formalizing the solution, Implementing the formalized solution through tools support, Testing the implemented solution and Documentation and submission of work), Support for Instructor (Active learning through tools, Active learning without using tools, Assessment tools, Curriculum and Threshold Concepts, Collaborative learning and other pedagogy ideas). Novice Programming Environment and Intelligent Programming Tutors.
Ref. [87] Reference, Keywords, Publication Source, Type (Conference or Journal), Type of study (Questionnaire, Literature Review, Interview, Experimental) and Number of participants.
Ref. [75] Publication year and geographic distribution (continent); Quality assessment (N Met criterion and N Did not meet) (Outcome measures; Background/literature review; Sample; Study design or methodology; Conclusions); Blended learning components (Online self-paced, face-to-face instructor-led, online collaboration, face-to-face collaboration, online instructor-led); Approaches applied by the reviewed studies.
Ref. [76] Country, Type of publication, (Matrix type of study vs learning approach) Type of study (Solution proposal, evaluation research, opinion articles, Systematic review) and learning approach (Collaborative; Cooperative)), Other significant contributions from the authors, TAC mentioned in documents organized by type cases) ( Ref. [77] Year, publication type, country, Teaching approach & study technique, Learners Context (university and country), Settings (formal or informal), Study impact/evaluation.
Ref. [78] Description, result, Quality analysis (Y, N, partial N/A) (Q1) research objectives definition; (Q2) sample description; (Q3) description of the experimental context (in this study, it is related to programming education); (Q4) description of the experimental design; (Q5) use of randomized controlled trial; (Q6) use of pre/post testing; (Q7) use of control/experimental group), Collaborative resources used in CSCL (Collaborative programming editor, Scaffolding for problem discussion, Peer review, Chat, Sharing techno logical resource, Pair-programming support, Physical, Discussion forums, Group task assignment, Individual contribution awareness, Automatic group formation, Programming observation, Group goals definition, Gamification, Algorithm sharing, Project-Based, Task recommendation, Help request, Voting-system), Outcome (Programming knowledge, User-acceptance, Specific metrics, Number of defects, Creativity index, Algorithm complexity, Code correctness, Motivation, Logical tests, Self-efficacy, Engagement, Self-regulated learning), Collaboration measurements techniques (Analysis of how students interacted with the platform, Interviews, Dialog analysis and group interaction observation, Peer evaluation, Number of algorithm lines, Group self-efficacy, Expert evaluation, Social Network Analysis, Multimodal data).
Ref. [ Ref. [83] Description, Quality Analysis (five items each criterion classified as yes, no and partially: research objectives defined, sample description provided, pedagogical context described, experimental design is presented, use of pre/post-testing), Features used to stimulate SRL (journaling, graphics, questions, worked-examples, social data, feedback, training, pseudocode, prediction), Programming Achievement (Positive or Positive/Neutral), Regulation of learning Development (Positive or Positive/Neutral), SRL Strategies stimulated (Self-assessment; Goal-setting and Planning; Seeking information; Keeping records and monitoring; Reviewing notes, Reviewing tests, Organization and Transformation).

Discussion
This umbrella review had as its first objective to review existing reviews on introduction to programming (CS1) in the context of higher education. ACM, IEEE, SCOPUS and WoS databases and the search strategy presented in Section 2.2 were used. After removing the duplicate records, the inclusion and exclusion criteria were used to discard articles: first, by type, then by abtract and, finally, by the full text. Our final records database consisted of 21 articles, as can be shown in Study selection flow diagram (PRISMA diagram adapted). Four research questions were stipulated and we now answer these questions: RQ1: What is the bibliometric information on the CS1 reviews, in the context of higher education (namely annual evolution, number and information on authors, types and sources of information, countries of affiliation, languages, keywords and most cited articles)?
The year with the most publications is 2019, with 6 publications, 15 articles are conference proceedings, ACM International Conference Proceeding Series is the most frequent publication source (n = 3), 18 articles are written in English, three is the most frequent number of authors per publication, eight out of 61 authors wrote two articles (three with affiliation from Colombia, three from Portugal, one from Ireland and one from New Zealand), Brazil is the most frequent affiliation country, Systematic review is the most frequent keyword. frequent and there are three articles cited in SCOPUS 96 or 95 times.

RQ2: What is the purpose of the reviews and what research questions do they set?
There are four different types of purpose: general, specific, student group and teacher directed. The most frequent propose is specific with nine papers, Blended learning, collaborative work, computational thinking, Collaborative learning with computer support, Intelligent programming tutors, MIT App Inventor, Robotics, Serious programming games and Self-regulation of learning. It was not possible to find a model for the research questions as they are so diverse. For this reason, we chose to list in attachments all research questions for all articles that had research questions and suggesting research questions for two articles that did not provide this methodology.
RQ3: What were the methods used to find the studies each review was based on (namely search strategy, databases used, eligibility criteria)?
ACM Library is the most used data base (n = 19), followed by the IEEE Digital library (n = 18). The search strategies is divided into two parts: one related to CS1 and another related to the specific objective of the literature review. The latter does not exist for reviews that have a general purpose. There are 45 different keywords to search for CS1 related articles, with CS1 (9 times). Inclusion criteria are varied and listed in Appendix B.
RQ4: What were the results found (number of records and which tables or divisions were made to catalog the articles)?
We chose to list (Section 3.4) by each of the four different types of purpose: general, specific, student group and teacher-directed and by reference, which divisions were elaborated by the review authors.
We found that there is an increase in the number of articles and literature reviews on the fundamentals of programming in higher education. We feel the need to group these studies together-to serve as tools for future studies, as well as a clear identification of what has been published. When analysing the methodology used in the literatureour sample-we found that there are very different perspectives, mainly because half of the studies have a specific purpose. Thus, the authors use a generic perspective (CS1, introduction to programming . . . ) accompanied by a specific perspective (people with disabilities or a specific application to be used in the teaching-learning process, for example). Another curious finding was that CS1 is still the most used acronym for this type of curricular unit, although it was used for the first time in 1978 and later many other curricular recommendation documents have been published with other names for these introductory courses. The proposed framework-general, specific, group of students and directed by the teacher-is expected to be a good starting point for those who intend to invest in research in this area. In addition, it is important to investigate in this area-to improve the teaching-learning process and to understand how it can be improved more and more.
There are always, in each study, limitations of the evidence included in the review. Studies are always being published, so a study like this needs constant updating. We think a lot about methodologie adopted and we are not always sure they are the best, namely in relation to research strategies and the use of other keywords, namely in other languages, to find a more comprehensive search.
As future work, it was realized that it would be interesting to create a taxonomy that would serve as a model for literature reviews in order to facilitate the work of authors and those who read these reviews and then use this taxonomy for introductory programming articles. This article can be an excellent tool for researchers in the area, finding the different approaches to the subject and different perspectives, as well as for those who intend to carry out a systematic review because the framework of different methodologies used by other authors in high quality works is here.

Data Availability Statement:
The data presented in this study, are available on request from the corresponding author.

Conflicts of Interest:
The author declares no conflict of interest.  Computational thinking A systematic review of computational thinking approach for programming education in higher education institutions RQ1 How has the use of CT been explored for teaching programming in HEIs? RQ2 What are the ways CT approach has been used to teach programming education in HEIs? RQ3 How has the use of CT approach for teaching programming impacted students learning experience?   "programming"

Appendix A
("blended learning" OR "blended course" OR "blended classroom" OR "blended class" OR "hybrid learning" OR "hybrid course" OR "hybrid classroom" OR "hybrid class") I1 ACM, ERIC, IEEE, SemanticS and Springer ("learn" OR "learning" OR "teach" OR "teaching") AND ("introductory programming" OR "CS0" OR "CS1") AND ("programming" OR "code" OR "coding") AND ("undergraduate" OR "higher education" OR "college" OR "university") AND ("computer" OR "computing") ("motivation" OR "interest" OR "engagement") ACM, IEEE, ScienceDirect, SCOPUS and Springer "introductory programming" OR "introduction to programming" OR "novice programming" OR "novice programmers" OR "CS1" OR "CS 1" OR "learn programming" OR "learning to program" OR "teach programming" E1. Less than four pages long (such as posters). E2. work in progress  ACM, IEEE and ScienceDirect ("teaching" OR "learning") AND ("programming" OR "programming languages") ("innovative" OR "innovation") AND ("case study" OR "case studies" OR "lessons learned" OR "experience report") ("computer science education" OR "software engineering education" OR "introductory computer" OR "introductory programming" OR "teach* programming" OR "learn* programming" OR "novice programming" OR "coding education" OR "cs1" OR "introductory computer science") ("intelligent tutor*" OR "adaptive tutor*" OR "cognitive tutor*" OR "smart tutor") I1. Relates to a completed IPT that has been used by students learning programming. I2. Relates to a programming language that has conditional and iterative control structures. I3. Tutors scripted programming, i.e., the student has to type lines of code as part of the tasks. I4. Meets a working definition of being intelligent by adapting some part of the tutoring process based on multiple metrics and/or an algorithm.