Application of Virtual Reality in Computer Science Education: A Systemic Review Based on Bibliometric and Content Analysis Methods

This study investigated the role of virtual reality (VR) in computer science (CS) education over the last 10 years by conducting a bibliometric and content analysis of articles related to the use of VR in CS education. A total of 971 articles published in peer-reviewed journals and conferences were collected from Web of Science and Scopus databases to conduct the bibliometric analysis. Furthermore, content analysis was conducted on 39 articles that met the inclusion criteria. This study demonstrates that VR research for CS education was faring well around 2011 but witnessed low production output between the years 2013 and 2016. However, scholars have increased their contribution in this field recently, starting from the year 2017. This study also revealed prolific scholars contributing to the field. It provides insightful information regarding research hotspots in VR that have emerged recently, which can be further explored to enhance CS education. In addition, the quantitative method remains the most preferred research method, while the questionnaire was the most used data collection technique. Moreover, descriptive analysis was primarily used in studies on VR in CS education. The study concludes that even though scholars are leveraging VR to advance CS education, more effort needs to be made by stakeholders across countries and institutions. In addition, a more rigorous methodological approach needs to be employed in future studies to provide more evidence-based research output. Our future study would investigate the pedagogy, content, and context of studies on


Background of the Study
Virtual reality (VR) has recently become a popular technology in different contexts such as entertainment, military, and education [1]. VR combines technologies to provide an immersive presence through highly interactive objects in a virtual environment but stimulates users' sensory awareness to perceive being in an almost natural environment. The use of VR in education to support training, teaching, and learning through 3D simulation and visualization of learning content in a virtual presence has grown recently [2]. This increasing VR application growth in the educational field is evident, as revealed by the literature, including a recent VR study in computer science education [3]. VR technology provides an opportunity to develop a state-of-the-art smart learning environment with a high level of interaction, engagement, and motivation for an enhanced learning experience [1][2][3][4][5][6][7][8]. This study refers to computer science (CS) education as the art and science involved in learning and teaching computer science, including computing, algorithmic and computational thinking [9]. For example, the science behind curriculum design, pedagogical approach, and instructional tools and techniques educators adopt to support computer science teaching and learning.
This study investigated the role of VR in CS education by conducting a comprehensive content and bibliometric analysis of relevant articles published between 2011 and 2020 in journals and conferences. Bibliometric and content analysis of articles focused on VR in CS education would provide a deeper understanding of the evolution of research conducted in this field and how VR applications have advanced CS education over the years [4,10,11]. From the standpoint of bibliometric mapping analysis, this study investigates the publication growth of studies on VR in CS education within the last 10 years, reveals the most active authors and affiliations contributing to the development of VR in CS education, and anticipates the future direction on the basis of the co-occurrence pattern analysis of current studies. In addition, this study explicates the role of VR in CS education from the perspective of methodological approaches used in studies related to VR in CS education [7], the kind of data collected for such studies, the sample size, and the types of data analysis conducted.
Research on VR in education has claimed several benefits, such as positively affecting users' attitude [12,13], presenting an effective and efficient learning and training environment [14,15], and increasing students' motivation to learn within a virtual environment [14][15][16][17]. Furthermore, many systematic review studies related to VR in education have been published in recent years. However, there have been only a limited number of such studies focused on computer science education. For example, Pirker et al. [3] conducted a systematic literature review of VR in CS education, focusing on the technology used to deploy VR applications for CS education, the learning objectives, and challenges recorded in studies related to VR in CS education. Pirker and colleagues revealed that VR desktop applications using Oculus Rift and HTC Vive dominate the technology currently used to deploy VR in CS education. On the other hand, the majority of studies on VR in CS education focused on cognitive learning with topics such as fundamental components of algorithms and object-oriented programming [3].
Similarly, Oyelere et al. [1] studied VR games in CS education, focusing on developmental features such as the technology, pedagogy, and gaming elements used in such studies. In terms of technology, Oyelere et al. [1] finding was in congruence with that of Pirker et al. [3], where Oculus Rift, HTC Vive, and PC-based applications dominate the technology aspect. Both studies show that mobile-based VR applications for CS education are still growing, with less than 15% of deployment of VR applications on mobile devices.
We could find only a few studies regarding recent studies that focused on content and bibliometric analysis of articles related to VR in education. For example, Arici et al. [11] conducted content and bibliometric mapping analysis of augmented reality (AR) in science education. Lorenzo et al. [17] investigated VR articles' scientific production for inclusive learning of people with autism spectrum disorder (ASD). Sobral and Pestana [18] studied a bibliometric analysis of articles related to VR application to learn about dementia from 1998 until 2018 by focusing on articles' intellectual structure and emerging trends. Lai et al. [19] conducted a bibliometric analysis of VR research in engineering education published and indexed in the Scopus database that spans over 26 years. Thus, Lai et al. [19] provided valuable insights in terms of article production, trends, and co-occurrence network of VR studies within the field of engineering. Another bibliometric study related to VR in CS field-specific was recently conducted by Enebechi and Duffy [20]. This study [21] focused on bibliometric analysis of VR and artificial intelligence (AI) articles in mobile computing and applied ergonomics.
While all these related studies highlighted above are relevant and provided essential knowledge about the field, our current research would expand on the existing research rather than re-inventing the wheel. For example, while the work of Pirker et al. [3] mainly focused on the technology used to deploy VR application for CS education, the learning objectives, and challenges recorded in studies related to VR in CS education, our research would address the aspect of methodological approach used in studies on VR in CS education, kind of data collected for such studies, the sample size, and types of data analysis conducted. The majority of these related studies analyzed a small sample size, limiting the study, and cannot justify the generalization of their findings. For example, Pirker et al. [3] analyzed 13 pieces of data, Lorenzo et al. [17] revealed 18 articles, Lai et al. [19] conducted bibliometric analysis on 274 articles, and Enebechi and Duffy [20] presented a content analysis of 8 papers. Our study took a different approach by analyzing more extensive data to discover more profound knowledge in the field. It is worth mentioning that our study drew motivation from [11] by focusing content analysis of variables such as materials and method trends, sample sizes, and method of an investigation conducted by articles on VR in CS education in the last 10 years. The authors hope that the approach used in this study would contribute to the existing knowledge in terms of unveiling how VR has supported CS education and what scientific achievement have been made in this field.
As a result of this comprehensive content and bibliometric analysis of studies on VR in CS education, we hoped that our findings would contribute to the existing knowledge by providing a deeper understanding of VR applications' role in honing CS education over the last decade. In addition, the authors believe that this study will unveil information regarding what scholars have made a scientific achievement in this field in terms of advancing teaching and learning of CS topics in the different contexts, which will serve as a boost for active researchers. In contrast, new scholars would derive motivation and valuable resources for future studies. To achieve objectives, this study set out to answer the following research questions: RQ1 How is the growth of research publication and citation of articles on VR in computer science education? RQ2 Who are the most active authors, institutions, and countries publishing articles on the use of VR in computer science education? RQ3 What co-occurrence patterns exist in studies on the use of VR in computer science education? RQ4 What is the trend of the research methodology employed in articles on VR in computer science education? RQ5 What were the most preferred data collection tools and sampling methods in articles on the use of VR in computer science education? RQ6 What were the sample sizes in articles on the use of VR in computer science education? RQ7 What were the most preferred data analysis methods in articles on the use of VR in computer science education?

Methods
The method explored in this study was centered on content and bibliometric mapping analysis. This study followed the recommended workflow for science mapping provided by Aria and Coccurullo [21] to conduct our bibliometric mapping analysis. In contrast, the approach shown by [11] was followed to present the content analysis, respectively.

Article selection process
The article selection process for this study includes 3 phases similar to the one presented by [4], namely, (i) literature search and data collection; (ii) data extraction, loading, and conversion; and (iii) data synthesis. A graphical representation of the data collection process is presented in Figure 1, showing detailed actions in each phase.
(i) Literature search and data collection This study obtained data from 2 databases, i.e., the Web of Science (WoS) and the Scopus databases. These 2 databases have been acclaimed to contain comprehensive data of scientific outputs relevant to this study [14]. To conduct an extensive data collection needed for this study, we define the search keywords to include "virtual reality" "VR", "computer science", and "computing education". A number of common protocols for data collection were applied to both databases. They include the same search keywords used in combination with the binary operators such as "OR" and "AND" across the 2 databases, limited time span to the period from 2011 to 2020, and language selected as "English". Table 1 presents details of the search protocol, how they were applied in each database, and the result obtained.
format, we conducted data extraction and conversion into a comma-separated values CSV file to merge the 2 datasets from WoS and Scopus. The process of merging the data is presented in Table 2, followed by executing command line instructions (CLI) shown in Figure 2. R-studio is an integrated development environment for R programming language (https://rstudio.com)) software was used to combine the data into a single CSV file before uploading it to biblioshiny (Biblioshiny is a web interface for bibliometrix r-package (https://www.bibliometrix.org/Biblioshiny.html)) for bibliometrix R-package [17].   TOPIC: ("virtual reality" OR "VR") AND TOPIC: ("computer science" OR "computing education").

Scopus
Applying the search keywords in quotation to Scopus title, abstract, and keywords field with binary operators and limiting the time span to 2011-2020.

1058
Applying additional conditions by limiting to only articles and conference papers.

962
After merging both files, we removed 49 duplicated documents.

Total
(ii) Data extraction, loading, and conversion After data from the independent databases were collected and downloaded in BibTex format, we conducted data extraction and conversion into a comma-separated values CSV file to merge the 2 datasets from WoS and Scopus. The process of merging the data is presented in Table 2, followed by executing command line instructions (CLI) shown in Figure 2. R-studio is an integrated development environment for R programming language (https://rstudio.com, accessed on 18 January 2018) software was used to combine the data into a single CSV file before uploading it to biblioshiny (Biblioshiny is a web interface for bibliometrix r-package (https://www.bibliometrix.org/Biblioshiny.html, accessed on 18 January 2018) for bibliometrix R-package [17]. 2 Save data in a directory with a name that says "rawData". 3 Open RStudio and import the bibliometrix library by running the script < library("bibliometrix") > in the command-line interface (CLI).

4
In Rstudio CLI, run the script < setwd ("C:/../ . . . / . . . /rawData") > to open the directory where data would be imported from and saved. Not that the ellipsis ( . . . ) indicates the paths to the directory and should be correctly inserted.

5
Download in BibTex format independently from databases (in this case, WoS and Scopus). 6 Save data in a directory with a name that says "rawData".

Findings from Bibliometric Mapping Analysis
This section presents our findings from the bibliometric analysis on the basis of the data generated from WoS and Scopus databases. This bibliometric analysis intends to provide insight into how studies on the use of VR for CS education have grown in the last 10 years. In addition, the result reveals authors, institutions, and countries who have been contributing to the field by actively publishing research related to VR in CS education. Furthermore, the result presents how studies on VR in CS education have had an impact in terms of their citations and authors co-occurrence pattern analysis. The section delineates the analysis of common keywords used in articles on VR for CS education, thereby After completing the steps in Table 2, we executed the line of commands (lines 6 to 15) in Figure 2 to complete the remaining process of data conversion and merging. This merging of the two converted points of data by running the command in line 11 of Figure 2A triggered the R-Function that identified 49 similar articles from WoS and Scopus databases. The identified similar articles were removed to avoid having duplicate data. Removing duplicate articles left the remaining data at 971, which was uploaded to biblioshiny for bibliometric mapping analysis. The search was conducted on 2 January 2021.

(iii) Data Synthesis
In Table 3, we present the synthesized data used for the bibliometric analysis. However, for the content analysis, 3 researchers screened the entire data by reading each paper's abstract to decide whether it was relevant or not. Further criteria for selecting relevant papers suitable for the content analysis included: Table 3. Data synthesis indicating the primary information about the data and document type. (i) the paper must focus on virtual reality for education in computer science education; (ii) the paper designed a study or developed a solution to facilitate CS education in a VR environment; (iii) the study reported any outcome by evaluating with users (students, educators, or experts); (iv) the paper is open access and could be downloaded for detailed review.

Main information about data
After applying the criteria, we arrived at 39 papers that met the content analysis requirements presented in Section 3.2.

Findings from Bibliometric Mapping Analysis
This section presents our findings from the bibliometric analysis on the basis of the data generated from WoS and Scopus databases. This bibliometric analysis intends to provide insight into how studies on the use of VR for CS education have grown in the last 10 years. In addition, the result reveals authors, institutions, and countries who have been contributing to the field by actively publishing research related to VR in CS education. Furthermore, the result presents how studies on VR in CS education have had an impact in terms of their citations and authors co-occurrence pattern analysis. The section delineates the analysis of common keywords used in articles on VR for CS education, thereby presenting the thematic area of the current research landscape and topic hotspots.  Figure 3 shows the articles' distribution in terms of the publication year regarding the article production and development across 10 years. The overall publication trend of articles related to VR in CS education shows that 2011 witnessed the highest production year, reaching 148 articles, followed closely by 135 articles in 2018. Educ Figure 3 shows the articles' distribution in terms of the publication year regarding the article production and development across 10 years. The overall publication trend of articles related to VR in CS education shows that 2011 witnessed the highest production year, reaching 148 articles, followed closely by 135 articles in 2018. The publication volume decreased from 2012 to 2015 and from 2019 to 2020. There was an increase in article production from 2016 to 2018 before the slight decline until 2020. This trend occurred probably because the selected articles were limited to only education, leaving out other domains, such as health, business, entertainment, and media.

Most Active Authors, Institutions, and Countries Publishing Articles on the Use of VR in Computer Science Education
Regarding authors' production over time, we investigated the top 20 authors. Our findings showed that most of those top authors were already publishing articles on VR in CS education by 2011. However, about half of those authors were not active from 2019. As shown in Figure 4, many articles related to VR in CS education were published between 2011 and 2020.
As we can see in Figure 4, the author Li Y. had the highest publication over time, having had several articles published yearly for 7 years from 2011 to 2020, except in 2013, 2014, and 2016. With the least productivity over time was the author is Dengel A., with publications only in 2019 and 2020.
We analyzed the top 20 authors' number citations across the production years (mindex) regarding their impact. M-index is calculated by dividing the total number of citations by the total number of years of production. In order words, this study measures the authors' impact by dividing the H-index by the total number of years of production. Note that the total years of production varied for different authors. Although the total number of years investigated in this study remained at 10, some authors did not start publishing from 2011; therefore, such an author's total number of years of production would count from the year the author published his/her first paper. For example, Dengel A. started publishing articles on VR in CS education in 2019; hence, the total number of years The publication volume decreased from 2012 to 2015 and from 2019 to 2020. There was an increase in article production from 2016 to 2018 before the slight decline until 2020. This trend occurred probably because the selected articles were limited to only education, leaving out other domains, such as health, business, entertainment, and media.

Most Active Authors, Institutions, and Countries Publishing Articles on the Use of VR in Computer Science Education
Regarding authors' production over time, we investigated the top 20 authors. Our findings showed that most of those top authors were already publishing articles on VR in CS education by 2011. However, about half of those authors were not active from 2019. As shown in Figure 4, many articles related to VR in CS education were published between 2011 and 2020.
As we can see in Figure 4, the author Li Y. had the highest publication over time, having had several articles published yearly for 7 years from 2011 to 2020, except in 2013, 2014, and 2016. With the least productivity over time was the author is Dengel A., with publications only in 2019 and 2020.
We analyzed the top 20 authors' number citations across the production years (mindex) regarding their impact. M-index is calculated by dividing the total number of citations by the total number of years of production. In order words, this study measures the authors' impact by dividing the H-index by the total number of years of production. Note that the total years of production varied for different authors. Although the total number of years investigated in this study remained at 10, some authors did not start publishing from 2011; therefore, such an author's total number of years of production would count from the year the author published his/her first paper. For example, Dengel A. started publishing articles on VR in CS education in 2019; hence, the total number of years remained at two. Therefore, the m-index would be the total number of citations in 2019 and 2020, divided by 2.  As shown in Figure 5, the authors' m-index was highest at 1.0 (to a single decimal). Therefore, the result indicates that Dengel A., with the highest m-index, remained the most impactful author at the end of 2020. This finding suggests that Dengel A. had had an unbroken research activity in the area of VR in CS education since the first publication and had received a significant number of citations.  As shown in Figure 5, the authors' m-index was highest at 1.0 (to a single decimal). Therefore, the result indicates that Dengel A., with the highest m-index, remained the most impactful author at the end of 2020. This finding suggests that Dengel A. had had an unbroken research activity in the area of VR in CS education since the first publication and had received a significant number of citations.  As shown in Figure 5, the authors' m-index was highest at 1.0 (to a single decimal). Therefore, the result indicates that Dengel A., with the highest m-index, remained the most impactful author at the end of 2020. This finding suggests that Dengel A. had had an unbroken research activity in the area of VR in CS education since the first publication and had received a significant number of citations.  Our analysis revealed some top universities regarding institutions (authors' affiliations) and countries fronting VR in CS education. As shown in Figure 6, some of these universities, to name a few, were the University of Southern California, USA; Aalborg University, Denmark; and University of Rennes, France. Our analysis revealed some top universities regarding institutions (authors' affiliations) and countries fronting VR in CS education. As shown in Figure 6, some of these universities, to name a few, were the University of Southern California, USA; Aalborg University, Denmark; and University of Rennes, France.  Figure 6 shows the USA as the most productive country in terms of publishing articles related to VR in CS education in countries. From the European continent, France, Denmark, Italy, the UK, Germany, and Spain made significant contributions. Only China and Japan made contributions regarding VR in CS education from the Asian continent.

Keywords Co-Occurrence Patterns of Studies on the Use of VR in Computer Science Education
A keywords co-occurrence pattern (KCP) focuses on understanding the knowledge components and knowledge structure of a scientific field by examining the links between keywords in the published articles within the same area [4]. Figure 7 focuses on keyword co-occurrence patterns of studies on the use of VR in computer science education. As observed in Figure 7, the root keyword in the field remains "virtual reality". Other keywords that are frequently used by articles on VR in CS education are shown in red color. For instance, we notice keywords such as gamification, simulation, higher education, mixed reality, serious games, and more. In addition, as expected, keywords that define the characteristics of virtual reality technology were seen to be strongly connected to the root keyword. For example, we observe a thick line connecting keywords such as immersion, interaction, and presence, to the root keyword "virtual reality". Moreover, virtualization, cloud computing, and virtual machine are keywords that show a strong connection. Other keywords that show a close relationship to virtual reality include augmented reality and computer science.  Figure 6 shows the USA as the most productive country in terms of publishing articles related to VR in CS education in countries. From the European continent, France, Denmark, Italy, the UK, Germany, and Spain made significant contributions. Only China and Japan made contributions regarding VR in CS education from the Asian continent.

Keywords Co-Occurrence Patterns of Studies on the Use of VR in Computer Science Education
A keywords co-occurrence pattern (KCP) focuses on understanding the knowledge components and knowledge structure of a scientific field by examining the links between keywords in the published articles within the same area [4]. Figure 7 focuses on keyword co-occurrence patterns of studies on the use of VR in computer science education. As observed in Figure 7, the root keyword in the field remains "virtual reality". Other keywords that are frequently used by articles on VR in CS education are shown in red color. For instance, we notice keywords such as gamification, simulation, higher education, mixed reality, serious games, and more. In addition, as expected, keywords that define the characteristics of virtual reality technology were seen to be strongly connected to the root keyword. For example, we observe a thick line connecting keywords such as immersion, interaction, and presence, to the root keyword "virtual reality". Moreover, virtualization, cloud computing, and virtual machine are keywords that show a strong connection. Other keywords that show a close relationship to virtual reality include augmented reality and computer science. Furthermore, Figure 8 presents a visualization of frequently used keywords in VR for CS education. It is clear from the size of the nodes that other related terms used for virtual reality, for example, "virtualization" and "virtual environment" were found to be highly connected to "computer science" and "education". In addition, some pedagogical concepts for teaching and learning, such as games, gamification, collaborative learning, and immersive learning, are visible in the network. Figure 8 also shows clustering of concepts where terms such as virtualization, virtual environment, computer science, and education form clusters depicted with different colors.
One way to examine how VR application has influenced CS education is to analyze trending topics over the period considered in this study. Figure 9 presents the trending topics or approaches scholars have explored to provide VR intervention for CS education.
This study analyzed the authors' keywords to determine what research hotspot in terms of topics and approaches have been explored by VR applications in CS education in the last decade. This analysis was conducted through the word cloud of authors' keywords, which gives a pointer to what has been the scholars' interest. This analysis also provides insight regarding the future outlook of VR interventions in CS education. Figure  9 delineates that virtualization, cloud computing, the virtual world, and virtual machine dominate VR studies in CS education between the years 2011 and 2015. In addition, slightly different changes were observed where keywords such as computer science education, serious games, and higher education emerged among the trending topics between 2015 and 2017. Furthermore, Figure 8 presents a visualization of frequently used keywords in VR for CS education. It is clear from the size of the nodes that other related terms used for virtual reality, for example, "virtualization" and "virtual environment" were found to be highly connected to "computer science" and "education". In addition, some pedagogical concepts for teaching and learning, such as games, gamification, collaborative learning, and immersive learning, are visible in the network. Figure 8 also shows clustering of concepts where terms such as virtualization, virtual environment, computer science, and education form clusters depicted with different colors.
One way to examine how VR application has influenced CS education is to analyze trending topics over the period considered in this study. Figure 9 presents the trending topics or approaches scholars have explored to provide VR intervention for CS education.
This study analyzed the authors' keywords to determine what research hotspot in terms of topics and approaches have been explored by VR applications in CS education in the last decade. This analysis was conducted through the word cloud of authors' keywords, which gives a pointer to what has been the scholars' interest. This analysis also provides insight regarding the future outlook of VR interventions in CS education. Figure 9 delineates that virtualization, cloud computing, the virtual world, and virtual machine dominate VR studies in CS education between the years 2011 and 2015. In addition, slightly different changes were observed where keywords such as computer science education, serious games, and higher education emerged among the trending topics between 2015 and 2017. Furthermore, it was observed that between the years 2018 and 2020, new keywords such as augmented reality, immersion, presence, gamification, game-based learning, and human-computer interaction were added to the trending topics. Therefore, topics such as immersion, presence, human-computer interaction, gamification, and game-based learning dominate the list of research hotspots in recent times. This finding suggests that one of the most appreciated learning and teaching approaches used by studies on VR application in CS education is game-based learning.

Findings from Content Analysis
This section presents the content analysis findings to address some of the research questions (RQ4 to RQ7). Moreover, an overview of the data analyzed in this section is presented as an Appendix A. In the Appendix A, information regarding the study focus Furthermore, it was observed that between the years 2018 and 2020, new keywords such as augmented reality, immersion, presence, gamification, game-based learning, and human-computer interaction were added to the trending topics. Therefore, topics such as immersion, presence, human-computer interaction, gamification, and game-based learning dominate the list of research hotspots in recent times. This finding suggests that one of the most appreciated learning and teaching approaches used by studies on VR application in CS education is game-based learning.

Findings from Content Analysis
This section presents the content analysis findings to address some of the research questions (RQ4 to RQ7). Moreover, an overview of the data analyzed in this section is presented as an Appendix A. In the Appendix A, information regarding the study focus and outcome are highlighted to showcase how the selected articles have employed VR in CS education.

Trends of the Research Methodology Employed in Articles on the Use of VR in Computer Science Education
According to Figure 10, 47% of the articles used a quantitative design approach, 16% used a qualitative design, 3% used mixed design, and 12% utilized a design and development research approach. In comparison, others may include review/meta-analysis research accounts for 5%.

The Most Preferred Data Collection Tools and Sampling Methods in Articles on the Use of VR in Computer Science Education
Data collection tools and sampling methods in research conducted on VR in CS education show that the questionnaire (46%) remains the most used tool. However, quite a number of studies (23%) either did not conduct evaluation or did not specify what method of data collection was used.
As shown in Figure 12, the use of interviews (13%) is still growing as fewer studies have been seen to use the method.

The Most Preferred Data Collection Tools and Sampling Methods in Articles on the Use of VR in Computer Science Education
Data collection tools and sampling methods in research conducted on VR in CS education show that the questionnaire (46%) remains the most used tool. However, quite a number of studies (23%) either did not conduct evaluation or did not specify what method of data collection was used.
As shown in Figure 12, the use of interviews (13%) is still growing as fewer studies have been seen to use the method.

Sample Populations and Sample Sizes in Articles on the Use of VR in Computer Science Education
According to Figure 13, the most commonly used sample size in articles published between 2011 and 2020 fell between 11-20 participants. Closely followed were 1-10 persons and 51-100 people. Although other studies utilized samples between 21-50 and 101-200 respondents, a few studies did not specify the sample size they used.

Sample Populations and Sample Sizes in Articles on the Use of VR in Computer Science Education
According to Figure 13, the most commonly used sample size in articles published between 2011 and 2020 fell between 11-20 participants. Closely followed were 1-10 persons and 51-100 people. Although other studies utilized samples between 21-50 and 101-200 respondents, a few studies did not specify the sample size they used. The findings show that most studies were performed using descriptive analysis regarding the most preferred data analysis conducted in studies focused on VR in CS education.

Most Preferred Data Analysis Methods in Articles on the Use of VR in Computer Science Education
The findings show that most studies were performed using descriptive analysis regarding the most preferred data analysis conducted in studies focused on VR in CS education.
Other preferred analysis methods, as shown in Figure 14, are meta-analysis and content analysis. Moreover, some studies adopted a theoretical approach while some other studies did not conduct any form of research, and therefore we categorized these types of studies as "others/not specified".

Most Preferred Data Analysis Methods in Articles on the Use of VR in Computer Science Education
The findings show that most studies were performed using descriptive analysis regarding the most preferred data analysis conducted in studies focused on VR in CS education.
Other preferred analysis methods, as shown in Figure 14, are meta-analysis and content analysis. Moreover, some studies adopted a theoretical approach while some other studies did not conduct any form of research, and therefore we categorized these types of studies as "others/not specified".

Discussion
The bibliometric method's potential is seen by earlier research [4]. It was opined that bibliometric study advances complement meta-analysis and qualitative research for the scientific evaluation of literature. This study delved into VR's role in CS education to provide a deeper understanding of the evolution of research conducted in this field and anticipate the future direction on the basis of the analysis of the co-occurrence pattern of keywords used in studies conducted in the last 10 years. The study contributes to knowledge by presenting valuable findings that can boost the morale of prolific scholars who have been contributing to this field and researchers and practicing managers who may be starting to research into VR for CS education. This current study obtained its bibliometric and content analysis data from the Web of Science and Scopus databases.
The bibliometric analysis of articles related to the use of VR in CS education, together with the methodological research trends over the last 10 years, was revealed. Bibliometric analysis results showed that the year 2011 was the highest in article production (148 articles). This result was closely followed by the year 2018 with 135 articles. This finding implies that between 2012 and 2017, articles related to VR in CS education dwindled. Regarding the authors' production over time, Li Y. had the highest number of articles produced in the field, which is not surprising as the author consistently published in 2011-2012, 2015, and 2017-2020.
Moreover, we analyzed studies' impact by investigating the number of citations obtained by authors within 10 years. The analysis was focused on the m-index of each author. Considering the 10 years duration in this study, we calculated the m-index by dividing the total number of citations by the total number of years authors have been publishing. For example, Dengel A. emerged as the most impactful author because this author had produced one paper per year for only two years. This means that Dengel's impact analysis was computed on the basis of the output of these two years. However, it was surprising to discover that Li Y., who had the highest number of articles produced over the years, was not as impactful as Dengel A., who had a limited number of articles published within just two years. Earlier studies have examined intrinsic factors affecting the number of citations of articles [22,23]; however, some indicators are not directly related to the quality or content of articles' extrinsic factors [24]. The previous finding reveals that price index, number of references, keywords, and length of studies are essential explanatory factors [24]. It can be concluded that it is likely that Li's articles are easily accessible to researchers via open access medium. The relevancy of their topic or even the quality of their paper in terms of content and presentation may account for the citations and rapid impact.
Regarding the institutions and countries contributing to VR in CS education, the results further showed that the University of Southern California, USA; Aalborg University, Denmark; and the University of Rennes, France, remain the top universities in terms of publishing VR in CS education articles. On the other hand, the USA emerged as the most productive country. However, other countries from Europe (France, Denmark, Italy, the UK, and Germany) and Asia (China) are making a significant contribution towards advancing CS education using VR technology. The co-occurrence pattern of authors' keywords revealed that VR characteristics are leveraged for CS education. For example, immersion, presence, interaction, and gamification are being explored in advancing CS education [1,16,18]. Moreover, these keywords also form the research hotspots in VR, primarily to support learning. Therefore, this study anticipates that VR in CS education would continue to be researched within the scope of these keywords [14].
The content analysis results showed that quantitative studies (47%) dominate the studies in terms of research methodology. The reason for quantitative method preference may be due to the simplified way of presenting quantitative research, as well as less time and effort required to conduct and analyze quantitative data [25]. It might also be the case that the generalization and replicability that the quantitative approach provides accounts for its dominance in the studies. The percentage for the use of mixed methods studies was meager, reflecting that the use of mixed approach studies presents methodological difficulties and challenges [12]. It is safe to conclude that only a few studies consider the potential of mixed-method research, which adds rigor and validity to research through triangulation and convergence of multiple and different sources of information [26,27]. Moreover, few qualitative studies have been conducted in the last 10 years. This may have been due to the rigor and non-use of numbers, making it difficult to simplify findings and observations [25]. On the contrary, Johnson and Christensen [28] assert that reliance on collecting non-numerical primary data such as words and pictures makes qualitative research well-suited for providing factual and descriptive information.
Regarding the frequency of the sampling size utilized over the years, the most used sample sizes were 11-20. We were surprised to find out that most published articles on VR in CS education were evaluated with about 11 to 20 participants. Since the research method's preference was quantitative research, we expected that many studies would have used more participants to arrive at a generalized outcome. Although studies that used 51-100 sample sizes were also seen in the result, one could have thought that 20 participants may be too small for a quantitative study. According to Faber and Fonseca [29], very small samples undermine the internal and external validity, while huge samples tend to transform minor differences into statistically significant differences.
Our findings revealed that the questionnaire is the most used data collection tool, while descriptive analysis remains the preferred data analysis method. One way to reflect on this result is that the questionnaire seems more straightforward, quicker, and cost-effective to collect data from participants. Moreover, the preference for descriptive analysis may be used to simplify data efficiently [30]. The researcher may have adopted this data analysis method to reduce the time and effort required to format and present beneficial, easily interpretable results to practitioners, policymakers, and other researchers to understand a phenomenon better.

Conclusions
This study provides a comprehensive view of scientific papers on VR in CS education published in peer-reviewed journals and conferences between 2011 and 2020. Two main approaches were explored to answer the research questions presented in this study. First, the bibliometric analysis answered the questions regarding the article production growth in the field within a decade, prolific scholars and their affiliations publishing to advance VR in CS education, and research hotspots in the field may guide scholar's future research focus. Second, content analysis of articles that met the inclusion criteria for this study was analyzed to provide a methodological overview of studies conducted on VR in CS education. Several findings were presented in this study. These findings show that VR research for CS education has fared well; however, some of the years (between 2013 and 2016) witnessed low article production. The study also revealed the prolific scholars and authors' impact analysis in this field and provided insightful information regarding research hotspots by analyzing the authors' keywords co-occurrence.
Regarding the scientific methodology and data sampling technique used by studies on VR in CS education, the most preferred is the quantitative method. At the same time, the questionnaire was the most used data collection technique. Moreover, descriptive analysis was mainly used to analyze data in studies on VR in CS education.
This study witnessed a limitation regarding the content analysis. It would be interesting to see the educational context where VR technology is being used and the learning contents deployed in the VR application for CS education. Nonetheless, this study contributes to knowledge in significant ways. The study revealed that pedagogical approaches such as game-based learning and gamification were explored for VR education in CS education. The findings from this study can provide insight into how VR technology research has progressed in a decade. Moreover, the result can be generalized since this study could obtain relevant data from two databases (WoS and Scopus) to conduct its analysis. The process for merging these data is another contribution as scholars interested in running a similar study would find this helpful study. Our future study would address the limitations by providing answers regarding the pedagogy, content, and context of studies on VR in CS education.
By implication, we conclude that findings from this study suggest that even though scholars are leveraging VR to advance teaching and learning in the field of CS, more effort needs to be made, especially from continents, countries, and institutions that were not reported among the top-20 list revealed in this study. In addition, a more rigorous methodological approach needs to be employed in a future study to provide more evident-based research output. For example, our study revealed only a few studies that used a mixedmethods approach, which has been more rigorous in terms of quality of scientific research.

Conflicts of Interest:
The authors declare no conflict of interest.
Demonstration and evaluation of the tool was expected to produce useful outcome.
Vallance [39] This study aimed to set a medium of collaboration within a 3D virtual world.
This study was still a work in progress, and hence a concrete result was not presented.
Arrington et al. [40] This study designed and implemented Dr. Chestr, a virtual human in a virtual environment game aimed at supporting the understanding and retention of introductory programming cources.
The study measured students' cognition, presence, usability, and satisfaction and found that students enjoyed the experience and were successfully engaged the virtual world.
Vanderdonckt and Vatavu [41] This study present a VR application where the user, a psychologist, controls a virtual puppet (a cartoon-like character in VR).
The study found that when receiving lectures in a virtual environment by a teacher, the child was calm, focused, and capable of working on his assignments without showing any disruptive behaviors.
Parmar et al. [42] The authors developed a VR tool-VEnvI-to support CS students in learning about the fundamental CS concepts such as sequences, loops, variables, conditionals, and functions.
Participants who tested the VR tool agreed that the visual aspect improved the overall learning experience.

Aim of the Study Results of the Study
Adjorlu and Serafin [43] This study investigated the feasibility of using VR to reduce disruptive classroom behavior of a child diagnosed with autism spectrum disorder (ASD).
The study provided guidelines to educators and instructional designers who wish to offer interactive and engaging learning activities to their students.
Berns et al. [44] A VR educational platform MYR was built to spark student interest in computer science by allowing them to write code that generates three-dimensional, animated scenes in virtual reality environment. The goal of the project was to gain insight into computing students' success, motivation, and confidence in learning computing.
Evaluation with CS students shows that MYR is hard for CS students to provide clear 3D representations for programming concepts; however, the study was able to derive some common figures.
Christopoulos et al. [45] Authors investigared what effect instructional design decisions have on motivation and engagement of students learning in virtual and physical world.
Evaluation of this tool suggests that users' experience is enhanced through the 3D animation.
Ortega et al. [46] The study developed a 3D virtual programming language to provide an interactive tool for beginners and intermediate students to learn programming concepts.
The study reported that the method creates fun and effective means of interdisciplinary study.
Sanna et al. [47] This study proposed a virtual 3D tool (touchless interface) to support people without any prior knowledge in code writing to promote user friendliness and usability experience.
Feedback from the workshop participants generally shows that they had a good experience.
Cleary et al. [48] This study explored a style of teaching youths how to write computer program using reactive programming in a 3D virtual environment.
The study tested educational virtual environments (EVEs) with pre-and post-test and found to be significantly effective.
Domik et al. [49] The authors created "Move the World" workshop in a summer camp to increase high school juniors' interest in computer science by leveraging math and virtual worlds.
Overall comments from participants of the workshop revealed that learning in the virtual world is appealing and inspiring.
Dengel [50] The study modeled three computer scienc topics-asymmetric encryption/decryption, and finite state machines in a 3 D immersive VR to teach these topics.
The study discusses students' preconceptions towards the inclusion of 3D virtual learning environments in the context of their studies and further elicit their thoughts related to the impact of the "hybrid" interactions Koltai et al. [51] This study used a VR game (Mazes) to teache CS concepts.
The study reported positive impact on computer science education by increasing engagement, knowledge acquisition, and self-directed learning.
The study generally indicated that participants perceived a high joy of use while playing FunPlogs, which indicated that despite the simple game concept, complex matters as the while-loop could be transported to programming laymen.
Banic and Gamboa [53] The study explored a summer course that uses visual design problem-based learning pedagogy with virtual environments as a strategy to teach computer science.
The study concluded that interactions in VR plays a crucial role in learner engagement.
Horst et al. [54] This study introduced a VR puzzle mini-game for learning fundamental programming principles.
The study outcome shows that the proposed module helps students learn stacks and queues while being satisfactorily usable.

Aim of the Study Results of the Study
Christopoulos and Conrad [55] Authors examined the impact that the virtual reality learning process has on university students who study CS and have almost no experience in the use of virtual worlds.
Results show that the self-overlapping maze is experienced as freely walkable while the map is mostly understandable.
Stigall and Sharma [56] This study designed a game theme-based instructional (GTI) module to teach undergraduate CS majors about stacks and queues.
The analysis of SEQ usability test shows good acceptance.
Serubugo et al. [57] This study investigated how working with VR setups can be walkable in small physical spaces or included in non-HMD participants using self-overlapping maze Analysis of the usability and likeability of the survey shows that students felt motivated and engaged in learning programming concepts.
Pilatásig et al. [58] This study designed a VR tool to assist in training and rehabilitation of hands and wrist The study reported that students gained cognitive thinking process and had a greater range of expressing sufficiently alternative to self-explanatory solutions.
Segura et al. [59] This study designed a VR application (VR-OCKS) to teach basic programming concepts such as flow statements and conditional selections.
The initial evaluation of this tool shows that it enhanced creative thinking of young children.
Pellas and Vosinakis [60] The authors explored a 3D simulation game to teach computational problem-solving.
Evaluation results demonstrated positive student perceptions about the use of gaming instructional modules to advance student learning and understanding of the concepts.
Stigall and Sharma [61] This study designed and developed two gaming modules for teaching CS students object-oriented programming (OOP) and binary search.
Result analysis suggests that participants showed similar connectedness in affiliative tour and competitive design.
Sharma and Ossuetta [62] The authors developed virtual reality instructional (VRI) modules for teaching loops and arrays that can provide a better understanding of the concept.
The study measured participants' intentions toward majoring in a computing discipline, attitudes toward computing, and overall satisfaction with the camp, and showed positive indication.
Ijaz et al. [63] This study proposed a VR exergaming platform that combines a recumbent tricycle and real-world panoramic images where the player can navigate real locations in a safe virtual environment This study argued that comparative studies are a useful method for analyzing benefits of different approaches to controlling virtual agents.
Hulsey et al. [64] This study reported the experience of a summer camp that introduced computing concepts to middle school girls in the context of an online, multiplayer, virtual world.
This study demonstrated that familiarity may reduce working memory load and increase children's spatial memory capacity for acquiring sequential temporal-spatial information from virtual displays.
Gemrot et al. [65] This study presents results of comparing the usability of an academic technique designed for programming intelligent agents' behavior with the usability of an unaltered classical programming language.
Outcome of the experiment with CodeSpells shows that students were able to understand and write basic Java code after only 8 h of playing the game.
Korallo et al. [66] This study examined the potential use of virtual environment in general computer knowledge in virtual environment.
Outcome of the study provide overview of the two reviewed approaches for implementing VR gestures, which may guide experts.