Towards the Sustainable Development of Digital Educational Games for Primary School Students in China

: Despite considerable discussion on the beneﬁts of digital educational games, empirical research on their effectiveness in primary schools in China is limited. This case study aimed to promote the sustainable development of digital educational games in primary schools in China by examining the effect of digital educational games on primary school students’ vocabulary acquisition and ascertaining their perceptions and attitudes toward this approach. Given the purposes of this study, an embedded mixed methods research design was employed. Two Grade 4 classes at a Chinese primary school were recruited in this study. During the quasi-experiment, the experimental class ( n = 50) was provided with educational game software, Quizlet, while the control class ( n = 50) was taught through a traditional teaching method. The results show that integrating educational games into language education in the primary schools was effective in improving students’ vocabulary acquisition. This study makes a case for further research of digital educational games in language classrooms at a primary level in China. Implications are made regarding the future sustainable implementation of digital educational games in primary schools in China.


Introduction
With the development of technology, the potential of digital games to facilitate language learning has been increasingly discussed [1]. A plethora of research suggests that digital games with educational purposes (hereafter referred to as digital educational games) can engage students in instructional activities, promote active learning, foster teamwork spirit, and thus help make the learning process more effortless, enjoyable, and productive [2]. However, currently, in China, there is a deficiency of relevant research in language classrooms of Chinese primary schools for fear of the possible drawbacks of digital educational games, and less is known about its implementation for language education at the primary school level in China.
In view of this gap, this case study focused on the effect of digital educational games on the English vocabulary acquisition of primary school students in China. Further, it attempted to reveal students' experiences of such language learning. We hope that findings from this study will provide guidance to English teachers in primary schools in China and contribute to the future implementation of educational game-based English education.

Literature Review
To date, digital game research remains a controversial topic among educational researchers [3]. On one hand, prior research suggests that digital games can be detrimental to students' studies, leading to negative impacts such as aggressive behavior, addiction, and social isolation [4][5][6][7]; on the other hand, some scholars have shown the potential positive impacts of such games in improving students' learning outcomes [8][9][10]. Despite these conflicting views, there is a growing consensus among researchers fact that digital games can exert a positive influence upon education if designed well and implemented deliberately [11][12][13]. In this vein, Kapp [14] holds that digital educational games, based on computer technology, aesthetics, and gameplay mechanics, are capable of engaging students and keeping them motivated. Recent research evidence suggests that, when used wisely, digital educational games can afford a new medium of learning, which then gives way to the emergence of digital educational games in various contexts [15,16]. For example, Yuda [17] examined the effectiveness of flash-based puzzle digital games of Japanese maps and found that educational games as digital education materials have an impact on improving pupils' spatial thinking, expanding their views, and supporting geographical education. J. Lee and F. Lee [18] designed a game-based learning platform named The Farm Rhapsody, based on the virtual and interactive learning environment. Through managing the operation of a farm, students can learn interdisciplinary knowledge about agriculture and develop their problem-solving ability.
Meanwhile, recent findings indicate that digital educational games have been a rising research direction in language research. Compared with traditional games, they may afford a multitude of potential advantages for language learning. Most of these advantages are related to the ability to create learning environments that contextualize knowledge and offer learners immersive experiences. For instance, Guillén-Nieto and Aleson-Carbonell [19] tested the effectiveness of an educational game, It's a Deal!, on developing students' intercultural communication competency. Their findings show that video games can be a perfect learning tool to facilitate intercultural communication between Spaniards and Britons in business settings in which English is used as the lingua franca. Mei and Yang [20] explored the integration of gamification, environmental education, and language education at the tertiary level. They designed a geolocation-based mobile augmented reality scavenger hunt for first-year students at a comprehensive university. The findings suggest that such an integration is effective and plausible and can benefit students' learning and improve their awareness of the environment.
Summing up, our literature review suggests that digital educational games are increasingly accepted as a promising tool in the modern language teacher's toolkit. However, with concerns about the negative influence of digital games on students' academic achievement and eyesight, primary school teachers in China are usually reluctant to integrate digital educational games into their classrooms, and most extant studies were conducted at tertiary level institutions or in laboratory settings [21].
Given the situation, this study investigated the application of an educational gamebased learning platform Quizlet in a primary school in China. The study aimed to address the following two research questions (RQ): RQ 1: Does the use of digital educational games have a positive influence upon the English vocabulary acquisition of primary school students in China? RQ 2: How do primary school students view the educational game-based vocabulary learning approach?

Research Design
Given the research questions, an embedded mixed methods research design was employed in this study. The feature of mixed methods design is that one dataset provides a supportive or secondary role in a study based on another kind of dataset [22]. Figure 1 provides a schematic overview of the research design. In this design, quantitative data analysis results are used to answer the research questions in general, and qualitative data are used to provide a detailed account of educational games in practice from the learners' point of view (Creswell & Plano Clark, 2011).

Case Study Site and Sampling Strategy
The present study was conducted in a public primary school located in Central China. The school has its own advanced multimedia network classrooms, and the teachers, in general, have adequate level of computer literacy (all teachers need to pass required tests or get relevant certificates as is stipulated in relevant regulations made by the educational administration department of the local province), which provides the necessary conditions for the study. Three English teachers voluntarily agreed to participate in the study. However, considering the aim of the study, we chose to conduct the study with a Grade 4 English teacher who was then teaching English in two classes.

Participants
The final sample after attrition comprised 100 Grade 4 students currently enrolled at the school. They were from two classes taught by the same teacher. All of the students from the two classes had received one year of English language study in Grade 3. Further, there were neither gender ratio nor age differences between these two classes. Details about the participants are provided in Table 1.

Materials
Given the aim of the study, Quizlet (http://www.quizlet.com/, accessed on 7 June 2020) was adopted as the educational game in this study. Quizlet allows users to create and study sets of flashcards online with a suite of engaging activities. Students can use it individually or collaboratively, either at home or in a classroom. It is an innovative online and mobile digital vocabulary-learning game that can provide students with different ways to interact with each word. For example, students can listen to a pronunciation model, learn the word's definition, spell the words they hear, check their progress, play review games, and take random practice tests. These pre-determined vocabulary-learning activities help expose users to necessary vocabulary with alternative interactions [23]. The activities used in the study were designed by the first author.
Match, Gravity, and Quizlet Live are the three games in Quizlet that help keep students engaged in language learning activities. Match has students quickly match terms to their respective definitions. If students match the term and its definition successfully, two corresponding pictures will disappear. Gravity is a game set in outer space. In this game, pre-determined words fall from the top of the screen at increasing speeds, depicted as asteroids. Students need to type in the definitions before they reach the bottom of the screen. To prevent asteroids from hitting the earth, students need to quickly recognize the vocabulary and type in the correct answer. Quizlet Live is also an exciting game. Teachers can set up different types of questions, including vocabulary and sentences with Quizlet Live. Students can be divided into several teams automatically. They must work together quickly to figure out the right solution for each question. The final victory requires close collaboration between team members. Students' interaction with Quizlet in the classroom is shown in Figure 2.

Procedure
There were two vocabulary lectures delivered to students each week, and each lecture lasted 45 min. During the quasi-experiment, the two classes were taught by the same English teacher, and the teaching content for the two classes was the same. The difference between the two classes was that the EC included educational games. Given the restrictive policies for the use of personal mobile devices in primary schools in China [24], students from the EC were required to go to the computer classroom to take the lesson. Detailed teaching procedures for the EC and CC are provided in Appendix A.
At the end of the experiment, a researcher conducted a post-test with participants to ascertain the effect of this project on their English vocabulary knowledge. Students in the EC were also invited to complete a Likert-scale questionnaire survey about its perceived usefulness, its perceived ease of use, their perceived enjoyment, their perceived engagement, and their intention to use the Quizlet app in the future. Two days after the intervention, five participants in the EC were randomly chosen to attend semi-structured interviews. They were asked to share their experiences and perceptions of learning English vocabulary with Quizlet.

Ethical Considerations
All procedures in the study conformed to the ethical standards of the local institution. Before the study, the researchers contacted the principal of the school and gained approval to conduct the study. English teachers from the school were approached and briefed about the purpose and content of the study. Given that the students were under 18, conversations with both the students and their parents were arranged. The researcher also sent letters to students' parents to inform them of this experiment's purpose and gain their written consent. Additionally, they were informed that participation in the one-month experiment would be completely voluntary, that a student could withdraw at any stage of the experiment without incurring any punishment, and that the collected data would not be anonymous, but would remain confidential and be accessible only by the researchers. When the study was completed, remedial sessions were arranged to make sure students from both classes had equal chances to be exposed to the two teaching approaches.

Research Instrument
In this study, data were collected through English vocabulary tests, questionnaire surveys, and semi-structured interviews.
First, a pre-experiment questionnaire survey was arranged to measure students' ability to use computers and the Internet, their frequency of computer use and playing digital games, their enjoyment of digital games, and their attitudes about English learning and the role of technology in vocabulary learning. This questionnaire contains 13 Likert-scale (1 = strongly disagree, 2 = disagree, 3 = agree, 4 = mostly agree, 5 = strongly agree) items adapted from Altiner's [25] pre-project survey.
Two English vocabulary tests that aimed to test students' vocabulary acquisition were then arranged, one before and one after the intervention. The two tests, each with a maximum score of 100 points, were designed in line with the teaching content covered in the four weeks. In order to ensure the accuracy of the results, factors that might influence the results of the test were controlled. For example, the difficulty of the two English vocabulary tests was similar, and the types of questions were identical.
Next, in order to gauge the students' perceptions of educational games, participants in the EC were invited to complete a post-questionnaire containing 27 Likert-scale items (1 = strongly disagree, 2 = disagree, 3 = agree, 4 = mostly agree, 5 = strongly agree), which were pertinent to five factors: the perceived usefulness, the perceived ease of use, their perceived enjoyment, their perceived engagement, and their intention to use the app in the future. All these items were developed on the basis of previous studies [26][27][28][29][30][31][32][33]. In order to help students to understand all the survey items easily and clearly, the questionnaire was translated into Chinese (see Appendix B for details).
After the experiment, face-to-face interviews were conducted to learn more about their experiences and perceptions related to vocabulary learning with Quizlet. The interview questions were also related to the five pre-determined themes, i.e., perceived usefulness, perceived ease of use, perceived enjoyment, perceived engagement, and intention to use the app in the future [20,34]. Each interview lasted approximately 15 min. The semi-structured interviews were recorded and transcribed later for subsequent analysis.

Data Analysis
The quantitative data were analyzed in two phases. In the first phase, quantitative data from students' test results were imported into SPSS 24.0 for statistical analysis. Afterwards, both independent-and paired-sample t-tests were carried out to analyze students' test scores. Specifically, an independent-sample t-test was used to examine differences between the EC and CC, and paired-sample t-tests were conducted to find the differences between the pretest and post-test scores of the EC. Following Hattie's [35] suggestion, Cohen's d values for statistically significant tests were reported as the measure of effect sizes, where values of 0.2 and below were interpreted as small, 0.2-0.6 as moderate, and 0.6 and above as large.
Next, quantitative data collected from the participants' pre-and post-experiment questionnaire surveys were also subjected to statistical analysis. Before the analysis, Cronbach's alpha coefficients were used to check the reliability of the questionnaire to ensure that the results were above the threshold value of 0.70, as recommended by Nunnally [36].
As for the qualitative data analysis, the transcripts of the interviews were imported into Nvivo and coded to obtain an in-depth understanding of the participants' experiences and perceptions toward educational game-based vocabulary learning. The researchers then grouped these transcripts thematically into the pre-designated categories.  Table 2, responses from the pre-experiment survey suggested that students were, in general, confident in their use of computers (mean (M) = 4.00, standard deviation (SD) = 0.99) and the Internet (M = 3.88, SD = 1.02). They also showed great interest in playing digital games (M = 3.90, SD = 1.11). However, the results also revealed that students had fewer opportunities to frequently use a computer (M = 2.98, SD = 1.10) or play digital games (M = 3.23, SD = 1.14). As for their attitude about English vocabulary learning, they agreed with the importance (M = 4.28, SD = 1.04) and usefulness (M = 4.28, SD = 0.93) of English vocabulary learning, but their responses showed that they held a less favorable attitude towards vocabulary learning (M = 3.28, SD = 1.22). They felt that memorizing vocabulary is not easy (M = 2.88, SD = 1.22). In addition, they did not have adequate knowledge about how to study English vocabulary effectively (M = 3.33, SD =1.14). In terms of their attitude about the technology-enhanced vocabulary learning, they, in general, agreed that computers and the Internet could help them improve their vocabulary (M = 4.15, SD = 0.80). Further, students also indicated that they hoped to be able to use online sources to improve their English (M = 3.63, SD = 0.98). In order to guarantee the study's reliability and effectiveness, the vocabulary level between the EC and the CC was compared with an independent-sample t-test. The average test scores of the CC and EC were 77.12 and 78.12, respectively. Though the mean score in the EC was slightly higher than the CC, no statistically significant difference was detected, suggesting that students from both classes were at the same level in English vocabulary learning.

Results of the Pre-and Post-Test Comparison
In this phase, independent-sample t-tests between the CC and the EC and pairedsample t-tests within each class were conducted to better evaluate the possible effect of the intervention.
First, to check the vocabulary development of students in the CC and the EC, a paired-sample t-test between the pre-test and post-test scores of students from the CC was conducted. Though the mean value of the post-test (M = 78.16, SD = 1.37) was higher than that of the pretest (M = 77.12, SD = 1.49), no statistically significant difference was detected, suggesting that there was no substantial improvement of participants' vocabulary in the CC.
Next, to examine the learning outcomes of students in the EC, another paired-sample t-test was conducted. The mean score difference between the pre-and post-test of the EC was 3.84, with a statistically significant improvement in the post-test, t(49) = 5.34, p < 0.05. The effect size for this analysis (d = 0.76) was found to exceed that in Hattie's work, which indicates that the educational game-based teaching method could effectively improve these students' performance in English vocabulary learning.
Moreover, an independent-sample t-test was performed to compare the learning outcomes of the CC and the EC. The result was found to be statistically significant: t(98) = 2.00, p < 0.05. The effect size for this analysis (d = 0.40) suggested the presence of a medium effect, which provides further support for the efficacy of an educational game-based approach.

RQ 2: How do Primary School Students View the Educational Game-Based Vocabulary Learning Approach?
Results of the Post-Experiment Survey Table 3 presents the descriptive statistics of the post-questionnaire items. With regard to participants' attitudes towards perceived usefulness, the mean score on each of the seven items is higher than 3.50, which indicates that participants generally agreed that Quizlet was a valuable tool for learners to learn and memorize English vocabulary.
Next, as for the questions evaluating the perceived ease of use, PEU6 was of the highest mean scores. We can infer therefore that most participants strongly agreed that they could easily interact with the user interface of Quizlet. Moreover, the mean scores of PEU1, PEU2, and PEU5 were also slightly above 4.00, suggesting that participants generally agreed that Quizlet was easy to use (M = 3.86, SD = 0.95) and that they had clear guidance (M = 3.95, SD = 0.73). However, there was less than complete agreement in PEU4, implying that a small number of them experienced difficulty using Quizlet.
As for the questions evaluating their perceived enjoyment, the mean scores were also above 4.00 (ranging from 4.67 to 4.76), suggesting that students strongly held that learning vocabulary with Quizlet was enjoyable. Moreover, the standard deviations ranged from 0.43 to 0.48, which indicates that there was no great discrepancy among students' perceptions towards this factor.
From the descriptive statistical analysis of their perceived engagement, the mean score of PEG3 was the highest among these five questions, which indicated that participants were engaged when playing the educational games, with the most rewarding experience being feeling happy. The mean score of PEG4 was slightly lower than that of PEG3, indicating that participants did not like being disturbed when they were playing the Quizlet games (M = 4.26, SD = 0.63). Students also showed positive attitudes towards PEG1 and PEG5, which suggests that many learners confirmed the belief that they lost Regarding the questions evaluating their intentions to use the Quizlet app again, participants showed a strong willingness to use it in the future and recommend it to their friends (M > 4.00).

Qualitative Results
In order to obtain a deeper understanding of participants' perceptions related to educational game-based learning, interviews were held with five students from each level around the five pre-determined dimensions: perceived usefulness, perceived ease of use, perceived enjoyment, perceived engagement, and intention to use the app in the future. These five interviewees were coded as Students 1-5. The results are, in general, in line with the participants' quantitative feedback First, in line with the positive feedback from the post-experiment survey, all participants found Quizlet useful because, when they played these games on this platform, their vocabulary, in terms of definitions, spelling, and pronunciation, was improved. One interview excerpt is listed below: By using this platform, I have learned many new words through flashcards. What's more, my pronunciation has also improved.

(Student 1)
Next, when asked the question about whether Quizlet is easy to operate, three interviewees provided a positive response. They reported that they could easily find the buttons that they needed. One excerpt is shown below: It [was] easy for me to find the button that I [wanted] to use. First you need to open a study set, and then you can see different kinds of functions on the Quizlet.

(Student 3)
Nonetheless, despite the agreement on the ease of use of Quizlet reflected in the survey, two interviewees reported negative experiences. They indicated that they had experienced some difficulties while using this platform. Therefore, they did not hold favorable attitudes towards Quizlet. Their feedback is listed below: However, one student complained about the games in the Quizlet app, indicating that the difficulty of the game might affect their perception in this regard.
I dislike "gravity", because I felt frustrated when I made minor punctuation mistakes. These mistakes [won't be] accepted by Quizlet. This game [was] difficult for me.

(Student 2)
Next, regarding engagement, four interviewees reported that they had been more concentrated in English class than ever before, which, to some extent, reveals the potential of gaming element in language classrooms. Furthermore, three interviewees expressed their intention to use Quizlet in the future. One of them reported that he had already shared the platform with his friends and they had played games on Quizlet together: However, one student expressed that he will not use Quizlet in the future, which points out practical constraints for implementing this approach. His reason is listed below: To sum up, both positive and negative responses were identified in these interviews. Overall, participants agreed that the use of Quizlet was useful for vocabulary learning. In their view, Quizlet was an engaging and effective learning method. Students reported that the Quizlet-assisted learning method also made them more motivated to learn English vocabulary. The major problems regarding Quizlet are related to technical issues, such as a slow Internet speed and an incompatibility between devices.

Discussion and Implications
The purpose of this study report was to ascertain the effect of digital educational games on the English vocabulary acquisition of primary school students in China and this particular cohort's perceptions of this approach. Employing mixed methods research, we conducted a case study with Quizlet. The overall findings support the efficacy of digital education games in language education at the primary level. Moreover, despite some negative feedback, participants exhibited favorable perceptions towards digital educational games in future language classrooms.

Effectiveness of the Digital Educational Game-Based Learning Approach
Compared with prior research findings of the positive impact of digital games-based learning on students' learning outcomes such as language awareness [37,38], knowledge acquisition [39] and metacognition [40], the experimental results reveal that though students from the EC and the CC both made progress, more significant improvement was found in the EC. Such results add to existing body of knowledge about the effectiveness of digital educational games learning approach, in particular, its efficacy in the vocabulary acquisition of primary school students in China.
Moreover, the post experimental data analysis results suggest that the effectiveness of the educational game-based approach in enhancing primary school students' vocabulary acquisition may be attributed to the fact that is helpful to maintain learners' motivation in goal-oriented tasks, thus enhancing the retention of knowledge. This finding also echoes Zou et al.'s [41] conclusion, which indicated that teaching language with information and communication technologies in a gamification context fostered language acquisition remarkably.

Students' Perceptions of the Digital Educational Game-Based Learning Approach
First, with regard to the perceived usefulness and ease of use of this learning approach, the post-experiment results provide further support to previous research about the usefulness of similar educational game-based activities in promoting students' motivation and self-exploration and active learning [42,43]. Most students agreed that Quizlet was useful and easy to operate. By using Quizlet, they could take control of their English vocabulary learning through games both in the classroom and at home. With the multimodal content delivered through Quizlet, the approach could also facilitate students' language acquisition and retention [44]. However, similar negative feedback as is revealed in pertinent studies was also documented [7]. Specifically, although participants generally agreed that Quizlet was easy to use, interview data also revealed that some technical problems regarding the usability of this platform may impede students' learning experiences [45,46].
Next, regarding perceived enjoyment and engagement, both quantitative and qualitative results indicated that participants enjoyed learning English vocabulary with Quizlet and were actively engaged in the learning process. Specifically, some participants mentioned that previously, it was no easy task to memorize English vocabulary; however, with the Quizlet-assisted vocabulary learning strategy, they became eager to participate in learning activities. These findings corroborate other research findings suggesting that game dynamics embedded within the approach can effectively prevent students from becoming distracted and make the learning process more exciting and interesting [13,47,48].
Last but not least, similar to previous research on the use of digital games in primary schools in other contexts [21,49], our results also revealed that students were mostly willing to engage with game-based learning in the future and would share the platform with their friends because of their positive experience.
Overall, our findings lent further support to the effect of digital educational games on English vocabulary learning in primary schools as the learning approach helps keep students engaged in their learning. This means that when teaching vocabulary, primary school English teachers in China may consider adding game dynamics to their instruction. Further, highly positive perceptions of the game-based vocabulary learning approach among the participants were also documented, suggesting students may be ready for the change.
For school leaders and teachers in China, given the research findings, a more inclusive attitude towards digital educational games is needed. Efforts should be made so that teachers can be aware of the potentials and issues pertinent to the implantation of digital educational games in classrooms. Teachers should realize that, although it might be time-consuming to prepare lessons with educational games, it would be worthwhile to spend time to design such courses. Additionally, digital educational games can keep students motivated in pre-set learning tasks. This is especially pertinent given the ongoing impact of the COVID-19.

Limitations
This study is an initial attempt to integrate digital educational games into English language education. However, the following limitations must be addressed. First, the present study is a case study conducted with only two classes of students from one primary school, which might influence the generalizability of its findings. Future studies may consider a larger sample size; moreover, teachers' and parents' perceptions of digital educational games can also be also be explored. Next, the long-term effects of the intervention were not examined. Future research can incorporate different time points after the intervention to further gauge the growth of students' vocabulary. Finally, due to the restrictive policy of digital devices on school premises, the intervention of the study had to be completed in a computer classroom; thus, the possibility of the Hawthorne effect could not be eliminated.  Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Conflicts of Interest:
The authors declare no conflict of interest.

Lesson outline (EC):
1. Presentation (10 min) The teacher uses word cards in Quizlet to display new words and teach students the meaning and the pronunciation of the new words by flipping word cards. 2. Practice (25 min) Ask students to practice using "match" on Quizlet by themselves. Ask students to play Quizlet Live in team mode. 3. Consolidation (10 min) Ask students to finish the "Test" of this unit on Quizlet.Reward students who get high grades.

Lesson outline (CC):
1. Presentation (10 min) The teacher uses word cards to display new words and teach students the meaning and the pronunciation of the new words by flipping word cards. 2. Practice (25 min) Listen to the tape. Ask students to read after the tape.Choose several students to read, and then have the whole class read together. 3. Consolidation Flash the word cards. Flash the word cards.Teacher asks the students, "What is this on this card?" Students answer, "It's chicken/milk . . . " Appendix B