Why Do Students Prefer Augmented Reality: A Mixed-Method Study on Preschool Teacher Students’ Perceptions on Self-Assessment AR Quizzes in Science Education

Abstract

Students’ perceptions on AR applications have gained researchers’ interest in the field of ICT-enhanced teaching and learning, especially in recent years. The current study investigates students’ perceptions concerning their learning and immersive experiences gained using AR quizzes with formative self-assessment purposes in a science education university course during one semester. The research followed the mixed-method approach, and the data were collected sequentially by questionnaires and focus group discussions. A descriptive statistical analysis and a thematic analysis were conducted, respectively. Fifty-one (51) students participated in the quantitative data collection procedure and ten (10) of them participated in the focus groups. The results indicate that students are in favor of AR quizzes which justify their stance based on the learning gains and the immersive experiences. AR was underlined to play a significant role by creating an engaging environment of immersion. The findings support the positive stances of students over the combination of AR and formative self-assessment and highlight the role of immersion supported by AR technologies. Additionally, based on the relatively long period of application, the findings create doubts concerning the influence of the novelty effect on students’ positive stances toward AR.

1. Introduction

During the COVID-19 pandemic and the emergency remote teaching, teachers and students at all educational levels faced new challenges. Teachers faced a situation in which their students needed additional support to replace the immediacy of communication classroom offers [1].

In this period, the author found himself in the same position while teaching a science education course to preschool teacher students. To support his/her students and overcome the obstacles of remote teaching and the lack of adequate communication, he/she developed a series of self-assessment augmented reality quizzes, using the GoMeta platform (Metaverse app). Upon return to the conventional educational settings, that material was offered to students as additional material for formative self-assessment purposes. However, is there something to be gained by the combination of self-assessment with augmented reality technology? As will be discussed below, both self-assessment and augmented reality technologies in educational settings seem to have an equally positive effect on students’ academic performance and experience.

The formative use of self-assessment represents a kind of assessment that works more as a learning strategy that could enhance and promote students’ learning [2] than as a substitute for other types of assessment. Self-assessment has a positive effect on students’ academic performance since it promotes self-control and reflection over their learning process [3]. More so, the use of self-assessment in higher education offers students the opportunity to develop skills relating to learning autonomy [4], commitment, engagement [5], and self-efficacy [6]. There are many ways that self-assessment can be delivered to students (paper and pencil, computers and the web, or internet-connected mobile devices). Despite the fact that the findings on utilizing technological ways to deliver self-assessment and their impact on students’ attitudes and performance seem yet inconclusive in the literature, “there are a considerable number of studies providing evidence that technology-mediated delivery modes have a positive impact on attitudes and performance” [6]. While research seems to support the use of mobile technologies, the author was not able to find any research that exploits augmented reality technologies in self-assessment applications or interventions in higher education settings.

During the last decade, there is growing interest among scholars and educators in the applications of augmented reality (AR) technologies in education [7,8]. Augmented reality is the technology that allows the overlay or projection of visual objects onto the real world [9]. The literature underlines the potential advantages of the use of AR in educational settings [9]. Empirical research suggests that the use of AR can be beneficial for both learning and learning outcomes. More specifically, most of the studies reported that the use of AR can promote enhanced learning performance (e.g., [10,11]), positive attitudes on learning [12], and learning motivation, and can “help students to understand” and “enhance satisfaction” [9]. Additionally, some researchers have supported that students consider AR technology both easy to use and enjoyable [13].

Despite the development of the field during the last decade and the promising results of the studies, some researchers [13,14] have raised awareness toward the novelty effect that may influence the positive results towards students’ learning attitudes and motivation. Garzón and Acevedo [15] seem to share their concern by reporting that 93% of the interventions included in their extensive review lasted less than one month, suggesting that “further studies are required to determine whether a novelty factor will continue to significantly affect results in longer-term applications”.

In the current study, a mixed-method research approach was followed to explore preschool teacher students’ perceptions on their learning and immersive experiences of using AR quizzes for formative self-assessment purposes in a science education course throughout a semester.

2. Materials and Methods

As noted in the introduction, the design and implementation of the self-assessment material and the pilot implementation took place before and during the fall semester of 2020–2021, respectively. Based on students’ feedback, the author made minor revisions mainly on technical issues (broken images/animation/connections). The following year, in the fall semester 2021–2022, the self-assessment material was offered through an e-class at the end of every week. It is during this semester that the current study was conducted, following a mixed-method approach. The data collection started when all the participating students used all twelve (12) AR quizzes following the content of 12 lessons over a period of 4 months.

2.1. Participants

Fifty-one (51) preschool teacher students (96% female and 4% male, representative of the sex distribution of all students enrolled in the course) participated in the research by completing the questionnaire, and ten (10) of them (all female) participated in the two focus groups conducted. Furthermore, 98% of them were first-year students, while 86% of participants were between 17 and 23 years old, 6% were between 24 and 30 years old, and 8% were 30 years old or more.

During their school years, most of these students (86%) did not have science as a major discipline, which indicates a weak background knowledge of science in the context of the Greek educational system. Regarding their stances on technology, 12% of students stated that they love new technologies and are among the first to experiment with a new one, 34% of them stated that they like new technologies and usually use a new technology before most people in their circle do, 40% tend to decide to use technology when most people already use a new technology, 8% are usually the last in their cycle to use a new technology, and 4% stated that they are cautious and use a new technology only when it is necessary.

2.2. Context

The implementation took place in a Department of Early Childhood Education within a first-year science education course which includes three kinds of topics: (a) introduction to theoretical aspects of science education, constructivism, and inquiry-based learning; (b) fundamental and other selected concepts and phenomena of science, suitable for early-years science activities; and (c) students’ alternative conceptions on the previously mentioned concepts and ideas. Introduction to science education comes first and lasts for three weeks (one three-hour lesson per week). After that, four sets of lessons concerning the content of science, the relevant students’ alternative conceptions (A/C), and suggestions for early-year science activities follow (see

Table 1. Description of the course structure by lesson.

Course Parts	Lessons
Introduction to theoretical aspects of Science education, constructivism, and inquiry-based learning	1. Why do we teach science in the early years? 2. How do students learn? Introduction to learning theories 3. What are students’ alternative conceptions and how can we handle them? Introduction to inquiry-based learning
State of matter, fundamental quantities, and mixtures	4—State of matter, mass, volume, density, mixtures 5—Students’ alternative conceptions on state of matter, mass, volume, density, mixtures
Pressure, buoyancy, sinking and floating	6—Pressure (including atmospheric hydrostatic pressure) and related phenomena 7—Buoyancy; sinking and floating 8—Students’ alternative conceptions on pressure, buoyancy, sinking and floating
Temperature and heat	9—Temperature; heat; thermal expansion; phase transitions; the case of water and the water cycle 10—Students’ alternative conceptions on Temperature; heat; thermal expansion; phase transitions; the case of water and the water cycle
Oscillations, waves, and sound	11—Oscillations, waves, and sound 12—Students’ alternative conceptions on oscillations, waves, and sound

).

The course is mandatory for all students; it is the first among several science courses in the department, and it is offered in the first semester of the degree’s curriculum.

2.3. Data Collection

First, data were collected through a questionnaire designed by the author to investigate students’ stances on the quizzes and their contribution to learning, as well as to explore how some characteristics of the AR quizzes played a role in students’ stances. Based on the results, a focus group protocol was designed to further explore students’ views and perceptions about their experience and the above-mentioned issues. All students were fully informed regarding the aim of the research; they participated voluntarily and consented to the use and publication of the data for research purposes. All the data were handled appropriately to protect the anonymity of all the participants.

2.4. Educational Design and Implementation

For the development of the quizzes, the author followed the same design approach (see Figure 1). First, he/she analyzed the content of the lesson to identify 10 to 15 key ideas and concepts for each lesson. Based on these key ideas and concepts, two types of closed questions were constructed, in the form of multiple choice and true or false questions.

In each question, a response was constructed that gives positive feedback to the participant for the right choice, a thorough answer to the question, and a continue-to-the-next question button. When the participant chooses a wrong answer, he/she finds himself in a loop starting with a feedback note (sometimes the same and sometimes different for every wrong choice depending on the case) with a hint aiming to guide the participant towards finding the right answer. Then, the participant returns to the original question and gives a new response. The loop stops when the participant chooses the right answer.

Then, the author used the GoMeta platform to create one quiz/experience for each lesson (experience is a GoMeta platform terminology for an AR scene sequence that is triggered by a QR code). The GoMeta platform allows the designer to create augmented reality experiences that contain 3D models and animations, 2D animations, images, GIFs, and sounds. The platform offers a variety of pre-designed scenes and multimedia that a designer/user can adopt for the creation of an experience by choosing the content of the scene. The designer creates a series of scenes and creates connections between them in order to give the player the sense of a unified experience. Participants trigger the AR experience by scanning a QR code generated by the platform. Each of the steps described above (question, positive feedback, and feedback on wrong choices) were placed in different scenes connected properly with each other (see Figure 2). A music theme was used as a sound background for the questions, while well-recognized sounds such as clapping or buzzers were used for right and wrong answers, respectively.

Every week, the trigger image of the experience/quiz related to the lesson was launched three days before the next one to the students by an announcement on the course e-class platform. Students were asked to participate in the AR quiz before the next lesson.

2.5. Data Analysis

A descriptive statistical analysis was conducted on the data of the questionnaires using IBM SPSS Statistics 27. The data from the focus groups were analyzed using thematic analysis [16,17], seeking to identify themes, categories, and sub-categories. According to Braun and Clarke [17] (p. 57), thematic analysis is a method ‘for systematically identifying, organizing, and offering insight into patterns of meaning (themes) across a data set’. To identify relevant themes, the author read the focus groups’ transcripts several times and collaborated with two external researchers to identify subcategories, categories, and themes. Atlas.ti was utilized for the analysis of the qualitative data and 37 codes were generated by the analysis. Three major themes emerged from these codes: (a) students’ perceptions towards their learning experience; (b) students’ perceptions towards their immersive experiences; and (c) reasons for recommending it to others.

3. Results

The results of the study are both quantitative and qualitative, and they will be presented in an integrated method based on the themes revealed by the analysis of the qualitative data. Since the qualitative data collected from the students provide more information about students’ perceptions, more emphasis will be given to the results coming for them [18,19]. However, it is important to mention that the results from the quantitative data will be well respected too [20]. The presentation of the results will also follow the sequential design used in this research [18] since that seems to support a better interpretation of the results.

3.1. Students’ Perceptions towards Their Learning Experience

The first theme includes two categories concerning the purpose of using the AR quizzes and the way students used them, and students’ perceptions regarding if and why quizzes had a positive effect on their learning.

3.1.1. Purpose and Learning Path

This category refers to students’ perceptions about the aim of AR quizzes based on the description that students provided during the focus groups of how quizzes are used by them and some ideas on how they intend to use them in the future to enhance their understanding. Students used them as a yardstick to their knowledge by checking the degree of their comprehension of concepts and of the course material. Mistakes in the quizzes led students to either review the specific topics again or simply revise their ideas within the game. Students also suggested that AR quizzes could be potentially used in the future as a means of refreshing or re-checking their content knowledge.

a. Knowledge of their level of understanding

A great majority of students (see Figure 3) believe that the AR quizzes helped them understand what they have or have not understood on the topic of the quiz. Of note, 44% of them responded that the AR quizzes helped them a lot, while 56% said that they helped them very much. It is interesting to mention that there was no negative response. Additionally, based on the qualitative data, students seem to view the quizzes as a yardstick for their understanding and they believe that they would not be able to see their mistakes without these quizzes:

1:29 I had a yardstick for myself, to compare what I’ve understood from the lesson. It was a measure of whether what I believed at the time was correct. Because until it’s time for a quiz or an exam, you can’t be sure that everything you’ve heard, whether you’ve memorized it, or you’ve absorbed it, or you’ve understood it correctly.

1:53 While sometimes I thought I knew something, that I got it, I went to it (the quiz) and said that “D., you’re missing it somewhere in 2,3,4 points”. If I hadn’t taken the quiz I wouldn’t have understood.

1:83 Many times, I studied and while I had an idea, an opinion when I did the quiz, I saw that I was completely wrong, and I was like... what have I been studying for so many hours?

Students prepared themselves similarly before using the AR quizzes. Based on the focus group analysis, students usually study using the material of the lesson and the books of the course and then challenge their knowledge by using the AR quizzes:

1:8 First I read the slides and then I did the metaverse quiz. However, the first two times, I did the quizzes before studying the slides. But then, when some more specific terminology of your course came in, I think that’s where I changed the order and said to myself: I should read the material first, what the teacher says in the lesson, and then do the metaverse.

1:56 It was definitely very helpful that you had to read the module; it was actually a prerequisite. That you had to have read the module, the e-class, if it’s relevant from course’s book and then go there and check yourself; what you understood what you didn’t

1:99 I was reading the slides on the e-class and I did the quiz another day on an unrelated day and time to see if I had memorized what I had read.

Although students seem to use the material similarly so far, the data revealed two different learning paths which students followed to handle the feedback from the AR quizzes. On the one hand, some students underlined the issues which troubled them during the AR quiz and returned to the learning material to clarify and study more carefully on these issues:

1:56 If there were some gaps, more than 2–3 questions you didn’t give the right answer to and there were certain things you didn’t understand, you could go back to the learning material and read it again. And do it again to see that you understood it.

1:85 It’s a test that confirms to us personally how much we have understood the lesson. And the good thing is that after I had studied the course, I did the quiz and I saw that sometimes I made mistakes, then I went back to the material, studied again, and tried to figure out where I made the mistake to get a better result.

1:99 When I had a few wrong answers I went home and looked for what I did wrong and redid the quiz.

1:101 I read during the day and did it at night. It helped me a lot because the next day when I didn’t know it, I would go in the material to see the mistake and figure it out.

On the other hand, some students clarified issues they faced within the quiz based on the feedback they had when they chose a wrong answer:

1:41 It was very helpful to understand because when you read why you chose the correct or the wrong answer (feedback) if you did it wrong you went back and answered it again correctly.

1:84 The truth is that most of the time what it showed was a direction, it didn’t exactly show you Wrong or Correct. It was giving you guidance that was helpful enough in combination with the reading helped you to understand.

1:44 Ιt was a way of playing and so through the game, we learned a lot about the lesson; it’s good that we could go back (to the same question) and figure out what we were doing wrong and so understand it better.

b. Use the quiz to refresh or re-check my knowledge

The focus group analysis revealed one more way in which the AR quizzes could be used. Some students suggested that the quizzes can also be useful after the lesson as a tool to refresh or re-check their knowledge (before their final exams, for example):

1:51 I’m also thinking about using it for the exams. Right now, I have it all together (if I get into the app I have on my phone) and I will definitely use it for my studying for the exam to do my checking. I will log into the app and re-take the quizzes in no particular order. Having them all in one place is very important. It’s not like “I did it and missed it”. They continue to be in the app. It’s really easy, I go into the app to find them, and I have them one after another.

1:113 The ideal would be every time we read each topic to do the quiz and then after we study all the topics to do the whole thing consecutively

1:114 Because it contains the essence of the course, I think doing the quizzes will be a good refresher in memory (before exams).

3.1.2. Contribution to Learning

This category refers to students’ perceptions concerning the potential effects of AR quizzes on their learning and the reasons for these effects. Students seemed to find AR quizzes very helpful in their study and supported the idea that the quizzes made learning easier. Additionally, according to the students, the AR quizzes seem to have had a positive effect on learning for diverse learning styles and on different kinds of topics, especially for the more challenging ones. Based on the results of the questionnaire (Figure 4), 50% and 48% of the participants stated that the AR quizzes were very much or a lot helpful in their study, respectively. It seems that almost all participants believe that AR quizzes have a positive effect on learning.

a. Easier way to learn

Based on several statements throughout the focus group discussion, it seems that students see AR quizzes as an easier and more effective way to learn:

1:49 Υou’re playing but you’re learning at the same time. You gain knowledge through it. It’s more enjoyable and easier.

1:89 Besides being like a game, I think it was easy for everybody, even if somebody hadn’t studied, he/she could use the quiz and gain knowledge.

1:97 Through the app it’s easier, I’ve memorized things without having to study for hours.

b. Different learning styles

Several statements of the students, regarding why the AR quizzes helped them personally to learn, reveal that the AR quizzes helped students who described their learning style in a completely different way:

1:92 Not everybody studies the same way. We see the quiz as a game, and we get something.

1:89 So, that’s the way I understand better. I’m more a visual and a math guy. The quiz helps me to keep it (the knowledge) in my mind.

1:100 The quiz helped me a lot despite the fact that I’m not a visual person. And when I made a mistake, I reread my notes so that the ones I made mistakes stayed with me.

1:105 We understand the lesson better, the picture stays, the repetition makes you get into the hole.

c. Helpful in the most challenging topics

Based on the questionnaire results, students seem to see AR quizzes as equally helpful in all three kinds of topics. As shown in Figure 5, regarding the three different sections of the course, 50–54% of students stated that AR quizzes are very much helpful, while 34–36% of students said that the quizzes helped them a lot.

Moreover, when students were asked if they found AR quizzes more helpful in a specific kind of topic than the others, they responded in two ways. AR quizzes were perceived by some students as equally helpful for all kinds of topics while, for others, they were more helpful for specific content which was more challenging for them:

1:11 It helped me more with the students’ ideas because the answers actually gave me feedback on what I should have in mind during the students’ answers, which was difficult at first.

1:31 It was equally useful for all three strands.

1:60 I found it useful in the parts where it was pure physics, i.e., where we were dealing with phenomena, basic concepts, etc. Because I was a “Theoretician” (author’s note: a student that studies Humanities as a major discipline in school) and didn’t do any physics after 10th grade, actually not even in 10th grade...I had some gaps even in concepts from everyday life. It helped me more in those subjects because they were more interactive, and I remembered some things in combination with the knowledge I got from the presentations and the book. It was more helpful in that part which was also more difficult for me than the other two strands.

1:61 I can’t pick out. I think it was the same across the bunch. It was probably more helpful for everyone where they had more difficulty.

1:103 I think more about the concepts (definitions, etc.) and students’ ideas. Personally, when it came to the students’ ideas, I had quite a hard time just reading the slides.

3.2. Students’ Perceptions towards Their Immersive Experience

Before presenting the second theme, it is important to explain why the study focused extensively on students’ immersive experiences. Both the feedback from the piloting implementation and the data collected through the questionnaire reveal a great preference of students for AR quizzes in comparison with more static and/or traditional quizzes (i.e., e-class quizzes). Based on the questionnaire results (Figure 6), more than nine out of ten students (see Figure 6) state that they definitely (72%) or probably (24%) prefer augmented reality quizzes over a more traditional/static one. Students’ perceptions on why they prefer AR quizzes to that extent and which specific characteristics played a positive or negative role in their experiences constitute the second theme that emerged from the qualitative analysis, which is presented below.

3.2.1. How Students Perceive AR Quizzes

That first category includes some characterizations that students give to the AR quizzes to explain why they prefer this type of quiz compared to others they have used for educational purposes, including multimodal ones.

a. Interesting and Entertaining

Based on students’ questionnaires (Figure 7), almost all students agree that AR quizzes are much (50%) or a lot (48%) more interesting than other quizzes used throughout their educational life. The same holds true with the data from the focus groups which support the same result since students referred to the AR quizzes as interesting and entertaining throughout the discussion:

1:2 It was very interesting and fun. At first, it scared me a little bit because I was like, “oh my god what is that?” But then I liked it and I couldn’t wait until Tuesday to do the metaverse.

1:18 It’s not monotonous and boring for someone who wants to learn.

1:40 It was indeed very interesting the way the quiz unfolded, it was very interactive, it was nice to see the faces in the answers when it was right.

1:66 Baboot (a/n the name has been changed because it’s a reference to a popular platform that offers interactive multimodal quizzes) was static, set up, nothing special. It was also instructive, but the Metaverse had something more playful that was more interesting.

1:88 I think it was an innovative application that intrigued students’ interest and made them realize what mistake they made.

1:116 I would like to add that it was fun and… funny.

b. Engaging—Not an assessment

Some students’ statements in the focus group discussion underlined the enhancement of their engagement during the use of AR quizzes and the feeling that they do not participate in an assessment procedure:

1:19 The other quizzes are more monotonous. I mean “answer to see if you got it right or wrong”, simple as that.

1:20 Yes, here we have the sentiment. So, it stimulates the emotional part, not just the cognitive.

1:21 Yes, it makes you more enthusiastic to answer the questions and not give up until the end. I mean it didn’t occur to me to say at some point “oh, stop it”, “it’s a bit boring” and “why did he give this to us?” Whereas in other quizzes... you feel like somebody’s checking you out.

c. Playful

Throughout the focus group discussions, students used to refer to the AR quizzes as a game or described their experience in a manner that indicates that perception (i.e., a losing or winning feeling):

1:17 Yes because let’s say when I do a correct answer it rewards me. With a little bell which we find mostly in children’s toys. It generally has game-like reactions.

1:49 It’s also a more pleasant way. You play but learn at the same time.

1:74 I sometimes get very enthusiastic. I had the feeling that I was winning or losing.

1:89 It was like a game, I think that’s why it was easy for everybody.

1:108 It was like a game, and I wanted to see something more.

3.2.2. Characteristics

In the second category, specific characteristics that had a positive or negative contribution to students’ experiences will be presented. Based on the findings, multimodality, an augmented reality environment, anonymity, some design characteristics, and the fact that it was easy to use played a significant positive role in students’ experiences with the quizzes. By contrast, unreliable internet and unreliable devices seem to negatively influence the experience.

a. Multimodality

Multimodality is revealed as a very important characteristic based on the results of both qualitative and quantitative data (Figure 8). Almost half of the students consider multimodality a characteristic that very much (47.9%) influences their preference for AR quizzes, while almost four out of ten suggest that it influences their preferences a lot (39.6%).

In the focus groups, students referred directly or indirectly to multimodality (sound, motion, images, cartoon, GIF, colors, etc.) several times as a very important characteristic of the AR quizzes:

1:14 The movements of all these objects, the movement combined with the music generally had a multimodal so to speak message to the student. I believe that it was very very nice.

1:42 Then we also enjoyed the faces in the game, the sounds and those were combined with the answers and we remember those as well (combined).

1:72 The cartoons were very important, beautiful, and clever. The GIFs were also very successful. I think it’s the picture first and then the words.

1:73 Those images, the vivid colours, the bright colours in some of the faces he was showing. The fact that he had sounds whether you did it right or wrong, The gifs were the best. It gave you the feeling of playing and learning.

1:78 Very important were the sounds when you answered Right or Wrong. It was very nice and helpful.

1:79 The Wrong (sound) haunts me in my sleep.

b. Augmented Reality

A great majority of students were also found to highlight the augmented reality environment as a very positive characteristic of their experience (Figure 9). About eight out of ten students stated in the questionnaire that AR influences them to prefer that kind of quiz over others very much (50%) or a lot (29.2%). The same holds true with the statements from the focus group analysis. Students seem to perceive AR as an advantage over other multimodal quizzes:

1:37 It would be interesting if it was just multimodal, but AR is one more advantage.

1:75 It’s very interesting that you move the mobile and the answer floats.

1:77 It’s more interesting if it appears on the wall and in the room. I think if it was a static background it wouldn’t draw as much interest. I mean it challenges you to do it and you learn more easily and you want to do it

1:111 The floating images are something out of this world for you. It’s like wearing VR glasses and doing it.

The focus group analysis revealed two other findings that could explain students’ preference and provide more information about the experience: immersion and the feeling of personal communication, and the interaction with the quiz within the physical world.

i. Immersion and Personal communication

Students’ statements indicate that the AR environment creates a feeling of presence of someone in the students’ physical world and of the possibility of a more personal communication between them and a teacher (who is represented by the quiz):

1:59 For me, the important thing was like having the professor in front of me. And the first time it felt like I was checking my knowledge and when I did it again and on the wrong answers...it felt like live; live teaching; like we were communicating. And it was too clever to include all these cartoons and phrases such as “you got it right”. It was very nice, I felt like I was having a conversation with somebody. I mean, it felt like something personal.

1:65 Compared to Baboot, which was something similar, the metaverse is more interesting. It has more interactivity, pictures, the fact that you have to move the phone to find the answer, the cartoons, and the sounds. It’s both more fun and more interactive and you get the feeling that someone is there with you.

1:67 There was someone there guiding you. I felt that there was somebody in there and actually giving me directions, telling me to come back, that was actually somebody there.

1:68 It was a bit weird at first, but you could see the question and behind them, your desk or your notes and you felt like you were with the professor, and he was asking you the questions.

1:69 It was like there was a professor there. It was like having the class at your house. He was guiding you to take the right steps, to pick the right answers.

ii. Interaction in the physical world

Another experience revealed from the focus groups analysis is the interaction with the quiz, not only in the mobile display but within the physical world, since students seem to move in their physical environments:

1:116 I’m sometimes standing up and looking for the questions and answers, that’s fun.

1:117 And I’ve done some weird moves to find the questions

c. Design

This sub-category includes several design characteristics found to play a positive role in students’ experience. These design characteristics are: feedback and looping, repetition, anonymity, short storyline, and direct and short questions.

i. Feedback and looping

Based on the results of the questionnaire (Figure 10), almost all students stated that the feedback received and looping design are characteristics that influence students’ positive stance toward AR quizzes very much (68.8%) or a lot (25%).

Students’ statements from the focus groups support the above and explain that feedback and looping helped them understand their mistakes and correct themselves while enhancing the interactive nature of the quiz:

1:41 It had the feedback, it helped you understand why what we chose was right or wrong. Very helpful and that when you were reading it, if you got it wrong it took you back and back to answer the question again.

1:70 It felt very much alive because I was getting an immediate answer. I didn’t wait to complete a cycle of answers to get feedback because the answer wasn’t with a tick and it was like someone was answering. That gave the feeling of communication

1:84 The truth is that most of the time what it showed was like a direction because it didn’t exactly show you True or False. He was giving you a guide that was quite helpful and after made you repeat the questions.

The focus group analysis revealed one more characteristic that positively influences their experience—the positive feedback. Students seem to be motivated by the positive feedback expressions when they chose the right answer by sound, images, and text, whether because it looks more like a game and/or because it supports them emotionally:

1:17 When I do a correct answer it rewards me. With a little bell which is mostly found in children’s toys. It generally has game-like reactions.

1:64 And it boosted our psychology where we didn’t have it.

1:115 “This app because it interested me made me feel more...happy every time I got the right answers.

ii. Repeat

Similar are the results of the questionnaire given the fact that they were allowed to repeat the quizzes as many times as they wanted (Figure 11). Almost all students expressed the importance, i.e., very much (56.3%) or a lot (35.6%).

Statements from the focus group support those results and explain that, in some cases, it is useful to return back to a previous topic to check understanding again:

1:43 It was very good because we could do it as many times as we wanted. Because generally you might have had a gap in the lesson, in a lesson, and you could go back to it again. That was a very positive thing.

iii. Short storyline and type of questions

Another characteristic is the short storyline. Focus group analysis revealed that a short storyline (a relatively small number of questions and feedback) is very important to keep the quiz interesting and to keep the feeling of a game:

1:15 If they ask me 30–50 questions, I might start to get tired and then I might lose the knowledge part.

1:81 I often had the feeling that there were not enough questions. But on the other hand, I think if there were more it would get boring. It left you with the feeling that it was little.

1:82 D: It might lose the essence of a game because if you get fed up with the game, it ceases to be a game.

Additionally, a student underlined that it is important for the questions to be short and direct:

1:94 It is very important that the questions are concise and direct

iv. Anonymity

About nine out of ten students support that anonymity is an important positive characteristic, i.e., very much (45.8%) or a lot (43.8%) (Figure 12).

The focus group analysis supports finding and disclosing another view whereby even the absence of comparison among the students (not grading) works positively in these quizzes:

1:24 Since it’s anonymous you don’t have the anxiety of shame or that I’m going to be offended now. If it wasn’t anonymoys maybe there would be anxiety of “oh don’t make a mistake, it will be seen now, what will others comment; or am I the 1% the 2%”. I was doing very well but because it’s anonymous, I felt very, very comfortable.

1:32 It has the advantage of anonymity. It’s not stressful. If it wasn’t anonymous I would have been very stressed.”

d. Easy to use

Students’ questionnaire results showed that the use of AR quizzes was not challenging for the great majority of the students (Figure 13). More than half of them stated that AR quizzes were very easy to use (54%), while about four out of ten (42%) participants considered them easy to use.

These results are supported by the focus group analysis. Several statements indicate that, students find AR quizzes easy to use, even on their first time:

1:45 It was too easy. Well, the first time I had a hard time...I said something like “we have to turn it like this”. However, I screenshot this QR code, I went straight in, and then it was too easy.

1:46 It was indeed very easy to use. Okay, the first time because we weren’t used to coming into contact with that kind of stuff it was a bit weird, you know “what’s he going to ask me”, asking each other ‘how did you do it? should I get in and do it?

1:47 It was very easy. I didn’t have a hard time from the beginning. It was fine.

1:90 I think any age can use it. We all have a mobile phone; why not?

1:91 For me it was too easy to do the quiz by just scanning

e. Unreliable internet or device

Students’ statements from the focus groups show that some students faced technological challenges that exclude them totally from the use of AR quizzes or diminish the immersive experiences due to unreliable devices or internet connection:

1:106 Perhaps some may not have access to devices or even reliable Internet. I think for some people the app wouldn’t open or the camera wouldn’t open, so they couldn’t see the site.

1:46 At the beginning we kind of missed the questions a little bit because we were moving the phone too fast. But then we realized that we had to wait a little bit, and something would pop up.

3.3. Recommend to Others

Students’ statements also revealed whether they would recommend the AR quizzes to other students or instructors and why. Most of the students were willing to recommend it to both students and instructors and even suggested that it can be a basic tool for all courses. They support the use by others using arguments from both their learning and immersive experiences presented above:

1:6 Yes, I would suggest it. I would recommend it because it’s a lot of fun.

1:27 “I think it’s a tool that could be used by all courses. Sure, maybe with a different question structure, in a different way but it could be used by all courses. Maybe, like embedding exercises.

1:80 I think it should be introduced in all courses as it has been done with e-class. It would be nice to do it in all courses.

1:93 And I would recommend it to my fellow students and also friends and family. I’ve already done it not only with students but also with others. When they saw me doing the quiz for the first time, I suggested they do it too. Then it became a habit. It’s good, not everyone reads the same way.

1:95 Memorization was unavoidable in high school, most of the lessons in school are theoretical and it’s hard to memorize everything. It would be more interactive if other teachers used it.

4. Discussion

This research, through questionnaires and focus groups, was instigated to reveal preschool teacher students’ perceptions on their learning and immersive experiences of using AR quizzes for self-assessment purposes. Both the qualitative and quantitative results indicate a positive response from the students regarding their learning and the immersive experiences using the AR quizzes, suggesting that they would recommend this kind of AR quizzes to their colleagues and teachers.

Findings concerning self-assessment seem to be aligned with the literature since students perceived the AR quizzes as part of a learning process [2], whereby any sense of participation in the assessment was diminished. It was perceived as a learning process in which students have the freedom to choose their path to understanding, as well as self-control and reflection over learning [3]. In parallel, students recognized that AR quizzes helped them to learn and understand in an easier way [9], thus accommodating their different learning styles.

Most of the design characteristics, such as feedback and repetition combined with multimodality, were found to play a positive role in students’ stances towards the AR self-assessment quizzes. This finding concerning the feedback [21,22,23,24] and repetition [25] is aligned with the literature since these characteristics are reported as common elements of gamified environments in educational settings with a similarly positive effect [26]. The findings of the current research also agree with the literature [8,9] concerning the preference of students for AR technologies, which is characterized by the students of the present study as interesting, entertaining, engaging, playful, and easy to use. More specifically, the findings are aligned with the results of Ređep and Hajdin [27], who report that “the main idea that extends through all analyzed papers is that the use of AR in teaching activities, and especially with game elements, improves student efficiency, encourages attention and interest, brings improvements in collaboration and perception of fun, improves teaching and learning processes”.

An important dimension that seems to have added value in that application is the element of augmented reality. Students mentioned that the quiz became part of their physical world which created a sense of personal communication and a feeling of real-world interaction. Immersion seems to be very influential and enhances their engagement and motivation. These findings are supported by the literature since the ability of AR to create immersive hybrid learning environments by combining digital and physical objects was one of the first affordances highlighted [28]. Additionally, Georgiou and Kyza [29] underline this kind of immersion and suggest that “there is a positive relationship between immersion and students’ learning gains”.

In contrast to concerns expressed in the literature [13,14] associated with the effect of the novelty on students’ positive attitudes toward the use of AR technologies in educational settings and the possibility of these positive attitudes being decreased over a long-term utilization of AR, the students of the current study seem to continue to have a very positive attitude toward augmented reality technologies, even after a relatively long period of application, as they become more familiar with it. Additionally, they propose the adoption of AR quizzes in other courses as a mandatory or main technique.

Finally, it is important to mention the limitations of the current study. The study was conducted in a specific educational context. The students of the department in which the implementation took place are 90% female—a gender distribution which does not reflect the gender distribution of other higher education departments. Moreover, in the current study, a specific platform, application, and augmented environment were used. Further studies could use the approach in different educational contexts and in other augmented environments.

5. Conclusions

The use of AR technologies in teaching and learning material with formative self-assessment purposes could be valuable to students’ academic performance, since both can support their engagement and help them learn in an easier way. The augmented reality environment seems to have an added value per se by creating a sense of personal communication that is found to be very engaging for students. As opposed to the novelty effect concerns reported in the literature, the novelty might have less influence, compared to what is implied by researchers, on students’ stances towards augmented reality applications, motivation, and learning.

Further studies are required to explore the potential provided by the combination of augmented reality affordances and self-assessment through other kinds of implementations. Additionally, long-term interventions or applications of AR technologies in educational settings could provide more evidence that supports or declines the novelty effect concerns and explores the learning gains from the use of AR compared to more traditional material or technologies.