The Motivation of Technological Scenarios in Augmented Reality (AR): Results of Different Experiments

Abstract

Augmented Reality (AR) is an emergent technology that is acquiring more and more relevance in teaching every day. Together with mobile technology, this combination arises as one of the most effective binomials to support significant and ubiquitous learning. Nevertheless, this binomial can only prove valid if the student is motivated to use it during the learning process. An attempt was made through the implementation of Keller’s Instructional Material Motivational Survey model o determine the degree of motivation of Pedagogy, Medicine and Art students from the University of Seville for using AR-enriched notes available by means of mobile devices in the classroom. Three applications designed for the subjects of Educational Technology, Anatomy and Art served to assess it positively in terms of the motivation raised by the participation in the experiment, as well as regarding academic performance improvement. It can additionally be stated that our main finding was a link between students’ motivation to use the enriched notes and the performance obtained in the subject in which they use them. Evidence was also found that the utilization of Augmented Reality benefits the learning process.

1. Introduction

Augmented Reality (AR) is becoming an emergent technology with great potential in various and largely different fields of application, such as geography, with regard to watershed control [1], medicine, cultural heritage [2] or industry [3]. It is likewise being used in the educational context [4,5,6,7,8,9], amongst other reasons because AR permits easy access to information—most often through mobile devices. It must be remembered that these devices are highly present in the life of university students all over Latin America [10,11].

According to Azuma [12], AR can be defined as a system with the following characteristics: it combines real and virtual; it is interactive in real time, and it is registered in 3D. AR technology consequently allows us to combine digital and physical information in real time by means of different technological media such as tablets, smartphones [13] or Augmented Reality glasses [3], to quote but three, thus providing the chance to create a new reality [14]. “Augmented reality visibly introduces the knowledge that students need to learn inside its real environment” [15].

Such a mixed integration of physical and digital reality may take place at various levels, ranging from the utilization of QR codes, the use of images, the exploitation of 3D objects and the Global Positioning System (GPS)-mediated mobilization of coordinates to thermal footprints [6]. Furthermore, one can classify AR systems depending on their location or according to images. “Location-based AR systems take advantage of the data about the position of mobile devices, determined by the Global Positioning System (GPS) or other Wi-Fi-based positioning systems… Instead, image-based AR focuses on image recognition techniques that serve to identify the position of physical objects in a real environment, so that the virtual contents related to such objects can be properly situated” [16].

The relevance of AR for learning has to do:

(a) with its specific characteristics: being a mixed reality, it offers the possibility for a real-time integration both of different information layers and of the various formats where that information may appear (texts, URLs, videos…). Furthermore, as an interesting, user-friendly technology, it allows us to enrich or alter the information about reality, adding further contents to it [6]; and (b) with the fact that the devices most frequently used for its observation, e.g., smartphones, are technologies within easy reach of university learners.

Numerous authors [17,18,19,20,21,22,23,24,25,26,27,28] have highlighted that AR utilization opens a wide range of possibilities for us, namely: (1) removing information that might hinder the assimilation of significant information by students; (2) increasing or enriching the information about reality so that it can be more easily understood by students; (3) being able to observe an object from different points of view, thus enabling the student to select the time and place of observation; (4) fostering ubiquitous learning; e) creating safe “artificial” scenarios, such as laboratories or simulators; (5) enriching the materials printed for students with additional information in different formats; (6) turning students into “proconsumers” of learning objects in AR format; and (7) achieving an improved learning through its use. Furthermore, AR can be used in a variety of disciplines and at different educational levels [29,30], even though university learning admittedly stands out as the context where it appears the most often.

As for its impact on education, the number of research studies carried out so far is still limited—even through that number is gradually increasing [31]. This possibly constitutes one of the most serious problems for its incorporation into teaching. Some aspects have been stressed in those works, however: that students show favorable attitudes towards AR and that its utilization enhances motivation towards learning [32,33]; that it helps create a constructivist learning context [16,34]; that it favors an active teaching environment [35]; that it reduces the cognitive load to which students are subjected in learning tasks [36,37]; that students have a high degree of acceptance as well as positive attitudes in relation to AR technology [38,39,40,41,42,43]; that it increases satisfaction levels amongst both students and teachers [24,44,45,46]; that it improves students’ spatial skills and orientation [47,48]; and finally, that its use enhances academic performance [26,30,49]. Additionally, it deserves to be highlighted that experiments are already being carried out for its incorporation into e-learning-based learning initiatives [31,50], as well as into others where students act as learning object producers [51,52].

In any case, we must bear in mind that the incorporation of AR into teaching faces a number of obstacles: this is a new technology; no development-focused educational experiments have been carried out yet; no learning resources and objects produced in AR format exist; teachers have a poor training; teachers need to develop positive attitudes towards AR incorporation into educational practice; no educational experiments have so far been performed in the development of AR-based learning objects; screens do not offer a suitable observation size; there are no conceptual frameworks which can help us find innovative practices in AR implementation; support centers for teachers need to be created so that they will find it easier to produce AR-based objects, as well as their maintenance in servers; a base/basic technology must be available for its observation by students; and an information overload clearly exists [28,52].

Concerning the last obstacle mentioned above, some precautions should be adopted when designing AR objects. Modules must have short and direct contents and be flexible as well as straightforward. They have to reflect an accessible and tolerant approach with a multimedia design, oriented to the student’s action and involvement. Tablets and smartphones do not constitute the goal of the educational process, and therefore, the methodology must focus on practicality and interactivity and adapt to the diversity of devices where it can be observed [53].

This paper shows an experiment carried out within a university context, additionally providing a detailed description of the AR products generated. This serves as the starting point to explain the educational research performed, where the Instructional Material Motivational Survey (IMMS) developed by Keller was used as the assessment tool [4]. The paper finishes with a summary of the results obtained as well as the corresponding conclusions drawn from them.

2. Materials and Methods

2.1. Research Aims

The different research works described below pursued the following objectives:

• Ascertaining the degree of motivation measured by means of Keller’s IMMS model and the dimensions shaping it (attention/relevance/satisfaction) that the utilization of AR objects generated amongst students.
• Analyzing if the degree of motivation (attention/relevance/satisfaction) measured using Keller’s IMMS model influenced students’ knowledge acquisition.

Below can be found the hypotheses that we pose in this study:

• H0 (null hypothesis): There are no significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.
• H1 (alternative hypothesis): There are significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.

2.2. Research Design

It must be stressed from the very beginning that the research was replicated with AR objects specifically made for investigation purposes that belonged to three different areas of knowledge: Educational Sciences; Fine Arts; and Medicine.

Two types of single-case pre-experimental design were used: one intended to analyze the relevance that the interaction with subjects had for the degree of motivation raised, which used a post-test measure; and the other referred to performance, which revolved around the analysis of academic performance and included pre-test and post-test measures [54,55]. More precisely, a treatment (the AR object created) was administered in both designs, complemented with a measurement (the student’s motivation to develop the experiment and to improve academic performance).

The experiment took place following the same sequences in all cases:

• Explaining to students what AR was all about;
• Showing the different objects produced and their possibilities;
• Specifying the website where they could download the different guides and apps for the objects produced;
• Encouraging students to work on their own using their mobile devices with the different objects; and
• Completing questionnaires

2.3. The Sample

All the students involved in the experiment were enrolled in degrees in Medicine (Anatomy) and Pedagogy (Educational Technology), and in the master’s degree in “Art: ideas and production” —all of them taught at the University of Seville—

Table 1. Number of participants in each one of the experiments.

Educational Technology			Medicine			Art
Total	Men	Women	Total	Men	Women	Total	Men	Women
302	77	225	50	21	29	20	5	15
	25.50%	74.50%		42%	58%		25%	75%

. The samples had a non-probabilistic nature; in other words, they were “convenience” or “causal” samples [56,57] determined by the researcher’s ease of access to the subjects making up the sample in question.

It is worth highlighting that the experiment with the Art group took place after a three-hour-long interaction of students with the learning objects, as opposed to the one-week-long interaction period in the other two groups.

2.4. The Objects Produced

The experiment involved three AR objects specifically developed for this study and referred to contents associated with (a) the discipline of Educational Technology: “ways to use video in teaching-learning processes”; (b) Medicine (more precisely, Anatomy); and (c) the field of Fine Arts. A program guide prepared for students told them where they had to download the apps which allowed for the operation of the objects as well as the actions that they could perform with those objects. This was all complemented with a reference bibliography for students to delve deeper into each thematic area.

The three aforementioned objects had different typologies; those related to contents framed within “Educational Technology” (educational roles played by video use) and “Anatomy” can be categorized as belonging to Type II [6] in the format of “notes enriched” with AR objects, transferring the former with a video resource, and the latter with a 3D object that the student could manipulate (Figure 1 shows images of the object created for the “Educational Technology” subject).

Four AR objects were created for the “Anatomy” area, namely: Coxofemoral2; Shoulde2r; Ankle2; and Cervical2. All of them shared the following features: 3D animations; stop and monitoring of the object; enlargement of the size of the component presented so that details become visible; and transfer to an information extending video located on YouTube (Figure 2).

The final object produced can be classified as belonging to the “Mixed Reality” category due to the combination of elements from both “Augmented Reality” and “Virtual Reality.” It shows the Church of the Annunciation in Seville, which belongs to the University of Seville, but initially was the “Professed House of the Company/Society of Jesus”. This church dates back to 1565, and its façade stands out as one of the masterpieces of Sevillian Mannerism. The application developed allows the user to view—on a marker—3D models of the sculptures that Martínez Montáñez made of San Ignacio de Loyola and San Francisco de Borja. These figures, sculpted between 1610 and 1624, are located at the Main Altarpiece of this church. Users can likewise have access to a gigapixel photograph of the Altarpiece, where an extremely high zoom allows them not only to look at every single detail but also to experience a 360º viewing of the central nave, the entrance, and the crossing. They can also watch a video filmed with the help of a drone that shows the interior of the central nave (Figure 3).

Most importantly, the material generated in this case included a series of questions designed to find out if students had paid attention during their interaction (Figure 4). A final result corresponding to the number of questions posed appears at the end of the questionnaire.

We can classify the material produced on this occasion as: Level-2 AR; Immersive virtual reality; 360° vision; 3D modelling; incorporation of audio and gigapixel which helps enlarge the image (Figure 5).

The production of these objects was possible thanks to the use of the different software programs listed in Appendix A, which also specifies the type of use given to each one of them. As can be seen, we used a variety of programs, including Metaio Creator, Eclipse and Ffmpeg, since the production of the respective objects required some specific characteristics. It deserves to be highlighted that all the programs were prepared in such a way that they could work on Android as well as on iOS devices. Appendix B contains the URLs which feature the videos showing the operation of objects and where the apps can be downloaded.

2.5. Information Collection Instruments

The basic tool to diagnose the degree of motivation raised by the participation in the experiment was the Instructional Material Motivational Survey (IMMS), developed by Keller (2010).

According to Keller [4], motivation “refers to the magnitude and direction of behavior. In other words, it has to do with the choice made by an individual in relation to what they experience or the goals that they will try to achieve or avoid, and the degree of effort that they will make for that purpose. Motivation depends on internal looks and external characteristics. Individuals respond to their context on the basis of internal reflections, impulses, perceptions and goals, and according to the perceived base and the real opportunities, together with the reinforcement of the external context”. (Figure 6). In turn, Cheng [41] stresses that this is an internal state or condition which leads us to action, which guides us and persists in our behavior, involving us in certain activities. It also refers—in the classroom context—“to subjective experiences, particularly students’ predisposition to take part in class activities and their reasons to do so” [41].

Keller used a number of different works [1,58,59] to formulate his ARCS model for the motivation applied to instructional design. He sees motivation in instructional contexts as being determined by the interaction of four dimensions: attention (A); relevance (R); confidence (C); and satisfaction (S) (http://www.arcsmodel.com/).

For Keller [59], Attention includes human characteristics such as orientation reflection, curiosity and the search for sensations. Relevance refers to those things which a person perceives as a tool to meet needs and to achieve personal Satisfaction, including the fulfilment of personal objectives. Confidence comprises several motivational constructs. Finally, Satisfaction constitutes the final step that will allow us to maintain the motivation to learn.

The model illustrates how “a person’s curiosity (“attention”) and the motives or values (“relevance”), combined with their hope of success (“confidence”) will determine which objectives acquire the highest prominence, consequently resulting in an intentional effort to reach the goal. Environmental influences such as teacher enthusiasm and social values, instruction quality, clear expectations and the availability of resources will also have some bearing on an objective effort and orientation towards some degree of fulfilment. “Individuals are thus influenced by their own knowledge, capacity and skills too” [1]. Therefore, this model assumes that, if the instruction strategy is considered useful to achieve the goals set, then that strategy acquires importance for students [60], and they will be likely to learn the contents and to acquire the skills.

Keller’s IMMS consists of 36 items which collect information about four dimensions: attention (12 items); confidence (9 items); relevance (9 items); and satisfaction (6 items). The instrument has a Likert-type construction with seven response levels (from 1 = “Extremely unlikely/disagree with” to 7 = “Extremely likely/I agree”).

As stressed by Loorbach, Peters, Karreman and Steehouder [61], the model formulated by Keller “has been used on countless occasions to implement motivation strategies for instruction materials—and to test their effects. Despite being originally designed to influence students’ motivation within a traditional learning environment based on a face-to-face teacher-student interaction, this model has also been applied and thoroughly tested in contexts that require other adjustments, e.g., computer-assisted teaching and virtual or distance education”. More specifically, Keller’s model proved helpful to check the degree of motivation raised by different technological resources amongst students. Examples are: videogames [62]; the inclusion of video in e-learning learning sessions [63]; audio podcasts [64]; Massive Open Online Courses [65,66]; or AR [67,68,69].

We obtained the reliability index for this instrument using Cronbach’s alpha—one of the most suitable tools to obtain it with Likert-type scales [70,71].

Table 2. Reliability index corresponding to the Instructional Material Motivational Survey (IMMS) instrument.

	Video Use	Anatomy	Art
Total for the instrument	0.940	0.898	0.951
Attention	0.876	0.704	0.893
Confidence	0.780	0.582	0.900
Relevance	0.764	0.746	0.789
Satisfaction	0.821	0.775	0.843

provides the values obtained both for the instrument as a whole and for the different dimensions shaping it, in all three applications.

The values reached allow us to state that the instrument has a high reliability level, both globally and in the different dimensions shaping it [70,72]. Only one dimension (Confidence) and one of the experiments (Anatomy) reveal values which can be considered moderate. The scores suggest high reliability indices in the other cases.

As for the tool created to evaluate academic performance, it actually adopted the format of two multiple-choice tests administered under the pre-test + post-test modality. These tests included 15 items which collected information about the three following categories extracted from Bloom’s taxonomy: Apply (4 items); Remember (6 items); and Understand (5 items). Both modalities contained the same items in the pre-test and the post-test, albeit in a different presentation order.

3. Results

We present the results of the average scores reached in each of the items of the instrument used. We present the total results and the results of the three experiments in Appendix C.

It becomes visible in most items that the scores obtained exceed the central value of the scale (3.5), which suggests to us that the experiments were positively valued; furthermore, the items receiving the lowest scores from students are those linked to negative attitudes towards the experiment.

As for the means and standard deviations obtained for the instrument as a whole, in its different dimensions, in its different applications, and in the application as a whole, see

Table 3. Means and standard deviations for the IMMS instrument and for its dimensions.

	Video Use		Anatomy		Art		Total
	M	SD	M	SD	M	SD	M	SD
Total for the instrument	4.39	0.58	4.31	0.71	4.55	0.61	4,44	0.63
Attention	4.49	0.62	4.49	0.71	4.66	0.67	4.28	0.67
Confidence	4.16	0.62	4.16	0.77	4.24	0.63	4.33	0.67
Relevance	4.63	1.01	4.60	0.85	4.85	0.98	4.64	0.95
Satisfaction	4.58	0.69	4.60	1.06	4.83	0.69	4.53	0.81

.

The average scores obtained tell us that students very positively assessed their participation in this experiment, regardless of the type of AR objects produced and of the knowledge area we were dealing with. They expressed a high level of satisfaction after taking part in the interaction experiment with AR objects. Most importantly, the items which obtained slightly higher scores were the ones which combined AR and VR objects.

The following two hypotheses were posed for the purpose of checking whether the pre-test and the post-test differed significantly in terms of scores obtained:

•

H0 (null hypothesis): There are no significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.

•

H1 (alternative hypothesis): There are significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.

Seeking to verify H0, a Student’s statistical t-test for related samples was applied to the pre-test and post-test scores. This statistical analysis separately focused on the group of students who interacted with the AR object “Video use” and those who did so with “Anatomy”.

Table 4. Student’s t-values (note: ** = significant at p ≤ 0.01).

	Student’s t-Value	Significance
Video use	16.107	0.000 (**)
Anatomy	17.323	0.000 (**)
Art	14.468	0.000 (**)

shows the Student’s t-values reached and their significance level. After applying Student’s t-test, we obtained the values of t = 16.107 for “Video use” and t = 17.323 for “Anatomy”. These values allow us to reject H0 (null hypothesis) and to accept H1 (alternative hypothesis) at a significance level of p ≤ 0.001 in both cases, which ultimately leads us to conclude that students’ participation in the learning experience with AR objects proved useful to acquire the contents offered and, therefore, for them to enhance their learning.

Pearson’s correlation coefficient served to verify whether the motivation shown by students and the levels of confidence, attention, satisfaction and relevance assessed by means of IMMS influenced the performance reached after their participation in the AR experiment. The results achieved in both experiments can be seen in

Table 5. Correlations between the performance and motivation shown by students and the levels of confidence (C), attention (A), satisfaction (S) and relevance (R), by identified t-values (note: significant at * = 0.05 and **= 0.01).

Pearson’s Correlation Coefficient	Global	Attention	Confidence	Relevance	Satisfaction
Video use	0.321(**)	0.348(**)	0.289(*)	0.256(**)	0.201(*)
Anatomy	0.52(**)	0.491(**)	0.323*(*)	0.321(**)	0.383(**)
Art	0.432 (**)	0.398 (**)	0.315 (**)	0.2989 (**)	0.356 (**)

.

It follows from the values obtained that: (a) the relationships between the different variables compared (performance and motivation; attention-confidence-relevance-satisfaction) are direct, i.e., if one of them increases, so does the other; (b) relationships are moderate; and (c) all of them are significant at a 0.05 or lower level.

4. Discussion and Conclusions

A number of conclusions can be drawn from our study:

• The diagnostic instrument—Keller’s IMMS—appears to be a good predictor to explain the motivation, attention, confidence, relevance and satisfaction raised by the interaction with AR objects. This is true regardless of the specific AR object that they interacted with and of the subject in which the students were enrolled. In this respect, our findings match those obtained in other works carried out by different authors [24,73,74,75,76,77,78,79].
• The instrument utilized has a high reliability index, once again showing a coincidence with the results achieved by other authors.
• Our first findings clearly suggest that students’ interaction with learning objects created by means of AR technology significantly improved the scores obtained in the pre-test that had been administered to them with the aim of checking those students’ previous knowledge in relation to the contents presented in the AR objects—regardless of the specific content developed. Therefore, AR arguably constitutes a useful means to facilitate learning and knowledge acquisition. These results also follow along the same lines as the conclusions drawn by previous authors [32,73,76,77,78,79,80].

As a limitation faced in this study, it is worth highlighting the impossibility of generalizing the results obtained. Being a pre-experimental design, its results must be adopted with caution. The preceding conclusions thus refer to the context where the research took place. Furthermore, the experiences of students with the AR objects produced were very specific and focused on three knowledge areas, to which it must be added that the duration of the experience was short.

This suggests the need to propose new research initiatives, either enlarging the sample or designing the sample in accordance with another methodological model. The possibility also exists to replicate the study with other groups of students and with students from other universities. We could additionally undertake longitudinal studies, in which students work with AR-based learning objects, and verify the extent to which the findings remain constant. It would finally be possible to enlarge the knowledge areas presented here or to add other cognitive variables worthy of consideration, such as cognitive styles.