Next Article in Journal
Deep Homography Estimation and Its Application to Wall Maps of Wall-Climbing Robots
Next Article in Special Issue
Augmented Reality Implementations in Stomatology
Previous Article in Journal
Frequency Hopping Signals Tracking and Sorting Based on Dynamic Programming Modulated Wideband Converters
Previous Article in Special Issue
Wrist Rehabilitation System Using Augmented Reality for Hemiplegic Stroke Patient Rehabilitation: A Feasibility Study

Appl. Sci. 2019, 9(14), 2907; https://doi.org/10.3390/app9142907

Article
The Motivation of Technological Scenarios in Augmented Reality (AR): Results of Different Experiments
1
Facultad de Ciencias de la Educación, Universidad de Sevilla, C/ Pirotecnia, S/N, 41013 Sevilla, Spain
2
Facultad de Educación, Universidad de Alicante, Campus Sant Vicent del Raspeig, 03080 Alicante, Spain
*
Correspondence: [email protected]; Tel.: +34-965-903-721
These authors contributed equally to this work.
Received: 19 June 2019 / Accepted: 17 July 2019 / Published: 19 July 2019

Abstract

:
Augmented Reality (AR) is an emergent technology that is acquiring more and more relevance in teaching every day. Together with mobile technology, this combination arises as one of the most effective binomials to support significant and ubiquitous learning. Nevertheless, this binomial can only prove valid if the student is motivated to use it during the learning process. An attempt was made through the implementation of Keller’s Instructional Material Motivational Survey model o determine the degree of motivation of Pedagogy, Medicine and Art students from the University of Seville for using AR-enriched notes available by means of mobile devices in the classroom. Three applications designed for the subjects of Educational Technology, Anatomy and Art served to assess it positively in terms of the motivation raised by the participation in the experiment, as well as regarding academic performance improvement. It can additionally be stated that our main finding was a link between students’ motivation to use the enriched notes and the performance obtained in the subject in which they use them. Evidence was also found that the utilization of Augmented Reality benefits the learning process.
Keywords:
mobile devices; augmented reality; education; higher education; students’ perceptions

1. Introduction

Augmented Reality (AR) is becoming an emergent technology with great potential in various and largely different fields of application, such as geography, with regard to watershed control [1], medicine, cultural heritage [2] or industry [3]. It is likewise being used in the educational context [4,5,6,7,8,9], amongst other reasons because AR permits easy access to information—most often through mobile devices. It must be remembered that these devices are highly present in the life of university students all over Latin America [10,11].
According to Azuma [12], AR can be defined as a system with the following characteristics: it combines real and virtual; it is interactive in real time, and it is registered in 3D. AR technology consequently allows us to combine digital and physical information in real time by means of different technological media such as tablets, smartphones [13] or Augmented Reality glasses [3], to quote but three, thus providing the chance to create a new reality [14]. “Augmented reality visibly introduces the knowledge that students need to learn inside its real environment” [15].
Such a mixed integration of physical and digital reality may take place at various levels, ranging from the utilization of QR codes, the use of images, the exploitation of 3D objects and the Global Positioning System (GPS)-mediated mobilization of coordinates to thermal footprints [6]. Furthermore, one can classify AR systems depending on their location or according to images. “Location-based AR systems take advantage of the data about the position of mobile devices, determined by the Global Positioning System (GPS) or other Wi-Fi-based positioning systems… Instead, image-based AR focuses on image recognition techniques that serve to identify the position of physical objects in a real environment, so that the virtual contents related to such objects can be properly situated” [16].
The relevance of AR for learning has to do:
(a) with its specific characteristics: being a mixed reality, it offers the possibility for a real-time integration both of different information layers and of the various formats where that information may appear (texts, URLs, videos…). Furthermore, as an interesting, user-friendly technology, it allows us to enrich or alter the information about reality, adding further contents to it [6]; and (b) with the fact that the devices most frequently used for its observation, e.g., smartphones, are technologies within easy reach of university learners.
Numerous authors [17,18,19,20,21,22,23,24,25,26,27,28] have highlighted that AR utilization opens a wide range of possibilities for us, namely: (1) removing information that might hinder the assimilation of significant information by students; (2) increasing or enriching the information about reality so that it can be more easily understood by students; (3) being able to observe an object from different points of view, thus enabling the student to select the time and place of observation; (4) fostering ubiquitous learning; e) creating safe “artificial” scenarios, such as laboratories or simulators; (5) enriching the materials printed for students with additional information in different formats; (6) turning students into “proconsumers” of learning objects in AR format; and (7) achieving an improved learning through its use. Furthermore, AR can be used in a variety of disciplines and at different educational levels [29,30], even though university learning admittedly stands out as the context where it appears the most often.
As for its impact on education, the number of research studies carried out so far is still limited—even through that number is gradually increasing [31]. This possibly constitutes one of the most serious problems for its incorporation into teaching. Some aspects have been stressed in those works, however: that students show favorable attitudes towards AR and that its utilization enhances motivation towards learning [32,33]; that it helps create a constructivist learning context [16,34]; that it favors an active teaching environment [35]; that it reduces the cognitive load to which students are subjected in learning tasks [36,37]; that students have a high degree of acceptance as well as positive attitudes in relation to AR technology [38,39,40,41,42,43]; that it increases satisfaction levels amongst both students and teachers [24,44,45,46]; that it improves students’ spatial skills and orientation [47,48]; and finally, that its use enhances academic performance [26,30,49]. Additionally, it deserves to be highlighted that experiments are already being carried out for its incorporation into e-learning-based learning initiatives [31,50], as well as into others where students act as learning object producers [51,52].
In any case, we must bear in mind that the incorporation of AR into teaching faces a number of obstacles: this is a new technology; no development-focused educational experiments have been carried out yet; no learning resources and objects produced in AR format exist; teachers have a poor training; teachers need to develop positive attitudes towards AR incorporation into educational practice; no educational experiments have so far been performed in the development of AR-based learning objects; screens do not offer a suitable observation size; there are no conceptual frameworks which can help us find innovative practices in AR implementation; support centers for teachers need to be created so that they will find it easier to produce AR-based objects, as well as their maintenance in servers; a base/basic technology must be available for its observation by students; and an information overload clearly exists [28,52].
Concerning the last obstacle mentioned above, some precautions should be adopted when designing AR objects. Modules must have short and direct contents and be flexible as well as straightforward. They have to reflect an accessible and tolerant approach with a multimedia design, oriented to the student’s action and involvement. Tablets and smartphones do not constitute the goal of the educational process, and therefore, the methodology must focus on practicality and interactivity and adapt to the diversity of devices where it can be observed [53].
This paper shows an experiment carried out within a university context, additionally providing a detailed description of the AR products generated. This serves as the starting point to explain the educational research performed, where the Instructional Material Motivational Survey (IMMS) developed by Keller was used as the assessment tool [4]. The paper finishes with a summary of the results obtained as well as the corresponding conclusions drawn from them.

2. Materials and Methods

2.1. Research Aims

The different research works described below pursued the following objectives:
  • Ascertaining the degree of motivation measured by means of Keller’s IMMS model and the dimensions shaping it (attention/relevance/satisfaction) that the utilization of AR objects generated amongst students.
  • Analyzing if the degree of motivation (attention/relevance/satisfaction) measured using Keller’s IMMS model influenced students’ knowledge acquisition.
Below can be found the hypotheses that we pose in this study:
  • H0 (null hypothesis): There are no significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.
  • H1 (alternative hypothesis): There are significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.

2.2. Research Design

It must be stressed from the very beginning that the research was replicated with AR objects specifically made for investigation purposes that belonged to three different areas of knowledge: Educational Sciences; Fine Arts; and Medicine.
Two types of single-case pre-experimental design were used: one intended to analyze the relevance that the interaction with subjects had for the degree of motivation raised, which used a post-test measure; and the other referred to performance, which revolved around the analysis of academic performance and included pre-test and post-test measures [54,55]. More precisely, a treatment (the AR object created) was administered in both designs, complemented with a measurement (the student’s motivation to develop the experiment and to improve academic performance).
The experiment took place following the same sequences in all cases:
  • Explaining to students what AR was all about;
  • Showing the different objects produced and their possibilities;
  • Specifying the website where they could download the different guides and apps for the objects produced;
  • Encouraging students to work on their own using their mobile devices with the different objects; and
  • Completing questionnaires

2.3. The Sample

All the students involved in the experiment were enrolled in degrees in Medicine (Anatomy) and Pedagogy (Educational Technology), and in the master’s degree in “Art: ideas and production” —all of them taught at the University of Seville—Table 1. The samples had a non-probabilistic nature; in other words, they were “convenience” or “causal” samples [56,57] determined by the researcher’s ease of access to the subjects making up the sample in question.
It is worth highlighting that the experiment with the Art group took place after a three-hour-long interaction of students with the learning objects, as opposed to the one-week-long interaction period in the other two groups.

2.4. The Objects Produced

The experiment involved three AR objects specifically developed for this study and referred to contents associated with (a) the discipline of Educational Technology: “ways to use video in teaching-learning processes”; (b) Medicine (more precisely, Anatomy); and (c) the field of Fine Arts. A program guide prepared for students told them where they had to download the apps which allowed for the operation of the objects as well as the actions that they could perform with those objects. This was all complemented with a reference bibliography for students to delve deeper into each thematic area.
The three aforementioned objects had different typologies; those related to contents framed within “Educational Technology” (educational roles played by video use) and “Anatomy” can be categorized as belonging to Type II [6] in the format of “notes enriched” with AR objects, transferring the former with a video resource, and the latter with a 3D object that the student could manipulate (Figure 1 shows images of the object created for the “Educational Technology” subject).
Four AR objects were created for the “Anatomy” area, namely: Coxofemoral2; Shoulde2r; Ankle2; and Cervical2. All of them shared the following features: 3D animations; stop and monitoring of the object; enlargement of the size of the component presented so that details become visible; and transfer to an information extending video located on YouTube (Figure 2).
The final object produced can be classified as belonging to the “Mixed Reality” category due to the combination of elements from both “Augmented Reality” and “Virtual Reality.” It shows the Church of the Annunciation in Seville, which belongs to the University of Seville, but initially was the “Professed House of the Company/Society of Jesus”. This church dates back to 1565, and its façade stands out as one of the masterpieces of Sevillian Mannerism. The application developed allows the user to view—on a marker—3D models of the sculptures that Martínez Montáñez made of San Ignacio de Loyola and San Francisco de Borja. These figures, sculpted between 1610 and 1624, are located at the Main Altarpiece of this church. Users can likewise have access to a gigapixel photograph of the Altarpiece, where an extremely high zoom allows them not only to look at every single detail but also to experience a 360º viewing of the central nave, the entrance, and the crossing. They can also watch a video filmed with the help of a drone that shows the interior of the central nave (Figure 3).
Most importantly, the material generated in this case included a series of questions designed to find out if students had paid attention during their interaction (Figure 4). A final result corresponding to the number of questions posed appears at the end of the questionnaire.
We can classify the material produced on this occasion as: Level-2 AR; Immersive virtual reality; 360° vision; 3D modelling; incorporation of audio and gigapixel which helps enlarge the image (Figure 5).
The production of these objects was possible thanks to the use of the different software programs listed in Appendix A, which also specifies the type of use given to each one of them. As can be seen, we used a variety of programs, including Metaio Creator, Eclipse and Ffmpeg, since the production of the respective objects required some specific characteristics. It deserves to be highlighted that all the programs were prepared in such a way that they could work on Android as well as on iOS devices. Appendix B contains the URLs which feature the videos showing the operation of objects and where the apps can be downloaded.

2.5. Information Collection Instruments

The basic tool to diagnose the degree of motivation raised by the participation in the experiment was the Instructional Material Motivational Survey (IMMS), developed by Keller (2010).
According to Keller [4], motivation “refers to the magnitude and direction of behavior. In other words, it has to do with the choice made by an individual in relation to what they experience or the goals that they will try to achieve or avoid, and the degree of effort that they will make for that purpose. Motivation depends on internal looks and external characteristics. Individuals respond to their context on the basis of internal reflections, impulses, perceptions and goals, and according to the perceived base and the real opportunities, together with the reinforcement of the external context”. (Figure 6). In turn, Cheng [41] stresses that this is an internal state or condition which leads us to action, which guides us and persists in our behavior, involving us in certain activities. It also refers—in the classroom context—“to subjective experiences, particularly students’ predisposition to take part in class activities and their reasons to do so” [41].
Keller used a number of different works [1,58,59] to formulate his ARCS model for the motivation applied to instructional design. He sees motivation in instructional contexts as being determined by the interaction of four dimensions: attention (A); relevance (R); confidence (C); and satisfaction (S) (http://www.arcsmodel.com/).
For Keller [59], Attention includes human characteristics such as orientation reflection, curiosity and the search for sensations. Relevance refers to those things which a person perceives as a tool to meet needs and to achieve personal Satisfaction, including the fulfilment of personal objectives. Confidence comprises several motivational constructs. Finally, Satisfaction constitutes the final step that will allow us to maintain the motivation to learn.
The model illustrates how “a person’s curiosity (“attention”) and the motives or values (“relevance”), combined with their hope of success (“confidence”) will determine which objectives acquire the highest prominence, consequently resulting in an intentional effort to reach the goal. Environmental influences such as teacher enthusiasm and social values, instruction quality, clear expectations and the availability of resources will also have some bearing on an objective effort and orientation towards some degree of fulfilment. “Individuals are thus influenced by their own knowledge, capacity and skills too” [1]. Therefore, this model assumes that, if the instruction strategy is considered useful to achieve the goals set, then that strategy acquires importance for students [60], and they will be likely to learn the contents and to acquire the skills.
Keller’s IMMS consists of 36 items which collect information about four dimensions: attention (12 items); confidence (9 items); relevance (9 items); and satisfaction (6 items). The instrument has a Likert-type construction with seven response levels (from 1 = “Extremely unlikely/disagree with” to 7 = “Extremely likely/I agree”).
As stressed by Loorbach, Peters, Karreman and Steehouder [61], the model formulated by Keller “has been used on countless occasions to implement motivation strategies for instruction materials—and to test their effects. Despite being originally designed to influence students’ motivation within a traditional learning environment based on a face-to-face teacher-student interaction, this model has also been applied and thoroughly tested in contexts that require other adjustments, e.g., computer-assisted teaching and virtual or distance education”. More specifically, Keller’s model proved helpful to check the degree of motivation raised by different technological resources amongst students. Examples are: videogames [62]; the inclusion of video in e-learning learning sessions [63]; audio podcasts [64]; Massive Open Online Courses [65,66]; or AR [67,68,69].
We obtained the reliability index for this instrument using Cronbach’s alpha—one of the most suitable tools to obtain it with Likert-type scales [70,71]. Table 2 provides the values obtained both for the instrument as a whole and for the different dimensions shaping it, in all three applications.
The values reached allow us to state that the instrument has a high reliability level, both globally and in the different dimensions shaping it [70,72]. Only one dimension (Confidence) and one of the experiments (Anatomy) reveal values which can be considered moderate. The scores suggest high reliability indices in the other cases.
As for the tool created to evaluate academic performance, it actually adopted the format of two multiple-choice tests administered under the pre-test + post-test modality. These tests included 15 items which collected information about the three following categories extracted from Bloom’s taxonomy: Apply (4 items); Remember (6 items); and Understand (5 items). Both modalities contained the same items in the pre-test and the post-test, albeit in a different presentation order.

3. Results

We present the results of the average scores reached in each of the items of the instrument used. We present the total results and the results of the three experiments in Appendix C.
It becomes visible in most items that the scores obtained exceed the central value of the scale (3.5), which suggests to us that the experiments were positively valued; furthermore, the items receiving the lowest scores from students are those linked to negative attitudes towards the experiment.
As for the means and standard deviations obtained for the instrument as a whole, in its different dimensions, in its different applications, and in the application as a whole, see Table 3.
The average scores obtained tell us that students very positively assessed their participation in this experiment, regardless of the type of AR objects produced and of the knowledge area we were dealing with. They expressed a high level of satisfaction after taking part in the interaction experiment with AR objects. Most importantly, the items which obtained slightly higher scores were the ones which combined AR and VR objects.
The following two hypotheses were posed for the purpose of checking whether the pre-test and the post-test differed significantly in terms of scores obtained:
H0 (null hypothesis): There are no significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.
H1 (alternative hypothesis): There are significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test.
Seeking to verify H0, a Student’s statistical t-test for related samples was applied to the pre-test and post-test scores. This statistical analysis separately focused on the group of students who interacted with the AR object “Video use” and those who did so with “Anatomy”. Table 4 shows the Student’s t-values reached and their significance level. After applying Student’s t-test, we obtained the values of t = 16.107 for “Video use” and t = 17.323 for “Anatomy”. These values allow us to reject H0 (null hypothesis) and to accept H1 (alternative hypothesis) at a significance level of p ≤ 0.001 in both cases, which ultimately leads us to conclude that students’ participation in the learning experience with AR objects proved useful to acquire the contents offered and, therefore, for them to enhance their learning.
Pearson’s correlation coefficient served to verify whether the motivation shown by students and the levels of confidence, attention, satisfaction and relevance assessed by means of IMMS influenced the performance reached after their participation in the AR experiment. The results achieved in both experiments can be seen in Table 5.
It follows from the values obtained that: (a) the relationships between the different variables compared (performance and motivation; attention-confidence-relevance-satisfaction) are direct, i.e., if one of them increases, so does the other; (b) relationships are moderate; and (c) all of them are significant at a 0.05 or lower level.

4. Discussion and Conclusions

A number of conclusions can be drawn from our study:
  • The diagnostic instrument—Keller’s IMMS—appears to be a good predictor to explain the motivation, attention, confidence, relevance and satisfaction raised by the interaction with AR objects. This is true regardless of the specific AR object that they interacted with and of the subject in which the students were enrolled. In this respect, our findings match those obtained in other works carried out by different authors [24,73,74,75,76,77,78,79].
  • The instrument utilized has a high reliability index, once again showing a coincidence with the results achieved by other authors.
  • Our first findings clearly suggest that students’ interaction with learning objects created by means of AR technology significantly improved the scores obtained in the pre-test that had been administered to them with the aim of checking those students’ previous knowledge in relation to the contents presented in the AR objects—regardless of the specific content developed. Therefore, AR arguably constitutes a useful means to facilitate learning and knowledge acquisition. These results also follow along the same lines as the conclusions drawn by previous authors [32,73,76,77,78,79,80].
As a limitation faced in this study, it is worth highlighting the impossibility of generalizing the results obtained. Being a pre-experimental design, its results must be adopted with caution. The preceding conclusions thus refer to the context where the research took place. Furthermore, the experiences of students with the AR objects produced were very specific and focused on three knowledge areas, to which it must be added that the duration of the experience was short.
This suggests the need to propose new research initiatives, either enlarging the sample or designing the sample in accordance with another methodological model. The possibility also exists to replicate the study with other groups of students and with students from other universities. We could additionally undertake longitudinal studies, in which students work with AR-based learning objects, and verify the extent to which the findings remain constant. It would finally be possible to enlarge the knowledge areas presented here or to add other cognitive variables worthy of consideration, such as cognitive styles.

Author Contributions

Conceptualization, J.C. and R.R.; methodology, J.C and R. R..; software, J.C.; validation, J.C. and R.R.; formal analysis, R.R.; investigation, J.C. and R.R.; resources, R.R.; data curation, J.C.; writing—original draft preparation, J.C. and R.R.; writing—review and editing, J.C. and R.R.; visualization, R.R.; supervision, R.R.; project administration, J.C.; funding acquisition, J.C.

Funding

This research was funded by the Ministerio de Economía y Competitividad del Gobierno de España [Ministry of Economy and Competitiveness of the Spanish Government] entitled: “Realidad aumentada para aumentar la formación. Diseño, producción y evaluación de programas de realidad aumentada para la formación universitaria [Augmented reality to enhance learning. Design, production and evaluation of augmented reality programs for university learning]” (EDU-5746-P – Rafodiun Project).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Programs used to create (the) AR objects
AUGMENTED REALITY
SOFTWAREWHAT IT IS USED FOR
Metaio CreatorAugmented reality programming.
Metaio SDKAR software development kit.
EclipseJava development environment. Exporting apk for Android.
XcodeJava development environment. Exporting .ipa for iOS. Uploading to the App Store.
Adobe After effectsVideo and sound post-production. Chroma, Rotobrush, Key Light.
Adobe PhotoshopImage post-production. Graphic designs. Photomerge. 3D Texturization.
Macromedia FireworksImage post-production. Graphic designs.
FfmpegProgramming on the codec to export 3g2 videos-AR.
MS PowerpointButtons in video format with transition effects.
Notepad ++Professional word processor for code retouching.
Metaio toolboxExtraction of a 3D-based marker.
Autocatch123DPhotogrammetry.
Artec Studio3D scanning.
AstrumCreator of Windows installer.
Android StudioJava development environment. Exporting apk for Android.
Blender3D retouching.
Gigapan StichGigapixel creation.
VIRTUAL REALITY
Unity 3dVirtual reality programming. Exporting apk for Android.
SDK Google CardboardVirtual reality software development kit.
XcodeJava development environment. Exporting .ipa for IOS. Uploading to the App Store.
Adobe PhotoshopEquirectangular creation. Graphics.

Appendix B

URLs to download markers and apps
ProgramURL
Ways to use video in teachinghttps://ra.sav.us.es/index.php/realidad-aumentada/49-video-integrado-en-apuntes-formas-de-utilizar-el-video
Coxofemoral2https://ra.sav.us.es/index.php/realidad-aumentada/20-coxofemoral
Shoulde2r,https://ra.sav.us.es/index.php/realidad-aumentada/24-shoulder
Ankle2https://ra.sav.us.es/index.php/realidad-aumentada/29-ankle
Cervical2https://ra.sav.us.es/index.php/realidad-aumentada/16-cervical
The Annunciationhttps://ra.sav.us.es/index.php/realidad-aumentada/23-anunciacion-ra

Appendix C

S-mean by items and for the instrument as a whole (note: 1 = ways to use video; 2 = Anatomy; and 3 = Art).
Item123T
When I first saw the lesson, I had the impression that it would be easy for me. (C)4.634.824.924.79
There was something interesting at the beginning of the AR lesson which called my attention. (A)4.784.824.914.84
This material has turned out to be more difficult to understand than I would like it to be. (C)3.683.563.423.55
After reading the information contained in the introduction, I felt sure that I knew what I was supposed to learn from this lesson. (C)4.524.584.634.58
Completing the exercises in this lesson gave me a satisfaction-of-attainment feeling. (S)4.234.384.494.37
It seems clear to me that the content of this material has to do with things I already know (R)5.085.105.215.12
I had to study so much information that it was hard to remember the key points. (C)3.893.504.013.80
AR technology called my attention. (A)5.285.345.435.35
There were no images, videos and texts available to show me how this material could be important for some people. (R)3.793.844.053.89
Successfully completing this lesson was important for me. (R)4.834.845.014.89
The quality of the AR material helped keep my attention. (A)4.794.604.894.76
The material was so abstract that I found it difficult to keep my attention on it. (A)3.213.383.013.20
While working on this lesson, I was sure I could learn the content. I4.914.465.024.63
I have enjoyed this lesson so much that I would like to learn more about this topic. (S)4.194.204.464.28
The images, videos and texts that I discovered through the lesson are unattractive. (A)3.603.703.513.60
The content of this material is relevant for my interests. (R)5.085.045.135.08
The way to organize the information using this technology helped keep my attention. (A)4.714.624.994.77
There are explanations or examples of the way in which people use the knowledge contained in this lesson. (R)4.614.565.024.73
It was hard to find the digital information associated with the real image. (C)3.593.663.143.46
The information discovered through the experiment stimulated my curiosity. (A)4.694.705.134.84
I loved studying this lesson. (S)4.834.884.994.90
So much activity repetition bores me. (A)3.893.923.613.81
The content and audiovisual material included in this lesson give the impression that it is worth knowing its content. (R)5.075.025.185.09
I have learned some surprising or unexpected things about AR. (A)4.614.505.014.71
After working on this lesson for some time, I felt sure I was going to pass a test about the content presented. (C)4.734.664.944.78
This lesson was not relevant for my interests, because I already knew more things about the content. (R)3.613.543.983.71
The achievements made helped me feel rewarded for my effort. (S)4.594.684.884.72
The variety of audiovisual material helped keep my attention in the lesson. (A)4.734.785.034.85
The audiovisual material is boring. (A)3.053.203.033.09
I could relate the content of this lesson to the things I had previously seen, done or thought. (R)5.085.025.145.08
There is so much content that it makes me angry. (A)3.873.824.073.92
I felt good to successfully complete this lesson. (S)4.714.685.144.84
The content of this lesson will be useful for me. (R)4.964.845.174.99
I was actually unable to understand some of the material included in this lesson. (C)3.063.183.013.08
The fact that the material was so well organized helped me feel sure I was going to learn it [this material]. (C)4.894.885.064.94
It was a pleasure to work on this such nicely designed lesson. (S)4.914.785.014.90
TOTAL4.394.314.554.44

References

  1. Pierdicca, R.; Frontoni, E.; Zingaretti, P.; Mancini, A.; Malinverni, E.S.; Tassetti, A.N.; Marcheggiani, E.; Galli, A. Smart maintenance of riverbanks using a standard data layer and Augmented Reality. Comp. Geosci. 2016, 95, 67–74. [Google Scholar] [CrossRef]
  2. Bekele, M.K.; Pierdicca, R.; Frontoni, E.; Malinverni, E.S.; Gain, J. A survey of augmented, virtual, and mixed reality for cultural heritage. J. Comp. Cult. Herit. JOCCH 2018, 117, 36. [Google Scholar] [CrossRef]
  3. Pierdicca, R.; Frontoni, E.; Pollini, R.; Trani, M.; Verdini, L. The use of augmented reality glasses for the application in industry 4.0. In Proceedings of the International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Ugento, Italy, 12–15 June 2017; De Paolis, L.T., Bourdot, P., Mongelli, A., Eds.; Springer: Cham, Switzerland, 2017; Volume 10324, pp. 389–401. [Google Scholar]
  4. Keller, J.M. Motivational design of instruction. In Instructional-Design Theories and Models: An Overview of Their Current Status; Reigeluth, C.M., Ed.; Lawrence Erlbaum Associates: Hillsdale, NJ, USA, 1983; pp. 386–434. [Google Scholar]
  5. Cabero, J.; Barroso, J. The educational possibilities of Augmented Reality. J. N. Approaches Educ. Res. 2016, 5, 544–550. [Google Scholar]
  6. Radu, I. Augmented reality in education: A meta-review and cross-media analysis. Pers. Ubiquitous Comput. 2014, 18, 1533–1543. [Google Scholar] [CrossRef]
  7. Bacca, J.; Baldiris, S.; Fabregat, R.; Graf, S. Augmented reality trends in education: A systematic review of research and applications. Educ. Technol. Soc. 2014, 17, 133–149. [Google Scholar]
  8. Cabero, J.; De la Horra, I.; Sánchez, J. (Eds.) La Realidad Aumentada como Herramienta Educativa; Paraninfo: Madrid, Spain, 2018. [Google Scholar]
  9. Del Río, L.; Sanz, C.; Búcari, N. Incidence of a hypermedia educational material on the Teaching and Learning of Mathematics. J. N. Approaches Educ. Res. 2019, 8, 50–57. [Google Scholar] [CrossRef]
  10. Mojarro Aliaño, Á.; Duarte Hueros, A.; Guzmán Franco, M.; Aguaded, I. Mobile learning in university contexts based on the unified theory of acceptance and use of technology (UTAUT). J. N. Approaches Educ. Res. 2019, 8, 7–17. [Google Scholar] [CrossRef]
  11. Loaiza, M.; Andrade Abarca, P.; del Cisne Salazar, Á. Determination of the innovative capacity of Ecuadorian universities. J. N. Approaches Educ. Res. 2017, 6, 57–63. [Google Scholar] [CrossRef]
  12. Azuma, R.T. A Survey of augmented reality. In Presence: Teleoperators and Virtual Environments; MIT Press: Cambridge, MA, USA, 1997; pp. 6355–6385. [Google Scholar]
  13. Vázquez Cano, E.; Sevillano-García, M.L. Ubiquitous educational use of mobile digital devices. A general and comparative study in Spanish and Latin America higher education. J. N. Approaches Educ. Res. 2018, 7, 105–115. [Google Scholar] [CrossRef]
  14. Morales, M.; Benítez, C.; Silva, D.; Altamirano, M.; Mendoza, H.M. Aplicación móvil para el aprendizaje del inglés utilizando realidad aumentada. Rev. Iberoam. Prod. Acad. Gest. Educ. 2016, 2, 1–18. [Google Scholar]
  15. Fombona, J.; Vázquez, E. Posibilidades de utilización de la Geolocalización y Realidad Aumentada en el ámbito educativo. Educ. XX1 2017, 20, 319–342. [Google Scholar]
  16. Maquilón, J.J.; Mirete, A.; Avilés, M. La Realidad Aumentada (RA). Recursos y propuestas para la innovación educativa. Rev. Interuniv. Formación Prof. 2017, 20, 183–203. [Google Scholar] [CrossRef]
  17. Wojciechowski, R.; Cellary, W. Evaluation of learners’ attitude toward learning in ARIES augmented reality environments. Comp. Educ. 2013, 68, 570–585. [Google Scholar] [CrossRef]
  18. Bacca, J.; Baldirisa, S.; Fabregat, R.; Kinshuk, K. Insights into the Factors Influencing Student Motivation in Augmented Reality Learning Experiences in Vocational Education and Training. Front. Psychol. 2018, 9, 1486. [Google Scholar] [CrossRef] [PubMed]
  19. Carozza, L.; Tingdahi, D.; Bosché, F.; Gool, L. Markerless Vision-Based Augmented Reality for Urban Planning. Comp. Aided Civil Infrastruct. Eng. 2014, 29, 2–17. [Google Scholar] [CrossRef]
  20. Cubillo, J.; Martín, S.; Castro, M.; Colmenar, A. Recursos digitales autónomos mediante realidad aumentada. Rev. Iberoam. Educ. Distancia 2014, 17, 241–274. [Google Scholar]
  21. Jeřábek, T.; Rambousek, V.; Wildová, R. Specifics of Visual Perception of The Augmented Reality in The Context of Education. Procedia Soc. Behav. Sci. 2014, 159, 598–604. [Google Scholar] [CrossRef]
  22. Barba, R.; Yasaca, S.; Manosalvas, C. Impacto de la Realidad Aumentada Móvil en el Proceso Enseñanza-Aprendizaje de Estudiantes Universitarios del Área de Medicina. Investigar con y Para la Sociedad; Bubok Publishing S.L.: Cádiz, Spain, 2015. [Google Scholar]
  23. Jamali, S.; Fairuz, M.; Wai, K.; Oskam, C.L. Utilising mobile-augmented reality for learning human anatomy. Procedia Soc. Behav. Sci. 2015, 197, 659–668. [Google Scholar] [CrossRef]
  24. Fonseca, D.; Redondo, E.; Valls, F. Motivación y mejora académica utilizando realidad aumentada para el estudio de modelos tridimensionales arquitectónicos. Educ. Knowl. Soc. 2016, 17, 1745–1764. [Google Scholar]
  25. Han, J.; Jo, M.; Hyun, E.; So, H. Examining young children’s perception toward augmented reality-infused dramatic play. Educ. Technol. Res. Dev. 2015, 63, 455–474. [Google Scholar] [CrossRef]
  26. Santos, M.; Wolde, A.; Taketomi, T.; Yamamoto, G.; Rodrigo, M.; Sandor, C.; Kato, H. Augmented reality as multimedia: The case for situated vocabulary learning. Res. Pract. Technol. Enhanc. Learn. 2016, 11, 1–23. [Google Scholar] [CrossRef] [PubMed]
  27. Tekedere, H.; Göker, H. Examining the Effectiveness of Augmented Reality Applications in Education: A Meta-Analysis. Int. J. Environ. Sci. Educ. 2016, 11, 9469–9481. [Google Scholar]
  28. Marín, V. The augmented reality in the educational sphere of student of degree in childhood education. case study. Pixel Bit Rev. Medios Educ. 2017, 51, 9–24. [Google Scholar]
  29. Wang, Y. Using augmented reality to support a software editing course for college students. J. Assist. Learn. 2017, 33, 532–546. [Google Scholar] [CrossRef]
  30. Prendes, C. Realidad aumentada y educación: Análisis de experiencias prácticas. Pixel Bit Rev. Medios Educ. 2015, 46, 187–203. [Google Scholar]
  31. Ibáñez, M.; Delgado-Kloos, C. Augmented reality for STEM learning: A systematic review. Comp. Educ. 2018, 123, 109–123. [Google Scholar] [CrossRef]
  32. Alkhattabi, M. Augmented Reality as E-learning Tool in Primary Schools’ Education: Barriers to Teachers’ Adoption. Int. J. Emerg. Technol. Learn. 2017, 12, 91–100. [Google Scholar] [CrossRef]
  33. Cózar, R.; De Moya, M.; Hernández, J.; Hernández, J. Tecnologías emergentes para la enseñanza de las Ciencias Sociales. Una experiencia con el uso de Realidad Aumentada en la formación inicial de maestros. Digit. Educ. Rev. 2015, 27, 138–153. [Google Scholar]
  34. Garay, U.; Tejada, E.; Maiz, I. Valoración de objetos educativos enriquecidos con realidad aumentada: Una experiencia con alumnado de máster universitario. Pixel Bit Rev. Medios Educ. 2017, 50, 19–31. [Google Scholar] [CrossRef]
  35. Chen, C.M.; Tsai, Y. Interactive augmented reality system for enhancing library instruction in elementary schools. Comp. Educ. 2012, 59, 638–652. [Google Scholar] [CrossRef]
  36. Fombona, J.; Pascual, M.J.; Madeira, M.F. Realidad aumentada, una evolución de las aplicaciones de los dispositivos móviles. Píxel Bit Rev. Medios Educ. 2012, 41, 197–210. [Google Scholar]
  37. Lee, I.; Chen, C.H.; Chang, K. Augmented reality technology combined with three-dimensional holography to train the mental rotation ability of older adults. Comp. Hum. Behav. 2016, 65, 488–500. [Google Scholar] [CrossRef]
  38. Akçayır, M.; Akçayır, G. Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educ. Res. Rev. 2017, 20, 1–11. [Google Scholar] [CrossRef]
  39. Akçayır, M.; Akçayır, O.; Pektas, H.; Ocak, M. Augmented reality in science laboratories: The effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. Comp. Hum. Behav. 2016, 57, 334–342. [Google Scholar] [CrossRef]
  40. Bicen, H.; Bal, E. Determination of student opinions in augmented reality. World J. Educ. Technol. Curr. Issues 2016, 8, 205–209. [Google Scholar] [CrossRef]
  41. Rodríguez, A.; Naranjo, M.; Duque, N. Prueba de usabilidad y satisfacción en objetos de aprendizaje con Realidad Aumentada en aplicaciones móviles. In Recursos Educativos Aumentados Una Oportunidad para la Inclusión; Baldiris, S., Ed.; Sello Editorial Tecnológico Comfenalco: Cartagena, Colombia, 2016; pp. 56–65. [Google Scholar]
  42. Cheng, K.-H. Reading an augmented reality book: An exploration of learners’ cognitive load, motivation, and attitudes. Australas. J. Educ. Technol. 2017, 33, 53–69. [Google Scholar]
  43. Joo, J.; Martínez, F.; García-Bermejo, J.R. Realidad Aumentada y Navegación Peatonal Móvil con contenidos Patrimoniales: Percepción del aprendizaje. RIED Rev. Iberoam. Educ. Distancia 2017, 20, 93–118. [Google Scholar]
  44. Martínez, S.; Fernández, B. Objetos de Realidad Aumentada: Percepciones del alumnado de Pedagogía. Pixel Bit Rev. Medios Educ. 2018, 53, 207–220. [Google Scholar] [CrossRef]
  45. Cabero, J.; García, F.; Arroyo, C. La producción de objetos de aprendizaje en Realidad Aumentada para la formación universitaria en el SAV de la Universidad de Sevilla. In Experiencias Interactivas con Realidad Aumentada en las Aulas; Villaustre, L., Del Moral, M.E., Eds.; Octaedro: Barcelona, Spain, 2016; pp. 19–30. [Google Scholar]
  46. Kim, K.; Hwang, J.; Zo, H. Understanding users’ continuance intention toward smartphone augmented reality applications. Inf. Dev. 2016, 32, 161–174. [Google Scholar] [CrossRef]
  47. Díaz-Nogueras, D.; Toledo-Morales, P.; Hervás-Gómez, C. Augmented reality applications attitude scale (ARAAS): Diagnosing the attitudes of future teachers. N. Educ. Rev. 2017, 50, 215–226. [Google Scholar]
  48. Carbonnel, C.; Bermejo, L. Landscape interpretation with augmented reality and maps to improve spatial orientation skill. J. Geogr. Higher Educ. 2017, 41, 119–133. [Google Scholar] [CrossRef]
  49. Del-Cerro-Velázquez, F.; Morales-Méndez, G. Realidad Aumentada como herramienta de mejora de la inteligencia espacial en estudiantes de educación secundaria. RED Rev. Educ. Distancia 2017, 53. [Google Scholar] [CrossRef]
  50. Toledo, P.; Sánchez, J.M. Realidad Aumentada en Educación Primaria: Efectos sobre el aprendizaje. Relatec. Rev. Latinoam. Tecnol. Educ. 2017, 16, 79–92. [Google Scholar]
  51. Cabero, J.; Marín, V. Blended Learning y Realidad Aumentada: Experiencias de diseño docente. RIED Rev. Iberoam. Educ. Distancia 2018, 21, 57–74. [Google Scholar]
  52. Barroso, J.; Gallego, O. Producción de recursos de aprendizaje apoyados en Realidad Aumentada por parte de los estudiantes de magisterio. Edmetic. Rev. Educ. Mediát TIC 2017, 6, 23–38. [Google Scholar]
  53. Gallego, O.; Barroso, J.; Marín, V. Análisis de la motivación de los estudiantes universitarios como productores de recursos educativos utilizando la Realidad Aumentada. Espacios 2018, 39, 8. [Google Scholar]
  54. Joo Nagata, J. Modelo de Realidad Aumentada y Navegación Peatonal del Patrimonio Territorial: Diseño, Implementación y Evaluación Educativa. PhD Thesis, Universidad de Salamanca, Instituto de Ciencias de la Educación, Salamanca, Spain, 2016. [Google Scholar]
  55. Sans, A. Métodos de investigación de enfoque experimental. In Metodología de la Investigación Educativa; Bisquerra, R., Ed.; La Muralla: Madrid, Spain, 2004; pp. 167–193. [Google Scholar]
  56. Albert, M.J. La Investigación Educativa; McGraw-Hill: Madrid, Spain, 2006. [Google Scholar]
  57. Alaminos, A. El muestreo en la investigación social. In Elaboración, Análisis e Interpretación de Encuestas, Cuestionarios y Escalas de Opinión; Alaminos, A., Castejón, J.L., Eds.; Marfil: Alcoy, Spain, 2006; pp. 46–67. [Google Scholar]
  58. Sabariego, M. El proceso de investigación (parte 2). In Metodología de la Investigación Educativa; Bisquerra, R., Ed.; La Muralla: Madrid, Spain, 2012; pp. 127–163. [Google Scholar]
  59. Keller, J.M. Strategies for stimulating the motivation to learn. Perform. Instr. 1987, 26, 1–7. [Google Scholar] [CrossRef]
  60. Keller, J.M. Motivational Design for Learning and Performance; Springer Science + Business: New York, NY, USA, 2010. [Google Scholar]
  61. Hood Cattaneo, K. Telling Active Learning Pedagogies Apart: From theory to practice. J. N. Approaches Educ. Res. 2017, 6, 144–152. [Google Scholar] [CrossRef]
  62. Loorbach, N.; Peters, O.; Karreman, J.; Steehouder, M. Validation of the Instructional Materials Motivation Survey (IMMS) in a self-directed instructional setting aimed at working with technology. Br. J. Educ. Technol. 2015, 46, 204–218. [Google Scholar] [CrossRef]
  63. Del Moral Pérez, M.; Guzmán Duque, A.; Fernández García, L. Game-Based Learning: Increasing the Logical-Mathematical, Naturalistic, and Linguistic Learning Levels of Primary School Students. J. N. Approaches Educ. Res. 2018, 7, 31–39. [Google Scholar] [CrossRef]
  64. García Martínez, J.; Rigo, E.; Jiménez, R. Multimedia and Textual Reading Comprehension: Multimedia as Personal Learning Environment’s Enriching Format. J. N. Approaches Educ. Res. 2017, 6, 3–10. [Google Scholar] [CrossRef]
  65. Bolliger, D.U.; Supanakorn, S.; Boggs, C. Impact of podcasting on student motivation in the online learning environment. Comp. Educ. 2010, 55, 714–722. [Google Scholar] [CrossRef]
  66. Castaño, C.; Maiz, I.; Garay, U. Diseño, motivación y rendimiento en un curso MOOC cooperativo. Comunicar 2015, 44, 19–26. [Google Scholar]
  67. León-Urritia, M.; Cobos, R.; Dickens, K. MOOCs and their Influence on Higher Education Institutions: Perspectives from the Insiders. J. N. Approaches Educ. Res. 2018, 7, 40–45. [Google Scholar] [CrossRef]
  68. Di Serio, A.; Blanca, M.; Delgado, C. Impact of an augmented reality system on students’ motivation for a visual art course. Comp. Educ. 2013, 68, 586–596. [Google Scholar] [CrossRef]
  69. Lu, S.; Liu, Y.-C. Integrating augmented reality technology to enhance children’s learning in marine education. Environ. Educ. Res. 2015, 21, 525–541. [Google Scholar] [CrossRef]
  70. Wei, X.; Weng, D.; Liu, Y.; Wang, Y. Teaching based on augmented reality for a technical creative design course. Comp. Educ. 2015, 81, 221–234. [Google Scholar] [CrossRef]
  71. O’Dwyer, L.; Bernauer, J. Quantitative Research for the Qualitative Researcher; Sage: Thousand Oaks, CA, USA, 2014. [Google Scholar]
  72. García-López, L.; Gutiérrez, D.; Pastor, J.; Romo, V. Validity and Reliability of a Questionnaire on Primary and Secondary School Teachers’ Perception of Teaching a Competence-based Curriculum Model. J. N. Approaches Educ. Res. 2018, 7, 46–51. [Google Scholar] [CrossRef]
  73. Mateo, J. La investigación ex-post-facto. In Metodología de la Investigación Educativa; Bisquerra, B., Ed.; La Muralla: Madrid, Spain, 2004; pp. 195–230. [Google Scholar]
  74. Lin, T.; Been-Lirn, H.; Li, N.; Wang, H.; Tsai, C.C. An investigation of learners’ collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system. Comp. Educ. 2013, 68, 314–321. [Google Scholar] [CrossRef]
  75. Bongiovani, P. Realidad Aumentada en la Escuela: Tecnología, Experiencias e Ideas. 2013. Available online: http://www.educacontic.es/blog/realidad (accessed on 3 May 2019).
  76. Chang, H.; Wu, K.; Hsu, Y. Integrating a mobile augmented reality activity to contextualize student learning of a socioscientific issue. Br. J. Educ. Technol. 2013, 44, E95–E99. [Google Scholar] [CrossRef]
  77. Buitrago-Pulido, R. Incidencia de la realidad aumentada sobre el estilo cognitivo: Caso para el estudio de las matemáticas. Educ. Educ. 2015, 18, 27–41. [Google Scholar] [CrossRef]
  78. Gazcón, N. Libros Aumentados: Extensión del Concepto, Exploración e Interacciones. PhD Thesis, Universidad Nacional del Sur, Bahía Blanca, Argentina, 2015. [Google Scholar]
  79. Mehmet, H. The classification of augmented reality books: A literature review. In Proceedings of the INTED 2016 Conference, Valencia, Spain, 7–9 March 2016; pp. 4110–4118. [Google Scholar]
  80. Marín, V.; Cabero, J.; Gallego, O. Motivación y realidad aumentada: Alumnos como consumidores y productores de objetos de aprendizaje. Aula Abierta 2018, 47, 337–346. [Google Scholar] [CrossRef]
Figure 1. Image of the Augmented Reality (AR) object and its utilization in “Educational Technology”.
Figure 1. Image of the Augmented Reality (AR) object and its utilization in “Educational Technology”.
Applsci 09 02907 g001
Figure 2. Pictures of one of the AR objects created for the Anatomy area.
Figure 2. Pictures of one of the AR objects created for the Anatomy area.
Applsci 09 02907 g002
Figure 3. Images of the object produced at the Church of the Annunciation.
Figure 3. Images of the object produced at the Church of the Annunciation.
Applsci 09 02907 g003
Figure 4. An example of the questions which appear in the program.
Figure 4. An example of the questions which appear in the program.
Applsci 09 02907 g004
Figure 5. Gigapixel image of the Church of the Annunciation.
Figure 5. Gigapixel image of the Church of the Annunciation.
Applsci 09 02907 g005
Figure 6. Keller’s ARCS (Attention, Relevance, Confidence, Satisfaction) model (Keller, 1983).
Figure 6. Keller’s ARCS (Attention, Relevance, Confidence, Satisfaction) model (Keller, 1983).
Applsci 09 02907 g006
Table 1. Number of participants in each one of the experiments.
Table 1. Number of participants in each one of the experiments.
Educational TechnologyMedicineArt
TotalMenWomenTotalMenWomenTotalMenWomen
3027722550212920515
25.50%74.50% 42%58% 25%75%
Table 2. Reliability index corresponding to the Instructional Material Motivational Survey (IMMS) instrument.
Table 2. Reliability index corresponding to the Instructional Material Motivational Survey (IMMS) instrument.
Video UseAnatomyArt
Total for the instrument0.9400.8980.951
Attention0.8760.7040.893
Confidence0.7800.5820.900
Relevance0.7640.7460.789
Satisfaction0.8210.7750.843
Table 3. Means and standard deviations for the IMMS instrument and for its dimensions.
Table 3. Means and standard deviations for the IMMS instrument and for its dimensions.
Video UseAnatomyArtTotal
MSDMSDMSDMSD
Total for the instrument4.390.584.310.714.550.614,440.63
Attention4.490.624.490.714.660.674.280.67
Confidence4.160.624.160.774.240.634.330.67
Relevance4.631.014.600.854.850.984.640.95
Satisfaction4.580.694.601.064.830.694.530.81
Table 4. Student’s t-values (note: ** = significant at p ≤ 0.01).
Table 4. Student’s t-values (note: ** = significant at p ≤ 0.01).
Student’s t-ValueSignificance
Video use16.1070.000 (**)
Anatomy17.3230.000 (**)
Art14.4680.000 (**)
Table 5. Correlations between the performance and motivation shown by students and the levels of confidence (C), attention (A), satisfaction (S) and relevance (R), by identified t-values (note: significant at * = 0.05 and **= 0.01).
Table 5. Correlations between the performance and motivation shown by students and the levels of confidence (C), attention (A), satisfaction (S) and relevance (R), by identified t-values (note: significant at * = 0.05 and **= 0.01).
Pearson’s Correlation CoefficientGlobalAttentionConfidenceRelevanceSatisfaction
Video use0.321(**)0.348(**)0.289(*)0.256(**)0.201(*)
Anatomy0.52(**)0.491(**)0.323*(*)0.321(**)0.383(**)
Art0.432 (**)0.398 (**)0.315 (**)0.2989 (**)0.356 (**)

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop