An App that Changes Mentalities about Mobile Learning—The EduPARK Augmented Reality Activity

Abstract

The public usually associates mobile devices to distraction and learning disruption, and they are not frequently used in formal education. Additionally, games and parks are both associated with play and leisure time, and not to learn. This study shows that the combination of mobiles, games, and parks can promote authentic learning and contributes to changing conventional mentalities. The study is framed by the EduPARK project that created an innovative app for authentic learning, supported by mobile and augmented reality (AR) technologies, for game-based approaches in a green park. A case study of the EduPARK strategy’s educational value, according to 86 Basic Education undergraduate students, was conducted. The participants experienced the app in the park and presented their opinion about: (i) mobile learning; (ii) the app’s usability; and (iii) the impact of the educational strategy in terms of factors, such as intrinsic motivation and authentic learning. Data collection included a survey and document collection of student reflections. Data were subjected to descriptive statistics, System Usability score computing, and content analysis. Students considered that the EduPARK strategy has educational value, particularly regarding content learning and motivation. From this study emerged seven supporting pillars that constitute a set of guidelines for future development of mobile game-based learning.

1. Introduction

In ‘traditional classrooms’ teachers have the central role of teaching, and students listen passively to them; mobile devices are not allowed, as they cause distractions and are associated with social networks [1,2]. However, the literature has been demonstrating that mobile devices are familiar to students [3] and that they have positive perceptions regarding the use of mobile devices to learn [4]. Facing these findings, if these technologies are properly used, they intrinsically motivate students, even with the mobile technology available in 2005 [5], and promote positive learning outcomes [6,7]. Work needs to be done to change the ‘traditional classroom’ mentality about mobile devices in order to integrate these ubiquitous technologies, with high potential in the promotion of quality teaching and learning, in educational contexts. Moreover, the proliferation of mobile devices has pushed their technological evolution, as they now commonly integrate a set of hardware, including sensors, such as video cameras or accelerometer, which can support the access to Augmented Reality (AR) experiences [8]. Due to its mobile nature, mobile devices can now include applications (app) with AR to facilitate learning in outdoor environments. The exploration of this technology in outdoor study visits is completely new for educational purposes and relatively little is known about the potential of AR to support teaching and learning with groups of students out of the classroom.

Like mobile phones, games are also associated with leisure time. Video games can be particularly highly motivating to the players, who can spend a substantial amount of time and resources with them [9]. In Education, games have long been studied to promote learning and the literature has revealed that, when games are carefully designed to scaffold adequate learning purposes, they can also enhance students’ motivation and engagement for learning [10,11].

In the same line of thought, people usually relate green parks to citizens’ leisure time, as they use them frequently for physical exercise. However, these spaces can also offer rich and authentic learning experiences, promoting the valorization of plants, habitats and their conservation, and positively influencing values, attitudes, and actions of park visitors [12].

Mobile devices, green parks, and games are all strongly associated with leisure, play, or distraction; nevertheless, they can be used together to promote serious and authentic learning and challenge conventional thinking about learning. That is the aim of the EduPARK project, thus contributing to the United Nations 4^th goal of sustainable development by ensuring inclusive and equitable quality education and promoting lifelong learning opportunities for all [13]. This work was developed under the EduPARK project that created an interactive app for authentic cross-subject learning, supported by mobile and AR technologies, in a game-based learning approach, to be explored by educational stakeholders from all school levels and by a wider community in a specific outdoor informal environment—a green park. Moreover, its aim is to contribute to the understanding of the educational value of the EduPARK strategy, involving a mobile game-based learning with AR app, for exploration in outdoor settings, according to Higher Education students (future teachers). For this purpose, a case study was conducted to analyze the opinion of Basic Education undergraduate students, who experienced the EduPARK activities, in what concerns: (i) mobile learning; (ii) the EduPARK app usability; and (iii) the impact of the educational strategy into: (a) learning value; (b) intrinsic motivation; (c) engagement; (d) authentic learning; (e) lifelong learning; and (f) conservation and sustainable habits. Student data were collected from an online individual questionnaire and from student group reflections on the topic. Those data were triangulated for content analysis.

The remaining manuscript starts with a theoretical contextualization on the potential of the trilogy of mobile devices, games and green parks to learning; hence, a trilogy that has the potential to change mentalities about how people can learn effectively. In the following section, previous related work that makes use of this trilogy, even if in an unconscious way, is also analyzed. In this section, the novelty of the EduPARK project and of this work is presented. Next, the case study methodology is explained. This section includes the case description on two levels: first, the EduPARK project and its mobile AR game-based learning app for exploration in the outdoors, second, the study participants. This section also presents the data collection and analysis techniques and tools. Likewise, results are presented and discussed in their respective sections. Finally, the Conclusion Section includes the EduPARK supporting pillars that emerged from this case study and that constitute a set of guidelines for future development of mobile game-based games for the outdoors. Consequently, this work contributes to the literature on mobile AR game-based learning in informal outdoor environments.

2. The Trilogy: Mobile Devices with AR, Games, and Parks in Education

This work is based on a conventional mentality change effort, in educational contexts. It is framed in a trilogy that articulates mobile devices, particularly their capacity to support new technologies such as AR, games, and green parks (Figure 1). All these elements are frequently associated with distraction, play time and leisure; however, as this section presents below, the literature has revealed that each of these elements can be used to enhance learning, in a mobile, cross-subject, and authentic manner. Adding AR to an everyday technology, such as mobile devices, constitutes an innovative use of technology for education and it has the potential to transform the way of thinking about how people can learn.

The first trilogy element in Figure 1 is ‘mobile devices’, which incorporate an increasing number of hardware and software that supports a wide range of functionalities. For example, several reports have been made regarding the youth frequently using mobile phones to access social networks and to socialize [2,14], even in classrooms and disrupting learning activities [1]. Therefore, the option may be of banning these devices from formal educational settings [2]. Consequently, the use of mobile devices for learning does not seem to emerge naturally and, thus, has to be demonstrated, both as a possibility and as an effective strategy for learning.

The use of mobile devices in education may leverage, e.g., from: (a) their reduced size and weight [15] or high portability, which supports extending learning beyond the traditional classroom environment [1,16,17]; (b) their capacity to support interactivity with others and with multimodal media content [2,18]; and (c) their potential to promote collaborative practices and innovation in education [1]. According to [2], when mobile devices are used for learning, the paradigm for their usage can be one of the following: (a) the learners and the learning experience are not mobile and the mobile devices could be replaced by desk computers, such as e-textbooks initiatives; (b) the learners are mobile, but not the experience, where the location of the learner does not influence the learning, such as anytime, anywhere approaches; or (c) both the learner and experience are mobile, sustaining contextualized and authentic learning. Learners using mobile devices to learn frequently reach higher cognitive and affective outcomes than those not using these devices [7,15], across a wide range of subject areas [2], although their use does not ensure success. Despite these results, mobile technology integration in schools is still limited [2]. Some of the challenges faced by mobile learning include: (a) their small screen size and computing processing limitations [16]; (b) limited connectivity [2,16]; (c) it may entrench digital divides regarding technology access, technological skills and learning competencies [19]; and (d) it requires high preparation from teachers [15].

With technology development, mobile devices can now support AR in an affordable way. AR systems allow overlapping virtual elements, such as 3D models, with real objects of the physical world, in real-time, producing a new experience [20,21]. The virtual objects are frequently triggered by: (a) image recognition, e.g., from 2D markers developed for this purpose or from real environment images; or (b) the user’s location, e.g., from GPS, Bluetooth, or wireless networks [22]. AR technology has been successfully used in several contexts, such as health professional training, complex equipment maintenance and repair or even marketing. More recently, AR’s potential for education has been studied and authors claim it is mature enough for adoption in formal contexts [17], as the literature has collected evidence of positive learning outcomes from the use of this technology [23]. For instance, AR can reduce student cognitive load through the annotation of real-world objects and environments [24] or it can better support the understanding of abstract concepts when students explore 3D AR models than when they use textbooks [25,26]. While there are boundless directions for the perceived educational value of AR, several researchers, such as [24], emphasize that AR has the potential to foster learners’ motivation, engagement and enjoyment in applying and discovering resources in the real world from a variety of perspectives that have never been implemented before. On the pitfalls side, AR systems’ usability has been reported as a challenge [24], as well as technical problems related with GPS [24,27,28]. As mobile apps using AR are relatively new, there is little research in this area and the existing one often relates with 3D educational models or mobile gaming [29].

The second trilogy element in Figure 1 is ‘games’ and their motivation and engagement qualities are recognized in the literature [10,11,17]. The use of mobile games in formal education has been shown to promote student engagement for deeper and authentic learning [16,30] as well as improved learning outcomes [10,23,31]. However, to be effective, game-based approaches need to be grounded in relevant learning theories [17], as well as carefully designed and integrated into the curriculum, for learners to achieve the desired learning goals [11,15,29]. The literature recommends that game-based approaches should: (a) fit student-lead strategies [10]; (b) activate prior knowledge [32]; (c) offer instant and quality feedback [10,26,32]; (d) balance game enjoyment and learning; and (e) provide a diversified gameplay experience and offer a good balance between players collaboration (within teams) and competition (among teams) [17]. To take advantage of this approach’s potential for contextual learning, games should prompt learners to explore or observe real world contexts [33]. The literature does not bear many reports of mobile AR games for learning in the outdoors.

The last trilogy element in Figure 1 is ‘green parks’, which are usually part of the citizens’ leisure time, who frequently use them for exercise activities. However, this type of parks has the potential to provide learning experiences that not only promote the importance of plants, habitats and their conservation, but also influences the values, attitudes and actions of their visitors [12].

In sum, in Figure 1, several elements usually associated with distraction, leisure and play time are integrated and articulated in such a manner that using some of the new potentialities of mobile phones, strolling in green parks and collective game playing can generate mobile, authentic and cross-subject learning. The EduPARK project is based on this trilogy, demonstrating that the combination of mobile devices, games, and parks can play a prominent role in changing conventional thinking about how people can learn.

3. Related Work

There is already a number of developed location-based AR games to be used in educational contexts, such as the ones that will be briefly described below. These are only examples of similar works to the one presented here.

In 2014, Srisuphab and colleagues [34] developed a context-aware self-guided app for Android, the ZooEduGuide, to promote learning about environmental protection and wildlife conservation during zoo visits. This app combines mobile devices and AR to support interactive exploration of the zoo. Its visitors may use the app to scan QR codes and access information about the animals in the format of rich and interactive multimedia content (illustrations, sounds and videos). In the educational game mode, zoo visitors can identify animals through their sounds and answer a trivia quiz. The games integrate a social and competitive feature, by being timed and having their scores ranked. Finally, in the visit zoo mode, zoo visitors can customize a zoo map, by adding their own points of interest and plan a visit path. AR technology is used to present to the visitor different points of interest of the zoo.

The Leometry [35] game is directed at 5^th and 6^th grade students to learn basic geometric shapes. The game uses storytelling to immerse players in the learning experience and presents them with geometry challenges. AR features of the game include a map showing spots with AR objects to overlay onto a real-world map. The game was tested by a group of 5^th grade students, who considered the game fun to play, appreciated its features and storytelling approach, and generally acknowledged its educational potential. The AR was pointed as a powerful motivator for learning, as well.

AR and mobile technologies, including environmental probeware, are explored in the EcoMOBILE [36] project to promote middle school student’s learning in outdoor contexts (field trips), more specifically related with water quality measurements. The EcoMOBILE experience involves accessing information, such as videos, 3D models, quizzes and AR visualizations, through GPS triggers, encouraging students to move between different hotspots in the field. Among the benefits of using these technologies in education, the authors identified the increased students’ motivation and the fact that these technologies allow students to progress at their own pace, while their teachers act as facilitators. The participating teachers seemed to be quite positive about the EcoMOBILE experience, however, they found was that some groups of students just sped through the activity without fully understanding the accessed contents.

Another learning system related with environmental issues is the AR butterfly [37]. It integrates AR, game-based and mobile learning approaches to motivate elementary students to learn about butterfly species, ecology and conservation in a school campus. The mobile app guides students to hotspots where real-world butterfly host plants are placed, so they can observe virtual butterflies around nectar plants when they approach their location. In this system, the AR contents are triggered by GPS coordinates. The app also gives access to a virtual green house where students can play a game that involves virtual butterflies breeding, supporting their learning about butterflies’ life-cycles. The authors found that using the AR butterfly ecological learning system can effectively promote students’ learning and that the game approach increased their interest in learning.

Hwang and colleagues developed an AR-based competitive game [33] that is based on the traditional board game concept. In this game, the board is a butterfly garden where the players move around after rolling a digital dice. The hotspots in the field offer students the opportunity to answer multiple-choice questions or to complete mini-games, such as a matching game or an AR-based shooting game. The study results revealed that elementary school students’ learning attitudes and performance on the field can be improved by the AR-based gaming approach, when compared to a conventional AR-based mobile learning approach. The authors recommend a structured design procedure for competitive AR-based gaming approaches in the outdoors:

(1)

Select the activities that require students to explore or make observations in real-world contexts;

(2)

Prepare a set of questions related to the real-world contexts for the competitive game; and

(3)

Determine the location and content for each AR-based events [33] (p. 1903).

SolarSystemGO [38] is a mobile AR game for astronomy learning in a ‘Planet Hunt’ challenge mode. This game aims at providing awareness of the vastness and proportionality of the Solar System objects, which are augmented objects placed on scale with the Sun position. No results with users were presented yet as the game is still under development.

In summary, the above presented studies provide important insights into the use of AR, mobile and game learning approaches in outdoor educational activities. All these studies present AR as a technology that can increase students’ interest and motivation, as well as promote self-learning, particularly when combined with game-based approaches. Mobile technologies allow learning activities in outdoor settings, as they are an appealing way of presenting information, and AR can support complex and abstract concepts’ understanding, while games can increase students’ will to overcome challenges and learning difficulties.

In line with the studies presented above, the EduPARK project developed an app that combines a technology, which is familiar to students, with outdoor learning strategies. As it will be explained in Section 4.1, the app targets groups of students and teachers, from basic to higher education, as well as tourists. Previous work involved the description of the app development and its assessment, including user testing and feedback [39,40,41,42], the refinement of the AR markers [43], as well as the development of the learning games, or guides, articulated with the Portuguese National Curriculum [44,45,46]. When a stable version of the app was achieved, the project team initiated the assessment of the EduPARK app usability and learning value, through a comparative analysis of the opinions of Basic and Higher Education students, which revealed the app promotes learning, enjoyment and high levels of usability [42].

One of the innovative aspects of the EduPARK strategy relies on moving learning beyond traditional classroom environments to a specific nature space—an urban green park in Aveiro—that visitors can physically explore at the same time that they make connections with curricular content. Moreover, the EduPARK app is the only one, according to this literature analysis, supporting cross-subjects and authentic learning. Thus, the project’s contribution to the existing body of research on educational AR games is highlighting how these strategies can foster situated, collaborative, and lifelong learning. This plays a central role in regional development and social innovation.

The present work contributes to the existing body of research on educational AR games by illustrating how the trilogy of mobile devices with AR, games and green parks can be used to promote authentic cross subject mobile learning, in order to change conventional thinking about how people can learn. This is achieved through a case study where the EduPARK strategy, which makes use of the trilogy elements, is assessed regarding its educational value, shedding light onto important matters that affect game-based AR mobile learning experiences in the outdoors. From this study, seven supporting pillars of the EduPARK project emerge, as will be presented in the last subsection, providing a theoretical framework for education. This framework can be useful for other initiatives aiming to take advantage of authentic learning opportunities offered by natural environments.

4. Materials and Methods

The EduPARK project follows the methodology of design-based research [47], since it integrates four cycles of refinement of a prototype, the EduPARK app, described in previous publications [42,48]. This work gives continuity to the work developed, through a qualitative interpretive methodology [49] of case study [50], by addressing the research question: ‘What is the educational value of the EduPARK strategy, involving a mobile game-based learning with AR app, for exploration in outdoor settings, according to Higher Education students (future teachers)?’ Thus, the main objective is to analyze the opinion of Basic Education undergraduate students, who have experienced the EduPARK activities, in what concerns: (i) mobile learning; (ii) the EduPARK app usability; and (iii) the impact of the educational strategy into: (a) learning value; (b) intrinsic motivation; (c) engagement; (d) authentic learning; (e) lifelong learning; and (f) conservation and sustainable habits. To reach this aim firstly the case study is described, for contextualization, followed by the data collection and analysis approach.

4.1. Case Study—Project and App Development

EduPARK (http://edupark.web.ua.pt/) is a research and development project that integrates a multidisciplinary team from the fields of education, computer science, and biology, from the University of Aveiro (Portugal), in agreement with the literature on developing educational games [11]. The EduPARK trademark was registered to protect intellectual property rights. The project general aim is to promote innovative practices of cross-subjects learning using mobile devices, AR, and games in the outdoors. The project leverages the rapid development of mobile technologies, which is allowing the integration of new and diverse functionalities in the devices to provide active and contextualized learning, as well as to encourage new teaching approaches. On the other hand, a European project found that 54% of the surveyed children claimed not being allow to use their smartphones at school [14]. Moreover, there is a gap between the use of mobile devices inside and outside school context, which may result in students’ disengagement [51]. This project intends to change this kind of mentality, which limits the use of mobile technologies in Education, by demonstrating the educational potential of these technologies. In this line, the EduPARK project was the winner of the 2018 ECIU Team Award for Innovation in Teaching and Learning. The jury pointed out that the project revealed a particularly high level of innovation through: a) the close collaboration in a multidisciplinary team of researchers; b) the educative use of AR in day-to-day technologies, such as the smartphone; and c) the challenging of conventional thought about the way people can learn.

The project team employed a design-based methodology, with four cycles of development, integrating users’ feedback, to create and improve one of its main products, the EduPARK AR app [42,48]. This is an interactive app with several guides/educational games, based on geocaching principles, and articulated with the Portuguese National Curriculum. The app target public includes students and teachers of all school levels, as well as the tourist/general visitor of a specific green park, the Infante D. Pedro Park, in Aveiro (Portugal).

The Infante D. Pedro Park was selected as the EduPARK educational laboratory because it is a large green area, in an urban context, with a high biodiversity and historical patrimony. The park main points of interest include a reasonable sized lake with fishes, birds, and amphibians, both exotic and native botanic species, and several historical constructions (such as a bandstand and a tea house). Therefore, this park comprises educational value as it encloses diverse opportunities to provide authentic and contextualized learning experiences within the curriculum. These experiences can raise awareness about the importance of biodiversity and may promote the need for everyone to adopt more sustainable lifestyles to support healthy ecosystems [12], since the ability to understand ecosystems is boosted by experiences in real environments influencing communities’ attitudes about nature [52].

The EduPARK app was developed using Unity 5, a popular cross-platform game engine. As for the AR marker detection, the Vuforia SDK for Unity was used, since Vuforia is currently one of the most widely adopted platforms for AR technology. The educational games are hosted in a database MySQL, due to its performance, scalability, and reliability. The app is available as a free download in the Google Play Store for Android devices. After the app is installed on the user’s device, it prompts the user to update the games and further use does not require an Internet connection. In the initial exploitation of the app (Figure 2a), the user can: (i) select the desired language, Portuguese or English; (ii) fill in a profile for the player or team of players; and (iii) select one of the available modes: the ‘game’ mode or the ‘explore freely’ mode.

In the game mode, players are prompted to select an educational guide or game, accordingly to their profile: 1st CBE (from 1st to 4th school-years); 2nd and 3rd CBE (5th and 6th school-years and from 7th to 9th school-years, respectively); Secondary (from 10th to 12th school-years) and Higher Education; or a tourist visiting the park. Each guide includes a different set of quiz questions, virtual caches, and paths through the park, fitting one of the most frequent genres of mobile AR games, the treasure hunt, which leverages a powerful player motivator: their curiosity [17]. The guide for the park visitor, as well as the ‘explore freely’ mode, is also available in English. As recommended by [11], the EduPARK games include pro-social content and no aggressive content at all.

At the beginning of the game, players are welcomed by the project’s mascot, a female monkey, that talks with a friendly human voice. The inspiration for the EduPARK mascot was the park’s informal name, as it is known as the ‘Monkey Park’ because a female monkey lived there for several decades. The mascot briefly explains the game structure, main aim of gaining score points by answering correctly the questions, among other relevant features. The use of this mascot suits the literature’s [11] recommendations for using animated agents in interactions with players and of using human voices. The app as both written and audio information so that special needs children, as well as early-school years students, who have difficulties in reading, are able to understand the instructions and play the game.

Most of the quiz questions are multiple-choice (Figure 2b). However, there is a variety of types of questions, as they may require different actions from the players, such as on-site observation, establishing relationships, and information interpretation or information selection. Nevertheless, this variety imprints dynamism to the game and scaffolds a range of different learning strategies. In addition, some questions may have more than one correct answer, which stimulates the players’ concentration in their response and negotiation within the group members before answering. After answering the app does not return back, so they cannot retry. Nevertheless, the players always get immediate feedback to their answers (Figure 2c), whether right or wrong, explaining the right answer or giving further information on the topic.

The game is organized in four stages, each one corresponding to a path with quiz questions, usually of multiple-choice to be answered whilst the players are in a specific zone of the park, according to the app’s map (Figure 3a). To better support the players’ orientation in the park, the app also includes a compass tool. Moreover, the mascot provides guidance to the players during the game, as suggested by [11], by giving:

• information about how to play the game, through a tutorial that is triggered the first time a new player profile accesses a game;
• guidance about the path in the park;
• educational content relevant for question answering (images, audios, videos, including information augmenting the reality); and
• feedback to the answers (as mentioned above).

The app prompts the players to search for AR markers in the park that will deliver information for answering specific quiz questions (Figure 3b). Markers are printed on plaques next to botanic specimens (Figure 4c) of different species, selected as representative of the park’s biological richness. The EduPARK project proposed to the Aveiro Municipality the installation of the plaques in the park. The plaques were planned and funded by the project and they serve a double purpose:

• AR trigger with the use of a mobile device with the app; and
• identification of 32 botanic species without the use of such devices [43].

All plaques have the same layout; however, the information in each one varies accordingly with the botanic specimen: the scientific and common names, its family (in biological classification), its origin and the AR marker, with the project mascot [43]. The AR content associated with each plaque includes resources about the identified species (texts, photos, videos, 3D models), as illustrated in Figure 4a (main menu) and Figure 4b (menu with information about the specimen’s leaf). The leaf feature is interactive, as it can be rotated with the finger to show its upper and lower surfaces. Additionally, the app also includes AR markerless tracking, increasing the number of opportunities of situated and authentic learning in the park (Figure 4c).

Some of the game mechanics incorporated to increase players’ motivation include receiving a clue/riddle to find a virtual cache (Figure 5a,b), at the end of each game phase. If the cache is found within a 5-minute period, the players win points and virtual bananas that can be traded for help with the following questions. Moreover, at the end of the game, the app displays data, such as the score, correct answers or the time on the game (Figure 5c). For a given player profile, it is possible to know information about the reached level, completed games, percentage of questions answered correctly, number of markers visited, and number of caches found.

As mentioned before, besides the game mode, the app has an ‘explore freely’ mode. This allows users to access AR content from the markers (in the botanical plaques) and from the markerless locations, without having to play the game.

The EduPARK project has been organizing several sessions of app exploration in the park, 75 so far, involving 1250 students from Basic to Higher Education, 280 teachers, and hundreds of tourists. Some groups included students with special needs, namely at cognitive level, vision impairment and mobility limitations. Consent of participation in the study was acquired at the moment of scheduling each activity, after informing that the participants would fill in an anonymous questionnaire at the end of the activity for research purposes, allowing systematic collection of data to better understand the benefits of mobile learning in outdoor settings.

4.2. Case Study—Participants

This study involved Higher Education students, particularly, 86 graduating students of the 2^nd year of the course of Basic Education, from the Department of Education and Psychology of the University of Aveiro. These students participated in the ‘Nature Integrated Sciences II’ curricular unit (NIS II), under the topic ‘Biodiversity and Conservation’, in two editions: 46 students in the 2017/18 academic year and 40 in 2018/19. This is a convenience sample.

Participating students were mainly female (80 females in 86 students). This woman dominated course is in line with the reality of Portuguese Basic Education schools as, in almost six decades, the proportion of female in-service teachers in this school level was always above 86% [53]. The most frequent ages of the participants were 19 and 20 years-old (37 and 27 students, respectively), although varying between 18 and 42, in line with what is expected from Higher Education undergraduate students. Moreover, their previous academic background was mainly Humanities (51 students) or Science (23), revealing a substantial variety. Possibly, most of the graduating students that participated in this study did not have a wide knowledge of the curricular unit topic ‘Biodiversity and Conservation’.

In each edition, graduating students had a preparatory lesson of two hours for a theoretical contextualization on mobile learning, game-based learning, and AR (consisting on a brief definition of each of those concepts, as the students were not familiar with them), as well as a first introduction of the EduPARK project, the app features, field trip preparation, and the assessment strategy. The theoretical contextualization did not include advantages nor constraints of mobile learning, game-based learning, and AR-based learning, so it would not create validity issues regarding data collection. This included information regarding the students’ views on mobile learning advantages and constraints, as it will be explained further in the following section (Section 4.3). Students’ previous experience with game-based and/or AR-based learning technologies was scarce. Some students reported they had already played Pokémon Go, but no other examples were mentioned. The field trip preparation involved: (i) the clarification of rules and work to be done during the activity; (ii) the definition of groups of work with three students; and (iii) the app installation in one Android device per group of students.

The preparatory lesson was followed by the field trip to the park, to play the EduPARK app game directed at Higher Education students. The field trip activity lasted between one hour and twenty-two minutes and one hour and fifty-five minutes. Some technical problems were reported by students in both editions, mainly related to computing processing limitations, in line with [16].

The assessment strategy was discussed with the graduating students in the preparation lesson and included: (i) the student elaboration of three original questions that can be used in a game-based mobile learning, which were orally presented and discussed in class; (ii) the filling in of an online individual questionnaire to evaluate the activity; and (iii) a reflection on this teaching methodology, written in groups as autonomous work. The topic ‘Biodiversity and Conservation’ weighted 20% of the whole curricular unit assessment.

Under the assessment point ‘(i) quiz-questions for game-based mobile learning’, graduating students produced questions for Basic Education children about a variety of topics, namely biodiversity, pollution and animals’ general features. They tried to be creative and to support learning through their questions and associated resources, namely AR content. For example, one group created a question for children in the 5th school-year, in an accessible level of difficulty, presented below:

• Introduction to the question: Biodiversity refers to the variety of species on our planet and it has been decreasing with species extinction. This is mainly due to mankind action, which has been destroying ecosystems.
• Question: Which of the following factor(s) is/are associated with biodiversity reduction?
• Answer option a) Tree planting
• Answer option b) Decrease in bee population
• Answer option c) Forest fires
• Answer option d) Greenhouses
• The answer is considered correct only if the user selects both options b) and c), gaining points.
• Answer feedback: Well done! [in case of right answer]/Ohhhh! [in case of wrong answer]
  • Bees allow plants to reproduce, so they improve food production. Forest fires destroy plants and animals. So, less bees and more fires lead to less species in a certain area.

This group of NIS II students referred that the introduction to this question should have associated a small video (about 1 minute) on the biodiversity topic and factors related to biodiversity reduction, to support students’ contextualization on the topic. Other resource could be AR associated with a flowering plant, such as olive tree or Indian rubber fig.

Assessment points ‘(ii) individual questionnaires’ and ‘(iii) group reflections’ were used for data collection in this case study and the techniques and tools are presented below.

4.3. Data Collection and Analysis

This research relies on data from two sources of evidence, survey and document collection.

Participating students were asked to fill in an individual online questionnaire (goo.gl/sEQqV4) about one week after the infield session. The response rate was 100%. The questionnaire includes mainly closed questions of multi-choice, item selection and a five-point Likert scale, although it contains open questions, as well. It consists of three parts: (i) students’ profile, including their use of mobile devices for learning and mobile learning advantages and disadvantages; (ii) EduPARK app usability; and (iii) EduPARK activity educational value. Parts (i) and (iii) were analyzed through descriptive statistics and content analysis. Content analysis regarding the advantages and constraints of using mobile devices to learn was based on the category used in a previous study (not published yet) with in-service teachers, and adapted according to this context specificities. The analysis of the part (ii) of the questionnaire was based on the ‘European Portuguese Validation of the System Usability Scale (SUS)’ by [54], who translated and validated the original instrument [55] for the Portuguese language. Thus, data collected from this part of the questionnaire were used to compute SUS scores, between 0 and 100, according to [55]. To adequately interpret the usability of the app, its SUS score was compared to the average SUS value (68) of 500 usability studies analyzed by [56]. The app SUS score was also placed in an empirically-defined qualitative classification system for SUS [57].

Document collection of the graduating student reflections was conducted to support the understanding of some of the survey results. Students were asked to include in their reflection the possible impact of the EduPARK strategy on: (a) learning value; (b) intrinsic motivation; (c) commitment; (d) authentic learning; (e) lifelong learning; and (f) conservation and sustainability habits. The groups of the written reflections were the same groups that explored the app in the park. Thirty group reflections were turned over. Qualitative data analysis was completed, in the logic of content analysis and resorted to the categorization of responses based on the levels of impact mentioned above with support from the literature [58,59,60,61]. A new category, ‘Extrinsic motivation’, was included due to the high frequency it appeared in students’ reflection. To decrease the risk of erroneous interpretation of the factors acknowledged by the students, in each document (group reflection) the coder coded only the existence or absence of each analysis dimension, even if the students refer to it more than once.

Data from the two sources mentioned above were triangulated to provide a comprehensive knowledge of the students’ opinions.

Data collected did not include personal data, and no individual participant can be identified through the data collected.

5. Results

This section provides a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

In this section, the results of the case study about the EduPARK strategy’s educational value, in the opinion of 86 Basic Education undergraduate students, are presented. This section is organized according to the data sources: the survey and the document collection of student reflections. The reflections provide in depth information regarding students’ choices in the questionnaire.

5.1. Survey Results

Figure 6 shows students reported frequency of mobile devices use to promote their own learning, with 69 mentioning they use them with high frequency. No student referred never using them for that purpose.

When questioned about advantages of the use of mobile devices to learn, all 86 graduating students mentioned at least one advantage, although 23 sentences were considered ‘dubious answers’ (see

Table 1. Advantages of using mobile devices for learning, mentioned by graduating students.

Category	Sub-Category	Analysis Dimension	Citations Examples	Frequency
Learning methodologies	Gains for learners	Motivation	‘It is a more iterative and interesting way of deepening knowledge and acquiring new knowledge’ Q11 ‘Draws more attention from the student so he/she becomes more captivated’ Q57	10	36
		Engagement	‘This is a different way of learning; thus, we won’t get bored so easily.’ Q7	1
		Learning	‘Knowledge broadening’ Q8 ‘… the use of these means of learning promotes authentic learning’ Q26	20
	Gains for learning practices	Diversification of learning strategies	‘Using mobile phones helps promoting formal learning in a more dynamic way and also in a way that is more fitted to today's lifestyle’ Q60	5
Technology support for learning	Mobile devices features	Resources availability	‘Greater versatility of resources for greater learning’ Q13	3	49
		Enhances information access (T)	‘It's faster to find answers’ Q1 ‘Great variety of contents’ Q2 ‘it is fast to get information with these devices and it is always up-to-date’ Q10	30
		Supports interactivity (T)	‘It is a more iterative and interesting way of deepening knowledge and acquiring new knowledge’ Q11	1
		Allows outdoor activities	‘They work anywhere making learning easier and possible at any time’ Q3 ‘This application can change the point of view of the community before a place that was once passing and walking and now, with the EduPARK, we can see that in every corner there is something to learn and thus enrich our knowledge‘ Q7	8
		It is eco-friendly	‘It does not waste paper’ Q22	1
	Users’ characteristics	Familiarity with the technology	‘We interact with something technological with which we feel at ease’ Q9	5
		Responsible use of mobile devices	‘It is necessary that our future students make a responsible use of mobile devices’ Q52	1
Total				85
Dubious answer			‘It’s easy’ Q5 ‘Mobility’ Q34 ‘Scientific papers’ Q35 ‘It is important’ Q37	23

Note: The notations ‘Q’ followed by a number indicate the questionnaire from which the citation example was taken.

). Nevertheless, 85 advantages were clearly identified and categorized.

Students identified diverse advantages that were categorized into two categories: a) 36 advantages for ‘learning methodologies’, with the sub-categories: ‘gains for learners’ (31), and ‘gains for learning practice’ (5); and b) 49 advantages of ‘technology support for learning’, with the sub-categories ‘mobile devices features’ (43), and ‘users’ characteristics’ (6).

Graduating students considered several ‘gains for learners’, such as the effectiveness of mobile devices when used for learning, that is, they considered mobiles really support their learning (20 mentions). Students also mentioned mobile devices increase motivation (10 mentions) and probably engagement (only one mention). The advantage regarding ‘learning practice’ mentioned by students was the ‘diversification of learning strategies’ (five mentions). This is in line with the literature, which points to mobile devices’ potential to promote innovation in education [1].

In what concerns the advantage ‘technology support for learning’, students valued the ‘mobile devices features’, particularly the fact that the mobile device ‘enhances information access‘ (30 mentions).

Other acknowledged features were as follows: (i) ‘high portability’ (eight mentions), allowing outdoor activities; (ii) ‘resources availability’ (three mentions); (iii) ‘supports interactivity’ (one mention); and iv) ‘it is eco-friendly’ (one mention). Under ‘users’ characteristics’, students’ ‘familiarity with [mobile device] technology’ (five mentions) and ‘responsible use’ (one mention) were also pointed out. It is worth noting that participating students considered that the use of mobile technologies in outdoor contexts for learning can lead to a change of mentalities in the community (see citation Q7 in Table 1).

When questioned about constraints on the use of mobile devices to learn, 18 graduating students did not identify any constraints and 68 students mentioned at least one constraint. In ‘dubious answers’ were included six sentences (see

Table 2. Constraints of using mobile devices for learning, mentioned by graduating students.

Category	Sub-Category	Analysis Dimension	Citations Examples	Frequency
Learning methodologies	At institution level	Technology ban (T)	‘Many times we cannot use them [mobile devices] in classes’ Q23	1	26
	At student level	Distractions (T)	‘Students can use mobile devices for other things, since they are not used to using them as a work tool’ Q59	13
		Lack of technology skills	‘Students must first be taught how to use mobile devices to take advantage of them to learn’ Q52	1
		Cognitive overload	‘difficulty in selecting the information we need’ Q6 ‘we often have access to too much information, right and wrong’ Q16	9
		Other methods depreciation	‘When we are looking for information, we only rely on what is written on the Internet for example, and we do not value books’ Q49	2
Technology support for learning	Technology related	Incorrect Internet information	Sometimes the information may be incorrect Q1	21	42
		Limited connectivity (T)	‘Sometimes it requires Internet so that learning is possible and complete’ Q3	4
		Visualization limitations (T)	‘Maybe the fact that the screen is smaller; we cannot do as many searches at the same time and with the same ease as with a computer’ Q12	1
		Computing limitations (T)	‘Sometimes they [mobile devices] are slow’ Q7	3
		Battery limitations	‘The battery may run out’ Q5	4
	Users’ characteristics	Unfamiliarity with technology	‘Some people still feel uncomfortable using certain devices’ Q27	1
		Technology perceptions	‘it is not seen as something useful for learning by some people, who continue to resort to methods such as books, which are quite important as well, but we have to recognize mobile devices usefulness’ Q81	1
		Health-related issues	‘Spending too much time using mobile devices’ Q15 ‘Impairs vision and causes dependence’ Q47	7
No constraints			‘None’ Q4	18
Total				68
Dubious answer			‘It is not always feasible’ Q18 and Q19	6

Note: The notations ‘Q’ followed by a number indicate the questionnaire from which the citation example was taken.

).

Similarly to the advantages, mobile learning constraints pointed out by the graduating students were distributed in two categories: (a) 26 in ‘learning methodologies’, with the sub-categories ‘at institution level’ (one mention) and ‘at student level’ (25); and (b) 42 in ‘technology support for learning’, with the sub-categories ‘technology related’ (33) and ‘users’ characteristics’ (9).

‘At institution level’, one student mentioned the mobile device ‘technology ban’, also reported in the literature [2]. ‘At student level’ the ‘distractions’ potentially supported by mobiles achieved some relevance (13 mentions). ‘Cognitive overload’, due to the high quantity and, frequently, bad quality of the information accessed through mobile devices, was also mentioned (by nine students). The ‘lack of technology skills’ of students from non-higher education contexts was mentioned one time. The depreciation of other learning methods, namely through printed books, was also a concern to two students.

In what concerns ‘technology related’ constraints, the most frequently referred one (21 times) was ‘incorrect Internet information’ revealing students’ concern with the reliability of the information they access through mobile devices. Other issues, also pointed in the literature include: (i) ‘limited connectivity’ [2,16] through no access to or slow connection to the web (four student mentions); (ii) ‘computing limitations’ [16], which results in slowness in task performance and even may turn the interactivity with the mobile device harder (three mentions); and (iii) ‘visualization limitations’ [16], due to the small screen of the devices, compared to the screen of a desk computer, with one mention by a graduating student.

Finally, some ‘users’ characteristics’ may also hinder mobile learning by students, particularly the development of ‘health-related issues’, such as the risk of promoting mobile devices dependence and vision impairment (seven mentions). Other users’ features were the possibility of ‘unfamiliarity with [mobile] technology’ and mobile ‘technology perceptions’ that associate these devices only with recreational activities (each one with 1 mention).

Another issue regarding which students expressed their opinion was the usability of the EduPARK app. For example, the most relevant usability issue was ‘The app soaks up memory resources, which sometimes causes the mobile device to crash’ (Q11). This indicates that the number and the file size of the multimedia resources of the app, as images or videos, should be reduced. The average SUS score [54,55] was 73.3, which corresponds to a good–excellent usability, according to Bangor and colleagues [57]. This is reinforced by most of the students’ comments that pointed the app has no usability issue to improve, for example, as illustrated by: ‘I enjoyed this app very much … the app is well-developed and I cannot identify any aspect to improve‘ (Q51). Additionally, the app usability ranged from 30.0 (achieved in two questionnaires) to 100.0 (attained in three questionnaires). These results support the claim that the app is easy to use by this cohort of higher education students, despite their varied academic backgrounds. Moreover, as this group experienced mobile learning integrated in their own formal education, one can claim that mobile learning was demonstrated as a possibility in formal contexts for these future teachers. Thus, the very experience of the EduPARK activity might have opened the door to a facilitated use of mobile devices in their own future teaching practice, particularly considering that these future-teachers were already frequently learning with mobiles.

Regarding graduating students’ evaluation of the EduPARK activity, they were initially challenged to rank its degree of relevance, on a scale of 1–5 (where 1 is not relevant and 5 is very relevant), with respect to: (a) learning value; (b) intrinsic motivation; (c) engagement; (d) authentic learning; (e) lifelong learning; and (f) conservation and sustainability habits. Students’ answers (Figure 7) reveal that they gave special relevance to the learning value of the app, since about 52% assigned level 5 of relevance (very relevant) and 33% assigned level 4 (relevant). This result shows that graduating students consider that this activity has a high impact on its participants’ learning and, hence, contributes to a positive perception regarding the educative potential of mobile learning strategies. As for impact on the participants’ intrinsic motivation and engagement, level 4 and level 5 of relevance were assigned in similar frequency, around 41%. These results reveal that these graduating students also perceive the EduPARK activity as highly motivating and engaging for learning. With regard to authentic learning and lifelong learning, most students attributed level 4 as relevant, with 50% and 38%, respectively. Notably, 31% of students took a neutral position regarding the value of the EduPARK activity for lifelong learning. Considering that most graduating students acknowledged the activity’s value for learning, but not all students recognized it in a lifelong perspective, this fraction of students fails to grasp the EduPARK activity potential for the regular citizens that might use the EduPARK app for an educational walk in the park, in their leisure time. Finally, in terms of conservation and sustainability habits, levels 4 and 5 of relevance are again expressed, around 34%–36%, so these graduating students recognize the EduPARK activity, of using a mobile app with augmented reality to play a game and learn in a green park, is relevant for environmental education.

5.2. Reflections

As mentioned above, 30 groups of students turned in their reflection about the possible impact of the EduPARK strategy on: (a) learning value (b) intrinsic motivation; (c) commitment; (d) authentic learning; (e) lifelong learning; and (f) conservation and sustainability habits. The category ‘Extrinsic motivation’ emerged from the data, as students made frequent references to it.

Table 3. Impact of the EduPARK activity, according to the graduating students.

Category	Analysis Dimension	Citations Examples	Frequency
Learning value—ability of the EduPARK activity to promote learning (49 assertions)	Content—ability of the activity to support substantive knowledge learning (e.g., about the flowers of a species).	‘There is acquisition of knowledge in the most diverse areas, such as Biology, Geology, History, Physics and Chemistry‘ R26	21 with positive assertions; one with critics
	Skills—ability of the activity to promote skills development (e.g., team work).	‘While walking through the Infante D. Pedro Park, we notice species of trees and plants that were unknown for us before. After this activity, not only we now know their name, but we can also identify them in other places.’ R13	Five with positive assertions
	Values—ability of the activity to stimulate participants’ appreciation of qualities that are important for society (e.g., conservation of the environment)	‘We brought with us [from the EduPARK activity] a luggage of values, both at the personal and at the collective levels’ R20	12 with positive assertions
	Not specified—ability of the activity to promote learning, without referring at which dimension(s)	‘In our view this mobile app is innovative, since it uses Augmented Reality, which is an added value for learning.’ R3	Eight with positive assertions; one with critics
Intrinsic motivation– ability of the EduPARK activity to promote participation for its own sake, i.e., for the pleasure and satisfaction derived from it (43 assertions)	Enjoyment—ability of the activity to be enjoyable/fun/ pleasant	‘The EduPARK app allows the student to learn in a more attractive, informal, and fun way.’ R24	24 with positive assertions
	Interest—ability of the activity to be interesting for itself	‘This activity, due to its good functioning and healthy app structuring, generated in all the elements of the group an interest for the proposed learning’ R1	Nine with positive assertions; one with neutral assertions
	Attention—ability of the activity to hold participant’s attention	‘The app holds children's attention and later it will be easier to teach varied contents since the children have previously been familiar with these same contents in a playful way.’ R23	Three with positive assertions
	Curiosity—ability of the activity to arouse curiosity (e.g., due to a novelty or surprising factor)	‘The interaction between the technologies and the context of the activity - the city green park - is an added value to arouse the participants' curiosity about curricular topics.’ R25	Two with positive assertions
	Stimulus—ability of the activity to be stimulating to participants (e.g., by challenging them to overcome their difficulties)	‘The questions and challenges posed by the Monkey mascot inspire in us a willingness to answer correctly, with the aim of overcoming themselves.’ R19	Four with positive assertions
Extrinsic motivation—ability of the EduPARK activity to promote participation due to external factors, i.e., the goals of participation extend beyond those inherent in the activity itself(54 assertions)	Free will—ability of the activity to let participants choose if and/or how to do it	‘For us, and for the people using the EduPARK app, it is really rewarding, as learning is faster and by on our own initiative.’ R12	Six with positive assertions
	Benefits—ability of the activity to be perceived as giving its participants gains beyond the activity itself	‘The use of the smartphone as an instrument in the activity may have been a factor to increase the motivation. This is not an instrument frequently used in the classroom for learning’ R6	Three with positive assertions
	Rewards—ability of the activity to offer rewards to its participants	‘We believe that extrinsic motivation elements, such as the points, bananas, etc., are of great help to motivate some participants, as this helps them to get into the learning dynamics’ R29	17 with positive assertions
	Competition—ability of the activity to promote participation due to people’s will to outperform other participants	‘when playing the quiz, and wanting to win, players will try to answer correctly to as many questions as possible, learning about some themes’ R7	10 with positive assertions
	Sanctions—ability of the activity to promote participation due to people’s will to avoid sanctions for not participating	‘The wrong answers will encourage participants to want to be better next time, even if in a different context.’ R19	Two with positive assertions;one with critics
Engagement—ability of the EduPARK activity to promote the participants’ active involvement (12 assertions)	Affective—participants’ positive emotional reactions and feelings towards carrying out the activity	‘As the activity was very fruitful, dynamic, and provided a good learning environment, the group liked it and it exceeded our expectations.’ R8	Three with positive assertions
	Compliance—participants’ obedience to the activity norms	‘This app contributes to the children’s engagement, since they receive points when they answer the questions correctly, having, for that, to explore the park and the markers of augmented reality.’ R27	Four with positive assertions
	Effortful participation—participants are invested in the activity	‘Although the activity takes some time to complete, we do not notice time passing.’ R2	Nine with positive assertions
	Cognitive—participants are mentally involved in meaningful processing, metacognition, etc.	-	0
	Disengagement—participants are not affective nor cognitively involved	-	0
	Not specified—ability of the activity to involve the participants, without referring at which dimension(s)	‘The fact that this app uses the best new technologies have to offer, including Augmented Reality, has prompted our engagement and motivation, leading us to achieve the victory with 300 points.’ R17	Three with positive assertions
Authentic learning—ability of the EduPARK activity to promote learning in a real-world context, in ways that are meaningful, useful and relevant to the participant	-	‘we believe that an application like EduPARK helps children to initiate meaningful learning about biodiversity, since students have to answer questions based on what they see (…) the app takes students to a park where they observe their environment, reflect and learn from it.’ R21	23 with positive assertions
Lifelong learning—ability of the EduPARK activity to support ongoing and voluntary learning throughout participants’ life	-	‘is a tool that, addressing different themes, from the arts to the various sciences, contributes to lifelong learning, arousing people interest and attention to their surrounding environment, at different ages.’ R15	13 with positive assertions
Conservation and sustainability habits—ability of the EduPARK activity to promote in its participants routines of thinking and behaving for sustainability	-	‘students realize the importance of the park's biodiversity and they can have a more real sense of what is happening around them, thus creating habits and responsibilities for the conservation and sustainability of the environment.’ R5	26 with positive assertions
		Total	221

summarizes the students’ views, with a total of 221 individual impacts identified and an average of 7.4 per document/group reflection.

Surprisingly, the most frequently mentioned impact of the EduPARK activity was the one that emerged from the data, ‘Extrinsic motivation’ with a total of 54 assertions. As there were only 30 student reflections, there were some reflections where several dimensions of extrinsic motivation were identified. For example, Reflection number 10 mentioned both the ‘rewards’ and the ‘competition’ dimensions: ‘In this case, the mobile app offers us several stimuli for our motivation such as the use of technology, the outdoors exploration, the group work, and the competition inherent to games with points/scores.’ From the analyzed data, it was clear that, generally, students from these two cohorts did not differentiate properly extrinsic motivation, intrinsic motivation, and engagement. Nevertheless, it was possible to code their assertions based on their explanations. For example, according to the students, the EduPARK activity has several features that can motivate participation, such as: (i) earning ‘rewards’ (17 mentions) like points or bananas during the game; (ii) getting involved in a friendly ‘competition’ (10 mentions), to see which group wins; (iii) allowing participants a certain degree of ‘free will’ (six mentions), as they can conduct the activity at their own pace, autonomously, and by their own initiative; (iv) gaining ‘benefits’ (three mentions) from participation beyond the activity itself, like being allowed to use mobile devices in formal education, which is not a frequent situation in schools; and even v) avoiding ‘sanctions’ (three mentions), such as the Monkey mascot sad face and sound when they answer incorrectly. Other important issue for motivation is the combination of collaboration and competition, as participants worked in groups, negotiated answers to gather points and reach higher score in the leaderboard, as illustrated by the following citation: ‘New technologies allied to a guided path Game Mode promote a healthy competitiveness that increases commitment, dedication, teamwork and a greater will to learn.’ (Reflection number 8).

The second most cited category was ‘Learning value’, with 49 mentions in its four dimensions of analysis. Students’ revealed that they give special relevance to the ‘content’ learning promoted by the activity, as it was mentioned 21 times, which can be illustrated with the following citation: ‘The experience with EduPARK makes learning more fun and dynamic, avoiding the classic model that children and adults have contact with. There is acquisition of knowledge in the most diverse areas, such as biology, geology, history, physics, and chemistry in an environment that is inspiring and fully linked to what is learned, which makes learning more consolidated, since direct observation is a key point’ (Reflection number 26). Also, quite present in students views it that the activity also promotes learning ‘values’ (12 mentions), particularly in what concerns sustainability, as we will discuss further, and learning ‘skills’ (with five mentions). Eight groups mentioned the activity has impacted on learning, without mentioning at each level. One group took a critical position: ‘It is important to mention that the use of mobile devices as a learning tool does not replace any other traditional learning ways, since they are considered as an additional resource to support and enrich the teaching and learning processes, in which the student is the main constructor of knowledge’ (Reflection number 21).

‘Intrinsic motivation’ gathered 43 mentions, being the ‘enjoyment’ of the activity the most relevant dimension for the students (24 mentions), as revealed by the following citations: ‘the way the app is developed allows students to learn the content in a playful and effective way’ (Reflection number 1) and ‘This activity was very motivating and interesting, due to the fact that it consolidates content from different disciplines and it puts us in contact with Nature’ (Reflection number 3). Other dimensions pointed out were ‘interest’ (10 mentions), ‘stimulus’ (four mentions), ‘attention’ (three mentions) and ‘curiosity’ (two mentions).

The category ‘Conservation and sustainability habits’ included 26 positive assertions. Graduating students acknowledged the relevance of the nature contact provided by the activity and its value for creating environmental awareness in its participants. They said, for example, ‘We consider that this app is highly recommended when teaching biodiversity, since, through this quiz-treasure hunt, students will learn about their environment in a meaningful way’ (Reflection number 21) and ‘this type of activities can create awareness of the need to take care of the park and its species’ (Reflection number 29).

Twenty-three student reflections considered the EduPARK activity had a positive impact on ‘Authentic learning’. As citations we can point out: ‘It is a new way of learning and captivating people, especially students, to put their knowledge and everything they learn in the classroom into practice’ (Reflection number 3) and ‘we believe that an app like EduPARK helps children to initiate meaningful learning about biodiversity, since the students have to answer questions based on what they see… the app takes students to a park where they observe their environment, reflect and learn from it’ (Reflection number 21).

The categories less relevant for these two cohorts of graduating students were ‘Lifelong learning’ with 13 positive assertions and ‘Engagement’ with 12. Regarding lifelong learning students mentioned: ‘we also consider that this game encourages learning throughout life, as its supporting app is available to any smartphone’ (Reflection number 7). This result suggests that the graduating students acknowledge the contribution of the EduPARK project to the area of Open Education. In what concerns engagement, the most relevant dimension was ’Effortful participation’ with nine positive assertions. Rule ‘compliance’ and ‘affective’ involvement in the activity achieved four and three mentions, respectively. Curiously, ‘cognitive’ engagement was not mentioned, nor was ‘disengagement’.

6. Discussion

This case study aims to contribute to the understanding of the educational value of the EduPARK strategy, involving a mobile game-based learning with AR app, for exploration in outdoor settings, according to 86 Higher Education students (future teachers), in what concerns: (i) mobile learning; (ii) the EduPARK app usability; and (iii) the impact of the educational strategy on six dimensions. The students experienced the EduPARK strategy under the curricular unit ‘Nature Integrated Sciences II’ in the academics’ years of 2017/18 and 2018/19, as described in Section 4.2.

Starting with the graduating students’ opinion regarding the first issue, mobile learning, they reported previous experience with the use of mobile technologies to learn. This indicates that students have a positive perspective on mobile learning, as the majority reported using mobiles to learn with high frequency, which would not happen if they held a negative perspective on mobile learning. These results are in agreement with [62], where 98% of higher education students reported using their mobile devices for academic purposes. However, the high report of mobile learning from these respondents may be, in part, imputed to the social desirability factor [63] and, thus, biased, particularly considering that students answered this question in a context where the teachers of their curricular unit used mobile devices to promote learning. Despite the possible impact of the social desirability factor, we might be facing a mentality shift concerning the use of mobile devices in education when this generation of candidates become teachers. Nevertheless, this issue needs further research to be clarified, for example, through a longitudinal study to analyze the inclusion of mobile technologies in the teaching practices of these two cohorts, under their early career stages.

Students’ positive view of mobile learning is also supported by the fact that they identified more advantages (85) than constraints (68) for this learning methodology. This is even more evident if we consider two aspects. Firstly, 23 student statements presented as an answer to the advantages question were of dubious interpretation, so it was not possible to include those items in the analysis and they were not accounted for as advantages. Secondly, there was not a single student mentioning that he/she could not identify any advantages, but 18 stated clearly that they did not consider mobile learning to have any constraints.

In what concerns advantages of using mobile devices to learn, the graduating students mentioned a wide variety of items, therefore no individual advantage was mentioned by the majority of students. However, they clearly valued advantages related to how technology supports learning, particularly regarding mobile devices supporting access to information and, even, how the development of resources for outdoor learning, taking advantage of mobile and AR technologies, can change people’s mentalities. This aspect has been pointed by the literature as well, namely in what concerns mobile devices supporting interaction with digital content [2,18]. In the ‘learning methodologies’ category, the learning and motivation mobiles provide were also valued. These survey results point to students’ high appreciation of mobile devices supporting content learning, which is in line with results from the group reflections regarding the impact of the EduPARK strategy on learning value. Here, students also valued content learning, in contrast with the low reference to other learning dimensions (skills and values) or even to learning without identifying the dimension. Surprisingly, in a generation that grew up in a society embedded in technologies, as most graduating students were 19–20 years old, the most exciting features of mobile devices that may support authentic learning, such as high portability, due to mobile devices’ reduced size and weight [1,16,17], and high interactivity [2,15,18], were mentioned by a minority of students. From these results emerges the relevance of working with future teachers to demonstrate to them how mobile devices can support other learning, besides content learning. In this way, we might be creating conditions for students’ future teaching practice to include strategies aligned with current and relevant pedagogic recommendations.

Despite graduating students’ positive views of mobile learning, they also identified a set of constraints, with no individual constraint being mentioned by the majority of students. Once again, the ‘technology support for learning’ category was the most valued, this time with ‘incorrect Internet information’ as the highest concern. On the ‘learning methodologies’ side, the distractions that may be promoted by mobile devices were a significant concern for the students, in line with [1]. From these results, it is possible to delineate strategies to overcome the identified constraints regarding the use of mobile devices to learn, such as teaching methodologies to find and recognize reliable information online. The ‘distractions’ issue is one that needs to be worked at several levels to support the change in current mentalities. Students, teachers, and other education stakeholders need to be made aware that mobile devices can be used to leverage authentic learning. Surprisingly, one of the constraints featured in the literature, the institutional ban of mobile technology [2], was not a major concern for these student cohorts, so they may not have felt this difficulty themselves. Likewise, a constraint recently valued by in-service teachers (paper not published yet) is the lack of resources, such as connectivity and the devices, which was not highlighted by the students. This may be due to the students’ lack of knowledge regarding the Portuguese educational reality at this level. Nevertheless, students’ lack of technology skills was also mentioned by one graduating student, suggesting some sensibility to the digital divides mentioned in the literature [19].

The second issue for students to pronounce their opinion was the EduPARK app usability, the app being an illustrative example of a mobile learning approach. According to the students, the app usability was good to excellent [54,55,57], revealing that mobile learning systems can be implemented in educational settings with a high usability, at least at the university level. The usability factor may have an impact on the users’ perspectives regarding mobile learning approaches and, thus, it may be a determining factor in the adoption of mobile technology for educational purposes.

Finally, students’ perspectives regarding the third issue, the impact of the EduPARK educational strategy on six dimensions, one of the most relevant results is the strategy’s potential to promote learning. Yet, some students revealed a critical position regarding this issue, by mentioning the relevance of the teacher role in the learning process, highlighting it cannot be not replaced by any technology, which is considered as an additional resource to support learning. Nevertheless, both survey and group reflection data reveal that graduating students are unanimous in considering the EduPARK strategy to have a high learning value. As discussed in the previous section, students focused particularly on the strategy’s potential for content learning, but also acknowledged its potential for skills and values learning, this last one in what concerns pro-environmental values.

In what concerns ‘conservation and sustainability habits’, students were nearly unanimous in acknowledging the EduPARK strategy potential in their promotion. This is an expectable result, as the strategy unfolds in a green park, requiring the analysis of issues contextualized in the reality of the park.

Students also considered that the EduPARK strategy promotes both intrinsic and extrinsic motivation. Although it has not been requested for reflection, students considered that the EduPARK strategy has the highest impact on extrinsic motivation, as they frequently made mention to the rewards offered, such as points and bananas, and the friendly competition spirit it created. These factors may have contributed to potentiate a genuine enjoyment of the activity by its participants, hence enhancing intrinsic motivation as well.

In the survey, students classified the EduPARK strategy as one that has a high impact in promoting participants’ engagement. However, in student group reflections this aspect was not as evident. In fact, qualitative data revealed that students confused the constructs of motivation and engagement, which creates validity limitations regarding the quantitative data for this topic. Nevertheless, when students successfully expressed that the EduPARK activity promotes participant engagement (which happened in 12 group reflections) the most valued dimension was the effortful participation, indicating investment in the completion of the activity. However, cognitive engagement was not mentioned by any group, which may indicate that the strategy fails to promote that type of engagement or that these cohorts of students failed to recognize this dimension in the strategy.

Regarding the impact of the EduPARK strategy on authentic learning, some graduating students appeared to have had a few doubts about this issue. In the survey, most students selected option 4 (out of a five-level scale), revealing a less intense agreement with the statement. However, in the group reflection, students were able to express an agreement about the strategy’s potential for authentic learning. Nonetheless, this issue needs to be asserted with caution and requires further analysis.

The dimension where students considered the EduPARK strategy to have less impact is lifelong learning, according to data from the survey and the group reflections. Students considered the strategy to promote learning, but not necessarily in a lifelong perspective. It is not possible to propose a justification for this result, as data collected in the group reflections regarding this issue were vague or of dubious interpretation.

This study results allow to answer the research question ‘What is the educational value of the EduPARK strategy, involving a mobile game-based learning with AR app, for exploration in outdoor settings, according to Higher Education students (future teachers)?’ According to the graduating students, the EduPARK strategy has educational value as it:

• is based on mobile technologies, which they also used themselves to learn and consider having more advantages than constraints associated;
• involves using an app easy to use; and
• has a high impact on its users’ authentic learning, particularly in the area of conservation and sustainability habits, on their intrinsic and extrinsic motivation, and on their engagement in learning as well.

The results of this case study need to be interpreted with caution, as they come from a specific case study to which the authors had access. Other cohorts of graduating students or in-service teachers might not share the study participants’ opinions on the educational value of the EduPARK strategy. Yet, the profile of the participating graduating students revealed two strongly female-dominated cohorts, in line with the Portuguese reality of the educational profession in the early years. Hence, one might argue that this case study results may be transferable to a wider group, bearing in mind the younger age and lack of teaching experience of the participants in this study.

7. Conclusions

This work was developed under a changing mentalities paradigm, in what concerns the educational potential of mobile and AR technologies, games and green parks, a trilogy of unconventional, but effective, educative approaches. With the aim of understanding the educational value of a strategy comprising all the trilogy elements, the case of the EduPARK was selected for analysis, particularly the perspective of students in graduation courses for future teachers. The results of two data collection origins support the overall educational value of the EduPARK strategy. This is a concrete example of how can mobile technologies be used to successfully promote learning, and hence this strategy is changing mentalities that associated the trilogy elements exclusively to distraction, play and leisure.

Now it is relevant to reflect on what conditions allowed the success of the EduPARK learning strategy. Hence, a set of supporting pillars of the EduPARK project emerges from this case study and constitutes guidelines for future strategy development combining mobile devices, AR, games and green parks. Consequently, this work contribution to the literature on mobile AR game-based learning in informal outdoor environments is summarized in the following subsection, which presents the EduPARK supporting pillars for the first time.

The EduPARK Project Supporting Pillars

The EduPARK project has been supported by seven pillars, summarized in Figure 8, in order to promote mobile, authentic, and cross-subject learning in a motivating way across formal, informal, and non-formal learning contexts and, consequently, to change mentalities about how people can learn using mobile technologies.

The first pillar is the use of mobile and AR technologies to promote learning. Mobile devices can place learning in a specific location, in order to support situated learning [19]. Our study showed that the participating students considered the EduPARK strategy has educational value; hence, the combination of mobile and AR technologies can be used to promote learning. The literature showed that mobiles can be associated with moving learning from the classroom to outside, for more authentic learning settings. However, the EduPARK team considers that these technologies can play a much more prominent role in education than that. In fact, the project is leveraging the rapid development of mobile technologies, which is allowing the integration of new and diverse functionalities (such as taking photographs or geolocalization) in the devices to provide active and contextualized learning, as well as to encourage new teaching approaches.

One of the new technologies that can be integrated in mobile devices is AR, as mobiles can support overlaying virtual elements onto real objects and locations, thus, augmenting the reality with additional information in compelling and meaningful ways [8]. For example, AR 3D models can be used to support visualization and comprehension of phenomena or concepts outdoors, in ways that are not possible with traditional books [24]. This technology combination can effectively engage and motivate students for learning in several school levels and contexts [24,25,41,64]. EduPARK is augmenting the Infante D. Pedro Park to support authentic and cross-subject learning. With the app, park visitors can easily access virtual information that complements and sustains the interpretation of real natural and social phenomena—the AR resources produced by the team—through a relatively inexpensive technology, the mobile device, owned by most of the Portuguese population, including young children [14].

The second pillar is outdoor learning, which combines learning with healthy lifestyle habits and with enjoyable time. This combination is relevant for formal education, for instance, during lecturing time, as well as for non-formal and informal education, for example, during leisure time or holidays.

Results point out two different perspectives about outdoor learning: first, the anytime/anywhere learning, including outdoors, by the use of mobile devices; and, secondly, students mentioned the EduPARK can support community learning, as its resources are available to the wider public in a green park usually used to walk, but now can also be used to learn.

As the educational laboratory of the EduPARK is a green park in the city, these activities are particularly important in what concerns conservation and sustainable attitudes of citizens about nature, contributing to promote smart, sustainable, and inclusive growth in society. The less frequent studies in the literature are the ones analyzing mobile learning in informal and formal learning contexts, the informal learning ones being the most predominant according to [6]. Oppositely, [7] found that most research on mobile learning took place in a formal educational context. Nevertheless, it seems that mobile educational resources that can be used across contexts, such as the EduPARK app, are not frequent. Moreover, [7] reported a higher impact of using mobile devices for learning in informal settings than in formal locations and recommended more research on mobile learning in contexts outside the traditional classroom setting.

Thirdly, game-based learning is a well-known promoter of learner motivation and engagement, as pointed by participating students and in line with the literature [29,64], particularly if the games are designed considering the desired learning objectives [11]. EduPARK mechanics of game play are: (i) scoring points whenever players get the right answers to the questions; (ii) gaining extra points when players answer all the questions of a game stage correctly; (iii) capturing bananas in virtual treasures hidden in the park; iv) solving riddles to find virtual treasures; and v) attaining several victories in different app games. These make this approach attractive, and are used with the aim of maximizing motivation, in line with the recommendation of [11] promoting the enjoyment of learning curricular content, whilst supporting an authentic learning that lasts.

Fourthly, EduPARK aims at promoting an authentic learning by establishing relationships between curricular concepts and real-life situations and through the use of resources already existing in the park to reach curricular learning objectives in an authentic and contextualized way [65]. Graduating students explicitly mentioned the EduPARK promotes meaningful learning namely about biodiversity, since the learners have to answer questions based on what they observe locally. In the EduPARK approach, students can establish connections between their environment (what they can see, observe closely, and even touch) and school learning content, through discussion with their classmates.

The EduPARK fifth pillar is cross-subject learning. Participating students acknowledged that learning within the EduPARK occurs in diverse subject areas, such as biology, geology, history, physics, and chemistry. The literature reveals that mobile learning research for cross-subject learning is not frequent [6] and, at a first stage, EduPARK also started to integrate only science subjects, but rapidly history issues joined the app content, as well as physical education, mathematics, visual arts, and languages. For example, some app challenges used flowers in the park to address issues about symmetry axis; or used historical monuments in the park to integrate geology issues or math problems.

Sixthly, the project is based on collaborative learning. Graduating students valued the EduPARK activity’s ability to promote team work for learning, as the game educational challenges can promote collaborative discussion of ideas among group members [41]. In fact, the project’s recommendation is to play the EduPARK game in small teams, although it is perfectly playable individually. This recommendation is in line with research that suggests collaboration is more effective than competition to reach learning achievements [17]. However, game-based learning also enhances competition among different groups, which may increase pupils’ engagement in challenging learning situations and improve their overall sense of enjoyment [17].

Finally, EduPARK follows the open education and Education for All philosophies, as shown by some of the participating students, in what concerns the wide availability of the app and the fact that it can promote learning in diverse public targets with different age ranges and learning needs. Being free is one to the main elements of openness [66] and the main product of the project, the app, is freely released in the Google Play Store (for Android devices). One might argue that its openness is limited as the app does not have a version for iOS operating systems. However, the game is designed to be played individually or in small groups, hence, not everyone in a group has to have an Android device. Moreover, the app has the potential to reach a large part of the population, so it is contributing to the aim of quality Education for All, reaffirmed in [67]. As mentioned above, the target public of the EduPARK app is quite wide as it encompasses: (i) students from the 1st Cycle of Basic Education (CBE) to Higher Education, including children with special needs; (ii) their relatives and friends, in addition to the ordinary citizen, or any Portuguese- or English-speaking foreign tourist, in a lifelong learning perspective; and (iii) education professionals, who participate in the project workshops or who provide the EduPARK activity to their students.

The great relevance and innovation of EduPARK is related to cross-subject outdoor learning in formal, informal, and non-formal contexts, supported by mobile technology, in an integrated perspective of Science, Technology, Society and Innovation [46]. The project has been organizing activities for students, teachers, and park visitors in order to collect systematic data to better understand mobile learning benefits in a rich and diverse outdoor environment, such as a green park. This way learning moves from traditional classroom environments to natural spaces that students can physically explore while making associations with curricular content. To date, the EduPARK project has organized more than 80 sessions of exploration of the park with the app, involving more than 1250 students, 280 teachers, and 360 park visitors. The game of the EduPARK app has systematically raised interest and enthusiasm from users, who learn in a fun way while strolling in the park.

As a final note, in technology-driven societies, such as the current one, the gap between the use of mobile devices inside and outside educational institutions can lead to students’ disengagement, impacting negatively on their academic success. The creation of a disruptive learning environment, leveraging from games’ motivational potential and from green parks’ authentic and contextualized learning opportunities, such as EduPARK, supports learning ecosystems to get smarter and more effective. The authors believe that many more people will explore this strategy with effects on future teaching and learning practices. This will surely contribute to challenging conventional thinking about the trilogy of mobile technology, games, and green parks, supporting an innovative perspective of education.