Augmented Reality in Lower Secondary Education: A Teacher Professional Development Program in Cyprus and Greece

Abstract

The current article provides an overview of a Teacher Professional Development (TPD) program that has been designed, pilot tested, and implemented to investigate the impact of augmented reality (AR) on: (a) Teachers’ level of technology (AR) acceptance, adoption of inquiry-based instructional approaches, and confidence towards teaching twentieth-first century skills in STEM-related courses; and (b) students’ potential enhancement of specific twentieth-first century skills and motivation and interest during a STEM- (science, technology, engineering, mathematics)-related course supported with AR. This article focuses on the teachers’ points of view concerning the impact of their STEM-related interventions on their students’ motivation and learning, as well as the factors that influence the teachers’ technology acceptance. The TPD program has been implemented in Cyprus and Greece with twenty-five lower secondary school teachers (20 in Cyprus and 5 in Greece). The research methodology applied is Educational Design Research (EDR), including an initial phase of the TPD program and a second (improved) phase. The data collection tools consisted of questionnaires, interviews, and observation of classroom interventions. Initial findings and their implications for teaching and future research are discussed, indicating the potential benefits and challenges surrounding the integration of AR within the educational process.

1. Introduction

During recent years, numerous studies highlight the current need for professionals in all fields equipped with twentieth-first century skills (i.e., critical thinking, inquiry, creativity, problem-solving, collaboration) [1,2,3], as well as teachers preparing their students for twentieth-first century professions [4,5,6]. STEM skills [7] (science, technology, engineering, mathematics) are among these key competencies necessary in a future knowledge-based society for employment, personal fulfilment and development [8,9]. However, there is a gap between the way STEM-related disciplines are taught in EU Lower Secondary Education and the twentieth-first century skills required by students to face real life situations in STEM-related studies and careers [10,11]. This means that teachers, still teach in the way they have been taught as students [12]. Therefore, school classrooms need to be upgraded through dynamic educational processes, allowing the adaptation of educational approaches and resources towards the promotion of these critical skills. This reframing could be accompanied with innovative pedagogical approaches and re-contextualized learning environments, as well as appropriate technological equipment (e.g., WiFi connection, mobile devices, modern computer devices) leading to smart learning environments, meaning that they fulfil the requirements of effectiveness, efficiency and engagement [13,14].

Changing teaching practices is proving difficult, while many teachers remain unprepared to effectively employ technology-enhanced teaching practices [15,16]. Many teachers are hesitant to use new technologies for reasons, such as: (i) Fear and lack of confidence in their use; (ii) lack of time or motivation for acquiring new technological skills and adapting new pedagogical strategies; (iii) lack of existing educational resources; and (iv) the fact that they feel uncomfortable with student-centered approaches enhanced by new technologies [15,16,17]. Additionally, the development pace of technology and innovative devices far exceeds the development pace of research studies aiming to identify effective ways to integrate each technological innovation within the educational process [18]. Thus, teachers, as well as other stakeholders involved in education (educational researchers/designers/developers, decision makers, school administrations, etc.), face challenges in deploying innovative smart learning environments and keeping updated with the latest technologies and devices [13,18]. For teachers to get convinced to introduce innovative technologies in their classrooms, they need to realize their educational added value and potential benefits in enhancing teaching and learning [19]. It is crucial for both future and practicing teachers to have opportunities for adequate training concerning the challenges of the twentieth-first century and the ways in which technology-enhanced learning can help learners cope with these challenges. They need to acquire knowledge relevant to emerging technologies in educational contexts, as well as develop skills in effectively applying these technologies [20,21]. Emerging technologies should be treated as a concept rather than sole technologies [22] offering new learning opportunities, but also creating new challenges in the educational process [22,23]. Although there have been small gains and interesting advances under specific situations and in relatively isolated circumstances, the educational process has yet to be transformed [13]. Teachers ought to be life-long learners, able to follow (or at least be aware of) future technologies and feeling adequately confident to try to learn and exploit them on their own.

One promising aspect recently explored by the research community, is the potential of integrating the technology of augmented reality (AR) and mixed reality (MR) in the Immersive Learning Landscape, as a means of enhancing students’ twentieth-first century skills and performance in STEM-related courses [22,23,24,25,26,27]. Teachers are key persons in the processes of designing and developing educational activities and material towards this landscape to exploit the potential of augmented reality—therefore, they could provide the necessary instructional support to their students, maximizing the impact of such technologies [21,24]. Additionally, teachers’ encouragement to recognize the added value that AR could have on teaching, learning, and assessment, as well as being informed about best practices in their exploitation as instructional tools are of critical importance [28]. Thus, the provision of teacher professional development (TPD) programs focused on twentieth-first century skills, including STEM skills [29] supported by emerging technologies, such as AR, and focusing on innovative pedagogical approaches is of critical importance [21].

There is an increased interest towards this field, however, due to the novelty of the technologies, research concerning augmented reality integration into teaching and learning is still relatively small in scale but with strong potentials [23,25,27]. Some studies have identified specific factors that may affect the acceptance of AR technology in education, such as curriculum, stability of the interaction, self-learning capability, parents’ involvement, students’ background, platform, and social factors [30,31,32]. The provision of high quality pre-service and in-service teacher training that will equip teachers with the required knowledge and skills to effectively apply AR into teaching and learning is of utmost importance to research efforts in this field [33].

The current research presents a Teacher Professional Development (TPD) program, developed within the context of the EU project Enlivened Laboratories within STEM Education (EL-STEM), which aims to familiarize teachers with the potential of AR technology for enhancing the teaching and learning processes in lower secondary STEM education. More specifically, the TPD program has been designed, pilot tested and implemented to investigate the impact of augmented reality (AR) on: (a) The teachers’ level of (i) technology (AR) acceptance, (ii) adoption of inquiry-based instructional approaches, and (iii) confidence towards teaching twentieth-first century skills in STEM-related courses; and (b) the students’ (i) potential enhancement of specific twentieth-first century skills, and (ii) motivation and interest during a STEM-related course supported with AR. The total research consists of an Educational Design Research (EDR) including two phases, an initial TPD (February 2019–May 2019) and an improved one (September 2019–February 2020), with a total number of twenty-five teachers involved in Cyprus and Greece. In this paper, the authors focus on the teachers’ technology acceptance (i.e., AR) and the teachers’ perspectives on how classroom interventions supported by AR technology in STEM-related topics impact their students’ motivation and learning. Finally, this paper concludes with a discussion of the initial findings, and the implications for teaching and future research.

2. Objectives and Research Methods

The main objectives of the broader study, under which the current research is implemented, are the design, pilot testing and implementation of a TPD program to investigate the impact of AR on:

• The teachers’ level of: (i) Technology acceptance (in this case augmented reality), (ii) adoption of inquiry-based instructional approaches, and (iii) confidence towards teaching twentieth-first century skills in STEM-related courses;
• The students’ (i) potential enhancement of specific twentieth-first century skills, and (ii) motivation and interest during a STEM-related course supported with AR technology.

In this study, the term STEM is adapted, not simply referring to the individual subjects of the acronym (science, technology, engineering, mathematics), but providing an engaging and integrated way of teaching and learning. Moreover, based on the California Department of Education [34] perspective on STEM-related education, it could include individual subjects, a stand-alone course, a sequence of courses, activities involving any areas, a STEM-related course, or an interconnected or integrated program of study. As far as the twentieth-first century skills are concerned, there are numerous frameworks suggested to categorize them [35,36]; in this study, we adopt a synthesis including three main categories: (a) foundational knowledge, (b) meta-knowledge, and (c) humanistic knowledge [37].

The methodological underpinning for the current research is Educational Design Research (EDR), mentioned in the literature as a methodology to “denote design research in the field of education, more related to the motive of improving practice, which could be conducted primarily to solve a problem, put knowledge to innovative use and/or increase robustness and systematic nature of design practices” ([38], p. 133). EDR incorporates the systematic study of design, development and evaluation of educational interventions (such as teacher professional development programs, teaching and learning processes, learning environments, teaching-learning materials, etc.) [39] aiming to make the educational research more relevant to practice and policy [40]. EDR is not considered as a single approach, but as a series of approaches, intending to produce results that account for and potentially impact learning and teaching in authentic environments [41]. Once a problem is identified, EDR tends to evolve through three main phases, each of which may be repeated multiple times: analysis, design, and evaluation [38]. Figure 1 explains how the EDR approach is applied in the current study, while the authors of this paper focus on the evaluation-reflection part.

This EDR study is related to the motive of improving practice, which could be conducted primarily to ([38], p. 133):

• Solve a problem: As generally stated, to decrease to some degree the gap between the way STEM-related disciplines are taught in EU Lower Secondary Education and the twentieth-first century skills required by students to face real life situations in STEM-related studies and careers;
• Put knowledge to innovative use: This is achieved through the design, pilot testing and implementation of a TPD program which aims to familiarize teachers with the potential of AR technology for enhancing the teaching and learning processes in lower secondary STEM education;
• Increase robustness and systematic nature of design practices: Establish a set of design principles for implementing teacher professional development programs for teaching STEM-related topics through inquiry-based approaches using AR technology.

It is important to mention that, given the complexity of an EDR approach, the current research could set the basis for wider and long-term research towards this direction in the future. At this stage, the suggested TPD program has been implemented two times, including the EDR phases of analysis, design and evaluation: (i) As a TPD program available for individual in-service Lower Secondary Education teachers of STEM-related courses in Cyprus (wide call for attendees during February 2019–May 2019); and (ii) as an improved TPD program in the context of specific schools in Cyprus and Greece (September 2019-February 2020), where cultural backgrounds appear to be similar [42].

The Research Questions (RQ) guiding the current research are the following:

• RQ1. Which factors influence the level of technology acceptance (AR) by teachers in their instructional approaches within a STEM-related course?
• RQ2. What is the effect of instructional approaches supported by AR in STEM-related courses on students’ twentieth-first century skills and motivation towards the educational process?

3. Data Collection and Analysis

The core target group of this research includes Lower Secondary Education teachers of STEM-related courses, having attended the TPD program. As already mentioned, in EDR, the data collection process is continuous and repeated [38]. The data collection process in this study has been taking place since the preparation phase, throughout the TPD program and during the final part, when some of the teachers finally designed and applied their interventions in their classrooms and/or school laboratories (guided field practice). The TPD program has been repeated twice: (1) An initial phase in Cyprus for individual in-service Lower Secondary Education teachers of STEM-related courses (wide call for participants), and (2) a second improved phase, both in Cyprus and Greece, in the context of specific schools, based on the Technological Pedagogical and Content Knowledge (TPCK) approach [43]. The total number of participating teachers is twenty-five, while six of them eventually applied AR in their STEM-related courses.

In addition to the teachers, students participating in the implementations of STEM-related courses supported by the technology of AR, are also a target group of this research. The implementations took place with students between 12 and 15 years old, which corresponds to Grades 1–3 of lower secondary education (in Cyprus and Greece: Gymnasium). The current research focused on three classes of twenty students (sixty students in total) that participated in the above-mentioned interventions implemented by the six teachers.

The current research uses multiple data collection tools with the purpose of: (a) Triangulation while seeking convergence of findings, and (b) expansion to extend the breadth and range of inquiry. Because of the complex nature of this study, the collection of both qualitative and quantitative data was deemed necessary in order to reach more in-depth results. It is necessary in data collection efforts, including personal views of the target groups, to apply more than one method of obtaining data, in order to ensure a broad perspective [44]. The results are not intended to be generalized at a population level but at a level of theory, meaning that it could be transferable to some extent and applied by other researchers in the field of educational research. Since teachers’ stated beliefs towards technology integration do not always align with their instructional practices [45], they have been triangulated to achieve better understanding, with [46]: (a) The responses to teachers’ questionnaires concerning the TPD program and Technology Acceptance (in this study, augmented reality), including their stated intention to apply AR in their classrooms, (b) teacher interviews’ content, (c) designed Lesson Plans (LPs), and (d) researchers’ observation of the instructional practices in authentic environments. Similarly, when referring to students, since their stated beliefs do not always align with their motivation, performance and twentieth-first century skills acquired, their statements have been triangulated with: (a) The students’ performance assessment (worksheets, achievements, tests, etc.) provided by their teachers, (b) interviews’ content, and (c) researchers’ observation of the learning process in authentic environments.

3.1. Data Collection Methods

All teachers participating in the study were asked to complete an anonymous online questionnaire based on the Technology Acceptance Model (TAM) [47] at the end of the TPD program. This questionnaire consists of two parts: (a) Questions concerning teachers’ background (personal data, personal beliefs/experiences on STEM concepts and pedagogical approaches, personal beliefs/experiences on innovative technologies); and (b) statements on a Likert scale from 1 to 5 (1—for strongly disagree and 5—for strongly agree) categorized to self-efficacy, social influence, anxiety, performance expectancy, effort expectancy, attitude towards AR and facilitating conditions. It is based on the TAM questionnaire [47], which has already been translated and administered to in-service STEM teachers [48], in the context of secondary education in Greece [49], and specifically referring to AR technology [50]. The questionnaire has been developed with the Lime Survey tool and has been distributed online, to enhance the data analysis process.

Semi-structured interviews took place with six teachers that expressed their willingness to share their views and experience concerning the TPD program, as well as their intention to apply AR in their classrooms and their confidence in teaching twentieth-first century skills. These teachers also integrated AR in their classrooms, and shared their experience through the interviews. Interviews with open-ended questions highlight issues that cannot be detected through the restricted questions/statements of the questionnaires and allow researchers to focus on specific issues [51]. Additional interviews informally took place during the TPD program (individual discussions, online forum comments, comments/feedback during the face-to-face workshops, etc.), as well as before, during and after the implementations in the teachers’ classrooms. A school principal was also interviewed after having attended some of the teaching interventions conducted by teachers in her school. The principal’s perspective was deemed very important, as she represented the background support necessary for the teachers to be able to implement innovative approaches in their classrooms. All interviews were conducted in Greek, in order to allow the teachers to freely express themselves (without any language constraints) and to encourage their feeling of comfort. Some of the questions that the teachers replied to are presented in Appendix A.

Finally, in this research, the observation was conducted in collaboration with the teachers and the students, though building solid relationships in their natural setting (school’s classrooms and laboratories). The research questions were used as specific indicators guiding the observation process on where to “start looking at”, however the “focus or stop action cannot be determined ahead of time” [52]. Observation involves active looking, informal interviewing and taking detailed notes [53], as well as an online collaboration environment (OneNote MS Office) for the preparation of the Lesson Plans. Observation is important when [54]: (a) Little is known about the topic under research, (b) the understanding of a concept/issue in a detailed way is valuable, and (c) it is important to study the topic under research in its natural environment, all responding to the aims of the current research.

In

Table 1. Data collection methods and relevant research questions.

Who?	No	Tools	RQ
Teachers who attended the TPD program	25	Technology Acceptance Model questionnaire (after attending the TPD program)	1
		Informal Interviews-Discussions (before, during and after the TPD program)	1
		Observation during the TPD program	1
Teachers having applied AR in their classrooms	6	Personal Interviews (before, during and after the educational implementations in STEM-related courses supported by AR)	1, 2
		Observation during the educational implementations in STEM-related courses supported by AR	1
		AR Lesson Plans designed and implemented in the teachers’ classrooms	1, 2
Students who attended the educational implementations in STEM-related courses supported by AR	60	Observation during the educational implementations in STEM-related courses supported by AR	2
		Interviews during and after the educational implementations in STEM-related courses supported by AR	2
		Worksheets, achievements, tests (performance data defined by their teachers)	2

, the different data collection methods are described, as well as the correlation with the target groups and the research questions mentioned above.

3.2. Data Analysis

It is important to mention that the current research data were collected as a part of a larger effort to answer the research questions. This research paper’s results and conclusions focus on the teachers’ perspective during the TPD program. Additional data are expected in the future, as well as the analysis of the collected data, needs to be continued in the context of the EDR approach for more in-depth conclusions and directions for future research.

The collected data were analyzed both simultaneously while being collected and retrospectively [55]. More specifically, simultaneous analysis occurred mainly during the observation processes, as the researchers kept notes in the authentic environments, where on-the-spot ideas emerged. Moreover, the simultaneous analysis took place during the activities throughout all the EDR phases, between the researcher and the teachers, as well as the students. This analysis provided constant feedback for changes and improvements concerning the integration of AR technologies in STEM-related disciplines. Finally, interviews were analyzed shortly after their conduct, in order for the researchers to be in a position to recall all necessary details. Both deductive and inductive approaches were employed to enhance the organisation of the data and highlight concepts that made these data valuable [56]. More specifically, from a deductive perspective, the collected data were analyzed according to existing theoretical frameworks, based on the research questions posed by the researchers. For example, as the literature review indicated that the integration of AR technologies in STEM education would allow teachers to teach STEM-related topics that are difficult to explain, data highlighting these processes were actively sought after. Inductive data analysis was also done, since it is also important for the completion of a mixed study, enabling the emergence of critical data through the patterns discovered during the analysis and the construction of a theory. For the initial analysis of the qualitative data collected, NVivo software has been used. It is critical to mention that at this point of the study, the data have been partially processed and further analysis is underway.

4. Description of the Teacher Professional Development Program

The TPD program focuses on how to effectively implement inquiry-based instruction within the school curricula through the functional integration of AR with existing core curricular ideas. The theoretical framework underpinning the TPD program is grounded on the interrelated bodies of Problem-Based Learning, Inquiry-Based and Contemporary Learning Approach, promoting scaffolding and collaboration in STEM education [21]. In addition, TPCK (Technological Pedagogical and Content Knowledge) [56] has been applied as a research framework for facilitating and assessing teachers’ professional development in the use of ICT in STEM education [21].

The main objectives of the TPD program were: (a) The utilization by participating teachers of existing AR Learning Objects (Los) in their classrooms, and (b) the development of their own AR Los and Lesson Plans (LPs) with appropriate tools and applications [24]. These objectives should be achieved through the development of a supportive culture, motivating teachers of STEM-related courses to effectively integrate AR with core STEM curricular ideas and to transform their classrooms and/or laboratories into a smart-learning environment [21]. A blended approach has been adopted, including both face-to-face training workshops to encourage communication, and develop a personal connection with the participants, as well as the EL-STEM project online course, to offer teachers time flexibility and ease of access to additional content (articles, videos, etc.). The TPD program structure and content is summarized in

Table 2. General description of the TPD program.

	Description
EL-STEM online course	The estimated time to navigate through the modules and get an overall idea regarding the content is about seven hours. Depending on the level of detail a teacher wants to reach, much more time may be necessary to study all the suggested additional content (readings, videos, etc.).
Face-to-Face Workshops	Five meetings, a total duration of fifteen hours (3 h each) First meeting: Introduction to STEM, inter-/multi-disciplinarily, trends in STEM studies/careers—introduction to STEM instructional and learning approaches—real STEM examples. Second meeting: Introduction to virtual, augmented and mixed reality—presentation of different applications/tools for creating augmented reality content (i.e., HP Reveal, Metaverse) – Real examples of using augmented reality in Education—Practice Workshop. Third meeting: Practice workshop with applications/tools (i.e., ARTutor [57], ready-made AR educational material)—action research: Link the workshop with the EL-STEM project and encourage the teachers to participate in the data collection process—teachers design their own STEM Lesson Plans on a topic of their choice (individually or in groups) applying augmented reality Technology. Fourth meeting: Development of AR Learning Objects—real examples of using augmented reality in Education—integration of AR Learning Objects within the STEM Lesson Plans designed. Fifth meeting: Discussion and Reflections on the applications applied in the teachers’ classrooms—sharing AR Lesson Plans and Learning Objects—encouragement to attend the online “AR STEM teachers’ community”

.

Between February and May 2019, twenty individual teachers participated and completed the TPD program to get equipped with the required knowledge and competences to scaffold their students in engaging with problem solving, inquiry-based activities and innovative technologies, such as AR, aiming to raise their interest in STEM and promote the attainment of important twentieth-first century skills [21]. Several teachers applied AR into their classrooms, while two of them from a public school in Cyprus, allowed the researchers to collect data in the context of their school unit.

After completing the initial phase, researchers took into consideration the feedback received and decided to repeat the TPD program with five teachers of a private school in Greece, that expressed their interest in the TPD program. The main factors that led the researchers to this decision were: (i) The support by the school administration in the school context (compared to individual teachers of the first phase that tried to apply AR in their courses without the necessary support structures in their school); (ii) the flexibility and opportunities for collaboration between teachers working in a specific school (compared to individual teachers of different schools facing difficulties in collaboration, such as time and location constraints); and (iii) the existing equipment in a private school compared to a public school. Thus, a total of twenty-five teachers completed the TPD program during the two phases, twenty in Cyprus and five in Greece.

During the face-to-face meetings, teachers were provided with ample opportunities for interactive and collaborative learning through the use of contemporary technologies and related equipment (i.e., smartphones, tablets, cardboards) and engaged in authentic collaborative educational activities. They also got familiarized with different tools and applications for developing AR Los within STEM-related courses, such as HP Reveal, ARTutor [57], Metaverse, Scratch and Unity. The instructional strategies applied during the TPD program, included open-ended investigations, AR visualizations, collaboration and reflection on one’s own and on others’ ideas and experiences, providing a learning environment that enhanced participating teachers’ understanding of STEM, AR supported education, and interdisciplinarity. Moreover, the learning environment served as a model to the teachers as to the kind of learning situations, emerging technologies and curricula they should employ in their own classrooms [58].

The final part of both TPD programs included guided field practice. Teachers were expected to expand upon the digital tools and the instructional material provided to them, and apply them in their own classrooms, through design and implementation of Lesson Plans and AR Learning Objects. These AR Los were not expected to be designed to enhance extra-curricular activities, but instead, to become an important part of the main curricula and also to consist of reusable educational material for other teachers to utilize. The researchers acted as mentors, providing their support to teachers. A total of six teachers finally integrated AR in their STEM-related courses (two from Cyprus and four from Greece).

5. Results

This section provides initial results concerning both teachers who had attended the TPD program, and students who had attended STEM-related lessons supported by AR technology, designed and implemented by teachers trained during the TPD program. It is important to mention that the current research is still in progress and the analysis of the collected data needs to be continued for more in-depth conclusions and directions for future research. This research paper’s results and conclusions focus on the teachers’ perspective.

5.1. Teachers

As already mentioned, twenty-five teachers participated and completed the TPD program. Fifteen were women and ten men. Around half of the participants (48%) were between 36 and 45 years old, while a smaller proportion were 35 or younger (22%), or older than 45 (30%). A high proportion of the attendees (more than 50%) were highly educated (at least Master’s degree), while all had a teaching experience of more than five years. All participants were teaching STEM-related courses, such as Mathematics, Science, Chemistry, Computer Science/Technology, Biology, Physics and Geography. Two of them were primary level teachers who asked to attend the training and were allowed by the project coordinator, in the context of promoting innovative technologies across educational levels. Another important characteristic was that a high percentage of the participants (more than 70%), had attended a TPD program in the past in topics such as STEM education, inquiry-based learning, e-learning, new technologies in education and teaching skills.

The following results are related to the first research question “Which factors influence the level of technology acceptance (AR) by teachers in their instructional approaches within a STEM-related course?”.

5.1.1. Teachers’ Initial Beliefs Concerning STEM Education, Interdisciplinarity and Innovative Practices

In the questionnaire given to the teachers at the outset of the study, eighty percent of the respondents gave responses indicating that they had already heard or had attended a relevant TPD concerning the topic of “STEM education”, thus, they knew what the acronym STEM (Science, Technology, Engineering, Mathematics) stands for, as well as alternatives of STEM, such as STEAM (A refers to Arts), STREAM (R refers to Reading), etc. However, when asked to indicate “How [they] understand/realize the concept of STEM”, their responses included key words such as: “innovative/better teaching”, “interdisciplinarity”, “new technologies”, “combination of lessons”, “something very progressive”, “technology in science”, “problem solving”, “the use of ICT in the learning process” and even “Virtual reality and AR”. This could be interpreted as a “blurred” understanding of the concept of STEM education.

All teachers mentioned that they were familiar with innovative pedagogical approaches, such as inquiry-based learning, and quite familiar with methods for supporting critical thinking, team collaboration and computational thinking. Additionally, they all had heard of innovative technologies, such as virtual reality, augmented reality and mixed reality, but did not know how to apply them in the educational process. As far as interdisciplinarity is concerned, during the interviews conducted at the beginning of the TPD, secondary education teachers seemed to feel safer when teaching their own course following the guidelines and indicators provided by the curriculum. Many teachers preferred teacher-centered approaches to keep the “class under control” and remain within the boundaries of the curriculum, since they felt that students may misbehave while working in groups or discover knowledge on their own. When referring to twentieth-first century skills, all teachers responded in ways that indicated some awareness of this concept, referring to key words, such as “problem-solving”, “critical thinking”, “computer skills”, “collaboration”, “teamwork”, etc. Forty percent of the teachers identified twentieth-first century skills as skills relevant to new technologies. Thus, similarly to the STEM concept, teachers had a “blurred” understanding of twentieth-first century skills. However, the majority (more than 85%) stated that their instruction included learning objectives designed to promote twentieth-first century skills and felt confident that they did help their students enhance their twentieth-first century skills.

In the question about the main challenges preventing teachers from making STEM-related courses more attractive and engaging for students, more than 65% mentioned the “lack of a single national policy concerning STEM-related courses”, “lack of professional development in the field of STEM education” and “lack of time to cooperate with other teachers”. Other challenges included the “lack of appropriate equipment” and “outdated textbooks”, as well as the fact that in both countries STEM-related courses are not included in the national curriculum.

5.1.2. Teachers’ Beliefs Concerning STEM Education, Interdisciplinarity, Innovative Practices and Augmented Reality during and after Attending the TPD Program

As far as the TPD program is concerned, more than eighty percent “strongly agreed” or “agreed” with the fact that they were satisfied with the content and the topics presented, as well as the overall implementation. Moreover, they “strongly agreed” or “agreed” with the fact that the pedagogical approaches presented are applicable in their schools, and that the software/applications presented could be exploited in their classrooms. The rest (20%) did not mention the reason for not agreeing, however, during discussions with them, teachers seemed to have higher expectations concerning the current situation of AR technology in education and/or expected to obtain professional skills concerning 3D design and programming in the context of the TPD program. This finding corroborates with the literature [18], which indicates that the development pace of technology and innovative devices far exceeds the development pace of teachers integrating each technological innovation within the educational process, something which creates feelings of anxiety and insecurity. Finally, eighty percent of the teachers having completed the TPD program mentioned that they intended to integrate AR in their courses, using the applications/tools they were familiarized with during the workshops. Only three expressed their hesitation, referring to the need for schools’ support (existing equipment and school administration persuasion on adapting innovative technologies) and lack of time.

During the design of STEM-related Lesson Plans, collaborating with others and discovering common axes between STEM-related courses proved to be quite difficult and time consuming for teachers. Individual teachers participating the TPD program during the first phase, mentioned various difficulties, especially in case they were not strongly supported by the school’s administration. Teachers’ collaboration in the context of the private school (second phase of the TPD program) proved to be more flexible and feasible, since most of the teachers had already been collaborating in the context of other school activities (such as EU projects, competitions, etc.), while they were also meeting every week during “teachers’ meetings” to discuss about any issues relevant to the educational process. Teachers of the private school had additionally a motivation, which is the added value they offer to the school through their activities (e.g., a classroom intervention support by AR is also a promotional activity for the school, thus, a teacher is rewarded for implementing it). Teachers’ support and positive attitude, in order to promote interdisciplinarity not only in STEM-related courses, but also in courses of more theoretical fields, such as history and language learning, proved to be of additional value. Moreover, the role of the school administration (i.e., school principal) seemed critical in promoting collaboration among teachers of different disciplines.

Finally, some teachers, including the school administration, were initially hesitant with the idea of asking students to use their mobile devices in the classrooms, being worried that students would make personal instead of educational use of the devices. This proved to be an unfounded concern, since students followed the teachers’ instructions, making reasonable use of their mobile phones during the STEM-related course. In the case of the private school, using mobile devices during the educational process was widely accepted, while the school provides the students with the necessary equipment (tablets) during their classes. It is important to mention that both in Cyprus and Greece, mobile devices (especially smartphones) are not allowed for personal use; they could only be used under specific circumstances for educational purposes with special permission from the Ministry of Education.

5.1.3. Teachers’ Beliefs and Practices Regarding AR (Technology Acceptance)

Teachers initially stated that they were already familiar with other technologies and used them in their courses, thus, they felt comfortable with the idea of using AR. Their initial expectations concerning augmented reality did not match reality. In fact, they believed that they could easily create impressive 3D content without spending considerable time—or just using ready-made material from various sources, thus, at some point, they were disappointed with the current situation of AR technology as an educational tool.

During the TPD program, participants were really impressed when they used AR for the first time. The more complex and “professionally-developed” the AR object appeared, the more interesting they found it to be. A significant proportion of teachers (more than 80%) confused augmented reality with virtual reality, therefore viewing simple digital objects, such as videos or additional information in the real environment, was not attractive for them. Teachers mentioned that AR is a promising technology that could be of additional value for their students in the future, provided that more educational content relevant to the curriculum becomes available, and that teachers get high quality training on AR-supported pedagogical approaches. Moreover, they mentioned that they needed less time than expected to get familiarized with the applications/tools for developing AR educational content.

At least eighty percent of the teachers who attended the TPD program mentioned that they intended to integrate AR technology in their classrooms, either in collaboration with other teachers (interdisciplinary approach) or in the context of their own STEM-related subjects. Some also stated that they could make their own subject more interdisciplinary by adding educational content from other disciplines using AR technology. For example, a teacher described a STEAM idea of teaching (during her own course) the Pythagorean Theorem (Mathematics) through the romantic story of Romeo and Juliet (Arts and Humanities), explaining the forces developed between the ladder and the balcony (Physics) and guiding the students during the process of drawing these forces using a math-software (Geogebra). A common comment expressed by the majority of teachers was that the school administration should encourage the teachers to attend TPD programs on the topics of innovation and new technologies in education. Moreover, it was pointed out that it is important for secondary education schools to have a high-level support (directorate of secondary education, ministry of education, etc.) in order to apply innovations in the educational process. Individual efforts, as well as projects, where a specific number of schools are involved, are of additional value but cannot bring a substantial change.

Table 3. TAM questionnaire statements.

Category	Statement
Self-efficacy (SE)	I feel confident that I can select the appropriate AR tool (application/software/online platform) to use in my teaching I feel confident that I can use augmented reality in my class I feel confident that I can use augmented reality in my courses/lessons to meet certain learning objectives I feel confident that I can create my own AR Learning Objects using non-programming tools (e.g., HP Reveal, Metaverse) I feel confident that I can create my own AR Learning Objects using programming tools (e.g., Unity)
Social Influence (SI)	I often exchange ideas with other teachers about augmented reality integration in teaching and learning Other teachers encourage me to integrate augmented reality in teaching and learning The school administration encourages me to integrate augmented reality in teaching and learning There are other teachers in my school who use augmented reality in teaching and learning Online Communities encourage me to integrate augmented reality in teaching and learning
Anxiety (AN)	The use of new technologies in teaching scares me The use of augmented reality in teaching scares me The use of new technologies in teaching stresses me out The use of augmented reality in teaching stresses me out If something goes wrong in the classroom while using augmented reality I will NOT know how to fix it
Performance Expectancy (PE)	Augmented reality helps students understand STEM-related concepts in more effective ways Augmented reality helps teachers to teach in more effective ways Augmented reality will change the way students learn in STEM-related courses Augmented reality will change the way I teach in my STEM-related course Augmented reality is a valuable technology for teachers
Effort Expectancy (EE)	Augmented reality is not conducive to students’ learning because it is NOT easy to use Augmented reality is not conducive to teaching because it creates technical problems It is easy to use augmented reality for teaching and learning Getting prepared for teaching a STEM-related lesson/course supported by augmented reality is time-consuming It is easy to think of activities supported by augmented reality with added value for my students in the context of STEM-related courses
Attitude Towards AR (AT)	The use of augmented reality in STEM-related courses excites me I like to use augmented reality for teaching My students like to use augmented reality for learning My students are willing to repeat AR experiences in their lessons/courses Getting prepared for teaching a STEM-related lesson/course supported by Augmented reality excites me as a process
Facilitating Conditions (FC)	The technical support in my school is adequate The technical infrastructure in my school is adequate Students use their own mobile devices in STEM-related courses supported by AR The instructional support for augmented reality in my school is adequate TPD programs within my school facilitate the integration of augmented reality in my teaching

includes the TAM questionnaire statements with an agreement scale from 1 to 5 (1—for strongly disagree and 5—for strongly agree) categorized to self-efficacy, social influence, anxiety, performance expectancy, effort expectancy, attitude towards AR and facilitating conditions.

Ιn the self-efficacy category, 50% of the teachers replied that they quite agree (level 3) with the statements “I feel confident that I can use augmented reality in my class” and “I feel confident that I can select the appropriate AR tool (application/software/online platform) to use in my teaching”. An interesting statement was “I feel confident that I can create my own AR Learning Objects using non-programming tools (e.g., HP Reveal, Metaverse)” where 60% replied that they agree or strongly agree (4 or 5), while in the statement “I feel confident that I can create my own AR Learning Objects using programming tools (e.g., Unity)”, only 35% of the teachers replied with 4–5. This indicates that teachers feel more confident when using non-programming tools/applications, while some that have the appropriate background (e.g., Computer Science teachers) are willing to experiment themselves with programming tools.

Social influence statements revealed that teachers have the support of the school administration to apply AR in their courses (this was also the reason for attending the TPD program). In addition, 60% of the teachers replied that they agree or strongly agree with the statement “I often exchange ideas with other teachers about augmented reality integration in teaching and learning”. This indicates that teachers who attended the TPD program, function as multipliers in their schools, exchanging ideas and collaborating with others. However, only 25% agreed or strongly agreed (4–5) with the statement that “Other teachers encourage me to integrate augmented reality in teaching and learning”. During the interviews, teachers were asked about this statement and they clarified that other teachers in their schools are not aware of AR, thus, they cannot encourage them to use this technology in their classrooms.

A critical factor is the anxiety of applying AR in the educational process. The majority of the teachers (85%) disagree or strongly disagree (1–2) with the statements “The use of new technologies in teaching scares me” and “The use of augmented reality in teaching scares me” (similarly for stresses me out), thus, they feel safe with new technologies and AR. During the TPD program, more details have been discussed concerning anxiety and some interesting statements were “I don’t feel anxious, but I feel overloaded with new technologies”, “I feel that we have to fit in the technology while this should apply the other way round”.

As far as the performance expectancy is concerned, quite all teachers (85%) agree or strongly agree with statements, such as “Augmented reality helps students understand STEM-related concepts in more effective ways” and “Augmented reality will change the way students learn in STEM-related courses” referring to their students. Additionally, none disagrees or strongly disagrees with statements referring to the added value in the teaching process, such as “Augmented reality helps teachers to teach in more effective ways”, “Augmented reality will change the way I teach in my STEM-related course” and “Augmented reality is a valuable technology for teachers”. Thus, teachers recognize the added value of AR and have expectations for the future.

In the effort expectancy category, all teachers agreed or strongly agreed (4–5) that it is time-consuming to get prepared for teaching a STEM-related lesson/course supported by augmented reality, while “It is easy to use augmented reality for teaching and learning”. Sixty percent quite agreed (3) that “it is easy to think of activities supported by augmented reality with added value for my students in the context of STEM-related courses”.

Concerning the attitude towards AR, the majority of the teachers (80%) agree or strongly agree (4 or 5) with statements, such as “The use of augmented reality in STEM-related courses excites me” and “Getting prepared for teaching a STEM-related lesson/course supported by augmented reality excites me as a process”. Similarly, they agree or strongly agree with statements referring to their students, such as “My students like to use augmented reality for learning” and “My students are willing to repeat AR experiences in their lessons/courses”. Combining the two categories of effort expectancy and attitude towards AR, teachers agree that much effort is necessary for preparing an AR supported STEM-related course, however, the experience is exciting for both students and teachers and they are willing to use AR in the future.

Finally, as far as facilitating conditions are concerned, all teachers agree or strongly agree that the technical infrastructure and support in their schools is necessary, while TPD programs could facilitate the integration of augmented reality in their teaching.

5.1.4. Teachers Having Applied AR in Their Classrooms

Taking into consideration the teachers that actually applied AR in their STEM-related courses (six of them were observed during their classes), technical issues were also under control (more than initially expected) thanks to close collaboration with the computer science teachers in their school, and assistance from the project consortium. Moreover, the better prepared a teacher was for dealing with technical issues (e.g., having a list of “to do” tips when a device cannot support an AR application), the smoother became the integration of technology into a STEM-related course. All teachers tried the AR content in their devices before implementing it in their classrooms. One teacher that omitted this action faced some issues with the school tablets because they were not updated on the day of the implementation.

The fact that teachers could not “save/store” data on their students’ progress (wrong/right answers, grades, etc.) was mentioned as a negative aspect of the existing AR applications. However, a teacher noted that using AR in her course gave her the opportunity to evaluate her students’ current “knowledge and skills level” in an easy, fast and pleasant way, avoiding the stressful process of a test or a worksheet. In fact, students indirectly got evaluated without them realizing it (participation in classroom, teamwork, etc.).

Teachers that applied AR in their courses, mainly used the technology as a tool for representing additional information and digital objects (videos, 3D objects, quizzes, 360° illustration, etc.) instead of promoting twentieth-first century skills. However, they recognized this limitation and mentioned that if they get more familiarized with the technology and have more options of existing AR educational content, they could apply the technology in more creative ways, aiming to interdisciplinarity and to the promotion of twentieth-first century skills. Finally, teachers mentioned that the additional time needed to design and prepare a STEM Lesson Plan supported by AR is well-worth it, given the positive impact that AR integration could have on students’ motivation and learning. The most difficult part for a teacher, they noted, is to make the decision and put an interdisciplinary idea into action. Reusability of the educational content in many classes and during other school years are some of the reasons that teachers believe it is worth trying.

5.2. Students

The second research question focused on students “What is the effect of instructional approaches supported by AR in STEM-related courses on the students’ twentieth-first century skills and motivation towards the educational process?”. About sixty students (three classes of 20 students) between 12 and 15 years old, which corresponds to students of the first and second class in lower secondary education (in Cyprus and Greece: Gymnasium) have been observed while participating in STEM-related courses supported by AR. The classes were characterized by their teachers as “average level”. Students worked in groups, following the AR Lesson Plans designed by their teachers in Mathematics, Physics and STEM topics. They were asked to use their own mobile phones to view the AR content or the school’s tablets and then, to put them aside in order to engage in relevant activities or to complete worksheets prepared by the teachers. At the end of the lesson, teachers discussed the experience with the students. The above-mentioned interventions were observed by the researchers.

Teachers highlighted positive aspects in their students’ participation and motivation during the educational process. Characteristically, they mentioned that students who were typically paying little or no attention in class, worked efficiently with their classmates when AR was introduced, trying to solve the worksheets’ exercises and in one case, even leading their team. According to the teachers, this was probably the first time that some of those students raised their hands to answer the teacher’s questions. Moreover, teachers mentioned as a positive aspect the fact that students could repeat the projection of the AR object as many times as they wished, even outside the classroom, just by using their mobile phones with the relevant trigger images. Getting deeper into unexplored worlds and concepts was also mentioned as a positive aspect of AR that could be of additional value in STEM-related courses (especially physics and chemistry). However, an important factor influencing the above-mentioned positive aspects and distracting students’ attention and interest is the stability/reliability of the AR application used to view the educational content. The more issues students faced (e.g., too much effort to focus the mobile device on the AR object), the more they got distracted and stopped trying.

Concerning the students’ progress, applying some interventions during the whole school year was not enough to reach safe conclusions regarding the impact of the EL-STEM approach on students’ learning. However, teachers noticed that some of their students could more easily recall the topics taught with AR (photographic memory), something which proved helpful in achieving higher scores in their exams. Moreover, teachers mentioned that completing worksheets with activities relevant to the AR technology, were motivating for students with lower performance to complete. Thus, their performance was surprisingly higher than expected, since those students usually submit “blank pages”.

6. Discussion

Many studies have highlighted the potentiality of AR technology in secondary education, as well as the fact that there is an increased interest and a need for further research in this field [23,26,27]. The provision of high quality pre-service and in-service teacher training that will equip teachers with the required knowledge and skills to effectively apply AR into teaching and learning is of utmost importance to research efforts in this field [31]. One innovative aspect of this research is the fact that teachers were encouraged to apply AR technologies using interdisciplinary approaches, instead of focusing on their individual subjects. Moreover, teachers were encouraged to choose between different AR tools/applications, selecting those that best fit their needs. School administration also consisted a critical factor for applying AR in STEM-related courses. Finally, the current research provides a context of empirical investigation through the Educational Design Research methodology, yielding a theoretical understanding of the fields of this research that can constitute the basis for future work.

The implementation of the TPD program confirmed the fact that changing teaching practices is proving difficult, while many teachers remain unprepared to effectively employ technology-enhanced teaching practices [15]. In addition, many teachers remain hesitant to use AR in their instruction for reasons, such as (i) lack of confidence in their use, (ii) lack of time or motivation for acquiring new technological skills and adapting new pedagogical strategies, (iii) lack of existing educational resources, and (iv) the fact that they feel uncomfortable with student-centered approaches enhanced by new technologies [15,16,17]. Moreover, teachers’ initial expectations concerning augmented reality did not match reality [15]. It is also important to mention the critical role of school administration for effectively applying AR in a school, as well as the need for the provision of technical support. Some of the obstacles, according to the participating teachers, that schools need to overcome to successfully integrate AR within the educational process are: (a) The WiFi connection (not all public schools in Greece and Cyprus have wireless internet), and (b) the usage of mobile phones/devices (such as tablets) in schools (mobile phones are not allowed by the Ministry of Education in Greece and Cyprus—they can only be used by students for educational purposes after receiving a special permission—while schools are not equipped with mobile devices).

Referring to the RQ1 “Which factors influence the level of technology acceptance (AR) by teachers in their instructional approaches within a STEM-related course?”, and taking into consideration the research of Dalim et al. [31], the factors mentioned in

Table 4. Factors influencing the level of Technology Acceptance (AR) by teachers.

Factor	Comments
Curriculum and National Policy	STEM-related courses are not included in the national curriculum of many countries (including Greece and Cyprus) The usage of mobile devices (necessary for integrating AR in education) is not allowed in some countries (including Greece and Cyprus) More AR educational content relevant to the curriculum is necessary Teachers need to get high quality training on AR-supported pedagogical approaches and tools/applications for using and/or creating AR educational content
Technical Issues – Equipment	Technical support provided in the school is of high importance (e.g., Computer Science teachers supporting teachers of other fields) Existing infrastructure is necessary to allow teachers to apply AR in their classrooms (e.g., WiFi connection, mobile devices, other AR equipment) The better prepared a teacher is for dealing with technical issues (e.g., having a list of “to do” tips when a device cannot support an AR application), the smoother is the integration of AR into a STEM-related course Teachers need to feel confident that they will be able to cope with technical issues that could arise during a STEM-related course supported with AR technology
AR tools/applications	The stability/reliability of an AR application/tool is important during the implementation Teachers feel more confident with tools/applications that do not require programming skills Teachers need AR tools/applications able to “save/store” data concerning their students’ progress (learning analytics)
School Administration	Teachers need the support of their school administration to apply not only AR technology, but also any innovation during the educational process Administration (i.e., school principal) plays a critical role in promoting collaboration among teachers of different disciplines School administration has a key role in encouraging teachers to attend TPD programs and experiment themselves with new teaching practices and technologies
Teacher’s confidence with technology	Teachers that have attended TPD programs in the past concerning innovative technologies, tend to feel more confident when dealing with any new technology, including AR Teachers of STEM-related courses, especially Computer Science teachers and Technology teachers, tend to feel more confident in applying AR and working as multipliers in their schools by helping other teachers Teachers with strong technological backgrounds are willing to get familiarized with AR tools/applications that require programming skills
Teacher’s motivation	Teachers with specific motives are more likely to try innovative technologies, including AR, in their STEM-related courses (e.g., teachers aiming to promote their schools by being promoted themselves as “innovative teachers”, which is important for their professional profiles) Since integrating AR within STEM-related courses has been characterized as time-consuming, teachers need incentives for devoting time on this technology (e.g., if they spend time on AR they can reduce their time on something else in the school context, such as administrative duties)

have been highlighted in this research as influencing to some extent the acceptance of AR technology by teachers within the educational process. In Table 4, these factors are listed along with some comments regarding the teachers’ statements which led to the prominence of the specific factors.

As Dalim et al. [31] mentioned, there are also other factors that could affect AR acceptance, such as parents’ involvement and students’ background. However, this study focuses on the teachers’ perspective.

Finally, referring to the RQ2 “What is the effect of instructional approaches supported by AR in STEM-related courses on the students’ twentieth-first century skills and motivation towards the educational process?”, the initial implementations within STEM-related courses, have indicated possible effects on the students’ twentieth-first century skills, such as collaboration and digital skills, as well as their motivation towards the educational process and their progress in STEM-related courses. However, this study focuses on the teachers’ perspective, thus, further investigation needs to be implemented towards this direction in order to reach some conclusions.

7. Conclusions

This research has highlighted some factors that may influence, to some extent, the acceptance of AR by lower secondary education teachers in STEM-related courses. As already mentioned, the current paper focuses on the teachers’ perspective, relevant to RQ1. However, the initial implementations within classrooms have indicated possible effects on students’ twentieth-first century skills as well as their motivation towards the educational process and their progress in STEM-related courses (RQ2).

One of the main objectives of the TPD program, hopefully, achieved to some extent, was for the teachers to realize that they are key persons in the process of designing and developing AR-supported educational activities and material within STEM-related disciplines, but also in the classroom implementation process where they need to provide the necessary instructional support to their students to maximize the impact of such technologies [21,24]. Teachers were also encouraged to recognize the added value that AR could have on teaching, learning, and assessment, while being informed about best practices in their exploitation as instructional tools [28]. The study’s aim was not to verify but to indicate what difficulties have been experienced by the teachers during the integration. Finally, an important conclusion concerning teachers is their belief that AR is a promising technology that could be of additional value for their students in the future, provided that more AR-enhanced educational content relevant to the curriculum becomes available, and that they get high quality training on AR-supported pedagogical approaches. As far as students are concerned, the initial AR interventions implemented by teachers in their STEM-related courses confirm existing studies indicate a positive impact on students’ motivation and classroom participation levels [22,23,25,26,27]. Additionally, there are some initial indications in the data that AR also led to improvements in learning outcomes, such as the fact that students could more effectively recall the concepts, they had been taught with AR. Further research focused on learners ought to be conducted in order to shed light into the actual impact of the AR interventions on student learning.

Referring to the limitations of the research, the subjectivity of the research participants, as well as their attitudes and perceptions, might have contributed to some bias that influenced the study’s level of validity. Moreover, participants may have “overtried” to respond to the questions, in an attempt to satisfy the researchers’ expectations. However, reliability was enhanced in the current research by applying multiple data collection methods and triangulation during the analysis process [38]. In addition, the fact that the process of iteration and analysis can lead to the collection of a large amount of data, which includes the risk of focusing only on data that specifically support the researcher’s theory [39], was also taken into consideration. This challenge was, to some extent, mitigated by clearly defining the scope and goals of each design enactment and ensuring that the data collection and analysis methods were aligned with the research questions.

The sample size employed in the study is too small to draw any generalizations, while also EDR studies need years of repeated applications to reach safe conclusions. The results are not intended to be generalized at a population level but at a level of theory, meaning that they could be transferable to some extent and applied by other researchers in the field of educational research. The results of this study could be transferable, while the researchers suggest further study into the subject, and present their conclusions as future directions.

It is important to point out that the current qualitative research is still in progress, and aims to highlight the added value of applying augmented reality within the educational process, as well as to identify the factors that influence the integration of AR technology in the educational process. The teachers who attended the TPD program functioned as multipliers in their schools, training other teachers on the topics of STEM education, interdisciplinarity, twentieth-first century skills and augmented reality, thus, wider involvement from teachers is expected in the near future. Moreover, authors plan to focus on specific case studies (educational interventions within STEM-related courses supported by AR technology), in order to investigate the acceptance of AR by teachers, as well as their instructional approaches and AR learning objects designed and integrated into their STEM-related courses. Finally, researchers aim to focus on how AR application can enhance students’ twentieth-first century skills.