Technological Acceptance, Motivation and Attitudes Towards Digital Assessment in Secondary Education Using Augmented Reality: Development and Preliminary Validation of a Scale for AR-STEM Contexts

Abstract

This study is set against a backdrop of interest in understanding how secondary school pupils perceive learning experiences based on Augmented Reality (AR) and digital assessment, with the aim of designing a useful tool for comparative and replicable research in AR-STEM contexts. To this end, an attitudinal questionnaire was developed and preliminarily validated through a quantitative study conducted with a sample of 199 students from various schools who worked on science curriculum content using an AR application created and validated for the explanation of key concepts. The instrument, designed ad hoc based on the TAM and IMMS models and peer-reviewed, comprised 35 items. Its reliability and construct validity were analysed using Cronbach’s alpha and exploratory and confirmatory factor analysis. The results showed favourable scores for motivation, technological acceptance and the evaluation of digital assessment, alongside high internal consistency and a stable three-factor structure. In conclusion, the instrument presented in full in this study provides preliminary evidence that it is a valid, reliable and useful tool, although the digital assessment dimension remains emerging and will require further validation with independent samples.

1. Introduction

The global crisis triggered by the pandemic has underscored the advantages of technological resources in a broad sense. In the educational sector, they enabled students to continue their learning remotely, obviating the need to physically attend educational facilities. However, this situation has also exposed significant limitations in this regard. The utility of technology as a tool that supports, but does not replace, in-person education and teaching staff has become evident. Nevertheless, it has been observed that there are still substantial barriers to implementing an educational system wholly reliant on technology (Tawil, 2020).

Some years ago, the OECD (2015) published a report titled “Students, Computers and Learning: Making the Connection” that presented the results of a comparative analysis of the skills students acquired in digital environments and the learning methods developed for their implementation. The findings indicated that indiscriminate use by students of technological resources for educational purposes not only had a negligible effect on their academic performance but could also be counterproductive. This raises questions about what might have been done wrong to cause a disconnect between the use of technology and learning outcomes. It also prompts inquiries into the most effective use of various types of technological resources in education. Subsequent studies in this area have also suggested that technological elements can have a positive impact, but only if the technology is used appropriately (Johnson et al., 2019). Therefore, in most cases the inappropriate use of technology in learning favours effects that are not as expected (Brown et al., 2020). In this context, educational experts like Nieto (2016) hypothesise that there exists a gap between conducting educational activities with technological resources and the assessment systems, which are not adapted and in most cases are still the traditional ones. This gap is posited as an explanation for the lack of connection between the use of educational technology and improvements in students’ academic outcomes.

Currently, there is a growing need for professionals specialising in areas related to Science, Technology, Engineering and Mathematics (STEM). In response to this need, universities are seeking to develop effective strategies to attract more students to STEM subjects and to strengthen their learning from secondary school onwards.

Today’s students demand more meaningful teaching and learning methodologies based on educational resources that are familiar, interesting, motivating and immersive. In this sense, immersive technologies promote experiential learning through interactive environments (Crogman et al., 2025).

Innovative methodologies such as gamification, enhanced by immersive educational technologies such as Augmented Reality (AR), have the potential to positively impact the promotion and improvement of students’ socio-emotional and cognitive development (Lampropoulos et al., 2023; Peikos & Sofianidis, 2024). However, this technology can also be used in combination with educational methodologies based on traditional analogue gamification processes, enhancing the educational experience whilst reducing pupils’ screen time. AR is an immersive technology that enables, via mobile devices, the creation of a mixed reality that integrates real-world elements with virtual ones (Abeywardena, 2023).

To situate this study within the AR-STEM literature, it is necessary to consider recent studies such as that conducted by Doğru et al. (2025a), who, using bibliometric networks and content analysis, summarise the main trends, the most common target groups and the methodologies, concluding that AR shows consistent potential to enhance motivation and the learning experience in STEM. Studies published in recent years on the use of AR have focused on general education, early years education and higher education (Magaña et al., 2025). Educational experts such as Garzón et al. (2019) highlight that international studies conducted in secondary education provide evidence of the relative advantages of incorporating AR into different educational settings.

The current literature shows a steady increase in educational initiatives based on the use of Extended Reality (XR) technologies, which include immersive technologies such as Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR), but it also reveals a marked heterogeneity in the approaches used to obtain empirical evidence regarding their effects and the methods for defining and measuring outcomes (perception, interaction, performance and analytics), which continues to hinder comparisons between studies and the generalisation of results (Li et al., 2025; Sakr & Abdullah, 2024). Recent studies on the use of XR in STEM education show the ability of VR and AR to induce improvements in student motivation, participation and skill acquisition, but also point out implementation challenges related to technical failures and physical discomfort and underline the desirability of moving towards standardized contexts that facilitate the evaluation and comparability of results (Zhang et al., 2024).

Similarly, the literature highlights an emerging trend towards the integration of XR/AR with data analysis to improve teaching and support educational decisions (learning analytics) and with data mining techniques to identify patterns and generate predictive models in education (educational data mining), with the aim of supporting pedagogical and assessment decisions. In this regard, methodological gaps and needs have been identified to consolidate concrete and more systematic measurement practices in immersive environments (Lampropoulos & Evangelidis, 2025). In the same vein, research is beginning to explore adaptive response approaches using AR aimed at optimising cognitive processes, such as that proposed in the study by Sun and Liao (2025), which highlights the need to develop more systematic assessment and feedback mechanisms tailored to the use of AR. With regard to instruments, there are some examples of specific scales designed to measure particular constructs within educational experiences incorporating AR technology, such as those proposed by Gandolfi and Ferdig (2025) and Huang and Shih (2025), with recent psychometric validations, which demonstrate progress but also fragmentation across constructs and contexts of application. In the specific context of AR-based educational assessment, the available evidence also points to gaps relating to its implementation and adoption, highlighting that the literature on the advantages and limitations of immersive technologies such as AR in assessment processes remains limited, and that further empirical research is required for their effective integration into both innovative and traditional educational contexts (Köroğlu, 2025). Similarly, digital assessment constitutes a domain with its own distinct facets, for which specific instruments have been developed in recent years, such as those proposed by authors including Siu et al. (2024) and Tat and Kiliç (2024), aimed at studying factors such as anxiety and affective states during online learning. Furthermore, the literature reviewed on AR based on motivational models such as ARCS reinforces the importance of measuring constructs such as student motivation using validated instruments that enable the improvement of the design and measurement of this type of educational experience (Prasetya et al., 2024). Similarly, recent psychometric validation of technology acceptance scales in specific secondary education contexts supports the view of acceptance as a measurable and comparable construct (Dou & Feng, 2025). Regarding the role of digital interaction and integrated formative assessment, recent evidence shows that interactive digital materials with integrated formative assessment can enhance self-regulation, engagement and positive indirect effects on performance, although they do not always lead to immediate improvements. These findings, drawn from higher education, offer a useful framework for incorporating formative assessments into immersive AR experiences, supporting student learning and engagement (Doğru et al., 2025b). Overall, the current literature review reveals that there are instruments designed to measure specific dimensions such as presence, usability, acceptance and motivation, but it also highlights the need for comprehensive, transparent, traceable and validated instruments that enable the joint study, within secondary education contexts and STEM subjects where immersive technologies such as AR are used, of students’ perceptions regarding technological acceptance and motivation, as well as the intervening factors that may be associated with digital assessment processes. These studies provide a solid theoretical foundation to justify the novelty and relevance of the present study within the AR-STEM research continuum.

Conceptually, technology acceptance and motivation serve as complementary frameworks for analysing the integration of AR in educational settings. Some studies that have analysed the use of the TAM model in the field of science, such as the one conducted by Sungur and Ateş (2023), conclude that technology-based STEM training has the potential to improve perceived usefulness and ease of use, as well as students’ attitudes and intention to use it, thereby reinforcing the model’s relevance for application in the study of educational technology adoption processes. The TAM model enables the identification of instrumental beliefs linked to perceived usefulness and ease of use, whilst the ARCS model, applied through the IMMS, allows for the description of motivational processes related to attention, relevance, confidence and satisfaction, which influence student engagement during the learning process. Additionally, perceived comprehension was integrated as a preliminary dimension designed to capture students’ subjective assessment of the extent to which the experience facilitates the assimilation of content, serving as an indicator of perceived pedagogical utility. The results of the factor analyses show that this dimension shares variance with motivational factors and factors relating to the evaluation of the experience; it is therefore interpreted as an attitudinal construct associated with the perception of learning, rather than as an objective measure of academic performance.

The overall aim of this study is therefore to design and conduct a preliminary validation of an attitudinal questionnaire that measures, among STEM students in secondary education, the degree of technological acceptance, the level of motivation, and perceptions associated with digital understanding and assessment, within the context of a learning methodology that utilises Augmented Reality (AR) technology. The instrument presented in this study is not designed to infer causal effects, but rather to capture students’ perceptions and attitudes regarding the educational use of AR and associated assessment components. Therefore, the overall objective is to contribute to educational improvement, contextualised within the scope of the Sustainable Development Goals (SDG 4) of the United Nations 2030 Agenda. The instrument designed and the data obtained in this research will be useful for the development of complementary studies leading to the systematic and replicable assessment of secondary school pupils’ perceptions and attitudes towards learning experiences based on Augmented Reality (AR), including aspects of technological acceptance, motivation, perceived understanding and attitudes towards digital assessment in STEM subjects.

In summary, this study is conducted with the following research questions (RQs) in mind:

RQ1. What underlying factor structure does the questionnaire present in secondary school students within the context of an AR learning experience?

RQ2. What evidence of internal reliability does both the global scale and the subscales show?

RQ3. To what extent is the proposed structure confirmed by CFA, and how robust is it in sensitivity analyses?

RQ4. Does any specific component emerge relating to attitudes towards digital assessment, and how should it be interpreted?

RQ5. What descriptive patterns do students’ responses exhibit at the item and subscale levels in the context of a learning experience with AR?

2. Materials and Methods

2.1. Study Typology and Sample

The typology of this study falls within the framework of the quantitative methodology, in which an ex post facto, prospective design was applied.

The sample used in the study (n = 199) consists of students at 16 Secondary Education centres (ten public schools and six semi-private schools) enrolled in a fourth-year Science course. Data were also collected on age, sex, prior knowledge of AR and the use of digital assessment resources. The student sample was selected from a group of students who had previously participated in a performance study related to another earlier phase of the research. These students had interacted in class with an AR application that represented a key concept of a Biology and Geology course.

2.2. Procedure

This study is part of empirical research which been addressed in several phases due to its inherent nature and complexity. The student sample participating in this phase had previously been involved in a performance study using an immersive educational resource. This resource was an AR application created ad hoc and validated in another prior pilot study, representing a key concept of the subject being taught. Although the AR application incorporated three-dimensional representations and a sequential narrative explaining the key concepts, it did not incorporate gamification elements into its design, as it lacked game mechanics and built-in assessment. For this reason, gamification was implemented as a complementary teaching resource through an external teaching strategy of a traditional analogue nature. After interacting with the app, the pupils answered a series of questions on paper, posed by the teachers, regarding the content covered. The gamified activity consisted of a competitive challenge, defined by correctly answering the greatest number of questions in the shortest possible time. Subsequently, the students were examined on the curricular concepts using an assessment system adapted to the methodology and immersive resource used (Delgado, 2021; Delgado-Rodríguez et al., 2023a).

For this phase of the study, which aimed to gather students’ opinions on specific aspects related to the methodology and technology used in class, an instrument based on a custom-designed questionnaire was created. The administration of the questionnaire also geared at understanding students’ attitudes towards the use of an innovative methodology based on an AR resource to improve understanding key subject concepts and of an adapted assessment system. Given that, attitude can be regarded as a latent construct inferred from the responses given to the questions posed in the items.

The instrument was created ad hoc to be integrated into the research process in order to determine both the level of motivation and the degree of acceptance generated in students by the use of technological resources in educational environments. It was based on a literature review, taking into account the guidelines for the development and validation of scales proposed by experts such as Cabrera-Nguyen (2010), and on an adaptation of the following theoretical models:

• The Technology Acceptance Model (TAM), established by Davis (1989) and widely used in research on technology acceptance (Venkatesh et al., 2003) for the Degree of Technology Acceptance dimension. This, in turn, comprises the Perceived Usefulness indicator (four items) and the Perceived Ease of Use indicator (three items).
• The Instructional Materials Motivation Survey (IMMS), proposed by Keller (2010), for the Level of Motivation dimension, which is composed of the following indicators: Attention (four items), Relevance (three items), Confidence (three items), and Satisfaction (three items).
• A third dimension was also added, initially called Degree of Understanding, consisting of the Ease of Understanding indicator (six items).

In total, the initial instrument consisted of 26 items (

Table 1. Structure of the initial questionnaire.

Framework	Dimensions	Indicators	Ítems
IMMS/ARCS	Level of Motivation	Attention	1–4
		Confidence	5–7
		Satisfaction	8–10
		Relevance	11–13
TAM	Level of Acceptance of AR Technology	Perceived Usefulness	14–17
		Perceived Ease of Use	18–20
Perceived comprehension	Level of Understanding of Key Concepts	Perceived Ease of Understanding of Key Concepts	21–26

Note: Pre-expert review version (n = 26).

).

To ensure the validity of the content, the initial version was sent to a group of 10 experts. Applying the Expert Competence Coefficient (K) and following the criteria established by Cabero and Barroso (2013), the opinions of only nine experts were taken into account. To analyse the responses provided by the panel of experts, the Coefficient (V) was used (Aiken, 1980). The Coefficient (V) was calculated based on the ratings given by the experts for each item in the questionnaire and was recorded using a five-category ordinal scale (1–5). Two dimensions were assessed: (1) the relevance and appropriateness of the items in relation to the study’s objectives and (2) their adequacy and clarity (adequacy of language, comprehensibility and ease of interpretation). The values obtained for each item and dimension are presented in

Table 2. Aiken’s V coefficients for each item (n = 26).

Item	Appropriateness and Relevance	Adequacy and Clarity
	V	V
1	0.8	0.7
2	0.8	0.7
3	0.8	0.7
4	0.7	0.5
5	0.7	0.7
6	0.7	0.6
7	0.8	0.8
8	0.8	0.7
9	0.8	0.8
10	0.8	0.9
11	0.9	0.9
12	0.8	0.6
13	0.8	0.8
14	0.8	0.8
15	0.6	0.4
16	0.8	0.7
17	0.8	0.7
18	0.8	0.5
19	0.8	0.8
20	0.8	0.6
21	0.8	0.6
22	0.8	0.6
23	0.8	0.6
24	0.9	0.9
25	0.8	0.8
26	0.8	0.8

.

With regard to the handling of missing data, one of the nine participating experts did not provide ratings for any item in the relevance dimension. Therefore, the calculation of (V) for that dimension was carried out using the ratings of eight judges. For the clarity dimension, the analysis was carried out using the ratings provided by all nine judges. Given the instrumental and exploratory nature of this phase (initial refinement of the questionnaire), an operational criterion was adopted whereby items with values (V ≥ 0.70) were considered provisionally acceptable, whilst items with (V < 0.70) were considered for revision. Furthermore, to strengthen the interpretation of the index, decision-making was supplemented by the qualitative evidence provided by the judges.

The results showed that, overall, the relevance dimension yielded adequate values, as most items met the established criterion (V ≥ 0.70). However, item 15 scored below the operational threshold (V = 0.60), suggesting that it required revision. Furthermore, this item was phrased in reverse, a characteristic that could contribute to interpretation problems. In the adequacy/clarity dimension, a set of nine items (4, 6, 12, 15, 18, 20, 21, 22 and 23) were identified as failing to meet the minimum criterion (V < 0.70). Consequently, these items were subjected to review, taking into account both the quantitative values and the qualitative observations provided by the judges, with the aim of identifying the nature of the problem (excessive length, semantic ambiguity, double meaning, syntactic complexity) and deciding whether to reformulate or, where appropriate, remove them.

In order to complement the quantitative analysis through a triangulation process, qualitative responses to two open-ended questions were also analysed, with the aim of identifying the need to modify items and assessing the advisability of adding or removing content. In summary, the experts noted: (a) that most items are relevant; (b) that some items are worded in a manner that is too long and complex for a Likert-type format, recommending a shorter and more direct wording; (c) that certain items (4 and 5) should not be worded in reverse; (d) that item 15 is perceived as confusing; and (e) that the wording should be adapted to incorporate inclusive language. It was also noted that, although the overall length of the items was considered adequate, it would be appropriate to include more items to avoid double questions and statements that incorporated more than one idea.

In short, the final version contains a greater number of items due primarily to two reasons: (i) the qualitative observations made by the expert panel, which recommended simplifying long statements, avoiding reverse formulations, and splitting items with more than one idea; and (ii) the need to broaden the coverage of some indicators to improve the representativeness of their content. Therefore, starting from the initial version (n = 26), items with a V < 0.70 were revised, and supplementary items were written and added to avoid duplicate statements and strengthen the measurement, resulting in a final version of 35 items, which was subsequently subjected to psychometric validation (EFA/CFA) on the student sample.

With the results obtained, the initial version of the questionnaire was modified according to the experts’ recommendations, resulting in a revised and corrected final questionnaire structured with a total of 35 items (See Appendix A). This complies with general recommendations on instrument design and with the ratio related to the minimum number of observations and participants needed for this type of study (Hair et al., 2018).

The final questionnaire included information on the study’s objectives, the voluntary nature of participation and the anonymous processing of responses. In accordance with school protocols and applicable regulations, written consent was not required. However, informed consent was implicitly obtained through the voluntary completion and submission of the questionnaire, after confirming that participants had understood the initial information and could raise any queries with the teaching staff. The questionnaire was administered to students online via Google Forms^®, without requesting personal identifiers or recording any information that would allow the students to be identified. In order to preserve the anonymity of the participants and ensure confidentiality, the data was stored with restricted access to the research team and analysed exclusively in aggregate form, with any sharing limited to anonymised results. It was decided not to seek individual written consent, as this would have been incompatible with the anonymous design of the study and would have introduced unnecessary identification of the participants. The maximum time agreed with the participating teachers for completing the questionnaire was 20 min.

Using the students’ responses, the reliability of the questionnaire was assessed using Cronbach’s alpha (Peterson, 1994; Sáez, 2017).

To determine construct validity, an exploratory factor analysis (EFA) was performed. The application of this type of analysis is fully justified, since one of the study’s objectives is to explore both the internal structure of the questionnaire through its main components and determine the existence of other possible factors in its underlying structure (Fabrigar et al., 1999; Hair et al., 2018). Additionally, a confirmatory factor analysis (CFA) was also conducted. Considering the recommendations of prominent experts like Schmitt et al. (2018), conducting both EFA and CFA on the entire sample is justified due to the model’s complexity, the intention to demonstrate the accuracy of its fit, and the pertinence of allowing comparability of the adjustment made with future research. Not dividing the sample into two parts also avoids reducing the sample size and, therefore, a loss of precision and accuracy in the analysed data caused by the random division of the main sample into two sub-samples that are smaller than the original (Fernández-Hernández et al., 2022; Schmitt et al., 2018).

2.3. The Instrument

Table 1 presents the initial theoretical structure of the questionnaire (n = 26) prior to expert review. After conducting the content validity process (Aiken’s K and V coefficients) and the qualitative review, the instrument was reformulated and expanded to a final version consisting of three dimensions, seven indicators, and 35 items, whose descriptive statistics and psychometric analyses are reported in

Table 3. Mean and standard deviation of the questionnaire items.

Indicator	Item	Abbrev.	M ± SD	Internal Consistency		Correlations
				Alpha	ω	Item-Total	MOT	ACC	EVA
Attention	Item 1	MOT01	3.55 ± 1.23	0.979	0.980	0.753	0.737	0.742	0.313
	Item 2	MOT02	3.30 ± 1.18	0.979	0.980	0.794	0.789	0.758	0.350
	Item 3	ACC01	3.90 ± 1.10	0.979	0.980	0.728	0.706	0.743	0.247
	Item 4	MOT03	3.58 ± 1.15	0.979	0.980	0.799	0.780	0.785	0.349
	Item 5	ACC02	3.90 ± 1.12	0.979	0.980	0.746	0.715	0.777	0.260
	Item 6	ACC03	3.61 ± 1.17	0.979	0.981	0.710	0.698	0.700	0.278
	Item 7	MOT04	3.68 ± 1.05	0.979	0.980	0.782	0.767	0.769	0.314
Confidence	Item 8	MOT05	3.35 ± 1.18	0.979	0.981	0.705	0.701	0.664	0.349
	Item 9	ACC04	3.33 ± 1.21	0.980	0.981	0.693	0.674	0.684	0.333
	Item 10	ACC05	3.53 ± 1.22	0.979	0.980	0.822	0.808	0.810	0.315
	Item 11	ACC06	3.41 ± 1.16	0.979	0.980	0.836	0.821	0.813	0.373
Satisfaction	Item 12	ACC07	3.67 ± 1.18	0.979	0.980	0.766	0.724	0.823	0.256
	Item 13	ACC08	3.61 ± 1.20	0.979	0.980	0.781	0.747	0.815	0.281
	Item 14	ACC09	3.52 ± 1.27	0.979	0.980	0.793	0.758	0.828	0.305
	Item 15	ACC10	3.42 ± 1.17	0.979	0.980	0.818	0.806	0.800	0.314
	Item 16	MOT06	3.43 ± 1.13	0.979	0.980	0.778	0.791	0.707	0.380
Relevance	Item 17	MOT07	3.45 ± 1.16	0.979	0.980	0.787	0.800	0.706	0.439
	Item 18	MOT08	3.47 ± 1.14	0.979	0.980	0.850	0.842	0.826	0.317
	Item 19	MOT09	3.36 ± 1.18	0.979	0.980	0.811	0.821	0.738	0.425
	Item 20	ACC11	3.73 ± 1.17	0.979	0.980	0.746	0.713	0.787	0.249
	Item 21	EVA01	3.08 ± 1.30	0.981	0.982	0.361	0.359	0.309	0.468
Perceived Usefulness	Item 22	MOT10	3.43 ± 1.19	0.979	0.980	0.830	0.829	0.777	0.418
	Item 23	MOT11	3.52 ± 1.18	0.979	0.980	0.818	0.805	0.800	0.335
	Item 24	MOT12	3.45 ± 1.19	0.979	0.980	0.830	0.824	0.785	0.414
	Item 25	MOT13	3.47 ± 1.16	0.979	0.980	0.794	0.799	0.724	0.449
	Item 26	EVA02	3.13 ± 1.22	0.981	0.982	0.401	0.422	0.303	0.468
Perceived Ease of Use	Item 27	ACC12	3.91 ± 1.12	0.979	0.981	0.745	0.732	0.737	0.281
	Item 28	MOT14	3.60 ± 1.19	0.979	0.980	0.846	0.854	0.788	0.370
	Item 29	MOT15	3.46 ± 1.17	0.979	0.980	0.837	0.848	0.770	0.388
	Item 30	MOT16	3.67 ± 1.13	0.979	0.980	0.814	0.814	0.780	0.306
Perceived Ease for Understanding Key Concepts	Item 31	MOT17	3.46 ± 1.19	0.979	0.980	0.836	0.857	0.753	0.377
	Item 32	MOT18	3.33 ± 1.21	0.979	0.980	0.823	0.842	0.736	0.424
	Item 33	MOT19	3.59 ± 1.16	0.979	0.980	0.844	0.861	0.767	0.388
	Item 34	MOT20	3.46 ± 1.16	0.979	0.980	0.813	0.818	0.760	0.360
	Item 35	MOT21	3.45 ± 1.23	0.979	0.980	0.795	0.816	0.704	0.425

Note. Table 3 shows statistics of the final version (35 items) resulting after expert judgment and revision of the instrument. Descriptive statistics are reported as mean ± SD. Cronbach’s α and McDonald’s ω correspond to internal consistency estimates computed under an item-deleted condition. Item-Total denotes corrected item-total correlations. SD = Standard Deviation; MOT = Motivation; ACC = Acceptance of AR technology (Technology Acceptance); EVA = Digital Assessment Acceptance. MOT, ACC, and EVA indicate Pearson correlations between each item and the respective latent dimensions. Values in bold indicate the highest and lowest means.

. Additionally, the following categorical variables related to students and educational centres were included: Gender and Age of the Student, Condition of Student with Special Educational Needs, Type and Geographic Location of the Educational Centre, Experience in the Use of AR Resources, Regular Use of Non-Immersive Technology, Regular Use of AR-Based Technology in the Classroom, Use of VR Tools in Class and Regular Use of Assessment Systems Adapted to New Technologies (Delgado, 2021; Delgado-Rodríguez et al., 2023a).

For the assessment of responses, a Likert-type scale was chosen, graded in response intervals from 1 to 5 with 1 indicating complete disagreement and 5 indicating complete agreement.

3. Results

For the analysis of the study’s different data, including the statistical analysis of the data obtained from the responses provided by the students in the attitudinal questionnaire, the IBM^® SPSS^® Statistics V.22 software platform was used for the Exploratory Factor Analysis (EFA). The IBM^® SPSS^® Amos V.28 platform was also used for Confirmatory Factor Analysis (CFA).

Regarding the demographic characteristics of the participants, the sample consisted of (n = 199) secondary school students, of whom 112 (56.3%) were male and 87 (43.7%) were female. In terms of age, the majority of students were 15 (44.7%) or 16 (49.2%) years old, while 6.0% were 17 years or older (Figure 1). Regarding the type of school, 123 (61.8%) students attended public schools and 76 (38.2%) attended private schools.

Finally, 76 (38.2%) participants indicated having some prior experience with AR resources in the classroom, while 123 (61.8%) students had not used them previously. Similarly, 111 (55.8%) students indicated a regular use of technology-based assessment systems, compared to 88 (44.2%) who did not use technology as an assessment tool (Figure 2).

With regard to RQ1–RQ5, the descriptive statistics for the students’ responses are shown below, along with the results of the EFA and CFA analyses and the reliability indices for the scale and its subscales.

The reliability of the instrument was established with the students’ responses to the questionnaire, using Cronbach’s Alpha coefficient (Peterson, 1994; Sáez, 2017) to determine internal consistency. The values obtained are shown in

Table 4. Cronbach’s Alpha Values Obtained.

Dimension	α	ω	AVE	CR	Correlation
					Motivation	Acceptance	Evaluation	Overall
Motivation	0.977	0.977	0.966	0.701	-	0.910	0.454	0.989
Acceptance	0.954	0.954	0.689	0.526	0.880	-	0.358	0.957
Evaluation	0.637	0.638	0.983	0.735	0.463	0.329	-	0.482
Overall	0.981	0.982	-	-	0.984	0.938	0.473	-

Note. α = Cronbach’s alpha; ω = McDonald’s omega; AVE = Average Variance Extracted; CR = Composite Reliability. Internal consistency was assessed using α and ω. Correlation coefficients above the diagonal represent Pearson’s r, and those below the diagonal represent Spearman’s ρ. Diagonal cells are not reported. Values in bold indicate the most distinctive results (maximum/minimum).

(Delgado, 2021).

A detailed analysis of the data obtained from the students’ responses revealed average scores above the scale’s theoretical mean (M = 3.0) in the three dimensions that make up the attitudinal questionnaire, i.e., level of motivation, acceptance of AR technology and understanding of key concepts, across all items considered (1 to 35). The highest score was 3.94 and the lowest was 3.10, corresponding to items 27 and 21, respectively (Table 1).

Regarding the students’ opinions, note should be made of their perception of the usefulness of this technology to capture and improve their attention on the subject matter, as they consider the former entertaining and at the same time useful for their learning process. They reported that it enables them to better understand and expand their knowledge of the subject’s key concepts and improve their academic results. They also consider that the use of a digital assessment system as a method that complements the use of an AR resource is important to improve their academic marks.

It should be noted that the analysis of the data obtained from the students’ responses shows low values in the standard deviations relative to the assessments made by the students, both in the partial assessments for each of the items (values between 1.05 and 1.27), and in their overall average (1.17). This indicates a mean homogeneity in the responses obtained, which in turn shows their evident uniformity and constitutes a clear indicator of their reliability.

Subsequently, with the objective of establishing possible relationships between the variables and dimensions or factors, and in accordance with relevant experts such as Hair et al. (2018), an Exploratory Factor Analysis (EFA) was conducted. This statistical analysis was selected because one of the study’s main objectives was to exploratively verify the internal structure of the questionnaire through its main components, and to determine the possible existence of other factors in its underlying structure (López-Aguado & Gutiérrez-Provecho, 2019).

The correlation matrix coefficients were calculated beforehand to verify the pertinence of performing an EFA, determining the relationships between pairs of variables. For this purpose, Bartlett’s Test of Sphericity and the Kaiser–Meyer–Olkin (KMO) index for sampling adequacy were used, obtaining a value of p < 0.05 (0.000) in the first case and p > 0.5 (0.956) in the second case, respectively, which verified compliance with the conditions for conducting the EFA.

Having confirmed compliance with the necessary conditions for the analysis, the EFA was carried out using the Principal Components method, where the first factor explains most of the variance of the variables. The Varimax rotation was used to facilitate the interpretation of the factors, which sometimes is not simple as the latter correlate with multiple variables. The objective was to ensure, as much as possible, that each of the selected factors was strongly represented by a specific set of variables, thus making the interpretation of their meaning more intelligible in theoretical terms. With this method four main factors were initially obtained.

However, a detailed analysis of the data allowed verifying that only three main factors should be extracted. This number of factors was deemed sufficient, as it significantly reduced the initial amount and also because it allowed explaining nearly 70% of the total variance. Specifically, the selected set of factors explains 69.2% of the variance: 61.0%, 5.1% and 3.1% for each of the three factors, respectively (Delgado-Rodríguez et al., 2023b). Moreover, the choice of three main factors is also justified as the selection of a fourth factor would not significantly increase the model fit and would introduce greater complexity for its correct interpretation. In order to interpret the factors obtained, recommendations by experts such as Lorenzo-Seva and Ferrando (2013), who recommend that each factor be represented by at least two theoretically related items, were also taken into account. On the other hand, the following criteria were applied to define the factors: (1) that in general, the items should present factor loadings greater than 0.40 (Hair et al., 2018); (2) if an item presented factor loadings greater than 0.40 on two factors, it was considered to contribute to the factor in which it had the higher loading, provided that the difference in the saturations of an item was greater than 0.10; (3) otherwise, the elimination of the item was proposed (Hair et al., 2018). Note should be made that item 23, with loadings of 0.569 (factor 1) and 0.551 (factor 2) and a difference of 0.018, is well below the 0.10 threshold, indicating the possible existence of a high cross-loading.

In order to obtain additional psychometric evidence, the EFA was re-estimated using a common-factor method, specifically the principal axis method, with Promax and Oblimin oblique rotations. Furthermore, the corresponding scree plot was analysed, along with a parallel analysis, to determine the optimal number of factors to retain. Consequently, the three-factor solution showed stability across the rotation methods. Promax and Oblimin explained virtually the same cumulative variance (72.12%) compared to 69.20%, with high principal loadings, comparable factor patterns and a minimal difference. The parallel analysis indicated that only the first three empirical eigenvalues exceeded the simulated ones, and the scree plot showed a clear inflection point after the third factor, confirming the retention of K = 3 (Figure 3).

It should be noted that item 23 showed significant loadings on more than one factor, thus meeting the criterion for cross-loading defined above. As its wording could be interpreted ambiguously, three alternatives were considered (rewriting, repositioning, or considering its possible removal). It was decided to test the removal of item 23 as the most conservative approach and to re-estimate the factor structure as a sensitivity analysis. Re-estimating the model after removing item 23 kept the explained variance and reliability coefficients stable (Δα ≤ 0.001), and, although the overall fit improved, the three-factor structure remained virtually unchanged. A comparison between the original structure and that estimated after removing item 23 shows virtually complete stability of the three-factor solution. In terms of cumulative variance, the difference is minimal: 72.12% with the item versus 72.07% without it, indicating no substantial loss of explanatory power. Therefore, it was decided to retain it.

Subsequently, the variables that best correlated with each of the factors were studied in order to name them. The results obtained allowed regrouping the items and naming the factors (components) as: (1) Level of Motivation; (2) Acceptance of AR Technology; and (3) Acceptance of the adapted digital assessment system (Delgado, 2021). In

Table 5. EFA—Factors, Items and Factor Loadings.

Dimension	Indicator	Estimate (95% CI)	Std. Estimate
Motivation	MOT01	0.918 (0.772 to 1.065)	0.746
	MOT02	0.953 (0.815 to 1.091)	0.801
	MOT03	0.917 (0.783 to 1.050)	0.795
	MOT04	0.816 (0.692 to 0.939)	0.778
	MOT05	0.833 (0.691 to 0.976)	0.708
	MOT06	0.896 (0.767 to 1.026)	0.798
	MOT07	0.936 (0.803 to 1.069)	0.808
	MOT08	0.972 (0.845 to 1.099)	0.855
	MOT09	0.983 (0.848 to 1.118)	0.830
	MOT10	0.998 (0.863 to 1.132)	0.841
	MOT11	0.971 (0.835 to 1.106)	0.823
	MOT12	1.005 (0.870 to 1.141)	0.841
	MOT13	0.945 (0.810 to 1.080)	0.813
	MOT14	1.032 (0.899 to 1.164)	0.866
	MOT15	1.006 (0.875 to 1.137)	0.859
	MOT16	0.936 (0.806 to 1.066)	0.826
	MOT17	1.031 (0.898 to 1.164)	0.865
	MOT18	1.028 (0.892 to 1.164)	0.851
	MOT19	1.006 (0.878 to 1.135)	0.869
	MOT20	0.957 (0.824 to 1.091)	0.826
	MOT21	1.017 (0.875 to 1.158)	0.824
Acceptance	ACC01	0.834 (0.702 to 0.965)	0.757
	ACC02	0.878 (0.746 to 1.009)	0.782
	ACC03	0.841 (0.699 to 0.983)	0.717
	ACC04	0.864 (0.716 to 1.013)	0.711
	ACC05	1.024 (0.887 to 1.161)	0.842
	ACC06	0.969 (0.84 to 1.099)	0.844
	ACC07	0.985 (0.851 to 1.119)	0.834
	ACC08	1.000 (0.864 to 1.136)	0.835
	ACC09	1.070 (0.928 to 1.213)	0.845
	ACC10	0.979 (0.847 to 1.111)	0.838
	ACC11	0.932 (0.796 to 1.067)	0.798
	ACC12	0.834 (0.702 to 0.967)	0.757
Evaluation	EVA01	0.765 (0.551 to 0.979)	0.593
	EVA02	0.961 (0.737 to 1.186)	0.792

Note. Estimate = unstandardized factor loading; CI = confidence interval; Std. Estimate = standardized factor loading. MOT = Motivation; ACC = Acceptance of AR Technology; EVA = Evaluation. Estimates are derived from the confirmatory factor analysis model and are reported with 95% confidence intervals.

, the said factors and the items associated with each one are represented, with factor loadings above 0.50 in all cases and a large number of factor loadings reaching high values above 0.70.

In addition to the anticipated dimensions, the EFA identified a third factor associated with the acceptance of digital assessment (EVA). This component was not a construct that had been fully defined a priori within the initial design of the instrument, but rather emerged empirically as a distinct factor based on the pattern of covariation among the items. However, given that in its current formulation it is based on a small number of items (two), its interpretation should be considered provisional. Consequently, as this is an emerging provisional factor, it is presented as a relevant but preliminary finding, the consolidation of which requires the expansion of the set of associated items and psychometric validation in independent samples in subsequent studies.

Given the very high internal consistency observed in the full version of the instrument, potential redundancy was assessed using the average inter-item correlation (AIC), and complementary reliability was estimated using McDonald’s ω. In order to improve the practical utility of the instrument and minimise duplication of content, a shortened version was derived from the original 35 items through proportional reduction by dimension, retaining 12 items: 6 MOT (MOT03, MOT02, MOT18, MOT21, MOT07, MOT01), 4 ACC (ACC11, ACC08, ACC07, ACC03) and 2 EVA (EVA02, EVA01), (see Appendix C). The selection was based on high principal loadings on the theoretical factor, substantial differences from the secondary loadings, adequate communalities and low complexity. The primary loadings ranged from 0.595 to 1.031, the differences from 0.358 to 0.896 and the communalities from 0.253 to 0.868. Reliability was high for the global scale (α = 0.926), MOT (α = 0.921) and ACC (α = 0.875), and acceptable for EVA (ρ_SB = 0.638).

As previously mentioned, a Confirmatory Factor Analysis (CFA) was also conducted, considering the relevance of allowing the comparability of the model with future related research enabling cross-validation with the data obtained in this study. The method used for parameter estimation in the CFA carried out was maximum likelihood, and the fit indices used were Chi-square (χ²), degrees of freedom (χ²df), Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Standardized Root Mean Square Residual (SRMR) and Root Mean Square Error of Approximation (RMSEA) using standard criteria for assessing fit (Goretzko et al., 2024; Hu & Bentler, 1999). The following values were obtained: χ²⁽⁵²⁴⁾ = 1478 (p < 0.0001), CFI = 0.86, TLI = 0.852, SRMR = 0.0449 and RMSEA = 0.0956; 90% Confidence Interval (0.0899–0.1010). In general, the data show an acceptable model fit. The factor loadings are appropriate, showing minimum values of 0.594 for item 21 and maximum values of 0.868 for item 33 (see flow diagram with Standardized Estimators in Figure 4).

Considering the factor loadings obtained, the items related to the level of motivation present estimators that vary mainly between 0.8 and 1.0, indicating generally high values. The confidence intervals for most items do not include the value 0, and p-values are consistently less than 0.001, suggesting that these estimates are statistically significant. The Standard Estimator varies but is generally around 0.8, suggesting good item reliability. This dimension suggests that the individuals assessed show a high level of motivation. The items related to the acceptance of AR technology also show high estimators, most of which are close to or exceed the value of 0.8. The p-values are <0.001, indicating statistical significance. The Standard Estimator for these items varies but generally remains around 0.75. This implies that the acceptance of this immersive technology is generally high among the students evaluated. The items of the dimension of acceptance of a digital assessment system have varying estimators, but all of them are significant with p-values < 0.001.

Given the high internal consistency and correlations between factors, two-factor models (G + MOT, ACC and EVA) were estimated as a sensitivity analysis to assess the possible predominance of a general factor. The full specification was not acceptable due to identification issues and negative latent variances, particularly in EVA, which consists of two items. After removing the specific EVA factor and retaining its items solely within G, the model converged adequately. The general factor exhibited high standardised loadings on MOT and ACC, whilst the specific factors retained moderate loadings on some of the items, suggesting relevant common variance, although insufficient to replace the correlated three-factor structure.

4. Discussion

As indicated in the Introduction and current state of the art, this study justifies the need for a validated questionnaire that can determine secondary education students’ perceptions of the use of general educational technological resources.

In particular, there is a need for tools developed on the basis of psychometric evidence of validity and reliability that enable us to characterise the perceptions and attitudes of students of STEM subjects towards the educational use of Augmented Reality (AR) in areas relating to technological acceptance and motivation, as well as in aspects linked to digital assessment, whilst avoiding the attribution of causal effects that have not been directly evaluated. All of this forms part of efforts to improve the quality of education in line with SDG 4 of the UN 2030 Agenda.

The analysis of the data obtained from the evaluations provided by the students through the attitudinal questionnaire designed ad hoc in this study allows us to determine that said questionnaire, as a research instrument, presents adequate technical characteristics for use in the evaluation of an innovative educational methodology.

The analysis of the responses to the items has also enabled verifying the functionality of the questionnaire to detect and determine the students’ opinion regarding the use of an innovative educational methodology. This methodology is based on the combination of an immersive AR technological resource and an adapted digital assessment system. In this sense, the analysed data reveal medium–high levels of the averages of students’ evaluations, with mean homogeneity associated with the responses in respect to the three dimensions that comprise the questionnaire: acceptance of the immersive educational technological resource, level of motivation and understanding of key concepts. This represents an indicator of the uniformity and reliability of the said evaluations. It also provides a validity argument to determine the acceptance of RA technology by students, in line with the results obtained by other related research in this field (Scherer et al., 2020). The reliability of the responses offered by the students allows us to verify the functionality of the questionnaire as a valid instrument to know their opinion.

The data analysed in this study on the factor structure of the questionnaire using Exploratory Factor Analysis (EFA) reveal the existence of a third factor associated with the acceptance of digital assessment (EVA), positively valued by the students in general. The data confirm that the set of the three factors, namely level of motivation, technological acceptance and acceptance of an adapted digital assessment system, better represents the internal structure of the questionnaire as a construct. However, the EVA dimension should be interpreted as a provisional emerging factor, as it initially comprises only two items. As a priority, it is advisable to expand the set of items in order to capture acceptance of digital assessment more comprehensively. To this end, it is proposed that sub-dimensions relating to equity, usability, accessibility, test anxiety, ethics/trust and the usefulness of the feedback be incorporated (Appendix C).

These data suggest that adapted assessment systems be incorporated as part of the educational methodological strategies based on the use of immersive AR technology. This finding is consistent with the hypothesis proposed some years ago by experts in digital technology such as Nieto (2016), in terms of enabling a positive effect produced by the use of an innovative methodology in combination with a digital evaluation system, in such a way as to generate profound methodological changes. These changes are based on specific skills and strategies related to the educational technological resource, which otherwise cannot be produced or quantified effectively with traditional assessment systems.

Similarly, these results are consistent with the views expressed by educational technology experts such as Spiteri and Chang Rundgren (2020). These experts uphold conclusions identical to the ones reflected in some relevant international reports, such as the one published years ago by the OECD (2015). This report raises the possibility that the widespread “disconnection” between the use of educational technology and gains in learning, and therefore, improvements in student performance, may be a direct consequence of the inadequate application of technological resources. These approaches also agree with other significant reports published recently, which also conclude that technology in general and immersive technology in particular, despite having a notable future projection in the field of education, is not capable of generating a greater impact on student learning on its own (Brown et al., 2020). Another conclusion derived from the EFA conducted in this study is the existence of a possible cross-loading in item 23. Following the recommendations of relevant experts such as Schmitt et al. (2018), who indicate the acceptability of fitting different models to the same data, and given the pertinence of allowing the comparability of the model developed in this study with future related research, the results of a Confirmatory Factor Analysis (CFA) have also been included. This allows cross-validation between the data obtained in this study and those obtained in future studies in which this instrument is administered to other different student samples. It is worth noting that not dividing the sample has avoided a reduction in the sample size and, therefore, a loss of precision and accuracy in the analysed data due to the random division of the main sample into two sub-samples smaller than the original sample (Fernández-Hernández et al., 2022; Schmitt et al., 2018). The results obtained in the CFA allow us to conclude that the individuals evaluated tend to show a high degree of motivation and acceptance. They also show a generally positive acceptance of the digital assessment system. The CFI and TLI means, and even the SRMR, indicate a good fit, but the RMSEA suggests that some adjustment of the model might be advisable, perhaps by modifying or removing a specific item, as previously mentioned.

Table A4. Final instrument.

ID Item	Abrev	Item	Planned Construction	PA1	PA2	PA3	EFA Factor	EFA Load	Second EFA Load	Delta EFA	CFA Factor	CFA Load
Item 01	MOT01	The Augmented Reality (AR) app I used in class helped me focus on the concept it explained.	Motivation	0.241275739	−0.002923951	0.596114606	PA3	0.596114606	0.241275739	0.354838867	Motivation	0.747426595
Item 02	MOT02	The AR app I used in class can help me get a better grade on the exam.	Motivation	−0.053106039	0.241471286	0.705328533	PA3	0.705328533	0.241471286	0.463857247	Motivation	0.791569298
Item 03	ACC01	This methodology using an AR app really caught my attention.	Acceptance	0.403227644	−0.164384132	0.56027751	PA3	0.56027751	0.403227644	0.157049865	Acceptance	0.753825811
Item 04	MOT03	This AR methodology helps me understand the concepts it explains.	Motivation	−0.1198654	0.122372018	0.903034466	PA3	0.903034466	0.122372018	0.780662448	Motivation	0.786734776
Item 05	ACC02	Overall, I find this methodology interesting.	Acceptance	0.4892383	−0.225797958	0.552369676	PA3	0.552369676	0.4892383	0.063131376	Acceptance	0.782656272
Item 06	ACC03	I think using this methodology is useful for my academic training.	Acceptance	0.050863752	−0.046486284	0.7946354	PA3	0.7946354	0.050863752	0.743771647	Acceptance	0.719257841
Item 07	MOT04	The content of the AR app I used seems quite accurate.	Motivation	0.24746441	0.096795901	0.519893405	PA3	0.519893405	0.24746441	0.272428996	Motivation	0.775658431
Item 08	MOT05	When I downloaded the AR app, I had the impression that it would help me better understand the concept it explained.	Motivation	−0.059205075	0.32058321	0.53080957	PA3	0.53080957	0.32058321	0.210226361	Motivation	0.707257807
Item 09	ACC04	I’m confident that I could easily pass an exam on the key concept the AR app explains.	Acceptance	0.37482183	0.209494485	0.169442612	PA1	0.37482183	0.209494485	0.165327345	Acceptance	0.708187455
Item 10	ACC05	While using the app, I felt it could be useful.	Acceptance	0.441360342	0.18280521	0.273216612	PA1	0.441360342	0.273216612	0.16814373	Acceptance	0.839939854
Item 11	ACC06	When I used the app, I was confident that I would learn what I needed to know about the concept it explained.	Acceptance	0.445803704	0.238659028	0.225143936	PA1	0.445803704	0.238659028	0.207144676	Acceptance	0.847151577
Item 12	ACC07	I find the AR app I used in class useful and engaging.	Acceptance	0.915696502	−0.200944576	0.101617343	PA1	0.915696502	0.200944576	0.714751925	Acceptance	0.831254729
Item 13	ACC08	Overall, I enjoyed using this AR app.	Acceptance	1.005044141	−0.063811164	−0.114123548	PA1	1.005044141	0.114123548	0.890920594	Acceptance	0.835539381
Item 14	ACC09	I’m happy to be able to learn with this AR app.	Acceptance	0.787159829	−0.003385843	0.065301581	PA1	0.787159829	0.065301581	0.721858248	Acceptance	0.844910062
Item 15	ACC10	The app helped me understand the concept it explains quite well.	Acceptance	0.437662694	0.216827959	0.238143973	PA1	0.437662694	0.238143973	0.199518721	Acceptance	0.835226705
Item 16	MOT06	I consider the academic results I obtained after using this app satisfactory.	Motivation	0.285538902	0.459339986	0.10874844	PA2	0.459339986	0.285538902	0.173801084	Motivation	0.799239603
Item 17	MOT07	I think it’s important to have an AR app like the one I used in class to better understand some key concepts.	Motivation	0.181914115	0.683233904	0.004070044	PA2	0.683233904	0.181914115	0.50131979	Motivation	0.808552813
Item 18	MOT08	I find the content of the app I used important.	Motivation	0.470485126	0.173259476	0.285161663	PA1	0.470485126	0.285161663	0.185323462	Motivation	0.855991562
Item 19	MOT09	I think understanding the content of the AR app will help me improve my exam grade.	Motivation	0.236785704	0.621441568	0.034641259	PA2	0.621441568	0.236785704	0.384655864	Motivation	0.830199732
Item 20	ACC11	I believe that for an AR app to be relevant to my learning, it also needs to be easy to use.	Acceptance	0.961365557	−0.184157632	0.015144083	PA1	0.961365557	0.184157632	0.777207924	Acceptance	0.798769845
Item 21	EVA01	I think it’s important to take the exam online.	Assessment	−0.162142956	0.521908307	0.044085854	PA2	0.521908307	0.162142956	0.359765351	Evaluation	0.605176948
Item 22	MOT10	Using this AR app allows me to better understand some important key concepts.	Motivation	0.424109181	0.471038446	0.009083812	PA2	0.471038446	0.424109181	0.046929265	Motivation	0.841934776
Item 23	MOT11	I believe that, overall, the AR app I used can be useful.	Motivation	0.520445775	0.31659319	0.051819716	PA1	0.520445775	0.31659319	0.203852586	Motivation	0.823077231
Item 24	MOT12	Using AR apps would motivate me in the learning process.	Motivation	0.557928589	0.445689324	−0.105831713	PA1	0.557928589	0.445689324	0.112239264	Motivation	0.841203598
Item 25	MOT13	I think I could achieve better results if teachers used AR apps.	Motivation	0.379514709	0.623729202	−0.136872524	PA2	0.623729202	0.379514709	0.244214493	Motivation	0.814587084
Item 26	EVA02	I think I can get a better grade if I take the exam online.	Assessment	−0.230755252	0.740394495	−0.056523979	PA2	0.740394495	0.230755252	0.509639243	Evaluation	0.773786879
Item 27	ACC12	It was easy to learn how to use the application.	Acceptance	0.595604596	−0.024517428	0.234957523	PA1	0.595604596	0.234957523	0.360647072	Acceptance	0.758950962
Item 28	MOT14	Its ease of use made it easier for me to understand the concept.	Motivation	0.439560903	0.408463444	0.076643384	PA1	0.439560903	0.408463444	0.031097459	Motivation	0.866071454
Item 29	MOT15	When the application is easy to use, my results improve.	Motivation	0.28484034	0.494635683	0.142088715	PA2	0.494635683	0.28484034	0.209795342	Motivation	0.859492718
Item 30	MOT16	I find the ease of use satisfactory.	Motivation	0.531202104	0.233092567	0.121212516	PA1	0.531202104	0.233092567	0.298109537	Motivation	0.825429926
Item 31	MOT17	Using AR can make it easier for me to understand.	Motivation	0.118495478	0.638333676	0.173614122	PA2	0.638333676	0.173614122	0.464719553	Motivation	0.862683024
Item 32	MOT18	It can help me improve my academic results.	Motivation	0.03534197	0.710494082	0.173276217	PA2	0.710494082	0.173276217	0.537217865	Motivation	0.847527647
Item 33	MOT19	This technology helps me understand the teacher clearly.	Motivation	0.443243461	0.480293513	−0.00250358	PA2	0.480293513	0.443243461	0.037050051	Motivation	0.870879174
Item 34	MOT20	I understood the concept better with AR than with a book.	Motivation	0.37275775	0.418401773	0.098238407	PA2	0.418401773	0.37275775	0.045644022	Motivation	0.826456361
Item 35	MOT21	I understood it better with AR than with a video.	Motivation	0.144926426	0.731501844	0.00529655	PA2	0.731501844	0.144926426	0.586575418	Motivation	0.82353913

(Appendix D) shows that items ACC02, MOT10, MOT14, MOT19 and MOT20 have relatively high cross-loadings, suggesting a possible conceptual overlap between dimensions. However, it has been decided to retain these items given their content relevance. With regard to item 23, it is necessary to distinguish between two levels of analysis: the overall fit of the model and the stability of the factor structure. The removal of the item resulted in a substantial improvement in the model’s fit indices (CFI = 0.999; RMSEA = 0.056). However, the factor structure remained virtually unchanged, both in terms of explained variance and the pattern of loadings, indicating that the three-factor solution is stable. For this reason, and in accordance with theoretical and content-related criteria, it was decided to retain the item in the final scale.

Furthermore, the data analysis supports the psychometric adequacy of the short version of the questionnaire, maintaining the original structural proportionality of the instrument. As a practical implication, a 12-item abbreviated version is proposed for educational contexts with time constraints.

In summary, regarding the underlying factor structure (RQ1), the results reveal a three-factor solution (motivation, technology acceptance, and digital evaluation) that is consistent with the theoretical framework and stable across different analytical procedures, thereby supporting the preliminary structural validity of the instrument. Regarding internal reliability (RQ2), both the overall scale and the subscales show high levels of internal consistency (α/ω), although the EVA dimension exhibits lower values, which is to be expected given its small number of items. Overall, the observed reliability supports the instrument’s suitability for preliminary use in comparative research. The proposed structure (RQ3) was confirmed via AFC with moderate overall fit indices, and sensitivity analyses (exclusion of item 23) suggest that the proposed solution is robust, given that excluding the item improves the fit but does not substantially alter either the structure or the reliability, suggesting the model’s stability. In addition, a specific component (RQ4) was identified, linked to attitudes toward digital assessment (EVA), which should be interpreted as an emerging and provisional factor since it consists of two items. This component suggests that, in educational experiences incorporating AR, digital assessment may constitute a distinct attitudinal factor that warrants further exploration and validation in future studies using different samples. Regarding the descriptive patterns present in the students’ responses (RQ5), they show favorable mean scores in the dimensions of motivation and technological acceptance, as well as a positive assessment of digital assessment, with moderate dispersion. These results suggest a generally positive acceptance of the AR-based experience, which should be interpreted as a reported perception and not as a causal effect on learning.

From a practical perspective, the instrument presented in this study provides a brief and traceable tool for teachers and researchers to observe the degree of acceptance, the level of motivation, and attitudes toward digital assessment in AR-STEM approaches in secondary education, facilitating agile and comparable diagnoses across groups, schools, or related educational experiences. The proposed 12-item abbreviated version improves feasibility in time-constrained contexts, and the preliminary identification of the EVA component offers guidance for designing and evaluating digital assessment systems that are more consistent with AR-based methodologies in terms of equity, usability, and accessibility.

Overall, the findings of this study are consistent with those reported by authors such as Doğru et al. (2025a) in identifying the potential of AR to enhance motivation for STEM learning. They also address the gaps identified in the literature review, offering an integrated and traceable instrument that combines: (i) the measurement of the degree of technological acceptance, aligned with TAM approaches in secondary education (Dou & Feng, 2025; Sungur & Ateş, 2023); (ii) the measurement of motivation levels, aligned with ARCS/IMMS (Prasetya et al., 2024); and (iii) a provisional and expandable emerging digital assessment (EVA) factor, which is consistent with recent studies on digital assessment (Siu et al., 2024; Tat & Kiliç, 2024). Therefore, this study addresses the existing methodological gap in XR/AR contexts, highlighted by authors such as Lampropoulos and Evangelidis (2025); Li et al. (2025), Sakr and Abdullah (2024) and Zhang et al. (2024), regarding the need for instruments that enable the collection of evidence and systematic measurement, overcoming the fragmentation by constructs when analysing specific previous scales such as those proposed in their work by Gandolfi and Ferdig (2025) and Huang and Shih (2025), within a combined framework of acceptance/motivation/evaluation. Furthermore, the methodological design incorporates gamification-based activities, thereby reducing screen time, and the findings are reported with a focus on transparency and replicability in terms of content validity and preliminary factor structure. All of this will facilitate comparability and the development of future studies on experiences based on innovative methodologies in the field of AR-STEM in Secondary Education.

5. Future Directions

While the findings presented are promising, they should be interpreted as an initial step in the process of validating this instrument for its application within the framework of active and innovative AR-STEM methodologies.

As explained, the short version of the questionnaire maintains a high level of reliability whilst reducing redundancy; however, it will require a further process of cross-validation using independent samples.

The evaluation of the two-factor model was constrained by the presence of a factor comprising just two items (EVA), which limits the ability to identify more complex specifications. It is recommended that this dimension be expanded and that the analysis be replicated on an independent sample. In this regard, bearing in mind that the EVA dimension should be interpreted as a provisional emergent factor as it initially comprises two items, it is proposed, as a priority for future research, to expand the set of items to capture acceptance of digital assessment more comprehensively, incorporating sub-dimensions related to equity, usability, accessibility, test anxiety, ethics/trust and the usefulness of the response. This expansion must first undergo content validation through expert judgement, cognitive interviews and subsequent psychometric validation in an independent sample using EFA/CFA and reliability indices (α/ω).

Given that the EFA and CFA were conducted on the same sample, it is advisable to repeat the validation on a different sample to ensure the stability and generalisability of the factor structure. In this regard, the fit of the confirmatory model should be interpreted as moderate and in need of improvement; therefore, it is recommended that, in future research, the set of items be refined (particularly: ACC02, MOT10, MOT14, MOT19 and MOT20) and the validation be replicated in an independent sample to optimise the factor structure and its fit indices.

Future research with larger and more diverse samples will strengthen the evidence base. Exploring the instrument’s applications in other educational contexts will also help establish its generalisability.

6. Conclusions

This study presents a fully designed and validated instrument to measure motivation, technological acceptance, and attitudes toward digital assessment in STEM subjects that integrate gamification processes based on immersive AR educational resources. The analyses supported a three-factor solution with high internal consistency, providing evidence of reliability. However, the high coefficients suggest that some items could be optimised to improve efficiency. Students reported generally positive perceptions regarding technological acceptance, motivation and perceived understanding through an educational methodology based on Augmented Reality (AR), as well as a favourable assessment of the associated digital assessment. The identification of a factor linked to digital assessment in the context of an AR educational experience constitutes an emerging and preliminary finding, suggesting the desirability of expanding research on technology acceptance to incorporate specific dimensions of digital assessment. These results should be interpreted as evidence of attitudes and perceptions rather than as direct estimates of causal effects on learning, which would require objective measures and experimental designs.

Overall, the instrument presented is a valid, reliable, and practical tool with a robust theoretical basis for studying student responses to AR-enhanced learning. With its application in future studies and with other student samples, it can contribute to a deeper understanding of how immersive technologies are received in educational settings and how they can support the design of more effective and efficient educational interventions.