International Comparative Pilot Study of Spatial Skill Development in Engineering Students through Autonomous Augmented Reality-Based Training

Abstract

Spatial ability is made up of several sub-components, such as the ability to perform mental rotation and object-based transformations. Together with each individual’s attitudes and general skill sets, this specific ability plays an important role in technical professions such as engineering. The components of spatial ability can be enhanced using targeted training or educational programs. This study analyses the levels of spatial skills in first-year engineering students at two universities, one in Spain and one in Peru. The purpose of the study is to establish the extent of symmetry between these study groups in terms of their spatial skills. Initial comparisons indicate that the Peruvian students have a lower level of spatial skill prior to training than their Spanish cohorts. AR-based training delivering representational system content was used with engineering students at both universities to boost spatial abilities. The results obtained indicate the training was effective, as both experimental groups made significant gains in their level of spatial ability. No difference was detected in either experimental group for the variable gender. The comparison of spatial ability gains between both countries is similar, although there is significant difference in the spatial ability component spatial visualization. In this instance, gains in this component were higher amongst the student population in Peru.

1. Introduction

Spatial awareness is one of the many components of human intelligence that enables us to recognize, shape, and manipulate images, figures, and objects [1]. For over a century, it has been held that this fundamental attribute bestows the vital skills required by anyone interested in pursuing a professional career in either science, technology, engineering, or mathematics (STEM) [2]. In the case of engineering, it has been widely demonstrated that spatial awareness has a direct impact on the success rates of first-year university students in engineering degrees. Additionally, spatial awareness has also been directly linked to better performance in courses taught in graphic engineering [3]. Furthermore, it has been demonstrated that spatial awareness also improves student performance in other subjects that are taught in the first year of engineering degrees [4,5]. However, the importance of spatial awareness stretches well beyond its impact on the success of future engineers. This ability is also vital in ensuring the talents of all professionals involved in STEM disciplines and for the technological development of a country [6]. A clear example of this is evident in the enormous efforts being made by countries in North America to formalize its teaching. In recent decades, these countries took the necessary steps to ensure that this ability was included in the educational curriculums. Consequently, it is now being regularly taught and assessed after having been severely neglected. Spatial awareness training has even appeared in projects such as “Training the Next Generation of STEM Innovators: Identifying and Developing Our Nation’s Labor Force” [7]. This has allowed students to practice the three main skills needed in STEM qualifications: mathematical skills, verbal skills, and spatial awareness. The activity “ENGAGE Engineering” project is also worthy of mention in relation to this topic. The idea behind this project is to address the issue of student retention in engineering degrees. To do so, the project targets undergraduate engineering students using a range of strategies, including offering spatial awareness skills training. There are currently 72 engineering schools from the US affiliated with this particular project [8].

Research has been conducted across all continents on the subject of how to improve spatial awareness in the fields of science and engineering [9]. Yet, despite this being the case, there has been a lack of consistency in terms of interest in and commitment to developing spatial awareness in students; rarely has it formed part of a general strategy for improving personal, professional and academic performance. Instead, most research has been restricted to examining specific and particular interventions, e.g., [10,11,12,13,14].

In the case of Spain, the Spanish government has awoken to the importance of including spatial skills training in engineering qualifications. Consequently, the Spanish Ministry of Science and Innovation (currently the Ministry of Universities) has ensured that the development of spatial skills in engineering students is explicitly mentioned in the official regulations governing engineering curricula [15]. In Spain, the task of improving engineering students’ spatial abilities has been addressed using training sessions employing technologies that have proven to be attractive to students [16], i.e., 3D solid modelling software [17]; the development of an interactive learning management system to develop spatial visualization skills [18]; or virtual reality-based training, interactive tools, and outdoor training [16,19].

In Latin America (and Peru specifically), the development and assessment of spatial skills has not been accorded the importance and recognition it deserves in either general or higher education. As such, universities are encountering two issues that stem from the absence of formal spatial awareness testing during the admissions process [20]: first, that incoming engineering students tend to possess poor spatial awareness skills; and second, that there are significant discrepancies in terms of students’ initial capabilities [21]. What this means in practical terms is that students lacking spatial skills are at a disadvantage when compared to their peers. It has been shown that first-year engineering students possessing better spatial awareness skills obtain higher grades in engineering graphics subjects and, likewise, in courses such as calculus, physics, or chemistry [22]. Students with only very basic spatial awareness skills achieve lower grades than classmates when taking any of the aforementioned subjects [23], which is a problem that becomes especially evident in the case of engineering graphics [24].

Spatial skills can be developed with training. Some authors, however, mention that both general training and indirect training prove effective when content overlaps with the core subjects being taught in engineering graphics [21,25]. Other authors argue that targeted training is more effective, i.e., training lasting one to three weeks that is aimed at a specific spatial awareness test [19]. This stance has been demonstrated in countless studies, in particular those offered by Michigan Technological University (Houghton, MI, USA) since 1993 [24].

This paper presents the effect of spatial skill training on the spatial skill gains of first-year engineering students at two universities located in different countries—one Spanish, and the other Peruvian.

Previous studies have compared the spatial skills of engineering students to establish the level of these skills amongst students from different countries. One of these studies compared freshmen students from the US with their cohorts in Europe, Brazil, and Africa. It demonstrated that training can enhance visualization skills in a relatively short period of time [9]. A similar study compared the levels of spatial abilities acquired by students at European and American universities from content taught as part of subjects taken in the first year of engineering studies [25]. Two conclusions were drawn from said study: On the one hand, that universities with smaller groups of participants demonstrate greater gains in the level of spatial skills; and, on the other, that the universities that emphasize descriptive geometry in their syllabus tend to achieve higher levels of spatial skills amongst women. A comparative study run at five universities in Austria and Germany looking at student progress in descriptive geometry subjects concluded that these subjects improve two factors—(i) spatial abilities and (ii) logic, which is required to resolve situations and make judgement calls [26].

This paper presents the results of a comparative study of first-year engineering students exposed to a short augmented-reality-based training course designed to improve their spatial abilities. The sample groups for this study are from the Universidad Católica San Pablo in Peru and the Universidad de La Laguna in Spain. The purpose of this study is to establish the symmetry between these two groups from different countries in terms of initial spatial skill levels and spatial skill gains following identical training sessions.

Despite being in the pilot phase, the study highlights indicators of interest that are worth studying in-depth through longitudinal studies. Studies of this nature could involve participants from all over the national territories of both countries. In the case of this particular study, the following observations were made prior to performing training:

• A significant difference exists in the initial spatial skill level of Spanish and Peruvian engineering students. The Spanish participants possess a higher level than their Peruvian cohorts.
• No differences in spatial skill levels based on gender were observed in either group, as found in studies run in developing countries over recent decades.

The analysis of results of training performed to improve students’ spatial skills reveals the following:

• Training was effective at both universities, as all students made significant gains in their level of spatial intelligence.
• No significant difference was found between the study groups at the two universities.
• In both groups (Spanish and Peruvians) there was no difference based on gender i.e., both male and female participants made the same gains.
• The training served to help both Spanish and Peruvian students improve mental rotation skills (spatial relations) to the same degree.
• A difference was detected between both groups with regards to improvements in the component spatial visualization. In this case, the experimental group in Peru made greater gains than the experimental group in Spain.

This paper introduces related research in Section 2, and contains a brief description of spatial ability, its components, how these components are measured, and the training methods used to improve them. This section justifies the importance of spatial skills in engineering and mentions relevant training based on augmented reality technology that has been used to improve this skill. Section 3 describes the study objectives. Section 4 details the methodology used to carry out the pilot study and includes a description of the training materials, measurement instruments, participant samples, and experimental design. The results and statistical analysis are described in Section 5. Finally, the critical discussion and conclusions are presented in Section 6 and Section 7, respectively.

2. Related Research

Spatial skill is defined as a combination of component abilities, i.e., the ability to imagine the movements of objects, mental rotation, and spatial transformations. Students’ spatial abilities can be enhanced through specific educational programs [27]. Engineering design activities can serve to engage students and thereby assist them in developing self-efficiency and an understanding of engineering design processes [28]. Nonetheless, traditional classroom settings usually lack design activities, and upon graduation, students have often not received the training needed to solve the design problems that they are likely to encounter in real life [29].

Integrated science, technology, engineering, and mathematics (STEM) education bridges separate disciplines. Research has demonstrated this increases academic achievement across related fields [29]. Findings also suggest that STEM activities containing engineering design practices help students to develop their spatial skills [30].

Spatial ability is important when it comes to solving mathematical problems, both in terms of understanding and configuring abstract spatial representation [31]. An 11-year longitudinal study performed using high school students in grades 9–11 provided evidence that spatial ability is important for developing STEM proficiency [2]. For his part, Lubinski claims that spatial ability is a personal distinction in human development and creative achievement [30]. According to this author, it would be difficult to conceive of someone being able to acquire advanced learning in STEM fields without first possessing spatial skills. In line with this, a study by Uttal and Cohen demonstrated the correlation between successful performance in STEM disciplines and spatial ability [32].

2.1. Spatial Abilities: Components, Measuring Instruments, and Training Methods

Martin Gutierrez provides an explanation of the terms spatial skill and spatial ability and their components [1].

Spatial visualization: “Spatial visualization skills are an important component of engineering because of their direct relationship to the graphical communication associated with design”. This skill is considered one of the most important skills in engineering and technical graphics [33]. Strong spatial skills have been shown to be associated with overall success and achievement, but also retention in engineering programs and success in mathematics [34].

Contrary to what many may think, spatial ability is not the sole domain of engineering and architecture [35]. As demonstrated by countless lines of research [36,37,38,39], spatial ability is one of the many components of human intelligence. However, there is no clear agreement on which sub-abilities fall under its umbrella. McGee distinguishes five components of spatial ability: spatial perception, spatial visualization, mental rotation, mental relation, and spatial orientation [34]. But, perhaps the most widely accepted theory is that offered by Lohman [37]. Also widely accepted is the Meta-analysis carried out by Linn and Petersen [40], which mentions three kinds of spatial ability—(i) spatial perception, which “requires participants to locate the horizontal or the vertical in a stationary display while ignoring distracting information”; (ii) mental rotation, which “involves the ability to imagine how objects will appear once they are rotated in two or three-dimensional space”; and (iii) spatial visualization, which “refers to the ability to manipulate complex spatial information when several stages are required to produce the correct solution.” Halpern and LaMay [41] add a further two distinct skills to the three just mentioned, thus contemplating a total of five components: the first, spatial-temporal ability, requires the ability to visualize changes that occur dynamically or during movement; the second, the generation and maintenance of a spatial image, is the ability to generate a specific mental image that is used to perform a specific cognitive task. Other authors have simplified the classification into only two components—spatial relations (which includes mental rotations and spatial perception) and spatial visualization [42,43]. In line with this simplified classification, Carroll states that spatial ability consists of two sub-abilities: spatial visualization; and mental rotation [44]. These are defined as follows:

• Mental rotation or spatial relation: “The ability to imagine rotations of 2D and 3D objects as a whole body.” Mental rotation is therefore the mental speed involved in twisting and turning simple shapes and recognizing them once in another position.
• Spatial visualization: “The ability to imagine rotations of objects, or their parts, in three spatial dimensions by folding and unfolding.” Visualization is therefore the ability to mentally manage complex shapes.

The literature on the subject of how to measure spatial ability is quite dense [35,45,46,47,48]. There are numerous measuring instruments available, yet not all use the same components of spatial ability.

Several studies measuring differences in spatial abilities between men and women have found that men have an advantage over women when it comes to mental rotation tasks [40,49,50]. Some authors have even suggested that the differences observed may have arisen due to the influence of people’s social status [51,52], or environmental and socio-cultural factors [53]. A good indicator would therefore be knowledge of the relationship between spatial skill and the typical tasks that men and women carry out in their day-to-day lives. Certain studies that have been performed along these lines have concluded that video games or technology-based training using engaging technologies could in fact serve as tools to improve people’s spatial abilities [54,55]. More recent studies are showing that in some areas of the world gender differences in spatial skill are not significant [16] at this time.

Spatial abilities can be developed with specific training [56]. The methodologies to do so can differ depending on the resources used, i.e., pen and paper sketches, multi-media platforms, video games, virtual reality, augmented reality, specific software, physical materials, etc. [11,57,58]. Nonetheless, as Mohler mentions “The most important aspect of activities designed to improve spatial ability is that they be context specific” [59].

Content related to technical graphics subjects (e.g., descriptive geometry, orthographic views, three-dimensional modelling, among others) has been used to improve engineering students’ spatial abilities. Several researchers acknowledge the impact of sketching on spatial ability [43,59,60]. Other studies conclude that solid 3D models and animation may help in developing visual perception abilities [33,61]. Recent studies have explored how 3D engineering modelling using the computer program SketchUp affects the development and improvement of students’ spatial thinking and visualization [57]. Virtual-reality-based technology has also proven to be a highly effective tool for spatial skills training [12,14]. An experimental study with secondary school students performed by Rafi et al. demonstrated that training based on an interactive virtual desktop environment for spatial training improved men’s abilities more than women’s [62]. More recent studies, however, have shown that the use of augmented-reality-based training or virtual-reality-based training generates the same level of gains in both men and women [16].

In the literature, several contributions exist indicating that short, remedial training sessions are effective when it comes to improving the spatial abilities of engineering students. Such sessions also ensure students reach an optimal level of understanding. This helps them to understand the content taught in graphic engineering (and in all engineering subjects).

2.2. Importance in Engineering Studies

Many authors have argued that spatial skills are of vital importance within the context of engineering, with some authors demonstrating this to be the case [56,59]. A number of studies assert that there is a direct relationship between academic performance, motivational beliefs, and self-directed learning [63,64] to the extent that a student’s ability to accurately visualize concepts in graphic engineering may affect their academic performance. Burton and Dowling determined that spatial visualization defined as “the ability to understand shapes and mentally rotate them in two dimensions whilst comparing to a model” can serve as a useful predictor of a student’s academic success [65]. This stance has been supported by other studies, which have indicated that a student’s ability to understand spatial relationships in the 3D influences their academic success [66]. These last two studies even showed that there is a direct relationship between the spatial skills of engineering students and their academic performance.

Understanding student motivation in graphic engineering subjects is essential if we are to understand potential problems relating to student retention rates. Engineering studies are regularly affected by the issue of student retention, particularly when it comes to first-year students [67]. According to Sorby, students who struggle with course content from the outset of a course are more likely to lose motivation and drop out of an engineering degree. However, there is much less likelihood of them dropping out if they go into a course with solid spatial skills and continue to make the effort to develop them as the course progresses [68].

2.3. Virtual Technologies and Spatial Abilities

Augmented reality (AR) is a variant of virtual reality (VR). These computer technologies can be used to create immersive worlds and experiences for users. In the case of VR, users cannot see the real world that surrounds them whilst immersed in the virtual reality system. AR, on the other hand, allows users to visualize virtual objects as they overlap or combine with objects in the real world. The goal of this technology is to create the sense that real-world and virtual objects coexist naturally in the same space. Therefore, AR should not be used as a substitute for a real experience, but rather as a tool to complement it. The concept of AR refers to “the extension of the real world by means of synthetic images, due to which it is not required that the scene is completely generated by a computer; however, the synthetic image is used as a complement to the real-world scenes” [69].

In the present study, augmented reality-based technologies have been used as a means to improve students’ spatial abilities. The authors decided to use AR-based technologies was because their motivational impact on learning is well-established [70,71,72], and because they have been demonstrated to facilitate student-centered learning [73].

It is possible to find many experiences involving the use of AR and VR in different educational levels. In the instance of higher education, there are studies that concluded that including AR in graphic engineering courses increased students’ interest and motivation [74,75]. Furthermore, courses based on virtual technologies aimed at improving spatial abilities, specifically visualization and rotation [76,77,78] now exist.

3. Study Objectives

In the university setting, engineering professors have long encountered the problem that students starting a degree in engineering or architecture often lack solid spatial awareness skills. This weakness acts as a barrier to success. It prevents them from understanding and practicing content using representation systems, as these systems require good levels of spatial skills.

The purpose of this study is to analyze the spatial skills of students starting technical degrees. The research detailed in this paper was carried out as a pilot study undertaken at two different universities, one in Spain and the other in Peru. Consequently, the target populations come from different social and cultural backgrounds. The research team performed identical experiments with each university population to improve their spatial abilities. These consisted of delivering a short and intensive AR-based training session. The choice to use AR-based training lies in that it has been previously validated as a reliable tool for improving spatial awareness [78]. The dual aims of this work are to

• Establish the symmetry or asymmetry in spatial skills of engineering students from different social and cultural backgrounds who are starting engineering degrees run at two universities in different countries;
• Analyze gains in spatial skills produced by tasks forming part of AR-based training sessions.

Furthermore, the effects of the variable Gender on spatial skills is also analyzed in this study; in other words, the authors analyze whether male and female participants have the same or different levels of spatial awareness skills prior to commencing training, and whether gender has any influence on the gains made.

4. Methodology

In this pilot study, the target population consisted of students enrolled in graphic engineering at two different universities—the Universidad Católica San Pablo (UCSP) de Arequipa, which is located in Peru; and the Universidad de La Laguna (ULL), which is located in the Canary Islands, Spain. The research team was interested in establishing the symmetry or asymmetry of learning gains that result from identical teaching materials when used by students from different universities who come from different social and cultural backgrounds. The Universidad de La Laguna is located in a historic city on the island of Tenerife. The island’s main economic activities center on the tourism industry and the hospitality industry. The majority of individuals attending the university are of middle class and are predominantly local residents who come from the eight islands that make up the Canary Island archipelago. The Universidad Católica San Pablo is a private university located in Arequipa, the second most important city of Peru. The city itself has the second largest economy in the country, which is driven by the commercial interchange, minerals extraction, and the textile and agricultural industry. Individuals attending this university are of upper class.

4.1. Research Design and Procedure

The experiment was run in an identical manner at each university. In the first few days of the academic year, students were invited to participate in training sessions to improve their spatial abilities. They were informed of the personal benefits of undergoing training sessions to gain these skills, in the sense that these skills could help them successfully study graphic expression. Participants who expressed interest in receiving training were fully informed of the nature of the study and prior consent was obtained before participating in the study.

The participants recruited for the study were divided into two groups—an experimental group (AR Group) that would undergo training and a control group (CT Group) that would not undergo any training.

The pilot study was designed so that data would be gathered in the first week of the academic course, taking advantage of the fact that students had not yet started classes or had any contact with any other university subjects. Although training sessions were designed so that they can be performed without the need for assistance from teaching staff, students were asked to undergo training onsite at each university so that the research team had full control over experimental conditions.

The training itself delivered a total of nine hours of self-study content spread over the course of five days (Monday–Friday). The training was divided into four 2-h sessions and a single 1-h session. A single two-hour training session was delivered each day (Monday–Thursday), except for the last day (Friday). On the final day, the single 1-h session was used solely for evaluation purposes. One training session (one day) is the equivalent to one level. Training required the use of an “augmented book” containing different sets of tasks designed to exercise and develop spatial awareness skills [79]. The underlying concept of the book consists of combining AR features with those of a physical textbook. In other words, digital content is associated with content taught using printed text in a paper book. In doing so, it is possible to greatly broaden the amount of didactics resources available. In order to do this, the research team developed a piece of software to access and display the content contained within the Augmented Book. This software is based on a personal toolkit used for AR marker tracking that allows 3D objects to be visualized in augmented reality [76]. To run the training, the computer software associated with the augmented book was installed on the computers in the computer laboratories at each university.

Prior to training, spatial ability pretests were administered to both the experimental group and control group at each university in order to establish spatial ability levels prior to training. Next, the experimental groups underwent the proposed training, whilst the control groups received no training. At the end of the five days, spatial ability posttests were administered to both the experimental and control groups to measure gains and establish spatial ability levels upon completing training.

4.2. Measuring Instruments

To measure spatial ability in terms of the two sub-skills outlined above, the research team selected the following two instruments in line with the classification offered by Carroll [44]: first, the mental rotation test (MRT) to measure spatial relation [48]; and second, a subset of the differential aptitude test (DAT5-SR) to measure spatial visualization [45] (see Figure 1).

• Orientation tests (MRT): As seen on the left in Figure 1, this instrument contains tasks that require users to imagine rotations and transformations in two and three dimensions. Some authors have pointed out that these tests may influence mental rotation and spatial perception factors.
• Three-dimensional visualization tests (DAT5-SR): As seen on the right in Figure 1, this instrument contains tasks that require users to visualize a three-dimensional object created from a two-dimensional pattern. Thus, students are required to mentally fold a flat object in order to choose the correct 3D object out of the alternatives provided.

4.2.1. Mental Rotation Test (MRT)

The research team selected the well-validated mental rotation test to measure the spatial relation factor [48], having taken into account the context and previous experiences of the field of engineering. In MRT, users are presented a figure. They are then presented with an image of the same figure that has been rotated by a certain angle together with alternative options. The MRT can capture response times and log how long it takes subjects to decide which rotated figure correctly matches the original. The MRT contains a total of 20 items. These items are divided into four subgroups, each containing five 3D figures. Each item is composed of an original sample figure and four additional figures—two matches the sample figure, but have been rotated by a certain degree, and two do not match the sample. The task consists of comparing the original sample figure against the four additional figures that have been provided and deciding which figures do not match original figure. The process of validation for this test indicated a high degree of reliability or consistence (0.88 from a sample of 3268 adults), which validates the MRT as a tool for measuring spatial skills. It has been designed specifically for measuring mental rotations and, as it presents a higher degree of difficulty than other similar tests, it has frequently been used at university level [80].

4.2.2. Differential Aptitude Test (DAT)

The differential aptitude test (DAT) consists of a battery of tests. The subset DAT5-SR contains 50 items. This subset was used in this study to measure spatial visualization. As mentioned above, this instrument measures students’ mental dexterity by tasking them with creating a three-dimensional mental image from an unfolded two-dimensional figure. This test has surpassed a reliability test, indicating that the 93% of variance is due to the real measurement and 7% due to errors. This demonstrates that it is a reliable and consistent measurement tool.

4.3. Participants

This research employed the use of the non-probability sampling technique of convenience sampling. In both universities the study participants were first-year students; however, in the Universidad Católica San Pablo (UCSP), students were recruited from the degree in civil engineering, whereas in the Universidad de La Laguna (ULL), students were recruited from electronic engineering and mechanical engineering. In both cases, the students had to be enrolled in computer-assisted graphic expression and design. The contents of this course revolve around learning different object representation systems, which in itself constitutes spatial awareness training. However, neither institution has set a minimum baseline for spatial awareness skills, thus creating the current vacuum that is preventing students from understanding content.

Table 1. Participant population and sample sizes by university.

		UCSP	ULL
Total Population		134	178
Experimental Group (n = 31)	Male	19	58
	Female	12	26
	Total	31	84
Control Group (n = 31)	Male	21	15
	Female	10	10
	Total	31	25

below presents the distribution of participants by university.

4.4. Training Material

The augmented book, which is designed to be used as didactic material, was created using Bloom taxonomy. The book was divided into five levels, each one containing a different array of exercises. Students were expected to complete one level per session. In the case of levels 1–4, students were given two hours per level, although in the case of the level 5 (evaluation), students were expected to complete all six exercises in just one hour without any help from a virtual model. The first exercise on each level and of each typology has a physical gesture related to it. Students were required to perform the gesture in order to find the solution to the exercise.

• Level 1 (Knowledge) contains three types of tasks. Identify vertices and faces in orthographic views and axonometric projections.
• Level 2 (Comprehension) contains two types of tasks. Identify orthographic views.
• Level 3 (Application/Analysis) contains two types of tasks. Identify the spatial relation between objects to identify how many objects are in contact with a selected object; identify the minimum number of views to define an object.
• Level 4 (Synthesis) contains two types of tasks. Sketch the missing orthographic view.
• Level 5 (Evaluation) contains one type of task. Sketch the axonometric perspective.

All of the elements required to perform the tasks are viewed using augmented reality.

Each page of the Augmented Book contains one exercise. In the upper-right-hand corner of the printed page there are a pair of fiducial markers (see Figure 2). When a printed page is captured onscreen via the computer’s webcam, the AR app detects these fiducial markers and identifies which corresponding exercise should be displayed. The user can then interact with virtual content and manipulate figures with their hands using the general markers, either from the printed book or on their smartphones (see Figure 3).

5. Results and Analysis

The SPSS statistic package was used to design the databases, and to capture statistical analysis and data correlations.

As mentioned previously, the AR-based training sessions were run at two universities and the total study population consisted solely of first-year students (Figure 4). At each institution, the population was divided into an experimental group (AR) and a control group (CT).

Table 2. Mean values and standard deviation for populations and study samples prior to training.

Group	Pre-MRT Mean Value (SD)	Pre-DAT5-SR Mean Value (SD)
Total population ULL (N = 178)	19.05 (8.40)	27.96 (9.20)
Group AR ULL (n = 84)	15.85 (6.55)	25.99 (9.46)
Group CT ULL (n = 25)	17.44 (9.82)	28.40 (10.17)
Total population UCSP (N= 134)	17.01 (7.46)	24.51 (8.27)
Group AR UCSP (n = 31)	18.52 (7.80)	26.42 (7.47)
Group CT UCSP (n = 31)	16.00 (7.46)	22.94 (8.14)

provides the descriptive data for the study groups prior to commencing training.

5.1. Analysis of Samples

First, the research team checked whether the data gathered on the population samples using the measurement tests follow normal distribution.

The Kolmogorov-Smirnov test was run on data from the MRT and DAT5-SR from each university. The results of the normality tests for both populations at both universities indicate that the data for DAT5-SR follow normal distribution (p-value = 0.200), while the data for MRT do not follow normal distribution (p-value = 0.004 and p-value = 0.011).

In order to perform the statistical analysis, it was necessary to establish two facts: (i) whether the experimental groups and control groups at each university were homogeneous in terms of their spatial skills, and (ii) whether they were representative of the student population at each university. The Kolmogorov-Smirnov test (sample of more than 50 people) and Shapiro-Wilk test (sample of less than 50 people) were used to test the normality of each sample.

Table 3. Normality test for samples from both universities (total population, experimental group and control group).

Test administered/	Group	Kolmogorov–Smirnov			Shapiro–Wilk
University		F	gL	Sig.	F	gL	Sig.
MRT_ULL	Total population_ULL	0.083	178	0.004
	AR_ULL	0.111	84	0.012
	CT_ULL				0.917	25	0.044
DAT5-SR_ULL	Total Population_ULL	0.045	178	0.200 (*)
	AR_ULL	0.082	84	0.200 (*)
	CT_ULL				0.965	25	0.526
MRT_UCSP	Total Population_UCSP	0.089	134	0.011
	AR_UCSP				0.908	31	0.012
	CT_UCSP				0.937	31	0.069
DAT5-SR_UCSP	Total Population_UCSP	0.070	134	0.200 (*)
	AR_UCSP				0.972	31	0.573
	CT_UCSP				0.981	31	0.828

* Greater than this value.

summarizes the result of the normality distribution analysis of the study samples from each university: total population, experimental group, and control group. It was observed that the data for spatial visualization (DAT5-SR) follow normal distribution in all three samples from both universities. However, the data for spatial relation (MRT) do not follow normal distribution. Next, the data that followed normal distribution (p-value > 0.05) were analyzed using parametric statistical tests (Student’s T-Test and ANOVA). Data that did not follow normal distribution (p-value < 0.05) were analyzed using nonparametric statistical tests (Kruskal–Wallis).

The ANOVA analysis of DAT5-SR data and the Kruskal–Wallis analysis of MRT data confirm that the two samples (experimental and control groups) were representative of the total population at each university. There were no significant differences between the groups prior to spatial training in UCSP, (F_2,131 = 1.432, p-value = 0.241 on DAT-5:SR and F_2,131 = 0.893, p-value = 0.411 on MRT). The same result was obtained for the groups in ULL prior to training (F_2,175 = 1.417, p-value = 0.244 on DAT5:SR and F_2,175 = 4.595, p-value = 0.096 on MRT).

From the results obtained, it was observed that there is no significant difference (>0.05) between either the mean values for either experimental group when compared against their respective control groups. This means both the experimental and control groups at each centre are statistically equivalent in terms of spatial visualization and spatial relation at the outset of this study. As a consequence, it is possible to state at this stage that any differences observed between the experimental and control groups following intervention are due to the AR-based training sessions, and not due to any other factors or variables.

5.2. Analysis of Spatial Awareness Skills of Both University Populations Prior to the Intervention

At this stage of analysis, the research team checked whether there is significant difference in the level of spatial awareness between the two university populations prior to receiving training. A student’s t-test on independent samples produced p-value = 0.001 and T = 3.419 upon comparing the DAT5-SR data of both populations. The Kruskal–Wallis test gave a value of χ² = 4.559 and p-value = 0.033 upon comparing MRT data of both populations. This indicates that there is significant difference in the levels of spatial skills between both sample groups prior to the AR-based training sessions. If we look at the measurement values shown in Table 2, it is possible to state that both the experimental and control group at ULL possess a higher level of spatial ability prior to commencing the experiment than their cohorts in Peru.

5.3. Analysis of Spatial Skill Gains in ULL Population

The objective of this study is to assess whether the students made any gains in spatial ability components after undergoing AR-based training. In a controlled experiment, students’ abilities were measured prior to and following the proposed training. The research team then compared the before and after results obtained for both the experimental and control groups at both universities.

Table 4. Data by gender: mean pretest and posttest gain scores (std.dev) for MRT and DAT tests.

Groups	Gender	Pre-MRT	Post-MRT	Gain MRT	Pre-DAT	Post-DAT	Gain DAT
Group AR ULL	Males (n = 58)	16.86 (6.98)	25.10 (7.43)	8.24 (6.21)	25.81 (9.31)	34.97 (9.75)	9.16 (5.31)
	Females (n = 26)	13.58 (4.85)	21.23 (8.84)	7.65 (6.51)	26.38 (9.96)	34.88 (8.89)	8.50 (5.26)
	Total (n = 84)	15.85 (6.55)	23.90 (8.04)	8.06 (6.27)	25.99 (9.46)	34.94 (9.44)	8.95 (5.27)
Group CT ULL	Males (n = 15)	19.93 (10.54)	25.87 (9.51)	5.93 (5.08)	30.80 (11.14)	35.17 (12.38)	4.47 (6.79)
	Females (n = 10)	13.70 (7.62)	16.40 (7.93)	2.70 (1.89)	24.80 (7.67)	30.90 (10.87)	6.10 (7.88)
	Total (n = 25)	17.44 (9.82)	22.08 (9.94)	4.64 (4.36)	28.40 (10.17)	33.52 (11.77)	5.12 (7.13)
Group AR UCSP	Males (n = 19)	19.47 (8.18)	28.00 (8.58)	8.53 (8.52)	24.58 (7.09)	38.74 (6.43)	14.16 (6.36)
	Females (n = 12)	17.00 (7.23)	21.58 (8.65)	4.58 (5.40)	29.33 (7.41)	40.67 (6.44)	11.33 (4.46)
	Total (n = 31)	18.52 (7.80)	25.52 (9.04)	7.00 (7.62)	26.42 (7.47)	39.48 (6.40)	13.06 (5.79)
Group CT UCSP	Males (n = 21)	17.14 (8.22)	19.05 (7.89)	1.90 (1.41)	21.33 (7.66)	23.14 (7.06)	1.81 (1.57)
	Females (n = 10)	13.60 (5.06)	16.90 (2.92)	3.30 (2.36)	26.30 (8.49)	28.00 (6.73)	1.70 (2.11)
	Total (n = 31)	16.00 (7.46)	18.35 (6.72)	2.35 (1.85)	22.94 (8.14)	24.71 (7.23)	1.77 (1.72)

shows the results of the pretests and posttests. The mean gain values for the pre- and post- tests of UCSP students who underwent training are 7.00 (Sd = 7.62) for the MRT and 13.06 (Sd = 5.79) for the DAT-5:SR, whilst the same values for ULL students are 8.06 (Sd = 6.27) for the MRT and 8.95 (Sd = 5.27) for the DAT-5:SR.

The research team proceeded to explore whether there is significant difference between the control group and the experimental group at the Universidad de La Laguna following training.

The following research hypotheses (HR) were defined to verify in the Universidad de La Laguna:

• HR1_ULL: the experimental group demonstrates an improvement in spatial visualization measured with the DAT5-SR following the proposed training experiment.
• HR2_ULL: the experimental group demonstrates an improvement in spatial relation measured with the MRT following the proposed training experiment.
• HR3_ULL: the gain scores in spatial visualization are equal for male and female participants.
• HR4_ULL: the gain scores in spatial relation are equal for male and female participants.

A one-factor ANOVA was performed on the gain values of the component DAT5-SR (F_1,107 = 8.586 p-value = 0.004), and a Kruskal-Wallis test on the component MRT (χ² = 6.968 p-value = 0.08). In both cases there is significant difference. Consequently, research hypotheses HR1_ULL and HR2_ULL are accepted. Thus, it is possible to state that the AR-based training did improve both spatial ability components in the ULL population.

When data was analyzed by gender, there was no significant difference in either of the two components. The ANOVA analysis of the spatial visualization component (DAT5-SR) and the Kruskal-Wallis analysis of the spatial relation component (MRT) produce p-values > 0.05 (F_1,82 = 0.275 p-value = 0.601 and χ² = 0.203 p-value = 0.652) when taking the variable gender into account. As such, the research hypotheses HR3_ULL and HR4_ULL are accepted. Thus, it is possible to state that male and female participants made the same gains in both components.

5.4. Analysis of Spatial skill Gains in UCSP Population

The analyses that were performed for ULL were repeated for data gathered from the Universidad Católica San Pablo. Again, the research team shall explore whether significant differences exist between the control group and the experimental group at the UCSP following training.

The following research hypotheses have been defined to verify in the Universidad Católica San Pablo:

• HR1_UCSP: the experimental group demonstrates an improvement in spatial visualization measured with the DAT5-SR following the proposed training experiment.
• HR2_UCSP: the experimental group demonstrates an improvement in spatial relation measured with the MRT following the proposed training experiment.
• HR3_UCSP: the gain scores in spatial visualization are equal for male and female participants.
• HR4_UCSP: the gain scores in spatial relation are equal for male and female participants.

A one-factor ANOVA was performed on the gain values of the component DAT5-SR (F_1,60 = 108.235 p-value = 0.000), and a Kruskal-Wallis test on the component MRT(χ² = 9.494 p-value = 0.002). In both cases there is significant difference. As such, the research hypotheses HR1_UCSP and HR2_UCSP are accepted. Thus, it is possible to state that the AR-based training did improve both spatial ability components of in the UCSP population.

When data was analyzed by gender, there was no significant difference in either of the two components. The ANOVA analysis of the spatial visualization component (DAT5-SR) and the Kruskal-Wallis analysis of the spatial rotation component (MRT) produce p-values > 0.05 (F_1,29 = 1.797 p-value = 0.191 and χ² = 0.661 p-value = 0.416) when taking the variable gender into account. As such, the research hypotheses HR3_UCSP and HR4_UCSP are accepted. It is therefore possible to state that male and female participants made the same gains in both components.

5.5. Comparison Spatial Skill Gains between ULL and UCSP Populations

In order to compare the resulting gains made by both experimental groups, the following research hypotheses were defined:

• HR5: There is no significant difference in spatial ability (MRT and DAT5-SR) between the experimental groups of each university.
• HR6: There is no significant difference in spatial ability gain scores by gender between the experimental groups of each university.

A two-way ANOVA compares the gain results for each component in both experimental groups taking into account the variable gender (

Table 5. Dependent variable: GAIN_MRT.

Source	Type III Sum of Squares	gL	Root Mean Square	F	Significance
Adjusted model	145.963 (a)	3	48.654	1.105	0.350
Intersection	4389.275	1	4389.275	99.712	0.000
UNIV	40.484	1	40.484	0.920	0.340
GEND	107.089	1	107.089	2.433	0.122
UNIV * GEND	58.742	1	58.742	1.334	0.250
Error	4886.159	111	44.019
Total	11,982.000	115
Adjusted total	5032.122	114

(a) R-squared = 0.029 (adjusted R-squared = 0.003).

and

Table 6. Dependent variable: GAIN_DAT.

Source	Type III Sum of Squares	gL	Root Mean Square	F	Significance
Adjusted model	449.277 (a)	3	149.759	5.122	0.002
Intersection	9712.699	1	9712.699	332.207	0.000
UNIV	320.366	1	320.366	10.958	0.001
GEND	63.175	1	63.175	2.161	0.144
UNIV * GEND	24.554	1	24.554	0.840	0.361
Error	3245.296	111	29.237
Total	15,335.000	115
Adjusted total	3694.574	114

(a) R-squared = 0.122 (R-squared adjusted = 0.098).

).

The results indicate that both experimental groups improved by the same amount in the component spatial relation (MRT) (F_1,111 = 0.92 p-value = 0.34). In other words, there is no significant difference for the component mental rotation. However, there is significant difference for the component spatial visualization (F_1,111 = 10.958, p-value = 0.001). This indicates that the study population in the Universidad Católica San Pablo in Peru has improved significantly more than that of the Universidad de la Laguna in Spain. The mean gain was 13.06 vs. 8.95, respectively.

In terms of gender, when both educational centers are compared there is no significant difference in gain values for MRT or DAT5-SR, which were p-value = 0.250 and p-value = 0.361, respectively.

6. Discussions

The authors’ initial suspicions in terms of the starting level of students in each country’s experimental group have been confirmed. In terms of participants’ spatial abilities, significant difference exists between the experimental group containing Spanish students and that containing Peruvian students. The research team identified that Spanish students possess a higher level of spatial skills at the onset of the course. It is important to examine the cause for the results obtained in this work. Specifically, we should analyze why Spanish students are at an advantage over their counterparts in Peru and establish what exactly has conferred them this advantage. It might be the education system and the secondary school syllabus; equally, it could be the result of the social environment or other types of activities, etc.

In general, the main research aim of this study has been to examine the effects of AR-based training on spatial abilities; more specifically, however, the goal has been to determine whether it affects gain scores in the components spatial relation (mental rotation) and spatial visualization. Based on the results obtained, it is possible to state that the training did have a positive influence on improving levels of spatial ability with regard to the two components spatial relations and spatial visualization. The p-values stand at well below 0.05 for statistical significance, which means that there is a more than 95% chance that students have improved their spatial abilities as a result of the proposed training. There is a significant difference between the experimental group and the control group when the spatial ability test scores are examined globally, versus when the spatial relation and spatial visualization test scores are examined separately. Therefore, there is a statistically significant difference between the spatial ability of the groups following the experiment. These findings are generally consistent with the results of similar training and research [16,76,80,81].

During the training sessions at the Universidad de La Laguna and the Universidad Católica San Pablo students improved their level of spatial intelligence. Furthermore, in both instances there was no difference based on gender; in other words, both male and female participants made the same gains. However, the question that must now be addressed is whether the effects of training led both experimental groups to improve to the same degree.

On reviewing the results, it was observed that there is no difference for the component spatial relation. In this instance, the training served to help both Spanish and Peruvian students improve mental rotation skills to the same degree. However, difference was detected between both groups with regards to improvements in the component spatial visualization. In this case, the experimental group in Peru made greater gains than the experimental group in Spain. This result requires further analysis to identify the underlying cause of said difference. If identified, this advantage could be used in future training sessions to confer its benefits to all.

Students at the Universidad de La Laguna began training the first week of term, prior to having started any course. The students from the Universidad Católica San Pablo underwent training two weeks before the beginning of the semester, so both experimental groups completed the experience under the same conditions.

The experimental groups’ gains for each training session were compared against the control groups. The results show that the control groups also have higher posttest than pretest scores, however this difference is not significant. The difference is the result of the well-documented phenomenon of test recall [80]. Put simply, during a post-test, individuals become familiarized with the task required of them, and consequently, they are able to submit answers faster.

The students from both experimental groups verbally expressed their satisfaction with the app and the design of the augmented book. It would be interesting to propose the creation of practical and theoretical didactic material in line with the guiding philosophy used to design this training and the technology that was implemented. The teaching staff perceived that students could work autonomously, and that, on the whole, teachers need not intervene during the training sessions. Future lines of research could examine the causes of additional gains in spatial ability (access to technology, entertainment habits, social environment, etc.).

7. Conclusions

Despite there being a significant amount of research on spatial abilities, there is still much to learn and research in terms of their role in engineering. Although the results of this study may have been somewhat expected, the comparison that was performed between both groups has revealed this type of AR-based training assisted both groups in achieving the same gains, even though students from Spain and Peru possessed different levels at the outset. The results that have been obtained in this study should help convince teaching staff in higher education of the benefits of making this type of intervention a mandatory component of the course curriculum.

The authors of this paper are of the opinion that this study could be broadened. The experiment could be run over the course of several semesters in order to analyze how it might affect student retention during the degree. It is even possible to run it at pre-university level. In such a case, it could be used to analyze whether this type of training serves to eliminate gender differences in spatial abilities. As demonstrated by other studies, such training could influence women’s participation in STEM qualifications and lead to greater involvement. In fact, some researchers have mentioned the need to develop spatial abilities so as to increase STEM learning achievements [2,82]. Teachers, however, are often slow to apply practices aimed at developing spatial abilities. In part, this is due to a lack of available resources, and limited career development opportunities [83] in this area.

Poorly developed spatial skills act as a barrier to success in science and engineering qualifications [2]. For this reason, it would be wise to try to include activities that develop these skills within the academic curricula.

The proposed training material presented in this paper has proven to be a viable method for developing spatial abilities in first-year students commencing university. It could potentially also prove viable for students in the final two years of secondary education (e.g., A-level studies in the U.K).