Virtual Reality Learning Activities for Multimedia Students to Enhance Spatial Ability

Abstract

Virtual Reality is an incipient technology that is proving very useful for training different skills. Our hypothesis is that it is possible to design virtual reality learning activities that can help students to develop their spatial ability. To prove the hypothesis, we have conducted an experiment consisting of training the students using an on-purpose learning activity based on a virtual reality application and assessing the possible improvement of the students’ spatial ability through a widely accepted spatial visualization test. The learning activity consists of a virtual environment where some simple polyhedral shapes are shown and manipulated by moving, rotating and scaling them. The students participating in the experiment are divided into a control and an experimental group, carrying out the same learning activity with the only difference of the device used for the interaction: a traditional computer with screen, keyboard and mouse for the control group, and virtual reality goggles with a smartphone for the experimental group. To assess the experience, all the students have completed a spatial visualization test twice: just before performing the activities and four weeks later, once all the activities were performed. Specifically, we have used the well-known and widely used Purdue Spatial Visualization Test—Rotation (PSVT-R), designed to test rotational visualization ability. The results of the test show that there is an improvement in the test results for both groups, but the improvement is significantly higher in the case of the experimental group. The conclusion is that the virtual reality learning activities have shown to improve the spatial ability of the experimental group.

1. Introduction

Virtual Reality (VR) technologies have been used for a long time to support the teaching-learning process in various educational environments at all educational levels. In this sense, Merchant, Goetz, Cifuentes, Keeney-Kennicutt and Davis [1] presented a meta-analysis about the effectiveness of virtual reality-based learning tools on the performance of high school (K12) and university students: a set of 69 studies involving more than 8000 students were meta-analyzed. The conclusions of this research revealed that all studies showed an improvement in learning, with experiences based on individual games providing the best results. Moreover, the broad review made by Akçayır and Akçayır [2] on the advantages and challenges associated with Augmented Reality (AR) for education, found an increase in the number of AR studies during the last years, being the most reported advantage of AR that it promotes enhanced learning achievement. With regard to the main challenges, they reported the usability issues and frequent technical problems. Other interesting work is that of Passig, Tzuriel and Eshel-Kedmi [3] whose goal was to examine the influences of a teaching process which takes place VR environment on children’s cognitive modifiability in the domain of analogical reasoning. Children participating in the VR environment showed the highest cognitive modifiability especially on the transfer problems compared to those not using the synthetic environment.

Not everything is advantageous when using VR. Petrakou [4] conducted an online learning course for university students consisting of creating a virtual world, demonstrating that VR facilitated online education. The course provided improved interactivity as it allowed synchronous communication and placed the student in a spatial dimension. However, the author also noted that in order to obtain greater benefits, students should be more familiar with the virtual world environments and improve their technical skills. In addition, it was also necessary to solve the technical problems associated with these computer-generated environments. This is not the only case, as the results of Dalgarno, Lee, Carlson, Gregory and Tynan’s [5] work coincided with Petrakou’s [4] findings that the use of immersive 3D virtual worlds requires levels of instructional and technical support that make them difficult to use.

In higher education, the use of VR is more widespread, especially in the health and engineering fields. Specifically, applications have been developed for simulation of surgical processes and other medical therapies [6,7]. Meanwhile, in the art world, Rubio-Tamayo, Gértrudix and Sáez-López [8] conducted a study evaluating the learning mechanics and resources that enable students to build and interact with their own learning in the context of studying historical heritage. In the case of engineering the use of this type of tools encourages collaborative and interdisciplinary work (Häfner, Häfner and Ovtcharova, 2013), helps in the understanding of complex concepts and allows the users to work in environments and conditions that, for various reasons, are not easy to recreate in a real world [9,10,11,12,13,14,15,16,17].

One of the recurring issues in works on VR-supported learning is how this technology empowers a variety of students’ skills. In this sense, Miyata, Umemoto and Higuchi [18] proposed the formation of interdisciplinary groups to promote creativity in developing VR applications and laboratory skills [19], for instance. Carbonell, Saorín, Melian, Cantero and Cantero [20] proposed a change in the teaching-learning strategy in which the student ceases to be a passive subject who is asked to exercise with a 3D-model to adopt a creative role, since the student creates the object of learning using 3D tools. They have measured the impact of this strategy on creative competence using the Abreaction Test of Creativity. The creativity is also focused on other the works [21,22]. Despite its relevance in autonomous learning and the skills of future engineers and designers, the enhancement of these qualities by virtual learning environments is not sufficiently contrasted. In fact, most of the work focuses on assessing the quality of immersion and interaction with the environment, as well as the comfort of use [23]. Aspects such as fostering imagination and creativity are often overshadowed, especially as they are particularly difficult to measure objectively [24]. Precisely, to mitigate this lack of objectivity, the work of Jimeno-Morenilla et al. [22], proposes the use of the Torrance test [25] to measure the impact of VR on students’ creative capacity.

Other more generalist research has focused on proposing methodologies and recommendations to include VR techniques in different areas. Thus, Bell and Fogler [26] presented a series of recommendations made by Felder and Silverman [27] that were added to Bloom’s taxonomy [28] to focus educational methods on students, depending on the teaching and learning styles of the instructor and student, which often do not coincide. The authors empirically demonstrated that the use of these recommendations allowed for better competence acquisition and increased student motivation. More recently, Hashemipour et al. [10] presented a Virtual Reality environment oriented to the field of mechanical and industrial engineering based on five modules. Each module aimed to support the learning of a component of industrial development. The experience was conducted in two phases: in the first phase, students used a virtual environment to perform certain tasks and, secondly, they performed the same tasks in a real hardware environment. Finally, the students were asked about their understanding of the experiment, as well as a series of questions about their virtual experience. The evaluation of the usability of the system followed the SUMI method [29]. In addition, five specific categories of virtual reality were added to this evaluation: clear entry and exit points, sense of presence, navigation and orientation, faithful viewpoints, and realistic feedback. In the same vein, Sutcliffe and Gault [30] proposed a series of criteria to evaluate the usability of a VR application. The twelve criteria, which should be scored by the users of the environment, are fundamentally oriented to the user interface in each of its aspects.

Moreover, the work of Rizzo et al. [31] outlined the application of a virtual environment for the study, assessment, and possible rehabilitation of a visuospatial ability referred to as mental rotation. The results showed that it appears that for low scorers on the selected pre-test, the use of VR tools produce higher gains on the post-test compared to high scorers on the pre-test. However, they consider that this observation needs to be interpreted with caution as it is derived from an early exploratory analysis on a small number of participants. Another work is that of Carroll, Johnston, & Hale [32], who explore how VR can be used to increase understanding of visual perceptual processes and train these skills using virtual training solutions. This work is part of a book with interesting contributions about Virtual Reality from a wide perspective [33]. Another work in the same direction is that of Carbonell-Carrera and Saorin [34] who present an immersive orientation experiment, in which an urban environment is displayed through a Smartphone installed in VR 3D glasses. The spatial orientation skill is measured using the Perspective Taking Spatial Orientation. The students participating in the experiment present a gain in spatial orientation skill.

Other studies use games to enhance learners’ spatial skills. For instance, Lin, Chen and Lou [35] developed a treasure-hunting game based on theoretical principles of spatial orientation and spatial memory. The authors conclude that the game is an efficient way of enhancing spatial skills over a very short period.

Some studies, however, fail to fully demonstrate the goodness of VR technologies in improving spatial ability. For example, the work of Dünser, Steinbügl, Kaufmann and Glück [36] presents a large-scale study on using 3-D computer technologies to investigate training aspects of spatial ability. They did not find concluding evidences, so they state that more research in this field must be done.

Most of teaching and learning experiences through VR, as well as methodological recommendations for its implementation, lack proven statistical tests that evaluate psychometric aspects, beyond the purely descriptive analysis of opinion survey results. One of the cases in which psychometric tests are used is the one of Jou and Wang [37] in which they considered assessing the influence of VR-based learning environments on the technical skills of knowledge, comprehension, simulation, application, and creativity. In the study, one of the variables measured was the usefulness of these environments in training students’ technical skills. The experimental design had a control group that was taught through a traditional teaching system and an experimental one with virtual reality tools. The analysis of the experiment was carried out by means of a quantitative study of the students’ answers in multi-response tests, as well as open-ended questions for four thematic engineering blocks. However, the tests carried out had very specific questions that did not follow any standard rules.

This research aims to follow a similar model to those proposed above, although standard tests will be incorporated. The design of the experiment will be conducted on a control group and an experimental group, and the evaluation will be carried out by means of contrasted psychometric tests. Specifically, the aim of the article is to evaluate the effectiveness of the use of virtual reality techniques for improving the ability of three-dimensional spatial perception. The experiment will be conducted on a sample of students in a subject on Computer Graphics in the Multimedia Engineering Degree and the Purdue Spatial Visualization Test—Rotation (PSVT-R) psychometric test [38] will be conducted before and after the experiment for each of the groups studied.

The two groups participating in the experiment, control and experimental groups, will carry out the same learning activity with the only difference of the device used for the interaction. In the case of the control group, they will use a traditional computer with screen, keyboard and mouse. In the case of the experimental group, they will use virtual reality goggles to insert a smartphone and use the phone gyroscope to interact. Therefore, the starting hypothesis in this research is that the use of a VR application will significantly improve spatial ability in one group of students over another group using no immersive 3D applications. The following sections will set out the context of the experiment, its implementation and results and, finally, they will allow verifying the hypothesis proposed.

After this first introductory part, the article is structured in the following parts: the second section shows the VR tool used to enhance the spatial ability. The third part explains the methodology used in this experience. The fourth section shows the results of the statistical study carried out to finally present the main conclusions of this research.

2. VR Tool for Spatial Ability Enhancement

Sketchfab [39] is a web-based application to publish and share 3D content on web, mobile, augmented reality, and virtual reality. It is one of the largest platforms for immersive and interactive 3D with over 1.5 million models published. Their technology is integrated with most 3D creation tool and publishing platform and is compatible with most browsers and virtual reality headsets. These features make it very easy to use the models created for this website on most devices, both mobile and desktop.

2.1. Models

Four polyhedral models have been developed for this experiment, similar to those used in the PSVT-R test, but not identical. The four developed models are shown in Figure 1.

2.2. Visualization and Interaction

The Sketchfab viewer has several interesting features. For the objectives of this project, the most interesting feature is the fact that it is multiplatform so that the models can be displayed in any device. In particular, it incorporates the option of presenting a stereoscopic view that can be displayed on a mobile device with the appropriate virtual reality goggles (an image of the goggles and the mobile device are presented in Figure 2). In addition, the virtual scene can also be viewed monoscopically on a computer screen without any changes.

When a mobile device is used for model visualization, the interaction is performed through the gyroscope and accelerometer of the smartphone, which act as a head tracker. This makes navigation very simple and intuitive. When a computer is used, the interaction is done through the keyboard and computer.

2.3. Navigation

Each of the four models can be visualized and manipulated in two ways: selecting a static pose or performing a set of pre-defined movements.

In the static pose, the model appears in a standard fixed position and can be manipulated interactively through navigation. In this mode, navigation is possible in two ways:

• Orbital navigation, which allows zooming in and out the object and rotating it around a predefined rotation axis.
• First person, consisting of a free navigation of the observer, in first person.

Alternatively, a set of pre-defined movements can be performed. It is made of a sequence of movements that has been pre-defined, which includes transformations of translation and rotation that allow all the faces of the object to be displayed. Specifically, the predefined movements are as follows:

• Models 1 and 2 have no predefined movements, so only orbital or first-person navigation is available.
• For model 3 there is a sequence of movements available, consisting of the following three transformations, performed sequentially: rotation of −90° around the Y-axis, rotation of −180° around the Z-axis, and rotation of −90° around the Y-axis.
• For model 4, the sequence of movements available is a little more complex: 100-unit translation on the Z-axis, −100-unit translation on the Z-axis and simultaneously −180° rotation around the X-axis, and finally a 90° rotation around the Y-axis.

The Sketchfab viewer also allows changing some of the view parameters such as enabling or disabling the edges display (wire mode) or changing the color and some of the lighting parameters, such as the diffuse and specular components.

3. Method

3.1. Academic Context

The experience was carried out in a group of Multimedia Engineering students at the University of Alicante, Spain. The main objective of the degree of Multimedia Engineering is “to train professionals in the ICT sector who are able to direct new projects in the field of multimedia, both in the sectors of digital leisure and entertainment and of content management for its dissemination in information networks [...] This training is focused on providing students with the skills to build digital systems for the management of multimedia information, provide technical support for multimedia projects in the field of culture, telecommunications, teaching or business, and create and support the technical elements involved in the creation of image and sound related to digital leisure” [40]. It has two specialties: Digital Creation and Entertainment, mainly oriented to the design and development of video games, and Content Management, oriented to the tools for the content management and dissemination, especially through the web.

The ability to visualize spatially is generally considered fundamental for graphics apprehension, especially three-dimensional solid modelling using computer-aided design software packages [41,42]. This is the case of Multimedia Engineering students, that have several subjects in their syllabus (such as Fundamentals of Graphic Design, Computer Modelling and Animation, Computer Graphics, Videogames and Virtual Reality) covering the following competencies, among others [40]:

• Know and apply basic computer graphics techniques, including 2D, 3D, rendering and lighting.
• Design, construct and animate three-dimensional models, including all stages required for the production of an image or infographic sequence.
• Design and develop video games and simulation systems.
• Acquire and apply the principles, techniques and technologies of Virtual Reality.
• Know, apply and optimize techniques for the development of complex games including advanced graphics, physical engines for videogames, artificial intelligence and network games.

Therefore, the concepts of three-dimensional object, transformations (such as translation, rotation and scaling), projection, volume, and others, are present in most subjects and areas of the curriculum. In this context, the need to develop spatial perception aspects is fully justified in the curriculum of Multimedia Engineering.

3.2. Participants

The group of students participating in this study are taking the final course of the degree. In particular, they are students in the eighth semester of the degree, coming to graduate in a few months. They are therefore highly trained students, who have already made use of this capacity for spatial perception in various subjects.

Due to the availability of virtual reality goggles (20 in total), the students were randomly divided into two groups: the experimental group, which used the goggles, formed by 20 students, although 18 students of the experimental group completed the whole experiment; the control group, which carried out the activities on a computer with a traditional screen, made up of the remaining 54. A total of 43 students in the control group completed the whole experiment.

Table 1. Enrolled students.

	Students	Female	Male	Age
Experiment	18	3 (16.7%)	15 (83.3%)	22.8
Control	43	4 (9.3%)	39 (90.7%)	22.5
Total	61	7	54	22.6

summarizes the distributions of the students in both courses, indicating the number of enrolled students, their gender distribution and their average age.

3.3. Instruments

The PSVT-R test [38] is intended to measure the capability of visualizing rotations. It consists of 30 questions showing first a three-dimensional object with its original view and a rotation of it, projected and in isometric view. Then, another object and five different rotations are displayed, and the student must choose the rotation that matches the given example. Only one of the five options is correct.

All the shown objects consist of simple geometric shapes formed by cubes and cylinders with some modifications. Transformations are combinations of 90 and 180-degree rotations around Cartesian axes. The simplest problems present a single rotation with respect to one of the axes, and the most complex ones contain rotations with respect to two axes. The more and less complex problems are distributed throughout the entire test and there is approximately equal number of problems of each difficulty level. A random example extracted from the PSVT-R test is presented in Figure 3.

3.4. Procedure

All the students of the eighth semester of the degree took part in the experiment, during the class time of the subject “Advanced Graphics Techniques”. In total, 74 students are enrolled in the subject, although not all of them participated in all the tests. Finally, 61 students completed the whole experiment.

The experimental group used the goggles. The mobile device was contributed by the students themselves, having a great variety of models, operating systems and browsers. There was no problem with any of the available devices. The control group carried out the activities on a computer with a traditional screen.

The experimentation was carried out in four stages that are described in the following sections: pre-test, training activity, free navigation and post-test.

3.4.1. Pre-Test

All participants, from both groups, performed the PSVT-R test in the same session and under the same circumstances. All participants had 25 min to answer all 30 test questions, as indicated in the questionnaire instructions. The pre-test took place in the second week of March 2017.

3.4.2. Performing the Training Activity

The next week, all the students carried out the same training activity, with the only difference that the students of the experimental group used the virtual reality goggles and their mobile devices for visualization, while those of the control group used a computer with screen, keyboard and mouse. The training activity lasted 40 min and took place in the classroom during a practical class session. It consisted of two parts:

• Firstly, models 3 and 4 were displayed following the pre-established sequence of movements, as explained in section 0 (20 min).
• Secondly, free navigation was carried out in the viewfinder, with the following requirements: the four models were to be displayed, the two navigation modes (orbital and first-person) were to be used, and the four elevations and two plan views of each model were to be displayed (20 min).

3.4.3. Free Navigation

The students were provided with the glasses for two weeks so that they could wear them outside the classroom. All students were asked to view the models prepared for the experiment during those two weeks and also to visit the Sketchfab website to view other open models shared by other users. The total time devoted to free navigation should be at least one hour. Students in the experimental group should always wear virtual reality goggles while those in the control group should always use their computers. They were asked not to exchange display devices.

3.4.4. Post-Test

One week after the free navigation period, all participants, from both groups, did the PSVT-R test again, under the same circumstances of pre-test.

A graphical representation of the experimentation is shown in Figure 4.

3.5. Design and Data Analysis

A non-equivalent control group design (Campbell & Stanley, 2015) with pre- and post-test control groups was used to test the research hypothesis.

To perform the statistical analysis, the GLM (General Linear Model) with repeated measures of the Statistical Package for the Social Sciences program (SPSS, version 21.0) has been used. This statistical consists of a multivariate analysis of variance (MANOVA) and a univariate analysis of variance (ANOVA). In this factorial analysis of variance, at least one factor is based on independent observations, and at least one of these factors is based on correlated observations (Gardner, 2003). The time in which the measures are taken (pre-test and post-test) was used as the intrasubject factor, and participation in the intervention (using the virtual reality goggles in the case of the experimental group or a computer with a traditional screen in the case of the control group) was the intersubject factor.

Additionally, interaction graphs were created to represent the differences observed between the experimental and control groups in the pre- and post-test situations and to observe the direction of the differences.

Finally, a comparison of averages was conducted to determine whether there were significant differences in the performances of the experimental and control groups.

4. Results

In this research, two groups have been compared that have received different methodologies. As stated in Section 3.5, the independent variable is the type of intervention: the group using VR (experimental), or the group without VR devices (control group). The dependent variables are the scores obtained by the individuals in the PSVT-R tests in the pre-test and post-test situations.

Firstly, in

Table 2. Descriptive statistics for the PSVT-R test.

Group	Score Mean		Standard Deviation		N
	Pre-Test	Post-Test	Pre-Test	Post-Test	Pre-Test	Post-Test
Control	21.12	21.98	5.905	5.958	43	43
Experimental	22.00	25.17	5.901	4.232	18	18

, we present the main descriptive statistics corresponding to the results of the test carried out before the experiment (pre-test) and those of the one carried out afterwards (post-test). The PSVT-R test consists of 30 questions, each one having a value of one point. Therefore, each participant can obtain a score between 0 (all answers are wrong) and 30 (all answers are correct). The answers in Table 2 are the mean of the scores obtained by the participants of each of the groups in the pre-test and the post-test.

The Box’s M test shows the homogeneity of the variance-covariance matrices (p = 0.220) thus showing that the sample of population participating in the research is normally distributed.

Once the normality of the sample has been verified, a comparison was made between the scores of both groups (scores belonging to the interval [0, 30]), before (pre-test) and after (post-test) the intervention, to evaluate the effect of the use of virtual reality glasses on the improvement of spatial perception of students.

The values of the inter-subject test (see

Table 3. Test of inter-subjects effects.

Source	gl	F	p	η2 Partial	Obs. Power a
Intersection	1	868,772	0.000	0.936	1.000
Group	1	1.770	0.189	0.029	0.258
Error	59	(59.493)

^a Computed using alpha = 0.05.

) indicate that the average of all observations differs from zero since the tests have been significant (p < 0.000) for the intercept, but not for group membership (p = 0.189), which allows us to state that there are no significant differences between the groups of students.

In furthermore,

Table 4. Test of intra-subjects effects.

Source	gl	F	p	η2 Partial	Obs. Power a
Spatial ability	1	17.759	0.000	0.231	0.986
Spatial ability * Group	1	5.824	0.019	0.090	0.661
Error (Spatial ability)	59	(5.794)

^a Computed using alpha = 0.05.

shows the effectiveness of the use of VR goggles through the testing of intra-subject effects. In the values resulting from this test, it can be observed that the effect of the interaction between the moment of evaluation (pre-test and post-test) and the application of the methodological intervention with the use of goggles is significant (p = 0.019). The observed power is 0.661 so the rejection of the null hypothesis of equality of averages is correct. The effect size (η²), proportion of total variation attributable to a factor or, also, the magnitude of the difference between the time of implementation [43], which produces the interaction between the test time and the application of the training activity is 0.090.

Finally, to check if there is any difference between the experimental group and the control group both in the pre-test and post-test periods, a t-test was carried out on the mean differences.

Table 5. Parameter estimation.

Time of Implementation	Group	t	p	Confidence Interval 95% Lower Upper		η2 Partial	Obs. Power b
Pre-test	Intersection	15.810	0.000	19.216	24.784	0.809	1.000
	Control	−0.533	0.596	−4.200	2.433	0.005	0.082
	Experimental
Post-test	Intersection	19.355	0.000	22.565	27.768	0.864	1.000
	Control	−2.060	0.044	−6.289	−0.091	0.067	0.526
	Experimental	0 a

^a the parameter has been assigned the zero value because it is redundant; ^b computed using alpha = 0.05.

shows the results of this test, evidencing that there are no significant differences in the pre-test time, and both groups can be considered to start from comparable situations as suggested by the inter-subject test. In terms of post-test time, the test shows a significant difference between the two groups (p = 0.044) and this difference is 2.06 points higher in the case of the experimental group.

Figure 5 shows the scores obtained by both groups before and after the intervention. It can be observed that the experimental group has a greater improvement compared to the pre-test after using virtual reality glasses than the control group whose improvement is minimal and not significant.

5. Discussion and Conclusions

The use of technologies based on Virtual or Augmented Reality is becoming more and more widespread, as devices increasingly offer a greater sense of immersion and their cost is more affordable. The rise of this type of technology has allowed a large number of applications to emerge at all educational levels in order to enhance student learning.

The study carried out prior to this research shows that most educational proposals that include the use of Virtual Reality have enjoyed great acceptance among students and that in some cases there has been an improvement in the learning process. However, many of these studies do not provide the statistical rigor needed to demonstrate such improvement, merely presenting satisfaction surveys that are either very specific or not recognized in the scientific community. Research using standardized psychometric tests to measure students’ abilities is very scarce.

This paper presents a methodological proposal that addresses the use of VR technologies in a course where it is expected that these technologies will have a concrete impact: improving the ability of spatial perception. One of the most important contributions of this research is the evaluation by means of a standard and prestigious psychometric test for measuring spatial ability: the PVSR-T test. The results of the experiments show a significant improvement (+2.06 points) of the experimental group compared to the control group, which supports the starting hypothesis of this research.

Although the proposal focuses exclusively on the area of spatial perception, this methodological approach could easily be extrapolated to the assessment of other skills through the use of other proven psychometric tests.

In this research, a low-cost technology was used that allowed the easy implementation of VR technology in the groups. For this same reason, the main limitation of this study is that these devices offer a low immersion sensation compared to others of higher cost.

Since the results have been positive using this technology, two types of experiments are planned in the future. On the one hand, we propose to extend the use of this methodology to other subjects where spatial perception plays an important role, such as “Computer Modelling and Animation” in the 2018/2019 academic year and to evaluate its impact on both spatial perception and the academic performance of students. On the other hand, it is proposed to make an experience of use with more advanced VR devices such as Oculus Rift©, Microsoft Hololens©, etc. and to check if the improvement of technology also implies an improvement in the spatial perception of students with respect to lower-cost devices.