Living in Virtual and Real Worlds: A Didactic Experience ^†

Abstract

During the last decade some technologies have achieved the necessary maturity to allow the widespread emergence of Virtual Worlds; many people are living an alternative life in them. One objective of this paper is to argue that the blending of real and virtual worlds has been happening for centuries, and in fact is the mark of “civilization”. This project presents a proposal to improve student motivation in the classroom, through a new form of recreation of a mixed reality environment. To this end, two applications have been created that work together between the real environment and the virtual environment: these applications are called “Virtual Craft” and “Virtual Touch”. Virtual Craft is related with the real world and Virtual Touch is related with the virtual world. These applications are in constant communication with each other, since both students and teachers carry out actions that influence the real or virtual world. A gamification mechanics was used in the recreated environment, in order to motivate the students to carry out the activities assigned by the teacher. For the evaluation of the proposal, a pilot experiment with Virtual Craft was carried out in a Secondary Educational Center in Valls (Spain).

1. Introduction: Living in Virtual and Real Worlds

During the last decade some technologies have achieved the necessary maturity to allow the widespread emergence of Virtual Worlds; many people is living an alternative life in them. Some authors have proposed that virtual and real worlds are not as different as expected [1,2]. But, how is to live in real and virtual worlds?

There is one initial question that has to be considered: is this new? Given its title, perhaps it can be inferred that the thesis of this paper is that there is a new revolution in the human history, which consist in living in virtual worlds. But this is not the case: living in virtual worlds is the most natural think for human beings since they created civilization.

One objective of this paper is to argue that the blending of real and virtual worlds has been happening since the dawn of civilization. The interleaving of the real world with virtual worlds has been happening for centuries, and in fact is the mark of “civilization”.

However, there is a revolution: the process has been enormously accelerated by the appearance of Information and Communication Technologies (ICT). In the near future many (if not all) areas of the human life will be profoundly changed by this revolution that allows a deeper blending of the real and the virtual world into a unique structure.

If we are going to speak about how the real and the virtual world can mix, first we have to define what the real world is, and what the virtual world is: how to distinguish between both. This may seem easy at the beginning, but it is not so easy.

The nature of the real world is very straightforward: just stop reading and look around. That is the real world. And then the virtual world: you can see an example, out of many, in Figure 1. And this is the first difference between both: there is just one real world, while there are many virtual worlds. Figure 1 is just an example of one of them.

If we want to speak about real and virtual worlds, we have to answer first what qualifies as “a world”. In a first (and simplified) approach we may define “world” as:

• a set of objects,
• one or several persons (we are interested in worlds populated by people),
• a set of interactions/relations between objects/persons: a person may possess an object, a person may speak to another person, an object may communicate with a person, an object may mediate the communication among persons, etc.

In a “world”, people can interact with objects and with the other people, triggering relationships. In the real world objects are material and relationships are mediated by the physical laws. On the other side, in a virtual world objects are not material and relationships are much more varied, and are mediated by the human though. The place where the virtual world exists is in our mind, and in the extensions of the mind: information carriers, such as paper and computers.

But things are a little more complex than that: virtual worlds are always mixed in some way or other with the real world. The main reason for this fact is that people lives in both simultaneously, and this has been happening for centuries.

1.1. An Initial Example

Let’s see an example of what we mean. One of the main activities that humanity has performed during history (and probably before) is commerce. That is clearly a real world activity. At the beginning barter was the way of performing commerce: one sheep may be exchanged for some sacks of wheat, and sheep and wheat are obviously real world objects. The amount of cattle and the yearly production of cereals have been used as a measure of “wealth” during centuries.

But carrying around sheep and wheat sacks is inconvenient, so humans found goods that made interchanges easier: for example, gold, silver, copper, cowry shells, etc. Gold looks also a real object, but it is not so real: it may be argued that is less real than a sack of wheat. The value of a sack of wheat is clear: it is related to how many people you can feed with it. But the value of a piece of gold is related to its scarcity, to the vanity of the people that uses gold for jewelry… gold is real, but its value is not so “real”.

Then, beginning in the European Middle Ages, we created the “bills of ex-change”, then the “promissory notes” and finally the “bank notes”. These were pieces of paper which could be exchanged for real gold coins. Since the middle of 20th century these pieces of paper were no longer exchangeable for real gold, but however continued being accepted as a means of interchange. They are real objects, but its value is completely virtual: it resides in the mind of the people that uses them. Moreover, in the last decades we have created the plastic money and the electronically kept bank accounts: this is completely virtual money. In fact, most of the money used today only exists in the “virtual world” (and that was true before the computer era, indeed). And finally, bitcoin and other virtual payment systems have carried this process to the extreme: today we can state that money lives in a virtual world. The moral of this example is that being real or virtual is not a black or white property, but there are levels of grey there.

1.2. A Way of Seeing What Is Happening

Let’s try to explain what is happening here. We have the real world, and one virtual world; in this case, the world of money. The first question is where is this virtual world located. It is located inside the head of people (and may work just living there), but it is also located in the extensions of brains, the so called information carriers: written characters in paper support and bits in computer support.

The main point is that there are many interactions between both worlds. Things that happen in the real world cause other things happening in the virtual world, and vice versa. That is the reason why the virtual world “is relevant”. Money is a virtual thing, but is relevant because events in this virtual world cause events in the real world (a sheep being transferred between two people, for example). Examples of such interactions are the monthly wage a worker receives, buying something using a credit card, transferring money between bank accounts on exchange of a service, etc.

Money is not the only example of a virtual world that exists since a long time and is quite relevant for the real world. What is the difference between a military medal and a virtual prize in a computer game? What is the “reality” of being a PhD? What is the difference of an academic title with being promoted to Ogre class 3 in a virtual world? Are nobility titles a real thing? In general, positions, titles, nominations, honors, live in virtual worlds, but have a big impact in the real world.

Summarizing:

• Creating and interacting with virtual worlds is the base of civilization.
• Virtual worlds live in the head of people.
• Information and Communication Technologies, which can be considered as “extensions of the brain”, are accelerating the creation and use of virtual worlds, thus creating deep changes in society and making civilization evolve quicker than ever.

1.3. Enabling Technologies for Mixed Worlds

As it has been mentioned in previous sections, information technologies have accelerated the creation and use of virtual worlds. Perhaps one of the first attempts to create virtual worlds that are isomorphic to the real world are the online multiuser games, in particular the so-called Massively Multiplayer Online Role-Playing Games (MMORPG), in which users interact with objects and other users in a scenery. The pioneers were the conversational adventures, where players used written dialogs to carry out missions. An example was the Multi-User Dungeon, one of the first computer based virtual worlds [3]. More recently, World of Warcraft [4] is a good example of successful MMORPG, with millions of players involved.

These games showed the possibilities of creating bonds between the real world and a virtual world. People “lived” intensely in the virtual world, feeling successes and failures in the virtual world as relevant as those in the real world. Is it more real to win a tennis tournament than defeating a dragon in the virtual world? Is tennis something more “real” than Starcraft [5]? And chess?

In fact, one interesting example of blending the real and the virtual world are the eSports. Starcraft is an example of an on-line game that has achieved the same relevance than mainstream sports: for some time, Starcraft was the “sport” with most audience in South Korea. It is broadcasted on TV in several countries, there are professional players, people gathers in big numbers to watch the games, and many people is making money (and fame) with their performance in the virtual world. This is mixed reality. Instead of watching real people in the real world, you have real people watching virtual games, with real players, achieving virtual goals and getting real prizes and fame. A quite mixed reality.

A second technology that has shown the possibilities of mixing real and virtual worlds is social platforms. Examples of some of the most successful social platforms are WhatsApp, Instagram, Twitter, and Facebook. Many times a group of friends meet in the real world, and then use a social platform to chat with each other and with other people who is not present: a mix of real and virtual worlds.

Social platforms and online games have some differences and similarities. In an online game users live an alternative life with its own rewards, recognition by their peers, etc. The emphasis is on objectives, achievements, rewards. In a social platform users live in the real world, but interact through a virtual world. Emphasis is on communication and sharing experiences, creation of groups, affective issues. In both cases, deep human needs are involved: the need to belong to a group and the need to be appreciated by peers.

Avatar worlds can be considered as the combination of online multiuser games and social platforms. They are commonly known as “virtual worlds”, but we can see them as a particular example of a virtual world, which is a more generic concept. They combine the advantages of both. A paradigmatic example is Second Life [6], or its open software version OpenSim [7]. Both involve exploration, meeting new people, finding objects, enjoying new experiences, building, creating your own spaces and experiences, and many more things. There are other such worlds, such as Mine-craft [8], and more recently High Fidelity [9].

Immersive 3D Virtual Reality is one technology that has very recently achieved the status of mass acceptance, and that will be used to interface with virtual worlds. Headsets such as Oculus Rift [10], PlayStation VR, Samsung Gear VR and the HTC Vive allow to enjoy an immersive experience in the virtual world, so improving its impact.

However, the most disruptive technology that will change the way we live, making possible to take a step forward in mixing the real and the virtual world is augmented reality. Augmented reality consists in adding virtual elements to the real world. It is based in some technological element that allows “seeing” the mixed world: a window to the mixed world. This window can be achieved through a smartphone (i.e., Pokemon Go [11]), projectors (i.e., MS Room Alive [12]) or some kind of glasses (MS Hololens [13]). Technologies that support augmented reality, and that have achieved maturity recently, are natural language recognition, gesture recognition, computer vision, and the Internet of Things. Technologies that are being developed and that may have impact in the future are ubiquitous surfaces, textile interfaces, eye lenses, artificial eyes, and brain interfaces.

Nowadays the main interface between the real world and virtual worlds is the smartphone. Certainly smartphones are widely spread, are portable, connected, and offer a rich set of functionalities for connecting the real and the virtual worlds: communications (access to Internet); position interface (GPS, compass, accelerometer); video interface (input and output); tactile input interface; audio interface; natural language interface; etc.

This is convenient for the moment, but the smartphone probably will be deconstructed in the near future. Communications and position interface probably will be kept in one gadget, while input/output interfaces probably will develop separately. The first step in this direction are bluetooth headphones and smartwatches. The next step probably will be glasses and textile interfaces.

2. Previous Work in Mixed Reality for Education

Mixed reality is the sphere that lies between the real world and the virtual world [14]. This technology is increasingly used in different fields such as entertainment, advertising, education, etc. Through different tools and techniques, reality is mixed with virtual elements that allow perceiving reality differently [15].

In recent years, mixed reality is becoming a popular practice. Applications such as Pokemon Go or applications with virtual reality glasses have appeared. The use of mixed reality in education avoids accidents or dangerous practices that can occur in reality, for example, some types of surgery, use of construction tools, contact with electricity, etc. [16,17,18].

In some projects various user interface controls (i.e., a joystick) are used as tangible objects to interact in the virtual world [16,19]. On the other hand, Mateu et al. [20] presents a toolkit to develop mixed reality environments with tangible objects and virtual worlds. Three projects are described: (i) Cubic; that allows the teaching of classification algorithms [21], (ii) Touch Eye: that allows the learning of the Catalan language [22], and (iii) Touch Book: that allows the student to carry out activities according to the knowledge acquired [23].

In the case of [18], they use Microsoft’s Holo Lens glasses, developing an application that simulates the use of a grinder and the protective objects that must be used. In [24], a study of learning chemical kinetics with mixed reality is carried out, using low-cost tools that easily exist in an educational institution (projector, webcam, computer).

Finally, in [25] a mixed reality technique different from those mentioned above is proposed. In this case the virtual world changes the real world and vice versa, but not visually, but in terms of academic actions. For example, students in the real world, using a mobile application, answer questions, which if they are correct provide a clue, and with it, they can find objects in the virtual world.

3. An Experience of Using Mixed Reality for Education

A pilot project using mixed reality for education has been implemented and tested in a secondary educational institution: the Institut Narcís Oller (Valls, Spain). The pilot project consists in a virtual world, where students can play and interact among them, and the real world: the real class-room. Things that happen in the classroom have effect in the virtual world, while things that happen in the virtual world have effect in the classroom. For example, if a student behaves adequately in the classroom may obtain “points” that will allow her to buy objects in the virtual world (such as dresses, appliances, etc.) Also, if the student uses educational resources in the virtual world, obtains “points” that will allow him to get “privileges” in the real world: for example, having extra time for the test, or being allowed to eat a sandwich while in the classroom.

Two applications were developed to implement such mixed reality. VirtualCraft (Figure 2) is a smartphone application that the students use in the class, which allow them to earn (or loose) points, and to acquire and use privileges. In the other hand, VirtualTouch (Figure 3) is a virtual world, based in the open software platform Opensim [7], that is connected with VirtualCraft.

There are three possible roles for a student: Knight, Priest and Count. For each role there are six levels, form “apprentice” to “master”. The teacher assigns a role to each student, depending on their personalities. Teams are formed with 4/5 students, mixing all roles. Each role has a set of privileges that can be acquired and executed provided that the student has accumulated enough points, and has evolved to the appropriate level.

The students use VirtualCraft at the classroom, by means of their smartphones. At the classroom they may receive points from the teacher if they perform adequately in their normal duties at the classroom. They may also acquire and use privileges, which are actions in the real world. For example, a priest of the “master” level may acquire and use the privilege of their team having five extra minutes for a test. Many privileges apply to the complete team, so the students are interested in the evolution of all the members of the team. The levels and points obtained at class have impact in the virtual world VirtualTouch.

The students use VirtualTouch at home, as a 3D virtual world. Each team has a parcel of land, where they can develop their own “home”. The first time the team enters the virtual world, they have to select a parcel of land. Once selected, other teams are not allowed to build or place objects in that parcel. Once the team owns a parcel, they can populate it with objects, such as buildings, plants, accessories, furniture, etc., in order to create their own “home”. Objects can be purchased in shops that can be found in the virtual world, using the “points” that are earned in the classroom. Each student can buy different objects, depending on her role and level. So, in order to have an “advanced” virtual home, the team is interested in the evolution of all their members in the real world (the classroom), as well as in obtaining enough “points” during the lectures. In addition, there are educational activities in the virtual world that can be used to obtain “points” that will be used to acquire privileges in the real world.

An initial evaluation of Virtual Craft has been made with 25 students of 2° ESO, in the subject of Social Sciences, for which students were distributed in 5 teams of 5 students each.

The results that the teacher has observed in the students, is that they reacted quite well when using Virtual Craft, improving their behaviour. The application proved to be a playful activity and a new way of teaching. The points system encouraged the students to participate, work and behave better in class. She also mentions that most students, specifically more than half of the class, were interested in the application and began to improve in every aspect.

4. Conclusions and Future Work

This paper proposes that mixing the real world with virtual worlds is a powerful tool for social changes. This “mixed reality” can be used in education for motivating students in their daily duties. A prototype of such ideas has been developed in terms of two applications: VirtualCraft, which students use in their “real world” (the classroom) and VirtualTouch, which students use in their “virtual world” (a 3D game-like environment). Events in the real world may influence the virtual world and vice versa.

An initial pilot test of these applications took place in a Secondary School in Spain. Although the initial feedback from the teacher is very positive, more extensive experiments are needed to get representative results. During the next academic course the system will be comprehensively tested in the same secondary school.

Finally, we want to thank the Intitut Narcis Oller at Valls (Tarragona) for all their support during the experiment. The work reported in this paper was partially funded by the Madrid Research Plan (project e-Madrid-CM / P2018/TCS-4307).