VoRtex Metaverse Platform for Gamified Collaborative Learning

Abstract

Metaverse platforms are becoming an increasingly popular form of collaboration within virtual worlds. Such platforms provide users with the ability to build virtual worlds that can simulate real-life experiences through different social activities. In the paper, we introduce a novel platform that provides assistive tools for building an educational experience in virtual worlds and overcoming the boundaries caused by pandemic situations. Therefore, the authors developed a high-level software architecture and design for a metaverse platform named VoRtex. VoRtex is primarily designed to support collaborative learning activities with the virtual environment. It is designed to support educational standards and it represents an open-source accessible solution developed using modern technology stack and metaverse concepts. For this study, we conducted a comparative analysis of the implemented VoRtex prototype and some popular virtual world platforms using Mannien’s matrix. Afterwards, based on the comparison, we evaluated the potential of the chosen virtual world platform and the VoRtex platform for online education. After an interactive demonstration of the VoRtex platform, participants were asked to fill out a questionnaire form. The aim was to enable participants to identify the main advantages of online teaching using the VoRtex platform. Finally, the authors analyzed benefits and disadvantages of collaborative learning between the metaverse platform and real-world classroom sessions.

1. Introduction

Over the years, people have used various technologies to adapt education to modern mediums. Online learning using personal computers (PC) and mobile devices has become very popular in the last few years. Modern learning platforms include sharing, monitoring, personalized experience, and different client applications (desktop, mobile, web-based) for accessing the content. Moreover, new mediums that we can use for online learning such as metaverse or virtual worlds (VWs) platforms are emerging. We can define VWs as follows: “Simulated persistent space based on the interaction with a computer, inhabited by several users, who are represented by iconic images called avatars, who can communicate with each other in a synchronized way” [1]. VW integrates several technologies into one platform, which includes audio, video, webcam, text chat, voice chat using voice over Internet protocol (VoIP), graphical editors, and digital avatars. For building VWs, we can use software development tools such as game engines. Game engines provide basic building blocks (rendering, object manipulation, scripting, and other computer graphics features) for VW development. In addition, for designing a high-quality computer graphics-based software, we should follow the best practices and standards regarding user experience design and software optimization [2]. Collaborative virtual environments (CVEs) are VWs that provide a richer collaborative arena for social encounters and community building. Another name for CVEs is virtual learning environments (VLEs), the universal term in the literature associated with education. Examples of CVEs include social entertainment, multiplayer games, interactive art, simulations, distance learning, architectural walk-throughs, cultural heritage, etc. CVEs can offer many features to students who need support in their learning process [3]. In the serious game domain and gamification, there are initiatives for interconnection with systems such as massive open online courses. Gamification is a method that provides an effective layer to learning in the context of engagement and motivation [4]. Furthermore, the VW solutions could provide another way to overcome boundaries during the COVID-19 lockdown while adding a new dimension where we can walk around and communicate with people (digital avatars).

Our idea is to provide a solution for educational purposes by incorporating collaborative learning in VW platforms. In this paper, we evaluated the user experience on the VoRtex game-based learning platform, an open-source project developed for educational purposes. Based on the results, we can use VWs to overcome boundaries in physical distancing while adding a new dimension using VWs. The article is organized in the following way. Firstly, we review related work regarding collaboration and education in VWs. Secondly, we describe the concept and purpose of VW platforms. Thirdly, we introduce a novel VoRtex software architecture for collaborative learning inside VWs. The software architecture included the authentication and recognition of users in a virtual environment using an access control system component. Moreover, we describe other platform components such as a virtual agent, a MicroLesson feature, and a networking system for communication between users. Afterwards, we compare VW platforms Vircadia (based on High Fidelity) [5] and Sansar [6] with a VoRtex prototype using a comparative analysis matrix developed by Mannien in his PhD thesis [7]. Based on the results, we provide ideas for future improvements and for building a VoRtex metaverse platform. In the end, we provide a survey where we evaluate the user experience in the VoRtex prototype for Windows PC.

2. Related Work

Over the years, there have been some studies regarding education through VWs with positive results. Virtual environments offer many assistive tools for students who need support in their learning skills development. For example, students can learn about a new subject and be engaged in tasks. An example is iSocial [8], a desktop application for skill development for students with Autism Spectrum Disorder (ASD) whose aim is to improve their social competencies by enabling social interaction in VWs. Stendal and Balandin [9] claim that people with ASD enjoy using VWs and feel more comfortable communicating in comparison to the physical world. They also highlight that VWs offer a venue for autistic people to be a part of a virtual society and not be dependent on social cues. Fernandez-Herrero and Lorenzo [10] introduced an educational VW software that serves as a tool for improving the social skills of students with ASD. They create a virtual school where the participants can socially interact with several avatars, with a female teacher and six children, and with similar ages to those of the children participating in the study. An educational VW software has been effective in the view of families and therapists, and it had a positive impact on the competencies of the experimental group. Krajčovič et al. [3] developed a case study for educational VW experience and lean management teaching method. It was concluded that the VW experience is an attractive teaching method in a university environment and that the rate of acceptance of the educational game in VW for the 5S methodology is high. In addition, students embraced the integration of VW technology into the process of teaching industrial engineering methods.

Warburton considered multi-user (networking) VWs as a new trend in the field of e-learning using the Second Life platform [11]. A VW was used to motivate students in conjunction with the e-learning software platform Moodle. An integration example of Moodle to the Second Life platform was Sloodle (Simulation Linked Object-Oriented Dynamic Learning) learning system for virtual environments [12]. Sloodle is an open-source solution that integrates the multi-user virtual environment Second Life and OpenSim with the Moodle learning management system (LMS). Sloodle establishes communication and interaction between the objects in Second Life and the materials that are downloaded from Moodle. Furthermore, Sloodle is used by hundreds of teachers and students around the world. Nunes et al. [13] have published research regarding Sloodle. In the paper, they introduced the main characteristics and evaluated the effectiveness of supporting tools in the teaching of algorithms in an engineering course. The solutions used in this research were OpenSim, Moodle, online software IDEOne for the algorithm exposure, and Sloodle. The analysis confirmed that the combination of a VW with Moodle is a valid alternative to the process of teaching, motivating, and stimulating immersion during student activities. Sloodle is currently the most well-known e-learning platform for VW-based education and solution that influences the development of the VoRtex prototype.

Greenwald et al. [14] introduced the CocoVerse framework for collaboration in VWs. The platform was an immersed virtual environment where the users could manipulate objects using hand-based tools. The platform was developed for the HTC Vive and tested on 30 selected candidates. The candidates had different tasks to perform within a VW and with the required collaboration between candidates. After testing, the majority of the candidates objected to the user interface, while some candidates did the task independently more easily. Passos et al. [15] described the CVE for the security training of agents at events. To create the environment, they used Autodesk 3ds Max for 3D model development and Unity 3D for creating the VW. CVE was there to provide the users with a training process where they could recognize a threat such as radioactive substances in the stadium. The procedure was applied at the FIFA World Cup 2014 and the Olympic Games in Rome candidature. The project went through four phases: task analysis using the CTA (cognitive task analysis) set method, the use case for creating a user interface (visual design), user interface prototype, and user evaluation. The user evaluation was done through the user surveys related to communication, collaboration, and management through a VW. The users had a good experience regarding the tasks in the simulation. As in the simulation development proposal, they included the additional option for non-verbal communication. Sharma et al. [16] introduced the Megacity VW for emergency training. Megacity provides collaboration between digital avatars (user’s projection in the world) and artificial intelligence (AI) agents. CVE was created in the Unity game engine while multi-user support was done using the Photon Networking Framework tool. During the test, Oculus Rift was used with the support of the keyboard and mouse; the simulation was tested with ten users and through three scenarios. The users made various decisions during the testing process and based on the results after the evaluation, the authors concluded that the simulation was not yet ready to replace the real-world training. Cruz-Benito et al. [17] introduced a Usalpharma software architecture for learning inside VWs. Usalpharma was a cloud-based architecture with the purpose to extend and enhance the training environments based on VWs. From a cloud perspective, the solution provided support for data analysis of training processes in health and enabled knowledge discovery inside VWs. This solution was based on a software architecture that could be deployed in a cloud with client-server schemes, consisted of several layers (data collection, data persistence, or analysis) connected and enabled the deployment of each layer even in different cloud environments with the potential to scale and apply various technologies to each segment. The primary focus of the Usalpharma solution was the data analysis component that supported the quality assurance training created in Second Life and cloud scaling. When comparing the software architectures, VoRtex and Usalpharma are similar solutions. However, the VoRtex platform provides an access control system to support identity management between users and intelligent agents to assist teachers during learning sessions. Zizza et al. [18] proposed another solution for the virtual learning environment over a high-speed network. They described the implementation of multiple learning modules developed using the High Fidelity VW platform [19]. Modules were implemented based on iSocial’s standards for virtual learning modules. The standards provide guidance such as the environments built with reduced distractions, guiding indicators to direct movement, and locking pods to help students position themselves while viewing a lesson. The instructor’s controls were implemented as a “tablet application” inside the High Fidelity environment. The experiment showed that the VW provided improved immersion in comparison to real-world classroom sessions.

Regarding VW engagement on skills development, Mystakidis [20] has researched gamification in social virtual worlds and its effects on student engagement. Based on the interactive VW experience, participants were able to reach high levels of engagement by integrating quest-based methods and virtual agent solutions inside VWs. Moreover, curriculum gamification in social VW opens new interdisciplinary cooperation that can enrich and differentiate in comparison to current online learning methods. Rodriguez Santiago et al. [21] present two gamification methods developed in the computer science (CS) department at the University of Barcelona, using the VW platform for educational purposes. In the first method, students first attended an online lesson about “Human Factors in Computing” and then competed in the gamified activity. After the first experience, most students had a satisfactory score (60% scored it between 7 and 10). The second method was an online lesson about the “Data Structure”. Students worked in different group sizes to improve their knowledge and skills on problem-solving in the subject. After the second experience, 66.7% of students scored their satisfaction higher than 6 (on a scale of 0 to 10). Gadille et al. [22] provided a multidisciplinary systemic analysis involving didactics, sociology, psychology, and management science on an individual, collective, and systemic scale in secondary school using VW. Pupils and teachers’ creativity appeared to be anchored within four main interdependent nurturing conditions, the personal inclinations and professional interactions in the sociotechnical network sustaining the VW. The pedagogical use of a VW provides novel learning engagement opportunities for the pupils through enhanced experiential learning and sustains the transformation of teachers’ professionality. Zhang et al. [23] developed a virtual laboratory (VL) with experimental hardware in the loop that enables remote access using a VR interface. VL human-computer interaction employs cameras to realize the interaction between the users and the VW and the Microsoft Kinect was used to track the human gestures and send results to the avatar. Using avatars, humans interact with virtual objects. In addition, voice commands were integrated to reduce difficulties in getting started with a VL platform. Two experiments were realized on the group of ninety-four students, using a simple gear train experiment and a planetary gear train experiment. The results indicated that the VL platform was well adopted by the students.

Based on related work, we can expect more VW projects from educational institutions in the future. Related work solutions are lacking features required by the educational standard and our goal with this paper is to provide free open-source solutions accessible to every university (teacher and student) using modern technology stack and metaverse concepts to enable a user-friendly interface that can be afforded by any student and teacher using moderate PC equipment.

3. VoRtex—Metaverse System Description

The VoRtex platform follows a user-centered design. We introduce a web platform, access control, and the MicroLesson feature as the main components (building blocks) of our architecture.

3.1. Architecture Outline

VoRtex software architecture (Figure 1 and Figure 2) is a cloud-based and immersive VLE with multi-user support that can collaborate and learn from remote instructors. The VLE for client desktop applications uses the Unity game engine with support for building desktop and VR clients. The software components follow a loosely coupled design where we can improve each element independently and quickly implement new features.

The software architecture consists of the following platform components:

• VW system (VWs produced by content creators or in this case teachers)
• Digital avatars (selection and customization)
• Intelligent agents (virtual assistants and chatbots)
• Software as a service (SaaS) and Platform as a Service (PaaS) interface for cloud infrastructure
• Learning material (presentations or video tutorials)
• Virtual classrooms (created VLEs for students and teachers)

The VW system should follow standards for content creation:

• Open space with a central area and clear boundaries in which users have the freedom to move and interact. A suitable environment where students can interact with objects and manipulate components.
• Lessons are uniquely designed based on the topic, with immersive and gamification features such as role-playing and teamwork collaboration.
• The lesson has a unique name and direct access to the VW learning environment.
• Each lesson should provide an interactive web page (SaaS component) where teachers have admin controls for manipulating lesson content.
• Allows users in an environment to communicate using the chat system or VoIP.

3.2. Web Platform

A web platform is a useful tool for teachers where they can make VLEs for students without knowing how computer graphics work and using a web browser application. In comparison to the client application, the web platform provides more flexibility regarding content creation. The software design of the web platform component is shown in Figure 3. The web platform provides a SaaS solution for building a virtual environment or VWs. The pattern management component is used to control the available features within each of the templates and provide associated components that are applicable in that context. The user management component is used to provide functionality for the provisioning of users and associated access rights according to the selected package. The content management module is devoted to handling all the content used within the platform such as 2D, 3D models, audio, and video files. Cloud PaaS provides the cloud infrastructure for the environment executed in the VWs. The content engine offers the framework for the management of all the media content, including the 2D and 3D models, sound, and video material. The version control management provides functionality for version control so that the developed VWs can be upgraded to the newest version and synchronized with already deployed VWs. The external service communication enables interfaces to services outside of the infrastructure. For building a web platform, we used web-based technologies such as JavaScript (three.js 3D library) on the front-end side and PHP with MySQL on the backend.

3.3. Access Control

The solution enables authentication and recognition of the user in a virtual environment, and for that purpose, we implement an access control system for identity management. Identity management is an essential feature in education to prevent false student presentations in VWs. VoRtex high-level layer-based architecture access control is represented with a centralized identity layer that is hosted on the secure cloud (server). For access to the platform, we use multi-factor identity verification with enforced access control policies where users can handle personal data without worrying about privacy issues. To preserve the integrity of data within each record, proof of integrity and validation are permanently retrievable from the cloud database. The proposed VoRtex platforms utilize multi-factor identity verification that includes:

• Facial recognition
• Fingerprints
• Speaker (voice) recognition
• SMS (messaging service)
• Email

For face recognition, we can use machine learning modules that use a web camera to recognize users. Regarding other access control features, fingerprints are unique and durable over the life of an individual, making them suitable for representing human identity. Moreover, modern laptops have a fingerprint reader embedded in the device. Voice has features that can be used to recognize the user, like a fingerprint. Using speaker voice (voiceprint) as input for access control simplifies the authentication process. The SMS and email approach provides the user with a pin code or link for accessing the platform.

Fraudulent activities are hard to prevent because of digital identities, so biometric or scanned documents for verification can be stolen and reused. It is essential to make it difficult for the users to represent themselves as someone else using a multi-factor identity, and to collect sufficient digital data to track the users and prove that they have done fraudulent activities. After being platform authenticated, the users can access different virtual classrooms based on an authorization from a moderator or in our case, teachers. The moderators can also use a cloud-based or private server to host created VLEs and upload tests on the secure cloud network with defined permissions for each student.

3.4. Intelligent Agents and MicroLessons

User experience inside VWs can be improved using intelligent agents (virtual assistants or chatbots). Supported by AI, a chatbot or a virtual assistant recognizes the content and understands what it means in a specific context that can be in the form of text, voice, or both. They can utilize information from different sources, using natural language processing (NLP) tools with real-time insights such as Microsoft LUIS, IBM Watson, Amazon Lex, Google Dialogflow, and more. Virtual assistants can increase productivity and automate the process, especially in industries such as education that rely on communication and handle a set of student issues and questions. We designed a component for developing NPCs inside the virtual environment (Figure 4). Based on software architecture, we can use multiple machine learning frameworks that enable us to add intelligent features such as emotion and video detection, speech and vision recognition, and language understanding of our platform. The intelligent agent component is used to enable the cognitive feature to Professor Vortex NPC that we use for tutoring VW users.

The VoRtex prototype adapts the MicroLessons feature. MicroLessons are sessions prepared and created by the teacher using a web platform or the client application on a desktop PC. The MicroLessons structure also includes quest-based challenges such as problem-solving puzzles for different use cases such as learning a new programming algorithm or exploring space to find required objects. Using quest-based challenges, the platform’s goal is to motivate the user and to improve the interface for collaboration during learning. In comparison to a webinar or video lectures, MicroLesson is a VW instance created by a teacher (session owner) and supported with the necessary tools for holding lectures such as presentation panels, video content, and real-time communication. Using the MicroLessons feature, users can understand the value of game-based learning and how to use VoRtex in the classroom. They can also understand how to navigate inside the VoRtex environment, or how to use the assistive tools and 3D assets included with the platform for problem-solving activities. MicroLesson practical examples are:

• Build 3D interactive simulation (introduction to game development);
• Bank resource management and critical thinking (introduction to finance and decision making);
• Real-world physics (a lesson about laws of physics);
• Scientific method (chemistry and biology);
• Flora and fauna exploration (vegetation and animal); and
• Earth exploration and time travel (geography and history).

Inside, MicroLesson students can experiment and practice, interact with different assets, and collect data in a safe environment. In a safe environment, users can move, fly, and interact with objects without the possibility of injury.

3.5. VLE User Interface

The VLE user interface can be classified into three groups:

• Non-immersive systems (desktop PC);
• Extended reality (XR), defined as systems that simulate 3D objects in the virtual or real-world; or
• Immersive systems that create a VW in a designed physical space (CAVE).

We can use the term virtual reality learning environment (VRLE) in the case where we use VR HMD. The proposed architecture should be adaptable for VR and other extended reality technologies such as augmented or mixed reality. We use a non-immersive and XR system to build the user interface (Figure 5). The system uses input processor controls to send information to the device (mouse or HMD trackers) in a compressed time frame. The simulation module represents the business logic of a VR system. It takes the user input and determines the actions inside the VW. The rendering module creates a 3D graphical environment for the end-user. Different rendering techniques are used for haptic or HMD devices. The VR environment also has a database that stores the objects from VWs.

3.6. Implemented Prototype Details

The goal of the prototype is to introduce the VoRtex platform and the following features:

• Networking using LAN network (server and clients),
• Communication using the chat system,
• Freedom of movement inside a VW,
• Freedom of choice of digital avatar representatives in a VW,
• Privileges between user roles,
• Object manipulation over interactable objects in a VW, and
• View perspectives (first-person and third-person).

Currently, there are two versions of the Vortex platform, Windows PC and Android Cardboard VR. Regarding the VoRtex Android Cardboard VR version, the prototype was developed using the Unity game engine. The user can control his digital avatar in the environment by moving his HMD on an object in the VW. The Windows version contains more features and better support. We describe the Windows version below.

For face recognition, we used a face recognition module with OpenCV, dlib (facial landmark library), and deep learning technology stacks. The developed module can recognize the user’s face with different facial expressions. We integrate this feature into the VoRtex prototype after the user enters the login credentials (username and password) and before he or she accesses the active session. Salts and random data are used to safeguard passwords in the database. The VoRtex prototype provides first-person and third-person view perspectives (Figure 6 and Figure 7). During lectures, the user can communicate with the teacher or interact with the presentation panel (right side of Figure 6) and other interactable objects such as the video panel (left side of Figure 7). The interactable object can be synchronized among users such as the presentation panel where the user can change the presentation slide and propagate this change to all users in the session. The teacher provides authorization for in-world content to students.

The VoRtex prototype contains a VW classroom (Figure 8) or MicroLesson with 3D material that supports learning in a real-life environment. The VW classroom contains a table, chairs, a blackboard, projectors, a video and presentation panel, and a virtual agent that assists the teacher during lectures. The users can interact with the intelligent agent using voice controls. Speech recognition (speech to text) software takes the user’s voice as input and determines which words were spoken as output that typically consists of a word graph. Based on the generated output, the event executes inside the VW. We use IBM Watson services in the Unity game engine. IBM Watson allows us to transform the client application into environments where the users and agents can interact using text-to-speech and speech-to-text services. The users can communicate using the chat system (bottom left in Figure 7). We implement a chat system as a separate component with the goal of high reliability and availability in a networking system and to avoid a single point of failure. This means that if a session stops, then we can use the chat system to check with other users what measures to take to fix the issue. The teacher cannot modify MicroLesson in the prototype version using the web platform via a web browser or the client application. We plan to include the web platform in a future version of the prototype.

4. Evaluation of Metaverse Platforms

From a comparative perspective, a VW platform for educational purposes should provide a graphical user interface and full control over the VW or digital avatar. The VW environment can be hosted in the cloud, server, or PC machine. The VW should contain learning content and tools for in-world object manipulation. We compare Vircadia (version 2021.1.3 Eos), Sansar (version R43.4.1), and VoRtex (prototype) VW platforms. Vircadia and Sansar were chosen because of the implemented features that are required for delivering an educational experience and popularity among VW users. We mentioned Zizza et al.’s education experience built upon the High Fidelity platform [18]. Vircadia and Tivoli Cloud VR [24] are open-source forks from the High Fidelity GitHub project [19]. However, both platforms have similar features, so we selected one for evaluation. WEe also wanted to include Fortnite Creative mode [25], this solution does not have VR support. In the end, we selected Sansar because of the scripting features and VR native support. Sansar platform uses photorealistic digital avatars [26]. The motivation for building the VoRtex prototype is to introduce features that are not implemented in the two mentioned platforms, while the goal is to build a better user experience with a focus on learning inside a VW. The involvement of virtual agents and real users provides diverse experiences based on the MicroLesson topic.

4.1. Technology Evaluation Method

The comparison is based on the common features for all platforms. In the VW platform comparison, we established a group of criteria based on Mannien’s matrix [7]. In his work, Mannien used perceivable user–user and user–environment interaction that can be utilized in multi-player video games and VWs. In addition, Reis et al. used Mannien’s matrix to compare VWs such as Second Life, There, and Active Worlds for educational development purposes [1]. Moreover, Lapin compared Multiverse OpenSim and Metaplace VW platforms for learning support in VirtualLife [27]. Lapin compared the following features: installation efforts, content generation, and data import from external tools. The criteria from the Reis and Lapin research were based on legacy platforms, while our version of Mannien’s matrix introduces new criteria features in the comparison process that are common for the new generation of VW platforms. Criteria are based on educational institution needs that can be included inside a modern VW. Furthermore, educational institutions have common interests regarding the possible issues in using a VW platform for education. The needs are represented through the following system requirements:

• Implement an access system for user authorization and authentication (security);
• Enable tools for creating a virtual classroom without additional programming skills (VW design);
• Enable the deployment of the virtual classroom to end-users on cloud servers or local (institutions) machines (scalability);
• Enable virtual assistance during the session that support teachers for predefined courses (content generation); and
• Design immersive environment that can be used via different user interfaces (Communication using PC desktop or VR devices).

In our comparison process, VW platforms were compared using the desktop personal computer with the Windows 10 operating system, Intel Core i5-8400 2.8 GHz processor, 8 GB 2400 MHz DDR4 RAM, and NVIDIA GeForce GTX 1060 6 GB DDR5 VR-ready graphics card. The results were achieved through platforms features testing; all features were selected based on earlier metaverse research [1,27]. Based on Mannien’s matrix comparison process [7], we used weight as a parameter to estimate features. Weight was ranked from 0 to 24 (not applicable, lowest score), from 25 to 49 (shortly applicable), from 50 to 75 (applicable enough), from 76 to 94 (applicable), and from 95 to 100 (an ideal platform). Moreover, we calculated the percentage of compliance for each factor. These values were generated based on the Quantitative Evaluation Framework (QEF) proposed by Escudeiro [28].

The QEF framework measures the quality relative to the value (100%), and the equation is:

1/∑_m PR_m × ∑_n(PR_m × PC_m) × 100 = Factor_n (%)

(1)

In the QEF equation, parameter m is the number of relevant criteria for the factor in the analysis, parameter PRm is the weight of criteria m (10), and parameter PC_m is the percentage of compliance with the criteria. For a better understanding of the equation, we provide an example below:

VW design (Vircadia) = 1/40 × (10 × 0.7 + 10 × 0.5 + 10 × 0.85 + 10 × 0.5) × 100 = 63.75%

The comparison matrix is shown in

Table 1. Mannien’s matrix for the relevant value assigned to virtual worlds.

Criteria		Vircadia (High Fidelity)	Sansar	VoRtex
Virtual world design	Online interaction	70	90	75
	AI in the world (chatbots or NPC)	50	50	80
	The existence of interactive objects	85	85	80
	3D models and environments	50	95	80
Communication	Audio	80	40	75
	Nonverbal	70	70	60
	Text	10	80	70
Avatar control	Move	90	90	90
	Sit	90	50	80
	Fly	90	0	50
Avatar design	Complex	60	90	80
	Configurable	30	80	70
	Upload custom model	90	60	80
Scalability	Multiple servers	90	75	80
	Limit of the world space	70	80	70
Security	Right on digital creations (DRM)	65	50	70
	Data transaction	80	70	50
	Identity management	50	50	90
Content generation	External tools	90	65	60
	Inside the world tools	90	65	60
Development support	Scripts (.NET or Python scripts)	90	70	10
	Open source projects	90	0	90

, and the percentage obtained for each factor is in

Table 2. Percentage obtained for each factor.

Factors (%)	Vircadia (High Fidelity)	Sansar	VoRtex
Virtual world design	63.75	80	78.75
Communication	53.3	63.3	68.33
Avatar control	90	47	73.3
Avatar design	60	77	76.6
Scalability	80	77.5	75
Security	65	56.6	70
Content generation	90	65	60
Development support	90	35	50
Total	74	62.67	69

.

Vircadia and Sansar provide advanced features including multi-user interaction and VW scalability (at least 30 users, average size of school class), which decrease the need for face-to-face communication. Moreover, both platforms offer useful features that we can use for VLE development. Sansar uses better lighting and rendering with photorealistic graphics in comparison with Vircadia. However, Vircadia has a better voice system. Both platforms enable the users to upload a completely custom digital avatar mesh built with 3D modeling software or customize the digital avatar by adjusting the facial features and clothing. Vircadia solution is focused on multi-user interaction using web-based clients. Sansar is Windows only and closed-source, while Vircadia is open-source. Vircadia open-source projects are developed in C/C++, JavaScript, Ruby, Python, and Go. From the user experience design perspective, Sansar provides Atlas for a scrollable list of experiences (or virtual environments), which can be sorted and searched by keywords. Sansar cloud server hosts both published and unpublished experiences. Vircadia has the flexibility for creating and distributing add-ons using the scripting features that enable gestures or voice control. Sansar uses a centralized commerce system. Vircadia and Sansar provide a solid foundation for building a VR environment and interaction. On both platforms, users can import 3D content created in Blender, Maya, or Paint 3D tools.

In comparison, VoRtex has less content than the other two platforms. However, VoRtex innovative design provides out-of-the-box assistive tools for teachers such as presentation and video panels that are not common features in the other two platforms. The VoRtex platform stores 3D object information, currency, and user identity on the blockchain [29]. In addition, Vircadia and Sansar do not focus on a specific niche while the VoRtex platform is targeting early adopters in the fields of online learning. Moreover, the solution to overcome the lack of content inside the VoRtex platform is a content creators component that motivates and engages the users in creating content for the platform [29]. We also included the data transaction in comparative analysis because it can support students to pay exam fees and enable an alternative way for payment by integrating cryptocurrency into the system. Regarding security and access, Vircadia and Sansar use a credential that includes a username and a password for signing into the platform. VoRtex provides more enhanced security measures in comparison to Vircadia and Sansar [29]. Scripting support is a standard feature for Vircadia and Sansar with an idea to automate common tasks in the VW. Scripting support includes writing scripts in high-level programming that we can attach to the 3D object inside the VW or modify the default virtual environment provided by the platform without using assistive tools via the graphic user interface. In the case of the VoRtex platform, scripting support is in the early stage of development, and we did not include it in the VoRtex prototype. Scripting support in VoRtex automates the MicroLesson creation process and 3D content manipulation in the VW. All platforms do not require HMD for interaction inside VW.

In comparison to other VW solutions [16,17,23], our platform does not require additional hardware interfaces for motion capture or cameras for tracking users. Moreover, our solution provides synchronized multi-user interaction in the collaborative environment and 3D photorealistic rendering. Teachers can share presentations and content inside the virtual world or use virtual agents for common questions. Furthermore, users can create content using the MicroLessons feature. Data management can be controlled over a blockchain network [29]. Usage of VR HMD is optional, and the solution is optimized for low-end PC devices (GTX 1060 6 GB or above for high-quality graphics). Regarding the user interface, it is intuitive and user-friendly for gamers and non-gamers, users can use VoIP, gestures, or chat boxes for communication with more than one participant during the session. The VoRtex solution is open-source and available for modification by end-users.

4.2. User Experience Evaluation

In this paper, we additionally organized user testing campaigns with a group of users that experienced the VoRtex platform in the early stage of development (prototype). We can find more about user demography inside virtual worlds from Spence’s research [30]. Based on research, virtual world platforms are mostly used for content creation and socialization (more than 50%) and 10% for education. In addition, platforms are mostly designed for younger audiences (10–15-year-olds). Second Life (design for content creators and serious gaming) is one of the most mature platforms where users are between 30 and 40 years old. Regarding similar platforms for content creators, users vary between 20 and 40 years. Moreover, the newer platforms mostly have a slightly younger audience. VoRtex is designed for content creators and online education and based on Spence’s research [30], our target audiences are between 18 and 38 years old (millennials and Gen Z).

The VoRtex prototype for Windows PC was demonstrated and tested mostly on users (participants) that attended engineering institutions as students or teaching assistants with experience in the e-learning and gamification process. In the survey, we included 24 users where 70% had experience in VW platforms such as Fortnite or multiplayer video games. More than 80% of participants had an engineering background and work in the industry or education sector. The prototype demonstrated the MicroLesson about VR and 3D technologies in a computer-generated classroom. Before the users accessed the environment, they created a login account and chose digital avatars that would represent them in a VW. Using the client-server networking system, the teacher created a server, and after, those students accessed the session as clients. The teacher that hosted the session had the privilege to allow them to manipulate VW content such as a video and presentation panel, where the users could change the presentation slides that were synchronized with the server and all clients during the session. Furthermore, the users could communicate over the chat system and ask a virtual agent (Professor Vortex) about their subjects of interest. Using the networking system, the users could synchronize with each author and follow what other participants were doing during the session. After testing the prototype, the users needed to answer a questionnaire provided by Google Forms. The survey consisted of a few questions regarding VW platforms, user experience, advantages and disadvantages for collaborative learning, and development potential of the following engineering education skills:

• Effective communication: present ideas in a confident, professional manner for future communications community;
• Competence in application and practice, properly utilize engineering techniques, as well as relevant tools and programs;
• Interpersonal and teamwork skills: the ability to function effectively as a team member or leader; and
• Problem-solving skills: how to identify and approach problems and define objectives.

Regarding participant background (Figure 9), most of them were software engineers (15), telecommunication engineers (2), electronic engineers (1), and assistant professors (3), while the rest were mathematicians (1), specialist doctors (1), and philologists (1). The results were organized in percentage charts and provided below.

The majority of participants have previous experience with multi-user environments. After the onboarding session, all participants understand how the platform works and can understand the usage of the VoRtex platform in collaborative learning and interaction between users (Figure 10).

The majority understand the usage of VW for education purposes. Based on previous experience while actively playing World of Warcraft, understanding social activities inside VW gives an idea of how we can change the context from pure entertainment into a collaborative environment (Figure 11). Afterward, in the test session each participant shared a real-world use case with VW solution:

• Collaborative work and training using 3D models (space traveling for astronauts, car manufacturing, industrial equipment, etc.);
• Demonstration and simulation (showing atom and molecules structure, flows of current, electromagnetic fields);
• Virtual surgery and dissection to teach medical students;
• Geography lessons and visiting destinations with our avatar;
• Problem-solving and interactive code review; and
• Conferences and meetups for sharing knowledge (multi-user sessions).

After an introduction to VWs and VoRtex platform design, all participants understood the goal of the demonstration (Figure 12). The next step is real-time interaction inside the platform using digital avatars and platform 3D assets.

After real-time interaction inside the platform using digital avatars and platform 3D assets, participants estimated overall experience on a scale from 1 to 5, 1 being the lowest score (low quality) and 5 highest scores (high quality). Based on results, 41.7% estimated VoRtex as a high-quality solution while more than 90% gave a score of 3 or larger (Figure 13).

Participants praised the networking and the presentation panel implemented in the VoRtex platform (Figure 14). Using the networking system, they could contact the teacher or other students without disturbing another participant inside the session. Both features are developed for educational purposes to simplify communication between students and teachers or multiple students.

Participants mostly praised communication inside the VoRtex platform using VoIP and internal chat. Of the sample of participants, 45.8% gave high-quality scores (Figure 15). Voice and chat features provided characteristics similar to popular online solutions (Skype, Teams, etc.). Overall experience provided a more immersive feeling in comparison to popular online solutions.

Regarding the MicroLesson feature, we included quest-based challenges inside the lesson about “Virtual Reality”. Participants could ask the virtual agent about VR-related topics and later answers question. MicroLesson is in the very early stage of development; using this feature, we can introduce gamified challenges related to puzzle-solving using the law of physics or in-world exploratory activities (treasure hunt, escape room, location analysis), and develop a game-oriented environment like with Fortnite Creative mode [24]. The participants recommended testing the MicroLesson feature with students inside other sectors like primary and secondary education. MicroLesson feature and problem-solving design were satisfactory for most participants (Figure 16).

Regarding platform extended usage, the majority see potential in 3D world-building and architecture design (Figure 17). We added storing of 3D content on the blockchain to protect created content from replication [29]. This enables many content creators to demonstrate assets and provide a concept for real-world activities.

After the survey, we interviewed the participants regarding their overall experience using the VoRtex platform in comparison to available teaching methods. We can provide the next conclusions from the participants regarding advantages and disadvantages over real-world classroom sessions:

• The virtual world can impact how to learn geography more efficiently. It is not the same when we see a photo or video of someplace as visiting some destination with our avatar. It is much more interesting to learn geography in the virtual world. In virtual worlds, learning becomes easier and more interesting because we feel like we are in a video game. This is a major VWs advantage in comparison to the real-world classroom session.
• Teachers noticed the advantage of using a platform for demonstration exercises via a VW application in comparison to the real-world classroom session. Using the MicroLesson feature, they can build a gamified interactive experience for computer science lessons related to programming and 3D graphics. During real-world classroom sessions, this lesson does not have many practical methods except writing software code; adding an element of visual programing can simplify many problem-solving algorithms.
• Education and problem-solving aspects using 3D avatars could be a good way to personalize the experience and attract users to support the platform. However, monitoring students’ activities is difficult because the focus is on the virtual environment. The VoRtex platform utilizes multi-factor identity, teachers are reliable over biometric checks. Real-world classroom sessions are more reliable for student activities monitoring.
• Regarding the digital avatar inside the VoRtex demonstration, many participants commented that the avatar design was cartoonish. Many participants prefer a photorealistic VW environment and digital avatar models. Using an existing avatar can make VW experience frivolous for many users, so adding realistic 3D-scanned avatars using photogrammetry provides a more immersive experience for an older audience. This feature is integrated into future software iterations.

In comparison to Nunes et al.’s research [13], we included 24 users while Nunes included 13 students. Nunes’s research is focused on a specific topic (computer engineering course within the algorithms discipline) while our MicroLesson is more general for a wider audience (VR and 3D technologies). We used similar questions to Nunes [13] to evaluate the user experience. In comparison to Sloodle, the VoRtex environment has less restriction caused by the number of available elements in the virtual environment. However, both user experiences provide a positive impact on participants and can be used for educational purposes.

4.3. Future improvements

Based on the provided results, the VoRtex platform in the first official version supports scalability [29] and photorealism [25]. For the development of a virtual environment, we plan to implement a visualization tool from scratch, which adopts the proposed features for the education sector such as the ISTE standard [31] and provides an improved VoRtex platform based on user experience feedback. Our next technology stack milestone is to port the VoRtex platform to Unreal Game Engine and enable digital avatar creation using MetaHuman Creator [32]. Photogrammetry model-based scanning is a very popular method for adding new 3D content [33] and was used to automatically clone a person’s face using structure from motion and map face over MetaHuman models. Furthermore, we plan to integrate real-time motion capture (body gesture and face movement) for more advanced users of modern deep sensor cameras and VR HMD devices [2,34].

A video showcase presentation of the VoRtex prototype with educational features is available online [35]. Regarding the next user testing campaigns, we will test the VoRtex platform with students in the primary and secondary education sectors.

5. Conclusions

In this paper, we have compared and evaluated the potential of a VW platform for educational purposes. Furthermore, we evaluated the user experience on the VoRtex platform. Firstly, we reviewed the related work regarding collaboration and learning in VWs. Secondly, we described the concept and purpose of VW platforms. Thirdly, we introduced the VoRtex software architecture for collaboration and e-learning inside VWs. Afterwards, we compared VW platforms Vircadia, Sansar, and the VoRtex prototype using Mannien’s matrix. Based on the comparison, features offered by the emerging VW platforms enabled simulation of real-life interaction and experience through the support of desktop PCs or VR devices. Afterwards, we provided a survey in which we evaluated the user experience in the VoRtex prototype for Windows PC. Based on the results of survey, we found that the users respond positively to the VoRtex platform, and the usage of VWs in online learning could overcome boundaries during the COVID-19 pandemic.

In the end of the comparison process, Vircadia provided an overall slightly better educational experience, based on the listed criteria, in comparison to the VoRtex and Sansar with a sufficiently suitable solution (74 scores) for educational purposes. VoRtex has less content than the other two platforms. However, the VoRtex platform regarding the results from Mannien’s matrix comparative analysis has a security advantage, the most important aspect in the educational sector [29]. In addition, VoRtex’s innovative design provides out-of-the-box assistive tools for teachers such as presentation and video panels that are not common features in the other two platforms. Planned improvements after the next technology milestone using Unreal Engine 5 and MetaHuman will bring the platform graphically to a hyper-photorealistic level and initially provide more assets to content creators [32]. In our future work, we will measure and evaluate the VW software architecture and improve the prototype to simplify the VW platform design process for educational purposes. Furthermore, one of the top priorities for moving forward is to have users carry out routine assessments of VoRtex to help the development of the platform and make it more feature-rich. VoRtex source code and project updates are available on GitHub [36].