Next Article in Journal
Collapse Analysis of a Two-Span Reinforced Concrete Bridge Model
Previous Article in Journal
A Novel Multi-Objective Harmony Search Algorithm with Pitch Adjustment by Genotype
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Immersive Virtual-Reality-Based Streaming Distance Education System for Solar Dynamics Observatory: A Case Study

1
Center of Human-Centered Interaction for Coexistence, Hwarangno 14-gil 5, Seongbuk-gu, Seoul 02792, Korea
2
NAVER Clova, Seongnam 13561, Korea
3
Hana Institute of Technology, Grace Tower, 127 Teheran-ro, Gangnam-gu, Seoul 06236, Korea
4
Artificial Intelligence and Robotics Institute, Korea Institute of Science and Technology (KIST), Seoul 02792, Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2021, 11(19), 8932; https://doi.org/10.3390/app11198932
Submission received: 1 September 2021 / Revised: 19 September 2021 / Accepted: 20 September 2021 / Published: 25 September 2021

Abstract

:
The combination of immersive virtual reality (VR) environments and distance education has led to a new educational paradigm. In this study, an immersive VR-based distance education system is proposed to enable multiple remote users to send, share, and experience images from the Solar Dynamics Observatory (SDO) via streaming. In contrast to the conventional system in which only experts use SDO data, the proposed system provides a head-mounted-display-based visualization platform that can be easily used by experts and non-experts. Real-time SDO image streaming must be possible to realistically observe changes in the Sun and increase involvement. Thus, multichannel-based SDO image transmission was applied to increase the network bandwidth utilization. To improve the social presence of participants, realistic avatar models controlled by the motions synchronized with the user are provided. In addition, free communication is possible through verbal interactions. This allows multiple remote users to participate simultaneously without having to be physically present. A user study with 20 subjects showed that the participants could observe SDO images in a more immersive manner by using the proposed system. In addition, they experienced social presence because of the user avatar models and an enriched educational experience by conversing with and listening to experts.

1. Introduction

The recent developments in virtual reality (VR) and networking technology have facilitated the use of VR in classrooms and distance education programs. VR in education promotes active interactions among students and avoids distractions in the classroom. Representative applications include medical education [1,2,3], vocational training [4,5], and virtual field trips [6,7]. VR in education can overcome physical (spatiotemporal) constraints and enable learning that is not influenced by the dangers or restrictions of the real world.
Distance education allows remote non-contact teaching and learning between teachers and students and between physically distant students [8,9]. It allows students to receive education at any time and place and adjust their progress accordingly. However, it is difficult to develop personal interactions between teachers and students and assess students’ abilities. Moreover, it is challenging for students to actively participate in the classroom, and teachers cannot immediately answer students’ questions. In addition, students should take greater responsibility for their learning, as they must decide when, where, and how to learn.
There are two main advantages of VR-based distance education. First, it resolves imbalances in educational opportunities between countries and regions. Rural areas that receive less support for educational infrastructure can benefit from experts. Physical constraints such as time and distance can be overcome by networking, and the imbalance problem can be solved by sharing various educational content. For example, massive open online courses (MOOCs) (https://www.mooc.org/, accessed on 1 September 2021) and other online programs provide various and rich educational content via educational platforms in which anyone can participate without qualification restrictions. Second, it provides a socially immersive virtual educational environment. Distance education generally uses videos by default, such as MOOCs or online education methods employing video conferencing tools. However, direct interaction between teachers and students is challenging compared with offline education [10]. For example, in video-conference-based education, the participants are displayed as thumbnails in a graphical user interface. This makes it easy to become distracted and lose focus. In contrast, immersive VR (IVR)-based distance education provides an improved educational environment using a head-mounted display (HMD) in which the remote participants feel like they are in the same space, in comparison to desktop VR and cave automatic virtual environment (CAVE) VR. Moreover, due to the recent global COVID-19 pandemic, there has been a need for non-contact distance education from elementary to higher education [11,12,13]. The need for team-based VR remote education that requires collaboration is also increasing [14,15].
VR education covers various disciplines including engineering, computer science, astronomy, biology, medicine, chemistry, manufacturing, physics, and surgical medicine [16]. Among them, the representative VR content includes applications related to astronomy. For example, Apollo 11 VR HD (https://immersivevreducation.com/products-vr-experiences/apollo-11, accessed on 1 September 2021) developed by Immersive VR Education allows ordinary people to vividly experience the process of lunar exploration performed by the first astronaut to land on the Moon. Split Light Studio developed an educational Solar System VR App (http://splitlightstudio.com/pages/movies_solar_system.html, accessed on 1 September 2021) that involves a tour of the planets and the Moon. Although it can help users understand the planets with colorful graphics, the contents are produced manually and cannot reflect the ever-changing appearances of celestial bodies.
On 11 January 2010, the U.S. National Aeronautics and Space Administration (NASA) launched the Solar Dynamics Observatory (SDO) (https://www.nasa.gov/pdf/418329main$_$SDO$_$PressKit.pdf, accessed on 1 September 2021) that can continuously observe the Sun, as shown in Figure 1. This allows anyone, and not just experts, to observe the changes in the Sun. In this study, an IVR-based distance education system was proposed to enable multiple remote users to share and experience SDO images in real time via streaming. The requirements for building the IVR-SDO distance education system are as follows:
R1: Low-barrier of entry for people who are interested in SDO. The system must be easy to use, even for non-experts. Only limited experts, such as space scientists or aeronautical meteorologists, have shown interest in existing SDO data. Although not well-known to the public, the SDO data provided on the web can be easily accessed by ordinary people. However, expert knowledge is still required to handle the data. Hence, it is necessary to lower the barrier to entry for the general public by providing a more accessible education method. Low-priced HMDs can replace expensive large-screen displays, and the limitations of place can be overcome by eliminating the need for remote visits using a network.
R2: Streaming immersive data visualization platform. The system should provide users with immersive data visualization of SDO images via streaming through an HMD-based visualization platform. Streaming is required because it is practically difficult to archive large SDO images at a local site. An HMD is used because in desktop VR and CAVE, the user recognizes the screen, thus lowering immersion. A virtual space can immerse users by freely adjusting the size of the Sun image and arranging it in various ways.
R3: Realistic avatar-based embodied interactions. To improve the social presence of participants, a realistic avatar model should be provided and controlled based on motions synchronized with the user. Furthermore, free communication using verbal interactions is possible.
R4: Physically distributed multiuser support for group learning. It should enable group learning in which many physically distant users can participate together without being physically present. This is achieved by making them feel as if they are receiving education in the same space.
The remainder of this paper is organized as follows. Section 2 introduces related work. Section 3 details the IVR-SDO distance education system. Specifically, the proposed system structure, network structure, and multichannel-based data transfer method for SDO image streaming are described. Section 4 presents the quantitative experiment results for the network utilization of the proposed IVR-SDO distance education system and the qualitative preference analysis results from user experiments. Finally, Section 5 provides the conclusions and directions for future research.

2. Related Work

2.1. Solar Dynamics Observatory

The objective of SDO is to help understand the Sun’s influence on Earth and near-Earth space by studying the solar atmosphere on small scales of space and time and in many wavelengths simultaneously [17]. SDO sends solar observation images to Earth at a continuous science data downlink speed of 130 Mbps, which are then stored. The SDO images captured at multiple wavelengths can be used to observe various events such as flares, active regions, and coronal holes by employing the web-based Hellioviewer (https://helioviewer.org, accessed on 1 September 2021). Furthermore, JHelioviewer (http://www.jhelioviewer.org/, accessed on 1 September 2021), the open source data visualization tool, has improved usability, focusing on space weather analysis [18]. However, the entry barrier is too high for non-experts to operate easily. When SDO images are visualized, the placement and resizing of the images are also limited by the monitor size. NASA provides SDO Solarium (https://www.nasa.gov/solarium, accessed on 1 September 2021) (Figure 2a) and the video wall of the NASA Museum Sun Video Wall (https://www.nasa.gov/content/goddard/sdo/national-air-and-space-museum-debuts-must-see-sun-video-wall, accessed on 1 September 2021) (Figure 2b) for the general public to easily experience the SDO images, which are again used only by experts. Because large displays are utilized for exhibition and education, SDO images can be experienced with greater immersion than when using a dedicated viewer.

2.2. Virtual Reality in Education

Kavanagh et al. conducted a systematic review of the motivations for using VR in education and addressed the potential problems and limitations [19]. VR education systems were found to improve the intrinsic motivation of students. In addition, they introduced and compared the latest VR technologies that can address the problems of VR education, including cost, user experience, and interactions. Carruth examined the factors influencing the effect of VR education and found their increasing use for studying and evaluating buildings [20], as students can explore problem spaces and test solutions in a virtual space. Furthermore, he reported that the effects of VR education appear in a wide range of areas, from surgery education to manufacturing and assembly.
Jensen and Flemming investigated the advantages and disadvantages of using HMDs in education and training [21]. HMD is ideal for teaching most techniques and enables immersive education, as corrections can be made using repetitive practice without any risk when learners are exposed to challenging educational situations. They noted that there were significant barriers to using HMDs remain due to cybersickness symptoms, lack of suitable software, and the technical limitations of peripherals. Nevertheless, VR as an educational tool is growing explosively and enjoying increased user preference with the emergence of affordable and easy-to-use immersive HMDs and peripherals. For example, in [22], the knowledge acquisition, satisfaction, and comfort of students when the same pharmacology artifact was displayed in CAVE2 and a mobile handheld device with stereoscopic lenses attached were investigated. The results showed that the students preferred the mobile handheld device over CAVE2, which is more cost-effective and accessible. VR-based education is also often used for training and learning complex operational tasks. Hou et al. suggested an efficient educational framework based on AR/VR for operating oil and gas facilities, which is a complex process that requires expertise [23]. According to the experimental results, compared to paper-based education, the AR/VR education system can save manpower, increase efficiency by reducing the movement distance during work, and shorten the learning time for improving learning performance. The AR/VR education system can enhance education quality to prevent errors and failures during work, considering effectiveness and safety concerns. However, it has the limitation that the data of the actual work environment cannot be synchronized to the mixed reality system in real time. Patle et al. showed that combining VR with operator training simulators enabled more effective training by providing the operator with a realistic experience [24].
VR-based education creates a virtual environment that supports multiple users. Senond Life (SL) (https://secondlife.com/, accessed on 1 September 2021), a leading online virtual world developed by Linden Lab, San Francisco, had approximately one million users in 2013. It allows users to create avatars and interact with objects or other avatars and meet and socialize with others, participate in private and group activities, build virtual assets and services, shop, and trade. Julian conducted a study on virtual learning experiences that allow students of English as a foreign language (EFL) to learn a second language in a 3D multiuser virtual environment, SL [25]. He observed whether language education and learning could be facilitated if SL provided visual and language support to EFL learners. In addition, he demonstrated that the virtual learning experience could be optimized by accepting the culture and world knowledge of learners and simulating real-life scenarios. Providing such an immersive collaboration environment can promote learner participation and motivation, make them actively learn, and increase creativity, making it different from traditional classroom-based education methods. Increasing the completeness of immersive simulation can further increase participation and produce positive learning effects.
Recently, many education systems using networks and cloud services have been developed. VR field trips compared the effects of a live teacher using networked depth cameras and a standalone pre-recorded teacher on student learning [26]. The video-based standalone education method was relatively simple to deploy because additional equipment or setting time was not necessary and could provide more content as less time was needed for communication with students. In contrast, the live teacher had the advantage of reducing misunderstandings by answering student concerns, making it more attractive to the students. Abichandani et al. performed solar energy education on virtual solar cells, solar modules, and solar arrays using a cloud-based VR education system [27]. Feedback from students indicated that their participation and knowledge acquisition were high. However, although this system is highly scalable and accessible as a cloud-based service, user immersion is low, given that it is a web browser-based desktop VR system. He et al. demonstrated that network-based VR could simulate a polluted gas diffusion process in real time in a cloud-computing environment [28]. They also simulated polluted gas diffusion in real time in a large-scale scene using an online PC, notebook, and mobile device. Ding et al. proposed a VR education system to scientifically and systematically improve the physical level of college students [29]. Users of this system can render scenes in the cloud and experience VR using a mobile device. Moreover, related data can be collected using the Internet of Things, and users can interact with VR scenes in real time.

3. IVR-Based SDO Streaming Distance Education System

3.1. IVR-SDO Streaming Education System Architecture

Figure 3 shows an overview of the proposed SDO distance education system. The system comprises an SDO client, a data relay server, and an SDO image transmission server. The SDO client comprises hardware such as the HTC Vive HMD, controller, and PC system. The HMD tracks the user’s head pose using a tracking sensor attached to the HMD while providing immersive visualization. Users can engage in voice conversations using a built-in microphone and earphones. The Vive controller tracks hand movements by 1:1 mapping of the user’s hand and offers a ray-pointing feature that helps explain the content when the controller button is pressed.

3.1.1. SDO Client

The main components of the SDO client are the I/O module, rendering engine, network module, and SDO image transfer module.
For generating realistic human avatars that represent users’ presence, the rapid 3D avatar creation method proposed by Lim et al. was used [30]. This method can quickly and automatically create a user avatar within 2 min using a single depth camera and turntable. For the simplicity of implementation, this study uses only a head and hand model instead of a full-body avatar model. The I/O module tracks the head and hand poses using an HMD and a controller to control the avatar according to the user’s motions. To support voice conversations between users, the voice I/O is controlled through the microphone and earphones attached to the HMD. The rendering engine consists of a sound engine and graphics engine, which enables verbal interaction by combining the avatar engine that controls the avatar model according to the user’s motions with the voice sent from remote users, which renders SDO images received from the network module in a 3D virtual space in an immersive manner.
The network module transfers the necessary data, such as SDO images, voices, and head and hand poses, to the network for SDO distance education. The SDO image transfer module merges the SDO images that are split-transmitted from the relay server via multiple channels to their original size and decodes them using the JPEG2000 decoder.

3.1.2. SDO Image Server

When a request is received by the SDO image transfer module, the SDO images in the repository are sent to the relay server. The images are sent with a frame size of approximately 1 MB using multiple channels. This method can fully utilize the network resources. The sender splits the total image into the maximum data size that can be sent for each channel, and the receiver merges them to the original size. This study transmits 10 types of 1 MB SDO images in 30 fps using six channels.

3.1.3. Relay Server

The SDO client and image server have considerable networking loads because large SDO images are transmitted. Furthermore, if the SDO image server also takes charge of user management, the complexity increases with the number of users. The relay server for each channel reduces this problem. For example, six-channel video relay servers, and a one-channel relay server are allocated for transmitting SDO images and data, respectively. Thus, independent relay servers are allocated to different types of media. Although the relay server suffers from increased transmission time compared to the direct data transmission method, it reduces the data transmission load of the SDO image server and achieves scalability by increasing the number of relay servers. Problem channels are restored without affecting the other channels using a relay server for each channel. This study used relay servers for six-channel video, one-channel voice, and one-channel avatar/hand motion data. Here, the difference between the SDO and other data channels is that the SDO video channel is only used for downloading as a simplex channel that only transmits data in one direction, whereas the voice, head, and hand channels are full duplex channels that transmit data in both directions.

3.1.4. SDO Data Specification

The SDO data, transmitted through the network in Figure 3, consist of SDO images, voices, and motion data of the avatar. The detailed data specifications are listed in Table 1. Figure 4 shows examples of 10 types of SDO atmospheric imaging assembly (AIA) ultraviolet wavelength images occupying most of the network bandwidth, and two types of illuminance/magnetism measurement images of the Helioseismic and Magnetic Imager (HMI).

3.2. Network Configuration for IVR-SDO Streaming Education System

Because the IVR-SDO distance education system requires a high bandwidth, it uses the Korea Research Environment Open Network (KREONET) (https://www.kreonet.net/eng/, accessed on 1 September 2021) as the backbone. KREONET supports high-performance network infrastructure to provide diverse science technology information resources, supercomputing, GRID, and e-science applications to approximately 200 research and development institutes. As shown in Figure 5, SDO images that are periodically transmitted from NASA connected to KREONET are stored in the network attached storage (NAS) of the Korea Astronomy and Space Science Institute (KASI), considered the Asian data hub. The SDO image server transmits user-requested SDO images stored in the NAS to the relay server. The SDO client receives SDO images via streaming by connecting to the relay server. Unlike SDO images, the voice and hand/head pose data are relayed through independent channels and transmitted to the SDO client.

3.3. Multi-Channel Based SDO Image Streaming

SDO image streaming is required for two reasons. First, it is practically impossible to archive large SDO images at a local site for each user because of limited storage. As shown in Table 1, transmitting SDO images requires a high bandwidth of 300 MB per second (2.4 Gbps). Second, to realistically observe the changes of the Sun in SDO images generated every 10 s, they must be sent and displayed at a rate of 30 fps or higher, similar to videos.
In this study, a multichannel transmission method was used to fully utilize the high bandwidth of KREONET to transmit large SDO images with up to 10 spectrums in real time. Because the size of an SDO image is approximately 1 MB, a network bandwidth of 300 Mbps or higher is required to transmit at 30 fps. This requirement can be satisfied by using KREONET, which supports a bandwidth of 10 Gbps or higher. The maximum transmission unit of the Ethernet is set to 1500 bytes by the IEEE 802.3 standard [31], and up to 9198 bytes can be transmitted depending on the compatibility of the router. If the transmitted data are of a larger size, the network splits the data, causing a delay. Because the method of sending data based on a single channel cannot use the total bandwidth, the bandwidth utilization is increased using multiple channels.
Figure 6 shows the concept of multichannel-based SDO image streaming. As shown in Figure 6a, the multichannel-based SDO image transfer module comprises a splitter and merger. The splitter in the sender splits the input SDO image into chunks and transmits them in parallel using the available multichannels. The merger in the receiver assembles the split chunks of the SDO image into the original SDO image, thus completing reception. An SDO header is added in front of each SDO image data, as shown in Figure 6b, to implement the SDO image transmission protocol for handling user requests and responses. The total SDO data with the SDO header added were split into chunks of unit size. Figure 6c shows the transmission of the split chunks of the SDO image using the available channels. This method shares channels instead of dedicating a specific channel for each SDO image. Network utilization can be improved by checking the currently available channels and by sending image data in parallel.
Figure 7 shows the SDO image header for transmitting the SDO images through the network. The fields of the SDO image header are as follows:
  • Timestamp—64-bit UTC information;
  • Type—16-bit SDO image information, indicating AIA or HMI;
  • Channel—16-bit AIA or HMI channel information;
  • Image data size—32-bit data size of SDO image;
  • Reserved #1, #2, #3—each 64-bit reserved space for expandability (e.g., channels other than AIA or HMI, different image data format, and different resolution).
Figure 8 shows the data transmission process of the split-and-merge approach for multichannel-based SDO images. The SDO image input to the transmission queue of the splitter (Figure 8a) is sent to the chunker where it is split into chunks of unit size (Figure 8b). The split image chunks are sent to the sender through an available channel among the multichannels (Figure 8c). If the transmission queue is not empty, the next SDO image is split and transmitted at every set time interval during the transmission of the previous image, and this whole process is performed in parallel (Figure 8d). The chunks that arrive at the merger of the receiver are delivered to the dechunker and assembled into the original SDO image (Figure 8e). The assembled SDO image is delivered to the receiving queue, which completes the transmission (Figure 8f).

3.4. Participant Interactions in VR Space

The education participants wore an HMD and earphones and stood with a controller in their right hand. An expert connects to the system and explains the SDO image through voice conversations. When the controller button is pressed, the object of explanation is indicated by ray pointing in a virtual space to improve the students’ understanding (Figure 9a). When students want additional information, they can ask questions via voice conversation while indicating a virtual object by ray pointing (Figure 9b).

4. Experimental Results

This section describes the implementation of the proposed distance education system, network utilization in multichannel-based data transmission, and the user experiment results.

4.1. Implementation for IVR-SDO Streaming Distance Education System

Each SDO client system has an Intel Core i7-6850K with 32GB memory and an NVidia GTX 1080 graphics card. The SDO clients used Vive HMDs with Vive controllers. The system was implemented using the Coexistent Reality Software Platform (CRSF) 2.0 [32] and Microsoft Windows 10. A relay server has an Intel Xeon W-2104 CPU with 32-GB memory running Windows server 2016. A SDO image server has an Intel Xeon W-2104 CPU with 32-GB memory running Centos 7. We developed all software programs (SDO client, SDO image server, and relay server) in Microsoft Visual Studio C++.
Figure 10 shows the network configuration of the proposed system. Every host is connected to KREONET, having a bandwidth of 10 Gbps or higher, using the Mellanox 40G network interface card. The SDO image server unidirectionally transmits 10 types of SDO images through the six channels at 30 fps to the relay server. When the three users access the relay server, the SDO image received through the six channels is merged. Then, the image is rendered in a virtual space. In contrast, the voice, hand, and head pose data are transmitted and received bi-directionally through the relay server. Figure 11 illustrates the educational experience when using the IVR-SDO streaming distance education system. With the help of a pointer, an expert explains the solar phenomena in detail. Students also look at the SDO images and ask questions while listening to the explanation. Remote users are placed in one virtual space; the user’s head and hand motions are tracked by the Vive HMD and controller and synchronized with their avatar. By pressing the controller button, the ray exiting the virtual hand is aimed at the desired point.

4.2. Network Utilization

Streaming large images for realistic SDO distance education requires a high network bandwidth. This study used the multichannel-based data transfer method mentioned in Section 3.3 to utilize the high-bandwidth network resources provided by KREONET. Figure 12 shows the data transfer rate and network utilization results according to the number of channels when the SDO images are split-transmitted in six channels from the SDO server to the relay server. The data transfer rate and network utilization for stable transmission were verified in an actual transmission experiment while increasing the number of channels by one to fully utilize the network bandwidth when large data such as SDO images are transmitted. The experimental results in Figure 12b show that the data transfer rate and network utilization increased linearly in proportion to the number of channels and 10 SDO image data can be transmitted without delay.
Figure 13 shows the changes in the network utilization of the relay server when the three participants, Users A, B, and C, enter the distance education system at regular intervals. When the education participants access the relay server, the SDO images are relayed. Hence, network utilization gradually increases at each point in time when the server is accessed. Quantitatively, the relay server receives approximately 287 Mbps of data from the SDO image server and relays three times (855 Mbps) the received data to the three participants.

4.3. Subjective Measures

In this section, user preferences such as the level of immersion, perceived engagement, comfort, and desirability of the IVR-SDO distance education system are examined, similarly as in [14,26], especially [26], who researched the efficacy of VR in distance team-based learning for students as an engaging platform. To acquire a subjective evaluation on the VR-TBL experiences, we utilized the well-established Likert-type questions. We modified the survey to suitably fit our research and performed two pre-tests with the volunteers before the user study to verify that the questionnaire reflects the intent of our research.

4.3.1. Participants

We recruited 20 unpaid participants (seven females) aged 24–54 years with a mean age of 29.9 (SD = 7.14) ) from Kwangwoon University, Center of Human-Centered Interaction for Coexistence (CHIC), and the Korea Institute of Science and Technology (KIST). All participants majored in engineering fields such as computer science, robotics, mechanical engineering, and electronic engineering. Before beginning the tests, participants were asked to answer some binary or 5-point Likert questions (i.e., 1 = strongly disagree to 5 = strongly agree). Participants reported some experience interacting in virtual environments (e.g., gaming). Most participants were not highly experienced with the VR education system (mean ± SD = 2.12 ± 1.6). Two participants had experience using a VR education system.

4.3.2. Apparatus and Procedure

We orchestrated the experiment to elapse 1 h per subject a day. Prior to the experiment, each participant was supplied with an overview of what the SDO is and the two different education methods over 20 min. After this, each participant filled out a survey prior to the experiment. The education method using the Hellioviewer for SDO images and IVR-SDO distance education system were randomly assigned to the participants to examine the satisfaction of the proposed system. The order of the two approaches was fully counterbalanced across the 20 participants. The education method using the Helioviewer allowed the viewer to observe solar events and SDO images based on the timeline, as shown in Figure 14. The participants can exchange questions and answers while selecting and observing the SDO images of various spectra. In testing the IVR-SDO distance education system, two remote students and one expert participated simultaneously while wearing a Vive HMD and earphones and using a controller for hand interaction. From the SDO image server, 10 types of images were streamed to the relay server at 30 fps, and the students and expert accessing the relay server can see the SDO images through the HMD. They can ask questions and receive explanations via voice conversation and feel the presence of other users through the head avatars. After all tests were completed, each participant was asked to answer a 16-question questionnaire with a 5-point Likert scale from 1—strongly disagree to 5—strongly agree.

4.3.3. Result and Discussion

In Table 2, questions 1–12 ask about preferences for the IVR-SDO education system, and question 13 asks about comfortableness for HMD-based learning, and questions 14 to 16 compare preferences between the IVR-SDO education system and Helioviewer. Table 3 summarizes the statistical comparisons between the viewer-based education and IVR-SDO education methods where there were statistically significant differences between the two education methods using Wilcoxon Rank Sum tests. According to the results on the preferences for the IVR-SDO education system, the participants gave a significant preference to the proposed method for all questions except Q4 and Q7.
The results demonstrate that the participants felt that they learned in the IVR-SDO education environment better than in the viewer-based education environment. In addition, the IVR-SDO education system provided the necessary tools for learning. Participants also reported that they were so immersed in the IVR-SDO educational environment that they forgot they were experiencing education in a virtual space. In the IVR-SDO educational environment, they could more confidently express their thoughts or ideas than in the viewer-based educational environment. This study indicates that participants felt that they were participating in the IVR-SDO educational environment compared to the viewer-based educational environment. Furthermore, they could easily use the technologies applied to the proposed educational environment and it offered a comfortable educational environment. All participants answered that education in the proposed environment was interesting, and most participants felt that they would receive education if provided in the IVR-SDO educational environment. Most participants agreed that the tool provided in the IVR-SDO educational environment was useful in aiding discussion among participants. Interestingly, all participants felt that learning in the IVR-SDO educational environment was realistic, and that the expert explanations were helpful.
Overall, all participants agreed or strongly agreed regarding satisfaction with the environment of IVR-SDO education, as shown in Figure 15. A possible explanation is that the IVR-SDO education system may provide the sense of being in the same place with other participants. This system also could allow participants to encourage their participation to more actively communicate and share ideas. These experimental results have some similarities with those of the teacher-guided educational VR research [26] on VR field trips. Having a networked education VR setup provides some benefits over the standalone approach of having a teacher’s recording in the VR space. Most notably, by streaming the teacher’s video live, the misunderstandings in communication between the students and the teacher can be resolved on the spot, leading to their reduction and higher gains in test scores. Our research also streams the SDO images live to be immersively visualized in the virtual space, peaking the participants’ interest further. Furthermore, similarly to the results from the VR-TBL experiences research, the feasibility of having a remote expert teach multiple participants while experiencing co-presence through an education platform was further verified. However, to answer the question of whether the HMD was sufficiently comfortable that it did not interfere with learning, only a relatively low percentage of participants preferred it because of the weight of the headset. This finding was similarly found in the VR-TBL experiences research and is one of the key issues that must be addressed for long sessions of education in VR.
For the questions comparing the preferences between the IVR-SDO education system and Helioviewer, the following results were obtained: nearly 70% of participants agreed or strongly agreed that, given the option, they would choose the proposed environment over the current dedicated viewer (Q14, mean ± SD = 4.20 ± 1.02). Most participants agreed or strongly agreed that learning in the proposed environment was more helpful (Q15, mean ± SD = 4.40 ± 0.75) and interesting (Q16, mean ± SD = 4.70 ± 057) than viewer-based learning.
The participants were asked to freely share their opinions after using the IVR-SDO education system. Positive answers include: “It was nice to be able to see more diverse images simultaneously and more realistically.”; “The education method was interesting, and the virtual environment was impressive.”; “The great advantage is that it provides a greater sense of immersion than a dedicated viewer, and that we can see various environments at once.”; “Above all, the greatest advantage is that it enables communication with experts and among class participants.”; “My concentration was definitely higher than when using the dedicated viewer, and I felt that it was more realistic.”; “It is more intuitive than a normal screen and makes learning interesting. I think it will be possible to experience realistic learning.” Participants provided a mixture of positive and constructive comments, including “If I could select things using the controller, I would have used it like a viewer.” and “It was a fresh experience. It would have been more interesting if 3D modeling of the sun was possible.” Comments expressing hope for improvement included “The HMD equipment was heavy” and “It was good in terms of immersion, but the dedicated viewer seems to be better for intuitive explanation (operation).”

5. Conclusions

Existing VR educational contents are mostly provided in a standalone form. For example, students study alone at home or individually experience the same content with friends in a classroom. Such VR education can help improve individual educational outcomes but lacks social education cultivated through collaboration. This can be overcome by distance-education-based VR using network technology. Compared with general online education, streaming educational content facilitates more vivid education. In this study, an IVR-based distance education system was proposed to allow multiple remote users to send, share, and experience SDO images via streaming. The proposed live-streaming-based SDO online education has the following advantages. First, it enables real-time interactions to encourage student participation, allowing them to share their opinions and overcome their fear or reluctance in asking questions. Second, it can improve the sense of realism more than pre-recorded content because the instructor can explain the material to the students while sharing screens in real time. Third, avatars and voice interactions with other remote participants provide a sense of immersion as if they are in the same place. Fourth, live-streaming-based education can provide a collaborative environment and allow participants to share their educational content. Furthermore, more enriching educational content can be provided by inviting experts on the given subject online. However, the following improvements in terms of usability should be made. Most students felt that wearing the HMD for a long time was difficult. In addition, they expected explanations on the SDO images would be displayed in the virtual environment, and that more diverse interactions using the controller would be added.
Currently, only simple interactions, such as pointing to SDO images, are possible. Hence, in the future, it is necessary to support more enhanced interaction functions, such as scaling for the part where a solar event occurs, as well as search and time machine functions for SDO images.

Author Contributions

Conceptualization, methodology, J.L.; funding acquisition, project administration, B.Y.; software, J.S., W.C. and J.L.; validation, W.C. and J.L.; writing—original draft, J.L.; writing—review and editing, J.L. and B.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by the Global Frontier Research and Development Program on Human-Centered Interaction for Coexistence funded by the National Research Foundation of Korea grant funded by the Korean Government (MSIP)(NRF-2010-0029759), in part by the Korea Advanced Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science and ICT (2020M3H8A1114928), and in part by the Korea Institute of Science and Technology (KIST) Institutional Program under Project 2E31063.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

This work was supported by KREONET (Korea Research Environment OpenNETwork), which is managed and operated by KISTI (Korea Institute of Science and Technology Information). The SDO data were (partly) provided by the Korea Data Center (KDC) for SDO in cooperation with NASA and KISTI (KREONET), which is supported by the “Next Generation Space Weather Observation Network” project of the Korea Astronomy and Space Science Institute (KASI). The authors thank Y. U. Kim and J. H. Jin for their support during the experiments.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Pottle, J. Virtual reality and the transformation of medical education. Future Healthc. J. 2019, 6, 181–185. [Google Scholar] [CrossRef] [Green Version]
  2. Ammanuel, S.; Brown, I.; Uribe, J.; Rehani, B. Creating 3D models from radiologic images for virtual reality medical education modules. J. Med. Syst. 2019, 43, 1–3. [Google Scholar] [CrossRef]
  3. Chávez, O.L.; Rodríguez, L.F.; Gutierrez-Garcia, J.O. A comparative case study of 2D, 3D and immersive-virtual-reality applications for healthcare education. Int. J. Med. Inform. 2020, 141, 104226. [Google Scholar] [CrossRef]
  4. Bakhos, D.; Galvin, J.; Aoustin, J.M.; Robier, M.; Kerneis, S.; Bechet, G.; Montembault, N.; Laurent, S.; Godey, B.; Aussedat, C. Training outcomes for audiology students using virtual reality or traditional training methods. PLoS ONE 2020, 15, e0243380. [Google Scholar] [CrossRef]
  5. Doganyigit, S.; Islim, O.F. Virtual reality in vocal training: A case study. Music. Educ. Res. 2021, 23. [Google Scholar] [CrossRef]
  6. Cheng, K.H.; Tsai, C.C. A case study of immersive virtual field trips in an elementary classroom: Students’ learning experience and teacher-student interaction behaviors. Comput. Educ. 2019, 140, 103600. [Google Scholar] [CrossRef]
  7. Klippel, A.; Zhao, J.; Oprean, D.; Wallgrün, J.O.; Stubbs, C.; La Femina, P.; Jackson, K.L. The value of being there: Toward a science of immersive virtual field trips. Virtual Real. 2020, 24, 753–770. [Google Scholar] [CrossRef]
  8. Sadeghi, M. A shift from classroom to distance learning: Advantages and limitations. Int. J. Res. Engl. Educ. 2019, 4, 80–88. [Google Scholar] [CrossRef] [Green Version]
  9. Bozkurt, A. From distance education to open and distance learning: A holistic evaluation of history, definitions, and theories. In Handbook of Research on Learning in the Age of Transhumanism; IGI Global: Hershey, PA, USA, 2019; pp. 252–273. [Google Scholar]
  10. Meneses, E.L.; Cano, E.V.; Mac Fadden, I. MOOC in higher education from the students’ perspective. A sustainable model? In Qualitative and Quantitative Models in Socio-Economic Systems and Social Work; Springer: Cham, Switzerland, 2020; pp. 207–223. [Google Scholar]
  11. Iyer, P.; Aziz, K.; Ojcius, D.M. Impact of COVID-19 on dental education in the United States. J. Dent. Educ. 2020, 84, 718–722. [Google Scholar] [CrossRef] [PubMed]
  12. Tekiner, H.; Kemaloglu, C.A.; Ali Tsaous, A. COVID-19 Era Requires Urgent Integration of Virtual Reality Simulations in Medical Education. HCA Healthc. J. Med. 2020, 1, 347–350. [Google Scholar] [CrossRef]
  13. Wang, X.; Chen, W.; Qiu, H.; Eldurssi, A.; Xie, F.; Shen, J. A Survey on the E-learning platforms used during COVID-19. In Proceedings of the 2020 11th IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, 4–7 November 2020; pp. 808–814. [Google Scholar]
  14. Coyne, L.; Takemoto, J.K.; Parmentier, B.L.; Merritt, T.; Sharpton, R.A. Exploring virtual reality as a platform for distance team-based learning. Curr. Pharm. Teach. Learn. 2018, 10, 1384–1390. [Google Scholar] [CrossRef] [PubMed]
  15. Harfouche, A.L.; Nakhle, F. Creating bioethics distance learning through virtual reality. Trends Biotechnol. 2020, 38, 1187–1192. [Google Scholar] [CrossRef]
  16. Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput. Educ. 2020, 147, 1–29. [Google Scholar] [CrossRef]
  17. Pesnell, W.D.; Thompson, B.J.; Chamberlin, P.C. The solar dynamics observatory (SDO). In The Solar Dynamics Observatory; Springer: New York, NY, USA, 2011; pp. 3–15. [Google Scholar]
  18. Müller, D.; Nicula, B.; Felix, S.; Verstringe, F.; Bourgoignie, B.; Csillaghy, A.; Berghmans, D.; Jiggens, P.; García-Ortiz, J.P.; Ireland, J.; et al. JHelioviewer-Time-dependent 3D visualisation of solar and heliospheric data. Astron. Astrophys. 2017, 606, A10. [Google Scholar] [CrossRef] [Green Version]
  19. Kavanagh, S.; Luxton-Reilly, A.; Wuensche, B.; Plimmer, B. A systematic review of Virtual Reality in education. Themes Sci. Technol. Educ. 2017, 10, 85–119. [Google Scholar]
  20. Carruth, D.W. Virtual reality for education and workforce training. In Proceedings of the 2017 15th International Conference on Emerging eLearning Technologies and Applications (ICETA), Stary Smokovec, Slovakia, 26–27 October 2017; pp. 1–6. [Google Scholar]
  21. Jensen, L.; Konradsen, F. A review of the use of virtual reality head-mounted displays in education and training. Educ. Inf. Technol. 2018, 23, 1515–1529. [Google Scholar] [CrossRef] [Green Version]
  22. Hanson, J.; Andersen, P.; Dunn, P.K. The effects of a virtual learning environment compared with an individual handheld device on pharmacology knowledge acquisition, satisfaction and comfort ratings. Nurse Educ. Today 2020, 92, 1–8. [Google Scholar] [CrossRef]
  23. Hou, L.; Chi, H.L.; Tarng, W.; Chai, J.; Panuwatwanich, K.; Wang, X. A framework of innovative learning for skill development in complex operational tasks. Autom. Constr. 2017, 83, 29–40. [Google Scholar] [CrossRef]
  24. Patle, D.S.; Manca, D.; Nazir, S.; Sharma, S. Operator training simulators in virtual reality environment for process operators: A review. Virtual Real. 2019, 82, 293–311. [Google Scholar] [CrossRef]
  25. Chen, J.C. The crossroads of English language learners, task-based instruction, and 3D multi-user virtual learning in Second Life. Comput. Educ. 2016, 102, 152–171. [Google Scholar] [CrossRef]
  26. Borst, C.W.; Lipari, N.G.; Woodworth, J.W. Teacher-guided educational VR: Assessment of live and prerecorded teachers guiding virtual field trips. In Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Tuebingen, Germany, 18–22 March 2018; pp. 467–474. [Google Scholar]
  27. Abichandani, P.; Mcintyre, W.; Fligor, W.; Lobo, D. Solar Energy Education Through a Cloud-Based Desktop Virtual Reality System. IEEE Access 2019, 7, 147081–147093. [Google Scholar] [CrossRef]
  28. He, Z.; You, L.; Liu, R.W.; Yang, F.; Ma, J.; Xiong, N. A Cloud-Based Real Time Polluted Gas Spread Simulation Approach on Virtual Reality Networking. IEEE Access 2019, 7, 22532–22540. [Google Scholar] [CrossRef]
  29. Ding, Y.; Li, Y.; Cheng, L. Application of Internet of Things and virtual reality technology in college physical education. IEEE Access 2020, 8, 96065–96074. [Google Scholar] [CrossRef]
  30. Lim, H.; Kang, J.; Ahn, S.C. Rapid 3D avatar creation system using a single depth camera. In Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan, 23–27 March 2019; pp. 1329–1330. [Google Scholar]
  31. IEEE Standard for Ethernet. IEEE Std 802.3-2012 (Revision to IEEE Std 802.3-2008); IEEE: New York, NY, USA, 2012; pp. 1–3747. [Google Scholar]
  32. You, B.J.; Kwon, J.R.; Nam, S.H.; Lee, J.J.; Lee, K.K.; Yeom, K. Coexistent space: Toward seamless integration of real, virtual, and remote worlds for 4d+ interpersonal interaction and collaboration. In Proceedings of the SIGGRAPH Asia 2014 Autonomous Virtual Humans and Social Robot for Telepresence, Shenzhen, China, 3–6 December 2014; pp. 1–5. [Google Scholar]
Figure 1. Solar Dynamics Observatory (SDO): Investigating the causes of solar variability and how space weather results from that variability.
Figure 1. Solar Dynamics Observatory (SDO): Investigating the causes of solar variability and how space weather results from that variability.
Applsci 11 08932 g001
Figure 2. People experiencing SDO images via facilities provided by NASA to the general public: (a,b) Solarium in the NASA SDO Exhibit; and (c) Sun Video Wall in the NASA museum.
Figure 2. People experiencing SDO images via facilities provided by NASA to the general public: (a,b) Solarium in the NASA SDO Exhibit; and (c) Sun Video Wall in the NASA museum.
Applsci 11 08932 g002
Figure 3. Overall system architecture for IVR-SDO streaming distance education.
Figure 3. Overall system architecture for IVR-SDO streaming distance education.
Applsci 11 08932 g003
Figure 4. Examples of 10 types of SDO AIA ultraviolet wavelength images and HMI illuminance/magnetism measurement images.
Figure 4. Examples of 10 types of SDO AIA ultraviolet wavelength images and HMI illuminance/magnetism measurement images.
Applsci 11 08932 g004
Figure 5. Network configuration diagram for IVR-SDO system education.
Figure 5. Network configuration diagram for IVR-SDO system education.
Applsci 11 08932 g005
Figure 6. Multichannel-based SDO image streaming: (a) multichannel data transmission using split–merge method; (b) splitting image data into chunks after combining with SDO header; (c) parallel transmission of the split chunks of SDO image.
Figure 6. Multichannel-based SDO image streaming: (a) multichannel data transmission using split–merge method; (b) splitting image data into chunks after combining with SDO header; (c) parallel transmission of the split chunks of SDO image.
Applsci 11 08932 g006
Figure 7. SDO image header format.
Figure 7. SDO image header format.
Applsci 11 08932 g007
Figure 8. Multichannel-based SDO image transmission using the split-and-merge approach: (a) start image transmission; (b) split the image; (c) transmit image chunks; (d) assemble image chunks; (e) complete image assembly; and (f) complete image transmission.
Figure 8. Multichannel-based SDO image transmission using the split-and-merge approach: (a) start image transmission; (b) split the image; (c) transmit image chunks; (d) assemble image chunks; (e) complete image assembly; and (f) complete image transmission.
Applsci 11 08932 g008
Figure 9. SDO image education and participant interactions in a virtual space: (a) SDO education using ray-pointing; and (b) listening to the explanation of an expert on SDO images through voice conversation.
Figure 9. SDO image education and participant interactions in a virtual space: (a) SDO education using ray-pointing; and (b) listening to the explanation of an expert on SDO images through voice conversation.
Applsci 11 08932 g009
Figure 10. Network configuration for IVR-SDO streaming distance education for user experiment. Users A and B are students, and User C is an expert.
Figure 10. Network configuration for IVR-SDO streaming distance education for user experiment. Users A and B are students, and User C is an expert.
Applsci 11 08932 g010
Figure 11. Experiencing the IVR-SDO streaming distance education system: (ad) Users A and B (students) receive explanations about the SDO images from User C (expert) and ask questions; (eh) display as seen from the viewpoint of User C (expert).
Figure 11. Experiencing the IVR-SDO streaming distance education system: (ad) Users A and B (students) receive explanations about the SDO images from User C (expert) and ask questions; (eh) display as seen from the viewpoint of User C (expert).
Applsci 11 08932 g011
Figure 12. Experiment results using the multichannel-based SDO image streaming module. The data transfer rate and network utilization increased in proportion to the number of channels: (a) multichannel-based SDO image streaming from the SDO image server to the relay server; (b) measurement results of data transfer rate and network utilization according to the number of channels.
Figure 12. Experiment results using the multichannel-based SDO image streaming module. The data transfer rate and network utilization increased in proportion to the number of channels: (a) multichannel-based SDO image streaming from the SDO image server to the relay server; (b) measurement results of data transfer rate and network utilization according to the number of channels.
Applsci 11 08932 g012
Figure 13. Increase in the network utilization of the relay server when three users connect to it one by one while SDO images are transmitted to the relay server.
Figure 13. Increase in the network utilization of the relay server when three users connect to it one by one while SDO images are transmitted to the relay server.
Applsci 11 08932 g013
Figure 14. Helioviewer: a solar and heliosphere image visualization tool that enables everyone anywhere to explore the Sun’s variability.
Figure 14. Helioviewer: a solar and heliosphere image visualization tool that enables everyone anywhere to explore the Sun’s variability.
Applsci 11 08932 g014
Figure 15. Responses to questions for each of the education methods.
Figure 15. Responses to questions for each of the education methods.
Applsci 11 08932 g015
Table 1. Data specification for IVR-SDO streaming education system.
Table 1. Data specification for IVR-SDO streaming education system.
Data TypeData SpecificationData Size/s
SDO image
  • 4096 × 4096 resolution
  • 1 MB/frame
  • 10 types
  • 30 fps
300 MB/s
Voice
  • 44.1 KHz sample rate
  • 2 channels
  • 2 byets/sample
  • 2048 samples
  • 45 fps
368.64 KB/s
Hand/head poses
  • Hand pose (22 joints × 2 hands × 4 × 4 pose matrix)
  • Head pose (4 × 4 pose matrix)
  • 60 fps
172.8 KB/s
Table 2. Subjective questionnaire.
Table 2. Subjective questionnaire.
No.Question Description
Q1I felt like I learned in this environment.
Q2This environment provided the appropriate tools necessary for me to learn.
Q3I forgot about my surroundings while experiencing this environment.
Q4I felt confident expressing ideas in this environment.
Q5I felt engaged in this environment.
Q6I found that the technology was easy to use.
Q7I found this to be a comfortable learning environment.
Q8This was a fun experience.
Q9I would take a lesson in this format if it was offered.
Q10The tools provided by the learning environment were useful during the discussion.
Q11The learning in this environment was realistic.
Q12The explanations of an expert in this environment were helpful.
Q13I felt that the HMD was comfortable enough that it did not interfere with learning.
Q14Given a choice, I would choose this environment over the current dedicated viewer.
Q15The learning in this environment was more helpful than the viewer-based learning.
Q16The learning in this environment was more interesting than the viewer-based learning.   
Table 3. IVR-SDO education and viewer-based education methods compared in terms of questionnaire responses; Wilcoxon rank sum tests.
Table 3. IVR-SDO education and viewer-based education methods compared in terms of questionnaire responses; Wilcoxon rank sum tests.
No.ZpNo.ZpNo.Zp
Q1−3.934<0.001Q5−4.053<0.001Q9−3.510<0.001
Q2−3.938<0.001Q6−3.891<0.001Q10−3.578<0.001
Q3−3.985<0.001Q7−1.867<0.062Q11−3.971<0.001
Q4−1.265<0.206Q8−4.035<0.001Q12−4.008<0.001
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lee, J.; Surh, J.; Choi, W.; You, B. Immersive Virtual-Reality-Based Streaming Distance Education System for Solar Dynamics Observatory: A Case Study. Appl. Sci. 2021, 11, 8932. https://doi.org/10.3390/app11198932

AMA Style

Lee J, Surh J, Choi W, You B. Immersive Virtual-Reality-Based Streaming Distance Education System for Solar Dynamics Observatory: A Case Study. Applied Sciences. 2021; 11(19):8932. https://doi.org/10.3390/app11198932

Chicago/Turabian Style

Lee, Joongjae, Jaeheung Surh, Wooseong Choi, and Bumjae You. 2021. "Immersive Virtual-Reality-Based Streaming Distance Education System for Solar Dynamics Observatory: A Case Study" Applied Sciences 11, no. 19: 8932. https://doi.org/10.3390/app11198932

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop