Virtual Reality, Augmented Reality, and Human-Computer Interaction

Dear Colleagues,

Extended Reality (i.e., Cross Reality, XR) is an umbrella term, which includes Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). All three technologies are related to each other, with VR and AR lying at opposite ends of the so-called Reality–Virtuality continuum (Milgram94). XR technologies offer the possibility of visualizing entities that are not perceptible in reality, such as structures and processes, using virtual overlays. This makes them interesting for a huge variety of application areas, for example, in assistance and training systems (Psotka95) (e.g., in the fields of production, logistics, and surgery), navigation systems (Krolewski11), virtual room design and furnishing (Kaleja17), and computer games (Fahn13). However, not only is the development of new XR technologies and applications a key issue, it is also of utmost relevance to ensure optimal utilization of the user’s cognitive resources and to be able to detect cognitive overloads at an early stage.

In this context, we must consider different input modalities for human perception. Visuals and sound modalities are the most important ones for human perception, and they are also prevailing in XR environments. These modalities allow representing a large fraction of a (real or artificial) world. They permit a good immersion of the user, e.g., have the user moving in the (real or artificial) world (wearing goggles) or have the user explore the world on some display device using input devices such as a mouse or position and orientation sensors integrated into the display device (such as in a smartphone or tablet).

However, when dealing with these input modalities, consistency of view and sound is an important topic. For a long time, sound has been considered a second-rate addition to VR. Today, we see the importance of including sound equal to visuals into VR or XR environments. Modern XR solutions require the rendering of a VR soundscape, including background as well as sound related to relevant actions such as speech and sound of phenomena that are relevant for experiencing visuals (Johansson19).

Some other modalities are only able to represent the world in direct contact (smell, taste, tactile information, temperature, gravity, pressure). Internal phenomena may well be directly caused by interaction with an external world (muscle tension, pain). In newer VR applications, some of these modalities are used, e.g., using wearable effectors for pressure, vibration, or temperature.

We notice that truly intuitive and usable virtual worlds are hard to design and implement taking into account the considerations described above. Researchers must solve multiple problems, e.g.:

• Development of new improved XR hardware;
• Development of new interaction paradigms;
• Supporting efficient collaborations in virtual worlds;
• Context awareness;
• Modalities (tactile and haptic simulated reality are now realistic options);
• Intuitive interaction and integrated haptic feedback;
• Scalability: from 2D to AR to VR;
• Ensuring real-time capabilities;
• Acknowledging safety aspects (e.g., use of AR in real environments);
• Transferring XR technologies to new application areas;
• Soundscapes (VR and AR are not only in a visual world);
• Support of impaired people;
• Translating reality (e.g., visuals or tactiles to replace sound);
• Evaluating XR settings.

In addition to these topics related to design and implementation of virtual worlds, user experience is another relevant field in XR: Experience is now generally considered the concept that joins perceived ease of use with all aspects of a human’s direct holistic reaction to any incoming stimulation. It may include some or all of the following:

• A meaning that a person attributes to what is perceived;
• The complex of emotions and feelings triggered;
• A positive or negative valuation in terms of being attracted;
• A tendency to act, including action of paying attention or focusing on the perceived phenomena; mental actions like information processing and decision taking; and physical actions like moving the head, closing the eyes, withdrawing the hand, walking, talking, etc. (Vyas11).

This Special Issue will provide an insight into the current state of the art of Extended Realities. It will describe current research on how to evaluate and guarantee their usability and provide a positive user experience. It will show recent works in the related fields as well as trends for future development.

Researchers are invited to submit recent unpublished work in the field of Extended Reality and Human Computer Interaction. The scope of contributions to this Special Issue includes but is not limited to the research problems listed above.

Related Work (excerpt)

Fahn, CS., Wu ML., Liu WT. On the Use of Augmented Reality Technology for Creating Interactive Computer Games. In: Shumaker R. (eds.) Virtual, Augmented and Mixed Reality. Systems and Applications. VAMR 2013. Lecture Notes in Computer Science, vol 8022. Springer, Berlin, Heidelberg. 2013. https://doi.org/10.1007/978-3-642-39420-1_37

Johansson, M. VR for Your Ears: Dynamic 3D Audio Is Coming Soon - A truly realistic experience in VR requires immersive audio. IEEE Spectrum, January 24, 2019

Kaleja, P., Kozlovská, M. Virtual Reality as Innovative Approach to the Interior Designing. Selected Scientific Papers - Journal of Civil Engineering 12. 2017. https://doi.org/10.1515/sspjce-2017-0011

Krolewski J., Gawrysiak P. The Mobile Personal Augmented Reality Navigation System. In: Czachórski T., Kozielski S., Stańczyk U. (eds) Man-Machine Interactions 2. Advances in Intelligent and Soft Computing, Vol 103. Springer, Berlin, Heidelberg, 2011. https://doi.org/10.1007/978-3-642-23169-8_12

Milgram, P. Takemura, H., Utsumi, A., and Kishino, F. Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies, 1994.

Psotka, J. Immersive training systems: Virtual Reality and Education and training. Instr Sci 23, pp. 405–431, 1995. https://doi.org/10.1007/BF00896880

Vyas, D. Designing for Awareness: An Experience-focused HCI Perspective. PhD University of Twente, 2011. Publisher: University of Twente, Enschede, Netherlands, ISBNs 978-90-365-3135-1

Prof. Dr. Achim Ebert
Prof. Dr. Peter Dannenmann
Prof. Dr. Gerrit van der Veer
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-anonymized peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Big Data and Cognitive Computing is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.