Search Results (3)

The popularity of the online teaching model increased during the COVID-19, and virtual reality online education is now firmly established as a future trend in educational growth. Human–computer interaction and collaboration between virtual models and physical entities, as well as virtual multi-sensory cognition, have become the focus of research in the field of online education. In this paper, we analyze the mapping form of teaching information and cue information on users’ cognition through an experimental system and investigate the effects of the presentation form of online virtual teaching information, the length of the material, users’ memory of the information, and the presentation form of information cues on users’ cognitive performance. The experimental results show that different instructional information and cue presentation designs have significant effects on users’ learning performance, with relatively longer instructional content being more effective and users being more likely to mechanically remember the learning materials. By studying the impact of multi-sensory information presentation on users’ cognition, the output design of instructional information can be optimized, cognitive resources can be reasonably allocated, and learning effectiveness can be ensured, which is of great significance for virtual education research in digital twins. Full article

Smartphones are a constant companion in everyday life. Interacting with a smartphone calls for a multimodal input and often leads to a multisensory output. Combining research in human-computer interaction (HCI) and psychology, the present research explored the idea that a smartphone is more than a smart object but represents an object to which people feel emotionally attached to and which is even perceived as a part or an extension of a person’s self. To this end, we used an established rubber hand illusion paradigm to experimentally induce body ownership experiences in young adults (n = 76) in a 4-level mixed-design study. Our results revealed that in contrast to a neutral control object participants indeed felt attached to a smartphone, perceived it as a part of themselves and felt the need to interact with the device. This was specifically pronounced when hedonic characteristics were evaluated as high and when its usage for social communication was highlighted during the experiment. Psychological mechanisms of the incorporation of technologies are discussed and connected to positive and negative effects of smartphone usage on human behavior, its implications for technology design and marketing. Full article

For years the HCI community’s research has been focused on the hearing and sight senses. However, in recent times, there has been an increased interest in using other types of senses, such as smell or touch. Moreover, this has been accompanied with growing research related to sensory substitution techniques and multi-sensory systems. Similarly, contemporary art has also been influenced by this trend and the number of artists interested in creating novel multi-sensory works of art has increased substantially. As a result, the opportunities for visually impaired people to experience artworks in different ways are also expanding. In spite of all this, the research focusing on multimodal systems for experiencing visual arts is not large and user tests comparing different modalities and senses, particularly in the field of art, are insufficient. This paper attempts to design a multi-sensory mapping to convey color to visually impaired people employing musical sounds and temperature cues. Through user tests and surveys with a total of 18 participants, we show that this multi-sensory system is properly designed to allow the user to distinguish and experience a total of 24 colors. The tests consist of several semantic correlational adjective-based surveys for comparing the different modalities to find out the best way to express colors through musical sounds and temperature cues based on previously well-established sound-color and temperature-color coding algorithms. In addition, the resulting final algorithm is also tested with 12 more users. Full article