Multimodal Technologies and Interaction

16 pages, 1832 KiB

Open AccessArticle

Multisensory Technologies for Inclusive Exhibition Spaces: Disability Access Meets Artistic and Curatorial Research

by Sevasti Eva Fotiadi

Multimodal Technol. Interact. 2024, 8(8), 74; https://doi.org/10.3390/mti8080074 - 19 Aug 2024

Viewed by 996

This article discusses applications of technology for sensory-disabled audiences in modern and contemporary art exhibitions. One case study of experimental artistic and curatorial research by The OtherAbilities art collective is discussed: a series of prototype tools for sensory translation from audible sound to [...] Read more.

This article discusses applications of technology for sensory-disabled audiences in modern and contemporary art exhibitions. One case study of experimental artistic and curatorial research by The OtherAbilities art collective is discussed: a series of prototype tools for sensory translation from audible sound to vibration were developed to be embeddable in the architecture of spaces where art is presented. In the article, the case study is approached from a curatorial perspective. Based on bibliographical sources, the article starts with a brief historical reference to disability art activism and a presentation of contemporary accessibility solutions for sensory-disabled audiences in museums. The research for the case study was conducted during testing and feedback sessions on the prototypes using open-ended oral interviews, open-ended written comments, and ethnographic observation of visitors’ behavior during exhibitions. The testers were d/Deaf, hard of hearing and hearing. The results focus on the reception of the sensory translation of audible sound to vibration by test users of diverse hearing abilities and on the reception of the prototypes in the context of art and design exhibitions. The article closes with a reflection on how disability scholarship meets art curatorial theory in the example of the article’s case study. Full article

► Show Figures

Figure 1

11 pages, 1027 KiB

Open AccessArticle

Micro-Credentialing and Digital Badges in Developing RPAS Knowledge, Skills, and Other Attributes

by John Murray, Keith Joiner and Graham Wild

Multimodal Technol. Interact. 2024, 8(8), 73; https://doi.org/10.3390/mti8080073 - 15 Aug 2024

Viewed by 365

Abstract

This study explores the potential of micro-credentialing and digital badges in developing and validating the knowledge, skills, and other attributes (KSaOs) required for diverse Remotely Piloted Aircraft Systems (RPAS) operations. The rapid proliferation of drone usage has outpaced the development of necessary KSaOs [...] Read more.

This study explores the potential of micro-credentialing and digital badges in developing and validating the knowledge, skills, and other attributes (KSaOs) required for diverse Remotely Piloted Aircraft Systems (RPAS) operations. The rapid proliferation of drone usage has outpaced the development of necessary KSaOs for safe and efficient drone operations. This research aims to bridge this gap by identifying the unique and specific KSaOs required for different types of drone operations and examining how micro-credentialing and digital badges can provide tangible evidence of these KSaOs. The study also investigates the potential benefits and challenges of implementing digital badges in the RPAS sector and how these challenges can be addressed. Furthermore, it explores how digital badges can contribute to the standardization and recognition of RPAS competencies across different national regulatory bodies. The methodology involves observational studies of publicly available videos of drone operations, with a focus on agriculture spraying operations. The findings highlight the importance of both generic and specific KSaOs in RPAS operations and suggest that digital badges may provide an effective means of evidencing mastery of these competencies. This research contributes to the ongoing discourse on drone regulation and competency development, offering practical insights for regulators, training providers, and drone operators. Full article

► Show Figures

Figure 1

27 pages, 455 KiB

Open AccessArticle

Evaluating Virtual Reality in Education: An Analysis of VR through the Instructors’ Lens

by Vaishnavi Rangarajan, Arash Shahbaz Badr and Raffaele De Amicis

Multimodal Technol. Interact. 2024, 8(8), 72; https://doi.org/10.3390/mti8080072 - 12 Aug 2024

Viewed by 487

Abstract

The rapid development of virtual reality (VR) technology has triggered a significant expansion of VR applications in educational settings. This study seeks to understand the extent to which these applications meet the expectations and pedagogical needs of university instructors. We conducted semi-structured interviews [...] Read more.

The rapid development of virtual reality (VR) technology has triggered a significant expansion of VR applications in educational settings. This study seeks to understand the extent to which these applications meet the expectations and pedagogical needs of university instructors. We conducted semi-structured interviews and observations with 16 university-level instructors from Oregon State University to gather insights into their experiences and perspectives regarding the use of VR in educational contexts. Our qualitative analysis reveals detailed trends in instructors’ requirements, their satisfaction and dissatisfaction with current VR tools, and the perceived barriers to broader adoption. The study also explores instructors’ expectations and preferences for designing and implementing VR-driven courses, alongside an evaluation of the usability of selected VR applications. By elucidating the challenges and opportunities associated with VR in education, this study aims to guide the development of more effective VR educational tools and inform future curriculum design, contributing to the enhancement of digital learning environments. Full article

► Show Figures

Figure 1

44 pages, 2589 KiB

Open AccessArticle

Enhancing Reflective and Conversational User Engagement in Argumentative Dialogues with Virtual Agents

by Annalena Aicher, Yuki Matsuda, Keichii Yasumoto, Wolfgang Minker, Elisabeth André and Stefan Ultes

Multimodal Technol. Interact. 2024, 8(8), 71; https://doi.org/10.3390/mti8080071 - 6 Aug 2024

Viewed by 716

Abstract

In their process of information seeking, human users tend to selectively ignore information that contradicts their pre-existing beliefs or opinions. These so-called “self-imposed filter bubbles” (SFBs) pose a significant challenge for argumentative conversational agents aiming to facilitate critical, unbiased opinion formation on controversial [...] Read more.

In their process of information seeking, human users tend to selectively ignore information that contradicts their pre-existing beliefs or opinions. These so-called “self-imposed filter bubbles” (SFBs) pose a significant challenge for argumentative conversational agents aiming to facilitate critical, unbiased opinion formation on controversial topics. With the ultimate goal of developing a system that helps users break their self-imposed filter bubbles (SFBs), this paper aims to investigate the role of co-speech gestures, specifically examining how these gestures significantly contribute to achieving this objective. This paper extends current research by examining methods to engage users in cooperative discussions with a virtual human-like agent, encouraging a deep reflection on arguments to disrupt SFBs. Specifically, we investigate the agent’s non-verbal behavior in the form of co-speech gestures. We analyze whether co-speech gestures, depending on the conveyed information, enhance motivation, and thus conversational user engagement, thereby encouraging users to consider information that could potentially disrupt their SFBs. The findings of a laboratory study with 56 participants highlight the importance of non-verbal agent behaviors, such as co-speech gestures, in improving users’ perceptions of the interaction and the conveyed content. This effect is particularly notable when the content aims to challenge the user’s SFB. Therefore, this research offers valuable insights into enhancing user engagement in the design of multimodal interactions with future cooperative argumentative virtual agents. Full article

(This article belongs to the Special Issue Multimodal Interaction with Virtual Agents and Communication Robots)

► Show Figures

Graphical abstract

16 pages, 7880 KiB

Open AccessCommunication

Multimodal Drumming Education Tool in Mixed Reality

by James Pinkl, Julián Villegas and Michael Cohen

Multimodal Technol. Interact. 2024, 8(8), 70; https://doi.org/10.3390/mti8080070 - 5 Aug 2024

Viewed by 1177

Abstract

First-person VR- and MR-based Action Observation research has thus far yielded both positive and negative findings in studies observing such tools’ potential to teach motor skills. Teaching drumming, particularly polyrhythms, is a challenging motor skill to learn and has remained largely unexplored in [...] Read more.

First-person VR- and MR-based Action Observation research has thus far yielded both positive and negative findings in studies observing such tools’ potential to teach motor skills. Teaching drumming, particularly polyrhythms, is a challenging motor skill to learn and has remained largely unexplored in the field of Action Observation. In this contribution, a multimodal tool designed to teach rudimental and polyrhythmic drumming was developed and tested in a 20-subject study. The tool presented subjects with a first-person MR perspective via a head-mounted display to provide users with visual exposure to both virtual content and their physical surroundings simultaneously. When compared against a control group practicing via video demonstrations, results showed increased rhythmic accuracy across four exercises. Specifically, a difference of 239 ms (z-ratio = 3.520, p < 0.001) was found between the timing errors of subjects who practiced with our multimodal mixed reality development compared to subjects who practiced with video, demonstrating the potential of such affordances. This research contributes to ongoing work in the fields of Action Observation and Mixed Reality, providing evidence that Action Observation techniques can be an effective practice method for drumming. Full article

► Show Figures

Figure 1

19 pages, 2134 KiB

Open AccessArticle

The Role of Audio Feedback and Gamification Elements for Remote Boom Operation

by Alissa Burova, John Mäkelä, Tuuli Keskinen, Pekka Kallioniemi, Kimmo Ronkainen and Markku Turunen

Multimodal Technol. Interact. 2024, 8(8), 69; https://doi.org/10.3390/mti8080069 - 1 Aug 2024

Viewed by 570

Abstract

Remote operations have been greatly enhanced by advancements in technology, enabling remote control of machinery in hazardous environments. However, it is still a challenge to design remote control interfaces and provide feedback in a way that would enhance situational awareness without negatively affecting [...] Read more.

Remote operations have been greatly enhanced by advancements in technology, enabling remote control of machinery in hazardous environments. However, it is still a challenge to design remote control interfaces and provide feedback in a way that would enhance situational awareness without negatively affecting cognitive load. This study investigates how different audio feedback designs can support remote boom operation and, additionally, explores the potential impact of gamification elements on operator performance and motivation. Due to COVID-19 restrictions, this study was conducted remotely with 16 participants using a simulated environment featuring a virtual excavator. Participants performed digging tasks using two audio feedback designs: frequency-modulated beeping and realistic spatialized steam sounds. The findings indicate that both audio designs are beneficial for remote boom operations: the beeping sound was perceived as more comfortable and efficient in determining the proximity of a hidden object and helped in avoiding collisions, whereas spatial sounds enhanced the sense of presence. Therefore, we suggest combining both audio designs for optimal performance and emphasize the importance of customizable feedback in remote operations. This study also revealed that gamification elements could both positively and negatively affect performance and motivation, highlighting the need for careful design tailored to specific task requirements. Full article

(This article belongs to the Special Issue Multimodal User Interfaces and Experiences: Challenges, Applications, and Perspectives)

► Show Figures

Figure 1

33 pages, 1947 KiB

Open AccessReview

Lessons Learned from Implementing Light Field Camera Animation: Implications, Limitations, Potentials, and Future Research Efforts

by Mary Guindy and Peter A. Kara

Multimodal Technol. Interact. 2024, 8(8), 68; https://doi.org/10.3390/mti8080068 - 1 Aug 2024

Viewed by 564

Abstract

Among the novel 3D visualization technologies of our era, light field displays provide the complete 3D visual experience without the need for any personal viewing device. Due to the lack of such constraint, these displays may be viewed by any number of observers [...] Read more.

Among the novel 3D visualization technologies of our era, light field displays provide the complete 3D visual experience without the need for any personal viewing device. Due to the lack of such constraint, these displays may be viewed by any number of observers simultaneously, and the corresponding use case contexts may also involve a virtually unlimited numbers of users; any number that the valid viewing area of the display may accommodate. While many instances of the utilization of this technology operate with static contents, camera animation may also be relevant. While the topic of light field camera animation has already been addressed on an initial level, there are still numerous research efforts to be carried out. In this paper, we elaborate on the lessons learned from implementing light field camera animation. The paper discusses the associated implications, limitations, potentials, and future research efforts. Each of these areas are approached from the perspectives of use cases, visual content, and quality assessment, as well as capture and display hardware. Our work highlights the existing research gaps in the investigated topic, the severe issues related to visualization sharpness, and the lack of appropriate datasets, as well as the constraints due to which novel contents may be captured by virtual cameras instead of real capture systems. Full article

► Show Figures

Figure 1

13 pages, 4027 KiB

Open AccessArticle

Effects of a 12-Week Semi-Immersion Virtual Reality-Based Multicomponent Intervention on the Functional Capacity of Older Adults in Different Age Groups: A Randomized Control Trial

by Li-Ting Wang, Yung Liao, Shao-Hsi Chang and Jong-Hwan Park

Multimodal Technol. Interact. 2024, 8(8), 67; https://doi.org/10.3390/mti8080067 - 1 Aug 2024

Viewed by 592

Abstract

Virtual reality (VR) exercise has been used as a strategy to promote physical health in older adults. Studies have revealed that the effects of exercise interventions vary across age groups. This study aimed to investigate the effects of a 12-week semi-immersion VR-based multicomponent [...] Read more.

Virtual reality (VR) exercise has been used as a strategy to promote physical health in older adults. Studies have revealed that the effects of exercise interventions vary across age groups. This study aimed to investigate the effects of a 12-week semi-immersion VR-based multicomponent exercise program on the functional fitness of young-old (65–73 years) and middle-old (74–85 years) adults. This study recruited two age groups (young-old adults, n = 49; middle-old adults, n = 37) and randomly assigned them to the experimental (EG) and control (CG) groups. EG participants performed a 75–90-min semi-immersion VR exercise routine twice weekly for 12 weeks, whereas CG participants maintained their original lifestyles without any alterations. The Senior Fitness Test was used to measure functional fitness by assessing upper- and lower-limb flexibility and muscle strength, cardiorespiratory fitness, and balance. EG participants exhibited greater improvements than their CG counterparts in certain functional fitness tests, specifically the Back Scratch, Arm Curl, 2-Minute Step, and 8-Foot Up-and-Go Tests. On comparing the age groups, a difference was exclusively noted in the effects of the Chair Sit-and-Reach Test. In the EG, the intervention significantly improved lower-body flexibility in young-old adults but elicited no such improvement in middle-old adults. Semi-immersion VR exercise improved the functional fitness of young-old and middle-old adults in the EG, with superior results in the former. Elucidating the impact of age-specific exercise interventions on functional capacity will help practitioners design age-specific exercise training content that enhances functional fitness in older adults of different ages. Full article

► Show Figures

Figure 1

24 pages, 2386 KiB

Open AccessReview

User Experience in Immersive Virtual Reality-Induced Hypoalgesia in Adults and Children Suffering from Pain Conditions

by Javier Guerra-Armas, Mar Flores-Cortes, Guillermo Ceniza-Bordallo and Marta Matamala-Gomez

Multimodal Technol. Interact. 2024, 8(8), 66; https://doi.org/10.3390/mti8080066 - 1 Aug 2024

Viewed by 644

Abstract

Pain is the most common reason for medical consultation and use of health care resources. The high socio-economic burden of pain justifies seeking an appropriate therapeutic strategy. Immersive virtual reality (VR) has emerged as a first-line non-pharmacological option for pain management. However, the [...] Read more.

Pain is the most common reason for medical consultation and use of health care resources. The high socio-economic burden of pain justifies seeking an appropriate therapeutic strategy. Immersive virtual reality (VR) has emerged as a first-line non-pharmacological option for pain management. However, the growing literature has not been accompanied by substantial progress in understanding how VR could reduce the pain experience, with some user experience factors being associated with the hypoalgesic effects of immersive VR. The aim of this review is (i) to summarize the state of the art on the effects of VR on adults and children suffering from pain conditions; (ii) to identify and summarize how mechanisms across immersive VR user experience influence hypoalgesic effects in patients with acute and chronic pain among adults and children. A critical narrative review based on PICOT criteria (P = Patient or Population and Problem; I = Intervention or Indicator; C = O = Outcome; T = Type) was conducted that includes experimental studies or systematic reviews involving studies in experimentally induced pain, acute pain, or chronic pain in adults and children. The results suggest an association between immersive VR-induced hypoalgesia and user experience such as distraction, presence, interactivity, gamification, and virtual embodiment. These findings suggest that hierarchical relationships might exist between user experience-related factors and greater hypoalgesic effects following an immersive VR intervention. This relationship needs to be considered in the design and development of VR-based strategies for pain management. Full article

(This article belongs to the Special Issue 3D User Interfaces and Virtual Reality—2nd Edition)

► Show Figures

Figure 1

21 pages, 16829 KiB

Open AccessArticle

Mindful Waters: An Interactive Digital Aquarium for People with Dementia

by Maarten Hundscheid, Linghan Zhang, Ans Tummers-Heemels and Wijnand IJsselsteijn

Multimodal Technol. Interact. 2024, 8(8), 65; https://doi.org/10.3390/mti8080065 - 26 Jul 2024

Viewed by 670

Abstract

Dementia can be associated with social withdrawal, mood changes, and decreased interaction. Animal-assisted therapies and robotic companions have shown potential in enhancing well-being but come with limitations like high maintenance costs and complexity. This research presents an interactive digital aquarium called Mindful Waters, [...] Read more.

Dementia can be associated with social withdrawal, mood changes, and decreased interaction. Animal-assisted therapies and robotic companions have shown potential in enhancing well-being but come with limitations like high maintenance costs and complexity. This research presents an interactive digital aquarium called Mindful Waters, which was developed to promote social interaction and engagement among People with Dementia. The pilot study involved interactive sessions at a community center and a care facility, with situated observations, video and audio recordings, and interviews to assess user engagement motivation, behavior, and user experience with Mindful Waters. The study revealed that Mindful Waters functioned well with People with Dementia and stimulated conversational topics about aquariums through engagement. User feedback was generally positive, with participants appreciating the visual appeal and simplicity. However, some participants with advanced dementia found it challenging to interact due to their mobility limitations, cognitive impairments, and the limited duration of interaction sessions. The overall results suggest that Mindful Waters can benefit dementia care; further research is needed to optimize its design and functionality for long-term placement in care facilities. Full article

► Show Figures

Figure 1

21 pages, 1037 KiB

Open AccessHypothesis

Serious Games for Cognitive Rehabilitation in Older Adults: A Conceptual Framework

by Diego E. Guzmán, Carlos F. Rengifo and Cecilia E. García-Cena

Multimodal Technol. Interact. 2024, 8(8), 64; https://doi.org/10.3390/mti8080064 - 23 Jul 2024

Viewed by 929

Abstract

This paper presents a conceptual framework for the development of serious games aimed at cognitive rehabilitation in older adults. Following Jabareen’s methodology, a literature review was conducted to identify concepts and theories that are relevant in this field. The resulting framework comprises the [...] Read more.

This paper presents a conceptual framework for the development of serious games aimed at cognitive rehabilitation in older adults. Following Jabareen’s methodology, a literature review was conducted to identify concepts and theories that are relevant in this field. The resulting framework comprises the use of virtual reality, integration of physical activity, incorporation of social interaction features, adaptability of difficulty levels, and customization of game content. The interconnections between these concepts and underlying cognitive theories, such as the cognitive reserve hypothesis and the scaffolding theory of aging and cognition, are highlighted. As we are in the early stages of our research, our goal is to introduce and test novel interpretations of current knowledge within this conceptual framework. Additionally, the practical implications of the conceptual framework are discussed, including its strengths and limitations, as well as its relevance for future research and clinical practice in the field of cognitive rehabilitation. It is hoped that this framework will provide a guide for the design and implementation of effective interventions to improve cognitive health and well-being in the older adult population. Full article

► Show Figures

Figure 1

Journal Menu

Journal Browser

Multimodal Technol. Interact., Volume 8, Issue 8 (August 2024) – 11 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI