102 Results Found

(1) Background: This study investigates whether audiovisual n-back training leads to training effects on working memory and transfer effects on perceptual processing. (2) Methods: Before and after training, the participants were tested using the audi...

The education of future teachers is going through a crisis related to audiovisual education. Following what is known as the 2015–2030 Agenda, the world of education is being confronted with a new learning model that must respond to the proposal...

In considering capacity measures of audiovisual integration, it has become apparent that there is a wide degree of variation both within (based on unimodal and multimodal stimulus characteristics) and between participants. Recent work has discussed p...

Unilateral hearing loss (UHL) leads to an alteration of binaural cues resulting in a significant increment of spatial errors in the horizontal plane. In this study, nineteen patients with UHL were recruited and randomized in a cross-over design into...

Introduction: Speech perception relies on integrating auditory and visual information, shaped by both perceptual and cognitive factors. Musical training has been shown to affect multisensory processing, whereas cognitive processes, such as recalibrat...

The social and cultural context after the COVID-19 pandemic is on many levels very different from the previous context. Studies carried out during 2020 and 2021 point to an unprecedented increase in the use of social media. This research delves into...

This study proposes a novel off-screen sound separation method based on audio-visual pre-training. In the field of audio-visual analysis, researchers have leveraged visual information for audio manipulation tasks, such as sound source separation. Alt...

Disabled children living in orphanages in low-income countries may not have access to the therapy they need. The COVID-19 pandemic has complicated the situation dramatically, making online training activities a possible innovative option to meet the...

Nowadays, audiovisual media play a central role in access to information and in personal relationships. Among the audiovisual media is cinema, which due to its heterogeneous nature, can fulfill diverse educational functions. The objective of this stu...

Robust speech recognition in real world situations is still an important problem, especially when it is affected by environmental interference factors and conversational multi-speaker interactions. Supplementing audio information with other modalitie...

This study investigates the effectiveness of immersive audiovisual simulations in eliciting emotional responses and replicating the psychological and cognitive demands of high-risk operational environments, particularly in firefighting scenarios. Con...

This article contributes to the discourse on how contemporary computer and information technology may help in improving foreign language learning not only by supporting better and more flexible workflow and digitizing study materials but also through...

Speaker diarization systems aim to find ‘who spoke when?’ in multi-speaker recordings. The dataset usually consists of meetings, TV/talk shows, telephone and multi-party interaction recordings. In this paper, we propose a novel multimodal...

Face-to-face communication is one of the most common means of communication in daily life. We benefit from both auditory and visual speech signals that lead to better language understanding. People prefer face-to-face communication when access to aud...

Education technology represents a system of influences on the trainee in the training process. It involves the management of a didactic process with the inclusion of stages of the trainees’ activities, organization and control. The problem of educati...

Audio-visual emotion recognition is the research of identifying human emotional states by combining the audio modality and the visual modality simultaneously, which plays an important role in intelligent human-machine interactions. With the help of d...

Multisensory integration is crucial for effective cognitive functioning, especially in complex tasks such as those requiring rapid audiovisual information processing. High-level martial arts routine athletes, trained in integrating visual and auditor...

The virtual courses developed by higher education institutions incorporate the video format as one of the most used resources in the delivery of their online training offer. Within the different types of audiovisual productions found in MOOCs, the in...

While VR-based training has been proven to improve learning effectiveness over conventional methods, there is a lack of study on its learning effectiveness due to the implementation of training modes. This study aims to investigate the learning effec...

The environmental quality of rail transit spaces has increasingly attracted attention, as factors such as train noise and visual disturbances from elevated lines can impact pedestrians’ psychological perception through the audiovisual environme...

Recognition of emotions and sentiment (affective states) from human audio–visual information is widely used in healthcare, education, entertainment, and other fields; therefore, it has become a highly active research area. The large variety of...

People rely on multiple learning systems to complete weather prediction (WP) tasks with visual cues. However, how people perform in audio and audiovisual modalities remains elusive. The present research investigated how the cue modality influences pe...

Background: Foreign body airway obstruction is a sudden emergency that can occur unexpectedly in healthy people, leading to severe consequences if immediate first aid is not provided. Unlike the Heimlich maneuver for adults, the first aid for infant...

Background: The aim of the work is to verify the usability of a stabilometric platform and audiovisual feedback in the group-based therapy of patients with vertebral algic syndrome, to analyze an immediate effect after a single therapeutic unit, and...

Speech recognition approaches typically fall into three categories: audio, visual, and audio–visual. Visual speech recognition, or lip reading, is the most difficult because visual cues are ambiguous and data is scarce. To address these challen...

Scene classification in remote sensing is a pivotal research area, traditionally relying on visual information from aerial images for labeling. The introduction of ground environment audio as a novel geospatial data source adds valuable information f...

Although isomorphic pitch layouts are proposed to afford various advantages for musicians playing new musical instruments, this paper details the first substantive set of empirical tests on how two fundamental aspects of isomorphic pitch layouts affe...

Automatic speech recognition (ASR) that relies on audio input suffers from significant degradation in noisy conditions and is particularly vulnerable to speech interference. However, video recordings of speech capture both visual and audio signals, p...

Multimodal emotion recognition has emerged as a promising approach to capture the complex nature of human emotions by integrating information from various sources such as physiological signals, visual behavioral cues, and audio-visual content. Howeve...

Voice loss constitutes a crucial disorder which is highly associated with social isolation. The use of multimodal information sources, such as, audiovisual information, is crucial since it can lead to the development of straightforward personalized w...

This paper describes a post-evaluation analysis of the system developed by ViVoLAB research group for the IberSPEECH-RTVE 2020 Multimodal Diarization (MD) Challenge. This challenge focuses on the study of multimodal systems for the diarization of aud...

Speech emotion recognition (SER) is an interesting and difficult problem to handle. In this paper, we deal with it through the implementation of deep learning networks. We have designed and implemented six different deep learning networks, a deep bel...

Identification of emotions triggered by different sourced stimuli can be applied to automatic systems that help, relieve or protect vulnerable groups of population. The selection of the best stimuli allows to train these artificial intelligence-based...

This paper investigates multimodal sensor architectures with deep learning for audio-visual speech recognition, focusing on in-the-wild scenarios. The term “in the wild” is used to describe AVSR for unconstrained natural-language audio st...

In this work we present a bimodal multitask network for audiovisual biometric recognition. The proposed network performs the fusion of features extracted from face and speech data through a weighted sum to jointly optimize the contribution of each mo...

Compared to traditional unimodal methods, multimodal audio-visual correspondence learning has many advantages in the field of video understanding, but it also faces significant challenges. In order to fully utilize the feature information from both m...

Artificial Neural Network (ANN) models, specifically Convolutional Neural Networks (CNN), were applied to extract emotions based on spectrograms and mel-spectrograms. This study uses spectrograms and mel-spectrograms to investigate which feature extr...

As a multimodal fusion task, audio-visual segmentation (AVS) aims to locate sounding objects at the pixel level within a given image. This capability holds significant importance and practical value in applications such as intelligent surveillance, m...

Speech emotion recognition (SER) classifies emotions using low-level features or a spectrogram of an utterance. When SER methods are trained and tested using different datasets, they have shown performance reduction. Cross-corpus SER research identif...

Deep learning (DL) models have emerged in recent years as the state-of-the-art technique across numerous machine learning application domains. In particular, image processing-related tasks have seen a significant improvement in terms of performance d...

Recently, automatic speech recognition (ASR) and visual speech recognition (VSR) have been widely researched owing to the development in deep learning. Most VSR research works focus only on frontal face images. However, assuming real scenes, it is ob...

Participation in evacuation training can aid survival in the event of an unpredictable disaster, such as an earthquake. However, conventional evacuation training is not well designed for provoking critical thinking in participants regarding the proce...

This paper presents GAVT, a highly accurate audiovisual 3D tracking system based on particle filters and a probabilistic framework, employing a single camera and a microphone array. Our first contribution is a complex visual appearance model that acc...

Traditional crafts exhibit tangible and intangible dimensions. Intangible dimensions include the practitioner’s gestural know-how in craft practice and have received smaller attention than tangible dimensions in digitization projects. This work...

This study aims to identify motivational and attitude variables among Chilean young English learners from Concepción. A child-appropriate Likert scale questionnaire was distributed to 137 students from the 3rd, 4th, and 5th grade of two state-...

With the spread of smart devices, people may obtain a variety of information on their surrounding environment thanks to sensing technologies. To design more context-aware systems, psychological user context (e.g., emotional status) is a substantial f...

There is a strong correlation between the like/dislike responses to audio–visual stimuli and the emotional arousal and valence reactions of a person. In the present work, our attention is focused on the automated detection of dislike responses...

Light and color present complex interactions whose understanding is not always intuitive. Over the past forty years, numerous studies have aimed to identify preconceived notions and the various models employed in schools. Numerous teaching and learni...

Emotion recognition in speech has gained increasing relevance in recent years, enabling more personalized interactions between users and automated systems. This paper presents the development of a dataset of features obtained from RAVDESS (Ryerson Au...

Auscultation of heart sounds is an important veterinary skill requiring an understanding of anatomy, physiology, pathophysiology and pattern recognition. This cross-sectional study was developed to evaluate a targeted, audio-visual training resource...