Search Results (120)

Automated bird monitoring plays a crucial role in ecoinformatics and biodiversity conservation. Despite significant advancements in fine-grained visual classification, targets in complex wild environments frequently encounter interferences such as long capture distances, severe visual occlusion, and strong background noise, rendering single-modal perception highly susceptible to performance bottlenecks. To advance research in audiovisual multimodal bird classification, this paper introduces AVB81, a multimodal dataset tailored for fine-grained bird recognition. Covering 81 bird species in North America, the dataset comprises 3247 fixed-length 10 s field videos, meticulously supplemented with 5418 independent audios and 7083 high-quality static images. Based on this dataset, we systematically conduct single-modal performance evaluations and cross-paradigm audiovisual fusion experiments, establishing a comprehensive and in-depth multimodal evaluation benchmark. The experimental results demonstrate that in video classification tasks, audiovisual multimodal fusion methods significantly outperform single-modal baselines. Notably, the mid-fusion strategy based on deep semantic spaces achieves the optimal performance. Furthermore, the experiments objectively reveal the formidable challenges associated with audio recognition within videos captured in complex wild habitats. Overall, AVB81 exhibits exceptional adaptability for multimodal research, providing a highly challenging evaluation benchmark for multimodal semantic modeling in complex scenarios and laying a solid data foundation for the development and validation of next-generation intelligent ecological monitoring systems. Full article

This study examined how exposure to constructive audiovisual news shapes people’s perception of societal issues in the Netherlands. An online experiment was conducted among 575 participants aged 18 to 90 years old. Participants were randomly assigned to watch an audiovisual news item, either constructive or nonconstructive news, about plastic waste in the ocean or the Dutch housing market. The study investigated how these different reporting styles affected participants’ perceptions of the main message, their awareness of the seriousness of the societal issue, and their evaluation of the journalistic quality of the news report. Results showed that, contrary to critics’ concerns, constructive news did not reduce perceived problem awareness: participants across conditions reported similarly high levels of awareness regarding the seriousness of the issues presented. Perceived journalistic quality also remained high in both constructive and nonconstructive conditions, indicating that incorporating constructive elements did not compromise credibility. Furthermore, constructive news appeared to encourage a broader, more solution-oriented perspective, prompting participants to consider opportunities and future prospects. Importantly, this broader perspective did not come at the expense of perceived problem awareness or journalistic quality. Overall, the findings provide empirical support for the value of constructive journalism in the Dutch media context. Full article

This pilot study examines how experimentally induced emotional states interact with the visual properties of urban environments to shape comfort perception. A controlled laboratory experiment was conducted with 17 participants assigned to one of four emotional conditions (Fear, Anger, Sad, Happy) through audio-visual induction. Participants evaluated 73 building façade and 42 pedestrian streetscape stimuli from three urban areas in Kitakyushu, Japan (Wakamatsu, Tobata, Mojiko) using a multi-method framework combining fractal analysis (D, Λ), six pedestrian visual metrics, webcam-based eye-tracking (Visual Attention Score, VAS), and self-reported comfort votes. Emotion induction was effective for Fear and Anger groups and partial for Sad and Happy groups, with the latter attributable to experimental fatigue. Cross-method correlation analysis revealed that fractal dimension D significantly predicted comfort vote consensus (Spearman r = 0.369, p = 0.013), while VAS showed no significant relationship with comfort votes (r = 0.097, ns) or with fractal dimension (r = 0.015, ns), confirming that visual attention and comfort preference are independent dimensions. For building façades, the ‘Complex but Organized’ fractal profile (D ≥ 1.70, Λ < 0.60) was the consistent comfort driver across all emotion groups. For pedestrian streetscapes, low spatial enclosure and spatially integrated tree canopy were the primary comfort predictors. Multi-method synthesis identified five empirical paradoxes and three design principles: (1) target D ≥ 1.70 with Λ < 0.60; (2) prioritize spatially integrated canopy over visible greenery quantity; and (3) leverage civic legibility as an independent comfort pathway. These findings support the development of emotion-independent frameworks for urban comfort evaluation. Replication with larger, more diverse samples is recommended. Full article

Specialized classrooms in architecture schools serve as the core spatial infrastructure of architectural education, supporting theoretical instruction, design tutorials, model making, and critique sessions. However, existing studios often follow rigid layouts that respond inadequately to the diversity of learning stages and usage scenarios. Based on embodied cognition theory, this study integrates spatial morphological analysis, stated preference surveys, and semantic differential questionnaires to establish a multidimensional evaluation framework covering psychological perception, spatial environment, audiovisual perception, professional facilities, and professional atmosphere. Four spatial types, classified through global case review, were examined under class-time and self-study scenarios. Statistical analyses reveal a contextual fit effect whereby the same spatial type exhibits divergent perceptual performance depending on the usage scenario, and a stage-dependent gradient in which environmental concern shifts from sensory qualities toward functional efficiency as academic level advances. Gender, visit frequency, and duration of stay showed limited explanatory power relative to spatial and scenario effects. Six evidence-based optimization principles and three tiered conceptual schemes are proposed for lower-year, middle- to upper-year, and advanced users, respectively. This study offers an empirically grounded reference for the evaluation and tiered optimization of architecture design studio environments. Full article

Urban waterfronts are vital public spaces that contribute to urban sustainability by providing residents with opportunities for recreation, social interaction, and nature experiences. Understanding user perceptions in these environments is essential for evidence-based design. Taking Taiyuan Fenhe Park in China as a case with local residents as respondents, this study investigated how objective audiovisual characteristics are associated with tourist loyalty through perceptual dimensions, while also examining interactive associations between visual and auditory elements. Data were collected at 539 spatial samples spaced at five-minute walking intervals. Methods included on-site acoustic measurements, panoramic imaging, computer-based visual and auditory quantification, and questionnaire surveys, yielding a total of 1768 valid responses. Visual features were quantified using semantic segmentation, object detection, and depth prediction, whereas the auditory environment was characterized by physical acoustic and psychoacoustic indicators. Three perceptual dimensions—environmental restorativeness (ERS), spatial vitality (SVS), and environmental controllability (ECS)—were extracted and tested as mediators within the stimulus–organism–response (S-O-R) framework. Results indicated that ERS, SVS, and ECS function as three parallel mediating constructs in the statistical model, with SVS showing the strongest statistical association with tourist loyalty. In addition, fluctuation strength exhibited a significant direct effect on tourist loyalty independent of these three perceptual dimensions. A total of 17 significant audiovisual interactions were identified, revealing both synergistic and antagonistic effects. These findings contribute to theoretical frameworks of multisensory integration and provide practical guidance for sustainable waterfront design. Specifically, zoning strategies and carefully selected audiovisual combinations are relevant to enhanced user experience and may contribute to long-term social well-being. Full article

In historic districts, the audiovisual environment plays an important role in shaping both cultural expression and spatial experience. However, the influence of acoustic and visual environmental factors on perceived street vitality remains insufficiently understood. Taking the Shangxiahang Historic District in Fuzhou as a case study, this paper employs on-site sound pressure level measurements, panoramic visual data collection, questionnaire surveys, principal component analysis, correlation analysis, and multiple regression analysis to systematically examine the effects of acoustic and visual environmental factors on perceived street vitality. The results indicate that traditional cultural sounds and natural sounds have a significant positive impact on perceived street vitality, while construction noise and tour guide’s horn sound exhibit negative effects. Regarding the visual environment, street and alley spaces, traditional architecture, greenery, and the sky are all important factors in promoting perceived street vitality. Further regression analysis reveals that the perception rate of street and alley spaces has the strongest influence, followed by the perception rate of traditional architecture, the perceived frequency of folk activity sounds, preference for greenery, and the perception rate of the sky. These findings demonstrate that perceived street vitality in historic districts does not depend on a single environmental factor but rather arises from synergistic interaction between culturally meaningful acoustic cues and legible spatial forms. These results offer practical implications for multisensory design and vitality-oriented regeneration in historic districts. Full article

Existing studies of street-related emotional perception mainly rely on static scene evaluations, which cannot capture the cumulative effects of environmental exposure during continuous walking. To address this limitation, this study proposes a method for predicting pedestrian physiological responses in sequential audiovisual street environments. Four real-world walking routes were selected, with outbound and return directions treated as independent paths, yielding eight paths and 32 valid samples. EEG, ECG, sound pressure level, first-person video, and GPS data were synchronously collected to construct a 1 s multimodal time-series dataset. Pearson correlation, Kendall correlation, and mutual information analyses were used to examine linear, monotonic, and nonlinear relationships between environmental variables and physiological indicators, and the resulting weights were incorporated into a Long Short-Term Memory (LSTM) model for multi-step prediction. Visual elements and noise exposure were the main factors influencing physiological responses. Among the models, the mutual-information-weighted LSTM performed best, achieving an R² of 0.77 for heart rate variability (RMSSD), whereas prediction of the EEG ratio (β/α and θ/β) remained limited. An additional independent street sample outside the training set was then used to generate a dual-dimensional EEG-ECG physiological response map, demonstrating the model’s potential for identifying emotional risk segments and supporting street-level micro-renewal. Full article

This paper presents a systematic review, conducted in accordance with PRISMA guidelines, synthesising evidence on how visual and spatial features of classical concert settings—such as performer visibility, seating position and sightlines, stage layout, lighting, and vibrotactile cues—shape listeners’ engagement and judgments. RILM, APA PsycNet, PubMed, and Scopus were searched for peer-reviewed experimental studies that manipulated or compared visual/spatial dimensions and reported subjective or physiological outcomes relevant to live, non-amplified contexts. Titles, abstracts, and full texts were screened, and data were extracted and analysed with respect to study design, stimulus environment, outcome measures, and main effects. Heterogeneity across studies precluded meta-analysis; therefore, a narrative synthesis was conducted. A total of 23 publications—22 experiments and one meta-analysis—met the inclusion criteria: the reviewed studies primarily examined issues related to visual presence and spatial configuration. Most studies relied on laboratory or home-based audiovisual reproductions, with only one study collecting data in a naturalistic performance setting. The evidence is limited by methodological heterogeneity, the predominance of simulated environments, and variability in outcome measures. Overall, visual and spatial factors substantially shape classical music listening and the audience experience, underscoring the need for more field-based and methodologically standardised research. Full article

Background: Attractiveness plays an important role in social interactions. However, it remains unclear whether presenting attractiveness information across multiple sensory modalities facilitates attractiveness evaluation, and how cross-modal congruency modulates this process. Methods: The present study used event-related potentials (ERPs) to investigate these questions. Participants judged the attractiveness of voices presented alone or paired with faces that were congruent or incongruent in attractiveness. Results: Significant differences were found between unimodal and audiovisual conditions, as well as between congruent and incongruent pairs, during both early perceptual (N1) and later evaluative (P3) stages. Both audiovisual conditions elicited larger N1 amplitudes than the auditory-only condition, and congruent pairs produced larger N1 amplitudes than incongruent pairs. At a later stage, the auditory-only condition produced larger P3 amplitudes than the audiovisual conditions. Furthermore, the interaction between voice attractiveness and visual context on P3 amplitudes was significant. Audiovisual incongruent pairs elicited larger P3 amplitudes than congruent pairs, but only when the voice was unattractive. Conclusions: The results demonstrate that redundant visual cues of attractiveness both accelerate and alter the perception of auditory attractiveness. These audiovisual integration effects occur across different processing stages and may reflect enhanced processing efficiency in multisensory social perception. Full article

Recent advances in extended reality (XR) have enabled immersive virtual flight experiences for applications such as entertainment and teleoperation support. However, XR-based flight systems that rely primarily on audiovisual cues often fail to evoke a compelling sense of flight and embodied sensation. This study investigates how adaptive wind feedback enhances subjective flight perception in a virtual flight simulation system, Drone Rider. We implemented direction- and velocity-adaptive wind feedback that synchronizes airflow intensity and direction with the user’s motion in the virtual environment, focusing on perceptual effects in a controlled manner to identify key design factors, rather than reproducing aerodynamically accurate airflow. To explore flexible system configurations, two fan installation positions were compared: front-mounted and bottom-mounted. A questionnaire-based user study revealed that adaptive wind feedback significantly enhanced the sense of flight, self-location, and agency compared with the constant-wind and no-wind conditions. However, no significant differences were observed between velocity-adaptive wind and direction- and velocity-adaptive wind conditions. Furthermore, wind delivered from beneath the user yielded flight sensations comparable to those generated by front-mounted airflow. These findings suggest that temporal coupling between airflow intensity and visual motion plays a central role in XR flight perception and provide practical design insights for immersive and flexible XR-based flight simulation systems. Full article

This paper proposes a human–system coupling framework for understanding interactive environments in which embodied human activity is continuously translated into perceptual feedback through computational systems. Rather than conceptualizing interaction as a sequence of discrete commands, the framework interprets interactive systems as perceptual mediation environments linking bodily action, computational interpretation, and perceptual response. The framework is illustrated through the EchoCycle installation, which converts mechanical energy generated by cycling into real-time audiovisual feedback. Observations from the installation suggest that participants initially engage in exploratory behavior and gradually develop more stable activity patterns as they adapt to the feedback provided by the system. In shared interaction contexts, the perceptual environment reflects collective activity, creating conditions under which behavioral alignment among participants may emerge. By framing interactive systems as continuous perception–action loops, this study highlights how computational mediation can shape both individual adaptation and collective interaction dynamics. The proposed framework contributes to human–computer interaction and interactive system design by offering an integrated perspective on embodied action, perceptual feedback, and responsive environments. Full article

(1) Background: This article explores the qualitative dimensions of bisexual identity through the lived experiences of bisexual individuals. (2) Methods: Drawing on an online questionnaire completed by 226 participants from a Spanish-speaking sample, the study uses a grounded theory-based analysis of participant narratives. (3) Results: The analysis identifies key components of bisexual identity, such as self-recognition, fluidity, and community belonging, as well as recurrent experiences of invalidation, promiscuity stereotypes, and intracommunity discrimination. The findings highlight the processes by which participants navigate and define their bisexuality, emphasizing the interaction between personal introspection, contact with audiovisual media, societal perceptions, and external validation in identity formation. (4) Conclusions: These results provide a nuanced exploration of how bisexual identities are constructed amid persistent challenges of invalidation, erasure, and limited community recognition. Full article

This paper introduces a biomimetic framework and novel brain-inspired AI (BIAI) models based on spiking neural networks (SNNs) for emotional state recognition from audio (speech), visual (face), and integrated multimodal audio–visual data. The developed framework, named eXCube2, uses a three-dimensional SNN architecture NeuCube that is spatially structured according to a human brain template. The BIAI models developed in eXCube2 are trainable on spatio- and spectro-temporal data using brain-inspired learning rules. Such models are explainable in terms of revealing patterns in data and are adaptable to new data. The eXCube2 models are implemented as software systems and tested on speech and video data of subjects expressing emotional states. The use of a brain template for the SNN structure enables brain-inspired tonotopic and stereo mapping of audio inputs, topographic mapping of visual data, and the combined use of both modalities. This novel approach brings AI-based emotional state recognition closer to human perception, provides a better explainability and adaptability than existing AI systems. It also results in a higher or competitive accuracy, even though this was not the main goal here. This is demonstrated through experiments on benchmark datasets, achieving classification accuracy above 80% on single-modality data and 88.9% when multimodal audio–visual data are used, and a “don’t know” output is introduced. The paper further discusses possible applications of the proposed eXCube2 framework to other audio, visual, and audio–visual data for solving challenging problems, such as recognizing emotional states of people from different origins; brain state diagnosis (e.g., Parkinson’s disease, Alzheimer’s disease, ADHD, dementia); measuring response to treatment over time; evaluating satisfaction responses from online clients; cognitive robotics; human–robot interaction; chatbots; and interactive computer games. The SNN-based implementation of BIAI also enables the use of neuromorphic chips and platforms, leading to reduced power consumption, smaller device size, higher performance accuracy, and improved adaptability and explainability. This research shows a step toward building brain-inspired AI systems. Full article

To address the escalating security challenges posed by unauthorized Unmanned Aerial Vehicles, this paper presents a Sim2real physics-informed audio–visual fusion simulation platform designed to enhance Counter-Unmanned Aerial Vehicle detection and tracking performance. The proposed method integrates two complementary sensing pipelines: a physics-based acoustic localization system utilizing Time Difference of Arrival principles and a deep learning-driven visual detection framework. To ensure robust surveillance against non-cooperative targets, these pipelines are not only fused through strict spatiotemporal synchronization but also mutually reinforce each other—acoustic data guides visual attention in low-visibility scenarios typical of adversarial intrusions, while visual detections refine acoustic parameter estimation. Building upon prior work in multi-modal perception, we extend the framework to dynamic environments characterized by configurable visual obstructions, including smoke and fog, which frequently compromise conventional optical anti-drone systems. Experiments demonstrate that the fusion system progressively adapts to degraded visual conditions, extending tracking continuity from approximately 50% coverage under vision-only operation to near-continuous target awareness, with a moderate trade-off in average angular precision when acoustic-only segments are included. Physical validation with quadrotor Unmanned Aerial Vehicles confirms the platform’s capability to bridge simulation-to-reality gaps. Our results highlight the system’s robustness against sensor degradation and its potential to accelerate the development of resilient multisensor Counter-Unmanned Aerial Vehicle systems while reducing dependency on costly field testing. Full article

Expressway traffic noise poses a critical threat to public health in developed high-density cities, causing chronic environmental stress in adjacent residential areas. While physical noise barriers are commonly used, the potential of audiovisual interactions in mitigating the adverse effects of traffic noise remains under-explored. Using immersive virtual reality (VR), this study examined the efficacy of visual greenery and auditory masking (birdsong) in promoting stress recovery, and tested whether audiovisual perception mediates the environment–restoration link. Following an acute stressor, 100 participants were randomly assigned to a 2 × 2 between-subjects experiment manipulating Green View Index (high vs. low) and soundscape composition (traffic noise vs. traffic noise plus birdsong), with 25 participants in each group. Restorative outcomes were assessed using self-reported measures and continuous physiological monitoring (heart rate variability [HRV] and electrodermal activity [EDA]). Results demonstrated that high-intensity visual greenery and natural sounds effectively enhance psychological restoration in noise-affected environments. Structural equation modeling revealed that audiovisual perception fully mediated the relationship between environmental features and restorative outcomes. The physiological outcome showed a distinct tiered restoration pattern, indicating that immediate psychological buffering can be achieved through natural sounds, while consistent visual reinforcement remained essential for deep physiological recovery. Consequently, soundscape planning in expressway-adjacent zones should integrate visual greening strategies to optimize the perceptual masking of traffic noise and enhance the environmental quality. Full article