Skip to Content
  • Article
  • Open Access

15 December 2025

19 Pages

Sensory Representation of Neural Networks Using Sound and Color for Medical Imaging Segmentation

,
and
1
IT Department, Computer Engineering, School of Engineering, University of Minho, 4704-553 Braga, Portugal
2
Algoritmi/LASI, University of Minho, 4704-553 Braga, Portugal
3
Faculty of Sciences and Technology, University of Azores, 9500-321 Ponta Delgada, Portugal
*
Author to whom correspondence should be addressed.
This article belongs to the Section Medical Imaging

Abstract

This paper introduces a novel framework for sensory representation of brain imaging data, combining deep learning-based segmentation with multimodal visual and auditory outputs. Structural magnetic resonance imaging (MRI) predictions are converted into color-coded maps and stereophonic/MIDI sonifications, enabling intuitive interpretation of cortical activation patterns. High-precision U-Net models efficiently generate these outputs, supporting clinical decision-making, cognitive research, and creative applications. Spatial, intensity, and anomalous features are encoded into perceivable visual and auditory cues, facilitating early detection and introducing the concept of “auditory biomarkers” for potential pathological identification. Despite current limitations, including dataset size, absence of clinical validation, and heuristic-based sonification, the pipeline demonstrates technical feasibility and robustness. Future work will focus on clinical user studies, the application of functional MRI (fMRI) time-series for dynamic sonification, and the integration of real-time emotional feedback in cinematic contexts. This multisensory approach offers a promising avenue for enhancing the interpretability of complex neuroimaging data across medical, research, and artistic domains.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.