Next Article in Journal
Impact of a Season of Bike Patrol on Police Officers’ Level of Fitness: A Pilot Study
Next Article in Special Issue
The Brief Form of the Test of Gross Motor Development-3 for Individuals with Visual Impairments
Previous Article in Journal
A Long-Term Incidence of Heart Failure and Predictors Following Newly Developed Acute Myocardial Infarction: A 10 Years Retrospective Cohort Study with Korean National Health Insurance Data
Article

Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution

Human Enhancement & Assistive Technology Research Section, Artificial Intelligence Research Lab., Electronics Telecommunications Research Institute (ETRI), Daejeon 34129, Korea
*
Author to whom correspondence should be addressed.
Academic Editors: Ali S. Brian and Pamela Beach
Int. J. Environ. Res. Public Health 2021, 18(12), 6216; https://doi.org/10.3390/ijerph18126216
Received: 23 April 2021 / Revised: 3 June 2021 / Accepted: 3 June 2021 / Published: 8 June 2021
(This article belongs to the Special Issue Movement Studies for Individuals with Sensory Impairments)
Visual-auditory sensory substitution has demonstrated great potential to help visually impaired and blind groups to recognize objects and to perform basic navigational tasks. However, the high latency between visual information acquisition and auditory transduction may contribute to the lack of the successful adoption of such aid technologies in the blind community; thus far, substitution methods have remained only laboratory-scale research or pilot demonstrations. This high latency for data conversion leads to challenges in perceiving fast-moving objects or rapid environmental changes. To reduce this latency, prior analysis of auditory sensitivity is necessary. However, existing auditory sensitivity analyses are subjective because they were conducted using human behavioral analysis. Therefore, in this study, we propose a cross-modal generative adversarial network-based evaluation method to find an optimal auditory sensitivity to reduce transmission latency in visual-auditory sensory substitution, which is related to the perception of visual information. We further conducted a human-based assessment to evaluate the effectiveness of the proposed model-based analysis in human behavioral experiments. We conducted experiments with three participant groups, including sighted users (SU), congenitally blind (CB) and late-blind (LB) individuals. Experimental results from the proposed model showed that the temporal length of the auditory signal for sensory substitution could be reduced by 50%. This result indicates the possibility of improving the performance of the conventional vOICe method by up to two times. We confirmed that our experimental results are consistent with human assessment through behavioral experiments. Analyzing auditory sensitivity with deep learning models has the potential to improve the efficiency of sensory substitution. View Full-Text
Keywords: sensory substitution; auditory sensitivity; cross-modal perception; generative adversarial network; visual perception sensory substitution; auditory sensitivity; cross-modal perception; generative adversarial network; visual perception
Show Figures

Figure 1

MDPI and ACS Style

Kim, M.; Park, Y.; Moon, K.; Jeong, C.Y. Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution. Int. J. Environ. Res. Public Health 2021, 18, 6216. https://doi.org/10.3390/ijerph18126216

AMA Style

Kim M, Park Y, Moon K, Jeong CY. Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution. International Journal of Environmental Research and Public Health. 2021; 18(12):6216. https://doi.org/10.3390/ijerph18126216

Chicago/Turabian Style

Kim, Mooseop, YunKyung Park, KyeongDeok Moon, and Chi Y. Jeong. 2021. "Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution" International Journal of Environmental Research and Public Health 18, no. 12: 6216. https://doi.org/10.3390/ijerph18126216

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop