Next Article in Journal
Early Identification of Fragile X Syndrome through Expanded Newborn Screening
Previous Article in Journal
Ethanol Conditioned Taste Aversion in High Drinking in the Dark Mice
Open AccessArticle

Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain

1
Department of Electrical and Computer Engineering, Southern Illinois University, Carbondale, IL 62901, USA
2
Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN 37232, USA
*
Author to whom correspondence should be addressed.
Brain Sci. 2019, 9(1), 3; https://doi.org/10.3390/brainsci9010003
Received: 14 November 2018 / Revised: 12 December 2018 / Accepted: 25 December 2018 / Published: 2 January 2019
(This article belongs to the Section Systems Neuroscience)
Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features which are subsequently classified by a multimodal classifier. The decision-integrating model, inspired by integration in primary cortical areas, classifies unimodal stimuli independently using unimodal classifiers and classifies the combined decisions using a multimodal classifier. The multimodal classifier models are implemented using multilayer perceptrons and multivariate statistical classifiers. Experiments involving the classification of noisy and attenuated auditory and visual representations of ten digits are designed to demonstrate the properties of the multimodal classifiers and to compare the performances of multimodal and unimodal classifiers. The experimental results show that the multimodal classification systems exhibit an important aspect of the “inverse effectiveness principle” by yielding significantly higher classification accuracies when compared with those of the unimodal classifiers. Furthermore, the flexibility offered by the generalized models enables the simulations and evaluations of various combinations of multimodal stimuli and classifiers under varying uncertainty conditions. View Full-Text
Keywords: multisensory integration; multimodal object classification; feature and decision integration multisensory integration; multimodal object classification; feature and decision integration
Show Figures

Figure 1

MDPI and ACS Style

Amerineni, R.; Gupta, R.S.; Gupta, L. Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain. Brain Sci. 2019, 9, 3.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop