Next Article in Journal
Pixel-Level Fatigue Crack Segmentation in Large-Scale Images of Steel Structures Using an Encoder–Decoder Network
Next Article in Special Issue
Feasibility of a Mobile-Based System for Unsupervised Monitoring in Parkinson’s Disease
Previous Article in Journal
Quadcopters Testing Platform for Educational Environments
Previous Article in Special Issue
Step Length Is a Promising Progression Marker in Parkinson’s Disease
Article

Multimodal Classification of Parkinson’s Disease in Home Environments with Resiliency to Missing Modalities

1
School of Computer Science, Electrical and Electronic Engineering, and Engineering Maths, University of Bristol, Bristol BS8 1UB, UK
2
Translational Health Sciences, University of Bristol Medical School, Bristol BS8 1UD, UK
3
Movement Disorders Group, Bristol Brain Centre, North Bristol NHS Trust, Bristol BS10 5PN, UK
4
Department of Human Centred Computing, Monash University, Melbourne, VIC 3000, Australia
*
Author to whom correspondence should be addressed.
Academic Editor: Markos Tsipouras
Sensors 2021, 21(12), 4133; https://doi.org/10.3390/s21124133
Received: 30 April 2021 / Revised: 9 June 2021 / Accepted: 10 June 2021 / Published: 16 June 2021
(This article belongs to the Special Issue Sensors and Sensing Technology Applied in Parkinson Disease)
Parkinson’s disease (PD) is a chronic neurodegenerative condition that affects a patient’s everyday life. Authors have proposed that a machine learning and sensor-based approach that continuously monitors patients in naturalistic settings can provide constant evaluation of PD and objectively analyse its progression. In this paper, we make progress toward such PD evaluation by presenting a multimodal deep learning approach for discriminating between people with PD and without PD. Specifically, our proposed architecture, named MCPD-Net, uses two data modalities, acquired from vision and accelerometer sensors in a home environment to train variational autoencoder (VAE) models. These are modality-specific VAEs that predict effective representations of human movements to be fused and given to a classification module. During our end-to-end training, we minimise the difference between the latent spaces corresponding to the two data modalities. This makes our method capable of dealing with missing modalities during inference. We show that our proposed multimodal method outperforms unimodal and other multimodal approaches by an average increase in F1-score of 0.25 and 0.09, respectively, on a data set with real patients. We also show that our method still outperforms other approaches by an average increase in F1-score of 0.17 when a modality is missing during inference, demonstrating the benefit of training on multiple modalities. View Full-Text
Keywords: Parkinson’s disease; deep learning; multimodal data; missing modality; accelerometer; computer vision; variational autoencoder Parkinson’s disease; deep learning; multimodal data; missing modality; accelerometer; computer vision; variational autoencoder
Show Figures

Figure 1

MDPI and ACS Style

Heidarivincheh, F.; McConville, R.; Morgan, C.; McNaney, R.; Masullo, A.; Mirmehdi, M.; Whone, A.L.; Craddock, I. Multimodal Classification of Parkinson’s Disease in Home Environments with Resiliency to Missing Modalities. Sensors 2021, 21, 4133. https://doi.org/10.3390/s21124133

AMA Style

Heidarivincheh F, McConville R, Morgan C, McNaney R, Masullo A, Mirmehdi M, Whone AL, Craddock I. Multimodal Classification of Parkinson’s Disease in Home Environments with Resiliency to Missing Modalities. Sensors. 2021; 21(12):4133. https://doi.org/10.3390/s21124133

Chicago/Turabian Style

Heidarivincheh, Farnoosh, Ryan McConville, Catherine Morgan, Roisin McNaney, Alessandro Masullo, Majid Mirmehdi, Alan L. Whone, and Ian Craddock. 2021. "Multimodal Classification of Parkinson’s Disease in Home Environments with Resiliency to Missing Modalities" Sensors 21, no. 12: 4133. https://doi.org/10.3390/s21124133

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop