Classification-Based Screening of Parkinson’s Disease Patients through Graph and Handwriting Signals ^†

Abstract

Parkinson’s disease (PD) is one of the most common neurodegenerative diseases, affecting millions of people worldwide, especially among the elderly population. It has been demonstrated that handwriting impairment can be an important early marker for the detection of this disease. The aim of this study was to propose a simple and quick way to discriminate PD patients from controls through handwriting tasks using machine-learning techniques. We developed a telemonitoring system based on a user-friendly application for drawing tablets that enabled us to collect real-time information about position, pressure, and inclination of the digital pen during the experiment and, simultaneously, to supply visual feedback on the screen to the subject. We developed a protocol that includes drawing and writing tasks, including tasks in the Italian language, and we collected data from 22 healthy subjects and 9 PD patients. Using the collected signals and data from a preexisting database, we developed a machine-learning model to automatically discriminate PD patients from healthy control subjects with an accuracy of 77.5%.

1. Introduction

Parkinson’s disease (PD) is one of the most common neurodegenerative disorders in the world, second only to Alzheimer’s disease [1]. The differential diagnosis of PD is still an ongoing challenge for the scientific community: to this day, confirmation of the disease is available only postmortem and the rate of misdiagnosis is high; it has been estimated that 25% of diagnoses are incorrect [2]. The main cause of PD is the lack of dopamine production, and its main motor symptoms are bradykinesia, tremor, and rigidity [3]; neurologists rely on imaging techniques such as MRI (Magnetic Resonance Imaging), CT (Computed Tomography), or PET (Positron Emission Tomography), and patient clinical evaluations [3]. Machine-learning techniques have been studied to help the diagnosis of PD and have shown promising results. Pereira et al. presented a review on recent studies concerning computer-assisted methods to aid PD recognition [4], which included speech, gait, and handwriting analysis. This work is part of a home-monitoring project that aims to aid in PD detection through a combined analysis of graphological and vocal signals [5]. An accuracy of 98.5% was reached through the analysis of vocal data from 55 subjects: 18 healthy control subjects and 33 drug-free and newly diagnosed PD patients.

In this study, we focused on handwriting of PD subjects; handwriting requires a complex coordination of consecutive movements, and the motor symptoms of PD can provoke handwriting impairments in the size of letters, which is referred to as micrographia, and in the pressure and kinematics of the pen [6,7], together with a general difficulty in writing, which involves different graphological patterns. Since “graphology is a discipline that deals with the dynamic study of the graphic gesture” [8], we based our analysis on computational graphology. Several studies have investigated the most relevant writing features and tasks for the diagnosis of PD. Reference [9] presents the state of the art of these studies. It is possible to collect relevant information from drawings (Archimedean spiral [10,11,12,13,14,15], circles [16], meanders [13,14], etc.) and from handwritten words and graphemes. The drawing of an Archimedean spiral (spirography) is a common task for tremor and other movement disorder analysis [10]. Thanks to the development of digitizing tablet technologies, it is possible to analyze not only the offline images, but also the kinematic characteristics of the graphic signal and the pressure applied to the tablet [17,18]. “Online” data are those collected while the user writes, while “offline” data are those available after the writing is completed [19].

In the past decade, important databases have been constructed in order to study handwriting impairments in PD: the PaHaW database [11], which includes real-time data (pen position, pen pressure, and pen inclination) collected from 38 PD patients and 37 control subjects, and the HandPD [13] and NewHandPD [14] databases, which include offline images collected by Pereira et al.

Dròtar et al., analyzing the PaHaW database, obtained an accuracy of 85.61% [20]; they demonstrated the relevance not only of the on-tablet movements, but also of the in-air movements, i.e., the variation of the pen position while the pen is not touching the table. Considering only the spiral task, they obtained an accuracy of 62.8% [12].

The aim of this work was to analyze handwriting signals from both PD patients and control subjects and to propose a way to automatically distinguish these two classes. In order to collect the necessary data, we developed a telemonitoring system based on a user-friendly application for drawing tablets that enabled us to collect real-time information about the digital pen during the experiment and, simultaneously, to supply visual feedback on the screen to the subject. Through this system, data can be collected remotely in order to allow patients to execute tasks in the comfort and safety of their home, reducing the demand on hospital services. We decided to propose a protocol to explore writing and drawing impairments, including specific tasks for subjects who declared Italian as their first language, since, to our knowledge, the literature is lacking in automatic handwriting classification studies with Parkinson’s subjects whose first language is Italian.

2. Methods and Materials

In this study, we collected data from 22 healthy subjects and from 9 PD patients. The data from the PD subjects were collected thanks to a collaboration with the Casa di Cura Le Terrazze institute. All participants were right-handed except for one PD patient, with an age in the range of 60 ± 25 years. Information about subjects’ age, gender, dimensions of the hand, and level of education are collected in

Table 1. Subject data. For each group (healthy control (C) or subject with Parkinson’s disease (PD)), the age and the number of males and females are specified. Information about the anatomical measurements of the hand and the level of education is indicated only for the control group.

Group	Age (Mean ± SD)	Number Male/Female	Middle Finger–Wrist Distance (cm) (Mean ± SD)	Level of Education (ISCED 2011) (Mean ± SD)
C	55.8 ± 6.5	8/14	19.6 ± 1.8	5.1 ± 2.1
PD	69.3 ± 10	5/4	-	-

. The educational level was classified according to UNESCO’s ISCED 2011 (International Standard Classification of Education) [21]. This classification distinguishes nine levels of education, from early child education (level 0) to doctoral or equivalent level (level 8). These levels can be aggregated into three categories: low (0–2), medium (3–4), and high (5–8) [22]. The hand dimension was quantified by measuring the distance between the wrist and the top of the distal phalanx of the dominant hand’s middle finger.

Information about the PD patients is collected in

Table 2. This table shows, for every PD patient, the Hoehn and Yahr Scale level (H&Y), the side of the body affected, the years since the PD diagnosis, and the levodopa-equivalent daily dose (LEDD) assumed by the subject.

Patient ID	H&Y	Side	Years since Diagnosis	LEDD
#1	2	Left	3	450
#2	3	Left	2	750
#3	2	Bilateral	1	200
#4	3	Right	6	625
#5	3	Left	11	400
#6	3	Left	4	250
#7	4	Bilateral	4	400
#8	1	Right	1	50
#9	3	Left	13	900

. For every patient, the Hoehn and Yahr Scale level is indicated. The Hoehn and Yahr Scale is a clinical scale used to describe the progressive motor impairment of subjects with PD [23]. This scale goes from 1 to 5 in order of severity of the motor symptoms; the first level corresponds to only unilateral involvement with minimal motor dysfunction, while the fifth and last stage corresponds to a more serious level of motor dysfunction wherein the subject is confined to bed or wheelchair unless aided. Table 2 also indicates the side of the body affected by the motor dysfunction (left, right or bilateral if both sides are involved), years since the diagnosis of PD, and the levodopa-equivalent daily dose (LEDD) corresponding to each PD patient.

A commercial Wacom One drawing tablet with a screen was used for this test in order to be able to extract both “online” and “offline” features. Wacom tablets are widely used in handwriting movement analysis, as they offer high spatial and temporal resolution [9].

An application was developed by our team using the development platform Unity, which allowed us to collect information about pen position $\left(x,y\right)$, pressure, and inclination with a frequency of 133 Hz and, simultaneously, to supply visual feedback on the tablet’s screen to the subjects. For the protocol, the Wacom tablet was connected to a Lenovo Thinkpad T495 computer with Windows 10 as the operating system. The “Duplicate” modality was selected in order to have the same screen shown on the computer and the Wacom One tablet, as shown in Figure 1; Figure 1a shows the point of view of the operator, while Figure 1b shows the point of view of the participant.

The application has a start page where the participant’s ID can be entered and which includes a menu from which the user can choose which task to take. The data are saved locally in different.tsv files for every acquisition.

In order to analyze the data, we used the software MATLAB.

The protocol was divided into four parts: drawing an Archimedean spiral, writing the bigram “le” six times and two Italian sentences, drawing ten concentric circles, and writing seven lines of free text. For each part of the protocol, a different screen was shown to the subject: firstly, an image of an Archimedean spiral was shown and the subject was asked to trace it at a comfortable speed; secondly, a blank screen was shown and the subject was asked to write six times in cursive the bigram “le” and the two Italian phrases: “I fiori sono sul prato” and “Nel cielo ci sono le stelle”. On the third screen, a circle was shown and the subject was asked to draw ten concentric circles inside it. Lastly, a blank screen was shown and the subject was asked to write seven lines of free text in cursive. The overall duration of the test varied between 10 and 15 min from subject to subject. The subjects were given the opportunity to try the tablet before the test. During the execution of the tasks, the subjects were seated in a comfortable position on a chair without armrests, and the tablet was positioned on a table in front of them.

Features were extracted separately from each task.

Data were separated into components, i.e., lines that are traced without lifting the pen from the tablet. In order to do that automatically, indices of the samples where pressure went from positive to zero and vice versa were saved in a vector of markers. Both in-air and on-tablet features were extracted.

Figure 2 shows an example of the “le” bigram task, where the different components, automatically detected, are represented in different colors and the “in-air” points of the pen position are represented as blue points. For each component, the velocity was calculated as follows:

$$v=\sqrt{v_x{}^2+v_y{}^2},$$

(1)

where $v_x=\frac{x_{i+1}-x_i}{t_{i+1}-t_i}$ and $v_y=\frac{y_{i+1}-y_i}{t_{i+1}-t_i}$,

Where x_i, y_i, and t_i correspond to the pen position along x, pen position along y and time at a specific index i of the recorded data, respectively.

Acceleration and jerk of the components were also calculated. To analyze the spiral, the angular and radial velocity were calculated. Furthermore, the distance of the drawn spiral from the spiral guide was calculated using the following algorithm:

• For each point of the drawn spiral $\left(x_i,y_i\right)$, we found the spiral guide’s closest point to it, $\left(s{x}_i,s{y}_i\right)$.
• We calculated the distance of each couple of points $\left(x_i,y_i\right)$, $\left(s{x}_i,s{y}_i\right)$, as

  $$d_i=\sqrt{\left(x_i{}^2-s{x}_i{}^2\right)+\left(y_i{}^2-s{y}_i{}^2\right)}$$

  (2)
• We found the parameter

  $$p={\sum }_i{d}_i{}^2,$$

  (3)

  that describes how much the drawn spiral is distant from the spiral guide. A smaller value of p meant a higher precision.

Furthermore, the power spectral density between 4 Hz and 9 Hz of the absolute velocity of the pen during the spiral task was calculated as a feature.

In order to develop a model for automatic classification of PD, we used data from 36 PD subjects and 35 healthy control subjects from the PaHaW dataset and data from the 22 healthy control subjects and 9 PD patients that we collected in our database. Two tasks that the two databases have in common were analyzed: the guided spiral and the bigram “le”. The features that were considered are reported in

Table 3. Features extracted: 1 if the feature was extracted from the spiral task analysis, 2 if the feature was extracted from the “le” bigram task analysis.

Features	Task
Velocity: absolute, vertical, and horizontal	1,2
Acceleration: absolute, vertical, and horizontal	1,2
Jerk: absolute, vertical, and horizontal	1,2
Radial velocity	1
Angular velocity	1
Variation of velocity, acceleration, and stroke between components	2
Number of changes of direction in velocity	1,2
Number of changes of direction in acceleration	1,2
Number of changes of direction in jerk	1,2
Normalized in-air time (time in air over total time)	1,2
In-air velocity: absolute, vertical, and horizontal	2
In-air acceleration: absolute, vertical, and horizontal	2
In-air jerk: absolute, vertical, and horizontal	2
Power spectral density of the absolute velocity	1

.

After the feature extraction, we proceeded with the selection of the most significant features and the generation of the classification model. In Figure 3, a scheme of the workflow is presented.

3. Results and Discussion

In order to discriminate between PD patients and healthy control subjects, three models were constructed: one using only data from the spiral task, one using only data from the “le” bigram task, and one using data from both of them.

We selected the most discriminant features using a Mann–Whitney test (p < 0.05).

Table 4. Feature ranking. The left column presents the most discriminant features for the spiral task, and the right column presents the most discriminant features for the “le” task.

Spiral	le
skewness of the pen acceleration on the x axis	mean of the maximum pen velocity on the y axis of the single le component in the air
kurtosis of the pen velocity on the x axis	mean of the range of the pen velocity on the y axis of the single le component in the air
25th percentile of the power spectral density of the absolute pen velocity	maximum of the acceleration on the x axis in the air

presents the features with the lowest p-scores, divided per task.

A 10-fold cross-validation was conducted. Results are reported in

Table 5. Model used and classification accuracy, specificity, sensitivity, F1 score, and precision of the two tasks analyzed separately and of the two tasks combined.

	Spiral	le	Spiral and le
Model	Linear SVM	Linear SVM	Medium KNN
Accuracy	71.6%	75.5%	77.5%
Specificity	79%	73.7%	77.1%
Sensitivity	62.2%	77.8%	77.8%
F1 Score	65.9%	73.7%	75.3%
Precision	70%	70%	72.9%

. Accuracy, specificity, sensitivity, F1 score, and precision were calculated in terms of TP (true positive), FP (false positive), TN (true negative), and FN (false negative), as follows:

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}\ast 100$$

(4)

$$Specificity=\frac{TN}{FP+TN}\ast 100$$

(5)

$$Sensitivity=\frac{TP}{FN+TP}\ast 100$$

(6)

$$F1 score=\frac{TP}{TP+\frac{1}{2}\left(FP+FN\right)}\ast 100$$

(7)

$$Precision=\frac{TP}{TP+FP}\ast 100$$

(8)

Considering the two tasks separately, we obtained a higher accuracy for the “le” bigram tasks than for the spiral tasks.

Moreover, considering the spiral and the “le” bigram task separately, the accuracy that we obtained for the spiral (71.6%) was higher than the accuracy obtained for the spiral by Dròtar et al. (62.8%) and, similarly, considering only the “le” bigram task, the accuracy that we obtained (75.5%) was higher than the accuracy obtained by Dròtar et al. for this task (71%).

The confusion matrix for each one of the three models (spiral, “le” bigram and combined tasks) are reported on the Figure 4.

The highest accuracy (77.5%) and sensitivity (77.8%) were obtained by combining the two tasks, while the highest specificity was obtained when using only the data from the spiral task. The machine-learning algorithms that were employed were the support vector machine (SVM) for the spiral task and the “le” bigram task, and the medium k-nearest neighbors (Medium KNN) for the combined tasks.

4. Conclusions

In this study, an application is presented that allowed us to register data from tablets with a frequency of 133 Hz, in order to aid the recognition of PD through handwriting impairments. The tool that is proposed is simple and easy to use, allowing subjects to complete the test in the comfort of their home.

Data from 22 healthy subjects and 9 PD patients were collected and added to the PaHaW database [11,20], a pre-existing dataset that includes data from PD patients and healthy control subjects. Using only two of the eight tasks that the PaHaW database includes, an accuracy of 77.5%, was obtained, close to the 85.61% accuracy that Dròtar et al. obtained when considering all the eight tasks together [20]. We could not compare the other tasks because the first language declared by our subjects (Italian) was different from the first language of the PaHaW database’s subjects (Czech).

The major limitations of this study are linked to the limited number of subjects involved and to the fact that we compared data from different databases, collected under different experimental conditions, such as the position of the subject during the tasks, and acquired with different devices, which could lead to a bias in the measurements. Moreover, subjects’ characteristics such as age could lead to misclassification; for example, a control subject could present some tremor or bradykinesia not linked to Parkinson’s disease, and for this reason could be misclassified as a PD subject.

However, the protocol that we developed can be used in future studies to collect more data from Italian PD subjects, in order to be able to create a model using only data from our protocol, using the combination of six tasks proposed here (drawing of an Archimedean spiral, writing the bigram “le”, writing two Italian phrases, drawing ten concentric circles, and writing seven lines of free texts.

Moreover, this work is part of a home-monitoring project that aims to aid in PD detection through a combined analysis of graphological and vocal signals [5].. The sets of subjects tested for the vocal tasks and the graphological tasks were different from each other, so we could not create a classification model using combined vocal and graphological data, but the aim of this project is to continue to collect both vocal and graphological data in order to create a single, more complete, classification model.