Dynamic Handwriting Analysis for Neurodegenerative Disease Assessment: A Literary Review

Abstract

Studying the effects of neurodegeneration on handwriting has emerged as an interdisciplinary research topic and has attracted considerable interest from psychologists to neuroscientists and from physicians to computer scientists. The complexity of handwriting, in fact, appears to be sensitive to age-related impairments in cognitive functioning; thus, analyzing handwriting in elderly people may facilitate the diagnosis and monitoring of these impairments. A large body of knowledge has been collected in the last thirty years thanks to the advent of new technologies which allow researchers to investigate not only the static characteristics of handwriting but also especially the dynamic aspects of the handwriting process. The present paper aims at providing an overview of the most relevant literature investigating the application of dynamic handwriting analysis in neurodegenerative disease assessment. The focus, in particular, is on Parkinon’s disease (PD) and Alzheimer’s disease (AD), as the two most widespread neurodegenerative disorders. More specifically, the studies taken into account are grouped in accordance with three main research questions: disease insight, disease monitoring, and disease diagnosis. The net result is that dynamic handwriting analysis is a powerful, noninvasive, and low-cost tool for real-time diagnosis and follow-up of PD and AD. In conclusion of the paper, open issues still demanding further research are highlighted.

1. Introduction

1.1. Motivations and Purposes

Neurodegenerative diseases, such as Parkinson’s disease (PD) and Alzheimer’s disease (AD), affect the structure and functions of brain regions resulting in a progressive cognitive, functional, and behavioural decline. PD is caused by the degeneration of the dopaminergic nigrostriatal neurons of the basal ganglia, resulting primarily in motor deficits: akinesia, bradykinesia, rigidity, and tremor are typically observed [1]. AD, on the other hand, is characterized by short-term memory loss in its early stages, followed by a progressive decline in other cognitive and behavioral functions as the disease advances: therefore, the dominant feature of AD is mainly of cognitive nature [2]. Unfortunately, in the case of signs of brain degeneration, there is no cure and the gradual decline of the patient can only be somehow managed during disease progression. However, an early diagnosis of neurodegeneration would be crucial in the perspective of proper medical treatment to be administered and for improving the quality of life of the patient. In addition, the assessment of signs and manifestations of a specific disease is useful for its diagnostic differentiation with respect to similar disorders and for monitoring and tracking its progression as the disease advances. To this end, a special attention is devoted to mild cognitive impairment (MCI) signs, as an individual with MCI is at a high risk of developing dementia, especially of the Alzheimer’s type [3].

The evaluation of the patient’s clinical status and their responsiveness to medication is typically achieved via a clinical workup including a thorough medical history, a neuropsychological test battery, and rating scales. Mini Mental State Examination (MMSE) [4], for example, is used extensively to assess cognitive impairment. However, there is still no one certain test to determine if someone is affected by a neurodegenerative disorder and a precise diagnosis is possible only postmortem. Getting a reliable diagnosis can require months, and symptoms need to be constantly monitored. In addition, the traditional evaluations depend to some extent on the experience of the clinician performing the assessment, and this makes the determination of the exact type of disease as well as its degree of severity difficult. For these reasons, identifying accurate biomarkers for early and differential diagnosis, prognosis, and response to therapy is a primary goal of the research on neurodegenerative disorders today (e.g., References [5,6]).

Changes in the brain caused by neurodegeneration—brain atrophy, neuronal loss, synaptic dysfunction, etc.—particularly result in a dysfunction of the motor system as well as in impairments of the performance of previously learned motor skills. Therefore, a key role in the context of neurodegenerative diseases assessment can be assumed by handwriting. Handwriting, in fact, is a complex activity entailing motor as well as cognitive components [7], of which the changes are promising as a biomarker for disease assessment. First, handwriting exercises are already part of neuropsichological test batteries. For instance, the Clock Drawing Test (CDT), which is part of the Mini-Cog test, requires the patient to draw a clock from memory and to put the hands at a given time: the goal is to evaluate executive functions [8]. Second, it is worth noting that, in several studies, researchers examined handwriting difficulties by using writing tests: their results showed that these difficulties are well correlated to the disease severity as well as the concomitant cognitive impairment. For example, in the seminal paper by McLennan et al. [9], it was pointed out how micrographia, which is an abnormally small writing typically associated with PD, can be easily detected by simple pen-and-paper exercises. Other studies, e.g., References [10,11], used analogous tasks and found that agraphia, which encompasses a progressive disorganization of the various components of handwriting, is an early symptom of AD.

Although several advancements have been so far obtained through the analysis of static characteristics of handwriting, i.e., the ones that can be analyzed after the writing process has already occurred, with the advent of new technologies, novel, dynamic features of handwriting have been available to the research community. These features concern the dynamic characteristics of handwriting that can be acquired while the writing process still occurs. Typical acquisition tools are inexpensive commercially available digitizing tablets and/or electronic pens. Through these devices, one can measure not only temporal and spatial variables of handwriting but also the pressure exerted over the writing surface and measures of pen inclination and pen orientation. Moreover, these devices can capture pen movement not only while the pen is in contact with the writing surface but also when the pen is in close proximity of the surface, i.e., “in-air”.

In the context of neurodegenerative diseases assessment, dynamic handwriting analysis has been employed for studying several issues and has attracted considerable research interest from psychologists to neuroscientists and from physicians to computer scientists. A large part of the literature on this topic investigated fine motor control in healthy and unhealthy people. Examining changes in the handwriting of impaired patients, in fact, facilitates the understanding of the brain–body functional relationships and can lead to identifiable patterns of the sensorimotor dysfunction associated with PD or AD. Several other studies focused on the effects of medication on handwriting: these changes can provide a useful tool for monitoring and tracking disease progression. More recently, an increasing research effort has been made towards the development of an automatic tool for the discrimination between impaired subjects and healthy controls on the basis of dynamic handwriting features. The goal is to provide a complementary approach to the pathology evaluation performed by expert clinicians that is quantitative, noninvasive, and very low-cost.

This paper aims at providing an overview of the most relevant literature investigating the application of dynamic handwriting analysis to the assessment of neurodegenerative disorders. In particular, PD and AD, as the two most widespread and most extensively investigated disorders, are taken into account.

1.2. Related Surveys

It is worth remarking that surveys on this topic have already been provided in References [12,13,14,15]. Neils-Strunjas et al. [12] discussed papers focusing only on the static characteristics of handwriting in AD. Letanneux et al. [13] considered papers that focused both on static and dynamic features for PD assessment. In particular, the authors proposed to extend the concept of “dysgraphia” also to PD, as it encompasses all deficit characteristics of Parkinsonian handwriting. De Stefano et al. [14] and Impedovo and Pirlo [15] recently proposed surveys focusing both on PD and AD. In Reference [14], De Stefano et al. made a categorization of works into statistical and classification studies, based on the methodological approach followed by the reviewed experiments. However, while the paper extensively reviews works on AD and MCI as well, at the time of writing, less research was done on classification of PD, thus forcing the authors to exclude several findings that are currently available. In particular, only one classification study on PD is reviewed in the paper. Conversely, in Reference [15], the authors considered the problem at hand only from a pattern recognition perspective. For this reason, they did not consider a body of previous literature not using the established machine learning experimental workflow. This paper is intended to provide a more comprehensive overview of the topic, providing the reader with a broad and organized view covering a wider spectrum of methodological approaches and analyses. In particular, the present survey aims at covering papers using either statistical or classification approaches, starting from the earlier papers, which reports the first attempts to investigating dynamic handwriting analysis for neurodegenerative diseases assessment, to the very recent works. The topic received an exploding attention in the last few years; thus, this papers aims at covering also the very recent advancements achieved.

1.3. Structure of the Survey

The present survey is intended to provide the reader not only with a historic, state-of-the-art, and future perspective on the topic but also with some guidelines. These guidelines may be useful to the reader to enter this line of research or to easily compare their findings with the existing literature. For this reason, the literary review provided in this paper is divided in two parts. The first part, which is reported in the next section, describes the experimental design typically adopted: the process of dynamic handwriting analysis is sketched, and the main issues arising from its application to health care are pointed out. Almost all surveyed studies, in fact, share a common experimental design including data acquisition, feature extraction, and data analysis. In particular, different studies reported the results of the application of different techniques, depending on the research question to be investigated. The second part, which is reported in Section 3, discusses the main research questions that have been addressed. As previously mentioned, the literature on this topic mainly followed three research directions: providing insights into the motor control mechanisms of handwriting; monitoring and tracking disease progression and the responsiveness of patients to therapies; and providing novel instruments for the (possibly early) real-time disease diagnosis. The last section concludes the paper and provides some considerations about directions for further research on the topic.

2. Typical Experimental Design

The studies investigating the application of dynamic handwriting analysis in neurodegenerative diseases assessment typically follow a common experimental setup including data acquisition, feature extraction, and data analysis (Figure 1). These issues are discussed separately in the following subsections.

2.1. Data Acquisition

At this step, issues arising concern participant recruiting, apparatus choice, and acquisition protocol definition. The currently available datasets are also described.

2.1.1. Participant Recruiting

In recruiting study participants, three aspects should be taken care of. The first important issue is to have the groups under study balanced under some criteria. Besides their cardinality, the study groups should be balanced at least in terms of age; otherwise, handwriting changes may be attributed to age differences instead of underlying pathological conditions. A balanced education (typically expressed in years) should also be considered, as there is evidence that education can influence the dynamics of handwriting despite the presence of cognitive decline [16].

The second aspect, instead, is whether the patient is on/off medication. For example, some studies on PD dealt with patients under treatment of antiparkinson medication (e.g., References [17,18]). These studies showed how handwriting significantly changes depending on the level of medical treatment administered.

Finally, the third aspect concerns the disease severity, in accordance with some standard clinical score. The unified Parkinson’s disease rating scale—UPDRS (part V) score, corresponding to the Modified Hoehn and Yahr Scale [19], is a commonly used rating scale for describing how PD symptoms evolve during time. Conversely, standard assessments of probable AD include cognitive and functional tests such as the already mentioned MMSE or the Trail Making Test [20]. MMSE, in particular, is a 30-point questionnaire which includes questions and problems in many areas: from orientation to time and place, and attention and calculation, etc. Having data of patients at different degrees of disease severity can better support the early disease diagnosis or the multi-class classification problem.

It is also important to pay attention to individuals who suffered injuries that could have significantly affected their handwriting: these participants should be excluded.

2.1.2. Apparatus

Current technology makes available a multitude of devices for data acquisition, some of them providing immediate visual feedback to the writer. The dynamic handwriting data are generally acquired by using digitizing tablets and/or electronic pens. The dominant attributes acquired are the x- and y-coordinates of the pen position and their time stamps. Moreover, pen tablets usually capture more information than the pen trajectory, namely pen orientation (azimuth and altitude) and pen pressure. In addition, pen tablets also detect the pen trajectory while the tip is not in contact with the pad surface, allowing trajectory acquisition pen-ups. One measure, in fact, is the so-called button status, which is a binary variable evaluating 0 for pen-ups (in-air movement) and 1 for pen-downs (on-surface movement).

It is worth remarking that elderly people may be unfamiliar with technological tools: to make writing conditions as close as possible to the usual ones, writing with an inking pen on a sheet of paper fixed to the tablet is an effective option (e.g., Reference [21]).

Electronic pens (also called “smart pens”) have been also adopted in alternative to tablets. For example, in References [22,23] a Biometric Smart Pen (BiSP) biometric smart pen was used. The BiSP pen is a multi-sensor pen system which is capable of capturing position, acceleration, and tilt angle of the pen, as well as the pressure and vibration generated in the refill during writing and the grip pressure of the fingers holding the pen.

Contrary to other diagnostic methods, such as medical imaging, data acquisition through these devices can be carried out even in the patient’s home; moreover, the task performance is quite simple and natural and does not require timing or exhaustive repetitions.

2.1.3. Acquisition Protocol

A crucial step in designing a computerized tool based on handwriting concerns the choice of the most appropriate handwriting tasks to be administered for data acquisition. Some tasks, in fact, may be redundant with other ones; others may even introduce noise in the data. Some recent works [24,25,26], in fact, employed ensembles of classifiers, each built on the feature space of every single task, emphasizing how a performance-driven selection of a subset of tasks can improve classification performance against the use of all tasks simultaneously. Generally speaking, handwriting tasks can be classified into simple drawing, simple writing, and complex tasks: they are described in the following paragraphs. It is worth noting that, in order to allow participants to familiarize with the equipment, some preliminary trials are typically required before the effective experimental session is carried out.

Drawing Tasks

Spirals, as well as meanders and circles, have been frequently used for the evaluation of motor performance. Spiral drawing on a digitizing tablet, in particular, was pioneered by Pullman [27] for assessing tremor. In fact, it is particularly suited to study motor control deficits in PD patients. The task is very easy to perform and is usually well tolerated. In general, simple drawings have been used for trajectory, tremor, dimension, and velocity evaluations, e.g., References [23,28,29]. Fine motor control problems may be caused by a reduced capability to coordinate the fingers and wrist and by a reduced control of wrist flexion. In Reference [28], for example, MCI and AD patients produced less automated, accurate, and regular movement compared to controls when drawing a spiral with the dominant hand. Differentiation between MCI and controls increased when subjects were requested to press a device, with the nondominant hand, while drawing the spiral. In Reference [30], excellent sensitivity in discriminating AD and MCI patients from controls with in-air movement was observed: the task consisted in copying a simple 3-D house with two windows, a door, and a chimney. Drawing a simple figure is very easy to perform and is usually well tolerated by all subjects. Complexity increases in the intersecting pentagon copying task, which is part of the MMSE test. Patients, in fact, typically exhibit constructional apraxia: drawing may contain fewer angles, spatial alterations, lack of perspective, and simplifications [31]. Patients can be unable to perform the task even if they understand what they should do. In particular, they typically show different drawing strategies: some trace the contours of the figure to be copied, others put points first and then connect them with segments, and so on. These issues can be reflected in the dynamic features of handwriting [31,32].

Writing Tasks

No-sense words composed by one or more character repetitions, for example lll and lele, can be used, e.g., References [22,33,34,35,36]. These characters are easy to write in a recursive and continuous fashion. One of the most typical evidence of PD is rigidity and tremor; thus, in contrast to controls, which show an automated handwriting, PD may produce slower and more irregular movements. In addition, PD patients may write letters in a more segmented fashion and show micrographia over time within the task [35]. The difficulty to anticipate the upcoming letter, in particular, may be the expression of a general difficulty in producing simultaneous actions. For this reason, this task can be discriminant also in the case of AD [34]. Writing words/sentences is suited to assess agraphia. A sentence requires a high degree of simultaneous processing and may have a higher neuromotor programming load than a sequence of the same characters, since it also involves linguistic skills, attention, and memory (for example, in the case where the sentence must be remembered). It also provides the possibility to evaluate the motor-planning activity between a word and the following one (in general, a hesitation between two words could highlight the necessity to replan the writing activity, while fluid writing can reveal the presence of an anticipated motor plan). A sentence allows one to capture a large number of in-air movements between words; by contrast, a word could be written without leaving the pen from the tablet surface [37,38]. Some works, e.g., Reference [39], also consider handwritten signatures. A signature represents an automatic gesture rather than a programmed one, as it is repeated very frequently during the lifespan. Since it requires only a minimal consciousness, a signature can remain preserved even when the subject is no longer able to write. Therefore, signatures may be weak predictors of cognitive impairment. Nevertheless, a signature carries a huge amount of information about the person who signs. Indeed, features of handwritten signatures emphasizing subtle deterioration of signature apposition have been successfully used to differentiate among groups [39].

Complex Tasks

Finally, the handwriting task can be part of a more complex task also involving cognitive and functional issues. For example, handwriting has been examined together with a simultaneous hearing and tone counting or has been part of a functional task (e.g., copying a bank cheque [40]). In Reference [40], participants with MCI and AD showed a significantly longer in-air time than controls. Moreover, they exerted more pressure on the surface: mean pressure, indeed, provided the best information for classification. The well-known Clock Drawing Test involves not only executive functions but also numerical knowledge, visual memory, planning, reconstruction, and visuospatial abilities. When drawing the clock, people with better cognitive-functional level generally divide the circle into different quadrants, placing the numbers 12, 3, 6, and 9 first and then the others. Conversely, patients with dementia start writing from 1 or 12 (sometimes from 11), filling the whole space with the following numbers; often, the clock is filled leaving out either the first or last number. The hands are indicated with a simple segment, and this is not a sign of cognitive decline. Instead, missing the position of the hands is a typical sign of cognitive or neurological deficit. In Reference [30], excellent sensitivity and good specificity in discriminating MCI patients from controls were obtained with in-air time. Finally, Trail Making Test and Attentional Matrices explore cognitive abilities and executive functions, in particular attentional skills, visuomotor planning and problem solving. The examiner is interested in evaluating the time of completion and the number of errors. In the Trail Making Test, the test taker is asked to connect a sequence of numeric or alphanumeric targets. Recently, Reference [41] showed that features related to timing (including times between and inside circles and rates between and inside circles) and features related to mobility (including pauses, lifts, pressure, and size) provide additional information not captured by the traditional paper-based Trail Making Test. The Attentional Matrice test, instead, is a cancellation test in which the subject is asked to mark target digits assigned among several distractors. In Reference [42], it was shown how the perceptual decision while scanning, easily captured by in-air movement analysis, is impaired in cognitively deteriorated subjects.

2.1.4. Datasets

Unfortunately, very few datasets are currently available to the research community. A schematic description of each of them is provided in

Table 1. Datasets (PD = Parkinson’s disease; AD = Alzheimer’s disease; EC = elderly controls).

Dataset	Groups	Apparatus	Tasks	Reference
PaHaW	37 PD, 38 EC	Wacom Intuos 4M	Spiral drawing, repetition of characters, words, and sentence writing	Drotár et al. [38]
NewHandPD	31 PD, 35 EC	Biometric Smart Pen	Spiral and meander drawing	Pereira et al. [23]
ParkinsonHW	62 PD, 15 EC	Wacom Cintiq 12WX	Spiral drawing and stability test	Isenkul et al. [43]
ISUNIBA	29 AD, 12 EC	Wacom Intuos Touch 5	Repetition of a single word	Impedovo et al. [44]

.

The Parkinson’s Disease Handwriting Database (PaHaW) consists of multiple handwriting samples from 37 Parkinsonian patients and 38 age- and gender-matched controls. Subjects were requested to complete eight handwriting tasks in accordance with a prefilled template: drawing an Archimedes spiral; writing in cursive the letter l, the bigram le, and the trigram les; writing in cursive the word lektorka (“female teacher” in Czech), porovnat (“to compare”), and nepopadnout (“to not catch”); and writing in cursive the sentence Tramvaj dnes už nepojede (“The tram won’t go today”).

The original HandPD dataset comprises handwritten exams from healthy and PD people; thus, it was primarily intended for static analysis. However, the dataset was further extended for dynamic analysis purposes, comprising data from 66 individuals (35 healthy controls and 31 PD patients). The extended version is called NewHandPD. Each individual was asked to draw 12 exams, with 4 of them related to spirals, 4 related to meanders, 2 circled movements (one circle in the air and another on the paper), and left- and right-handed diadochokinesis. During the exam, the handwritten dynamics was captured by using the BiSP smart pen.

The ParkinsonHW database collects 62 PD patients and 15 healthy individuals. From all subjects, three types of handwriting recordings, namely Static Spiral Test (SST), Dynamic Spiral Test (DST), and Stability Test on Certain Point (STCP), were considered. The images of the spirals drawn by patients are also provided. In the SST test, three Archimedes spirals appeared on the graphic tablet and patients were asked to retrace them. Unlike SST, in the DST test, the Archimedes spiral just appeared and disappeared at certain time stamps. This forced the patient to keep the pattern in mind and to continue to draw. In the STCP test, there was a certain red point in the screen and the subjects were asked to hold the digital pen on that point without touching the surface. The purpose of this test was to determine the patient’s hand stability or hand tremor level.

Finally, the ISUNIBA dataset collected the data of 29 probable AD patients and 12 healthy controls, who were requested to write the word mamma (“mother” in Italian) over different writing sessions. This is one of the first words learned and one of the last words used before dying.

At the time of writing, Castrillón et al. [45] are developing a large set of Parkinsonian handwritten patterns, including samples from adult and young healthy individuals. Concerning AD, the Handwriting Analysis against Neuromuscular Disease (HAND) project, among its goals, intended to release a large dataset of a battery of handwriting tasks performed by elderly controls and by people suffering from MCI and neurodegenerative dementia [32,46].

2.2. Feature Extraction

The horizontal and vertical components of handwriting, as recorded by the tablet, are typically segmented into on-surface and in-air strokes in accordance with the button status. A stroke corresponds to a single trait of the handwritten pattern which is connected and continuous, i.e., between two consecutive pen-lifts. By using the Cartesian coordinates of the sampled points and their time stamps, several features can then be calculated for both on-surface and in-air strokes.

Kinematic features include number of strokes; tangential, horizontal, and vertical displacement, velocity, acceleration, and jerk; number of changes of velocity/acceleration (NCV/NCA); and NCA and NCV relative to writing duration. Displacement corresponds to the straight-line distance between two consecutive sampled points: it provides a good approximation of the pen trajectory. From displacement, velocity, acceleration, and jerk can be straightforwardly calculated as the first, second, and third derivatives of displacement, respectively. Analogously, displacement, velocity, acceleration, and jerk can be calculated with respect to both the horizontal and vertical directions. NCV and NCA are the mean number of local extrema of tangential velocity and acceleration, respectively.

Spatiotemporal features include stroke size and duration; speed and stroke speed; stroke height and width; on-surface and in-air time; total time; normalized on-surface and in-air time; and in-air/on-surface ratio.

In order to make use of the pressure signal, the following dynamic handwriting measures are also typically calculated: mean pressure; number of changes of pressure (NCP); and relative NCP. NCP was proposed in Reference [38], and its meaning is analogous to the concept of NCV/NCA, explained above.

In order to capture the randomness and irregularity of fine movements, which are difficult to analyze using only the abovementioned features, the following features can also be computed for both the on-surface and in-air horizontal and vertical components of handwriting [37]: Shannon and Rényi entropy; signal-to-noise ratio (SNR); and empirical mode decomposition (EMD). EMD iteratively decomposes the signal into so-called intrinsic mode functions (IMFs), which are functions that satisfy two requirements: (1) the number of extrema and the number of zero crossings are either equal or differ at most by one, and (2) the mean of their upper and lower envelopes equals zero.

It is worth noting that, to obtain complete statistical representations of the available features, statistical functions of the feature vector are also computed. They include means, percentiles, moments, and other statistical functions (range, median, mode, standard deviation, etc.). In addition, note that features are generally normalized before classification so as to have zero mean and unit variance.

An alternative approach to modeling the handwritten patterns is to use the Kinematic Theory of Rapid Human Movements [47,48] and, in particular, the so-called sigma-lognormal ($\Sigma \Lambda$) model [49]. This model has been used with successful results in many practical applications, for example, for developing an online signature verification system [50] and for analyzing graphomotor performance in kindergarten children [51]. The main advantage of this approach is that it is based on a physiological model of the human movement production which can lead to an improved characterization of the hidden specificity of the writers.

Finally, due to their increasing popularity, a robust alternative to more classic “hand-crafted” features is to use features automatically learned by deep learning models. Some works, in particular, used (possibly pretrained) convolutional neural networks for automatically extracting features from static images obtained by exploiting dynamic information of the handwriting, e.g., Reference [52].

A schematic overview of the features most commonly used in the different studies is provided in

Table 2. Most commonly used features: they are typically intended both on-surface and in-air.

Feature	Description	Observation
Direct
Position	$(x,y)$-coordinates of the sampled points	They are used to derive the geometrical pattern of handwriting
Time stamp	Temporal information of the sampled points	It is used to derive the temporal duration of the hand movement
Pressure	Pressure exerted over the writing surface	In patients, pressure takes on erratic values due to cognitive and muscular difficulties
Tilt-x	Angle between the pen and the surface plane	Their use is typically ignored and their usefulness is controversial
Tilt-y	Angle between the pen and the plane vertical to the surface
Button status	Boolean variable of whether the pen is on-surface or in-air	It enables the separation between on-surface and in-air movements. It has been shown how the two handwriting modalities carry on nonredundant information
Kinematic
Displacement	Trajectory during handwriting	It is generally used to derive other kinematic features
Velocity	Rate of change of displacement with respect to time	Patients suffering from neurodegenerative diseases do not write with the same constancy of healthy subjects but are affected by a lower writing speed, with continuous acceleration peaks
Acceleration	Rate of change of velocity with respect to time
Jerk	Rate of change of acceleration with respect to time
NCV/NCA	Number of local extrema of velocity/acceleration	These measures are tailored to capture the fluency of the handwriting movement. Highly automated movements are characterized by bell-shaped and smooth velocity and acceleration profiles
Spatio-temporal
Stroke size	Strokes’ path length	Patients suffering from PD can exhibit micrographia
Stroke height/width	Height/width of strokes
Stroke duration	Movement time per stroke	The average writing duration of a patient affected by a neurodegenerative disease is typically longer than in a healthy subject
Time	Time spent on-surface/in-air during writing
Entropy and energy
Entropy	Entropy-based features	These measures are tailored to capture the randomness and irregularities of fine movements
SNR	Signal-to-noise ratio
EMD	Empirical mode decomposition
Model-based
$\Sigma \Lambda$-based	Parameters of the $\Sigma \Lambda$ reconstruction of the handwritten pattern	The $\Sigma \Lambda$ model can help investigate the dynamics of handwriting during the generation of the action plan
Automatically learned
Deep-learning based	Features automatically learned by deep learning models trained on static representations of the handwriting dynamics	Their meaning is typically hard to interpret: they can express subtle between-group differences not captured by traditional hand-crafted features

. Some features provide different perspectives on the same aspect of handwriting, e.g., kinematic and spatiotemporal features are able to capture the fluency and (ir)regularities of handwriting movements, leading to similar results. Some others, in particular those automatically learned by deep learning models, are difficult to correlate with the other ones; however, they may provide novel and nonoverlapping information. In general, almost all features, either directly measured by the digitizing tablet or derived from them, have been used with promising results in every single study. Only the pen angle information is typically discarded: its applicability appears to be not useful, even if a very recent work applied it and reported encouraging results [53].

2.3. Data Analysis

The goal of this final step is to uncover useful patterns able to support decision making. Mostly, the literature investigating handwriting changes due to aging and relies on statistical analyses to perform this step. For example, the classic analysis of variance (ANOVA) is typically used to test group differences across different measures of handwriting.

In the last years, the studies focusing on the development of computer aided diagnosis systems have made use of machine learning and statistical pattern recognition strategies to discriminate between unhealthy and healthy subjects [54]. In a series of experiments, for instance, Drotár et al. found that support vector machines, fed with kinematic and spatiotemporal features, provide better prediction accuracy than other classic approaches, such as Naïve Bayes, e.g., Reference [37].

More recently, due to their increasing popularity in a plethora of recognition tasks, some works investigated the usefulness of deep learning approaches [25,55]. The features automatically extracted by a convolutional neural network can be used to feed a fully connected layer stacked on top of the convolutional base or a more classic statistical classifier.

It is worth noting that, since the data at disposal are typically small, several resampling methods are usually adopted to achieve more reliable evaluations of the classification performance, such as cross-validation and leave-one-out [56].

A simple analysis, based on the visual inspection of the performed task and the velocity and pressure profiles of handwriting is sketched in Figure 2.

3. Research Directions

The studies involving dynamic handwriting analysis for neurodegenerative diseases assessment can be broadly classified in accordance with the disease taken into account: Parkinson’s and Alzheimer’s disease. For each of them, different lines of investigation can be identified: they are discussed in the following.

3.1. Parkinson’s Disease

The papers focusing on Parkinsonian handwriting can be further classified depending on three main research directions:

• Disease insight: the first category (including the oldest papers) have been devoted to providing an insight into the fine motor control of handwriting and its relationship with the concomitant impairment. The main goal is to better understand the involved mechanisms underlying PD;
• Disease monitoring: other papers studied the effects of medication on handwriting with the aim to evaluate the effectiveness of handwriting analysis on monitoring disease progression;
• Disease diagnosis: the third category (including the most recent works) investigated the use of handwriting as an inexpensive objective tool for automatic disease diagnosis.

This section is structured in accordance with this classification. A schematic overview is provided in

Table 3. Summary of studies on PD (EC = elderly controls; YC = young controls; and SZ = schizophrenia patients).

Reference	Groups	Tasks	Main Features	Main Findings
Disease insight
Phillips et al., 1991	7 PD, 7 EC, 7 YC	Meander writing	Kinematic and spatiotemporal	Dynamic analysis can be useful to characterize PD handwriting
Teulings and Stelmach, 1991	6 PD, 6 EC	No-sense word and a sentence	Kinematic and spatiotemporal	PD patients experience problems with the production and regulation of force amplitude
Fucetola and Smith, 1997	20 PD, 20 EC	Figure drawing	Kinematic and spatiotemporal	PD patients have difficulties in adjusting the size of their drawing to compensate for distortions in visual feedback
Oliveira et al., 1997	11 PD, 14 EC	No-sense word	Kinematic and spatiotemporal	PD patients write with a normal amplitude when given external cues
Teulings et al., 1997	17 PD, 12 EC	Lines, circles, and no-sense words	Kinematic	PD fine motor control problems may be caused by a reduced capability to coordinate the fingers and wrist and by a reduced control of wrist flexion
Van Gemmert et al., 1998	9 PD, 9 EC, 9 YC	Sentence writing	Kinematic and spatiotemporal	PD patients are vulnerable to a moderate level of secondary task load
Van Gemmert et al., 1999	13 PD, 15 EC	Lines, circles, and no-sense words	Spatiotemporal	PD patients may have trouble in maintaining a constant force amplitude
Swinnen et al., 2000	13 PD, 13 EC	Triangle drawing	Kinematic	PD patients benefit from practice to alleviate their basic motor deficits
Van Gemmert et al., 2001	7 PD, 7 EC	No-sense word and sentence writing	Kinematic and spatiotemporal	PD patients show micrographia when cuncurrent processing load increases
Teulings et al., 2002	11 PD, 16 EC, 10 YC	No-sense word	Spatiotemporal	PD patients rely on previous or ongoing feedback to program subsequent strokes
Van Gemmert et al., 2003	13 PD, 13 EC	No-sense words	Spatiotemporal	PD patients produce inadequate stroke sizes when the size equals or exceeds 1.5 cm
Caligiuri et al., 2006	13 PD, 10 SZ, 12 EC	“hello hello”	Kinematic	PD patients exhibit impaired movement velocity and velocity scaling
Ponsen et al., 2008	13 PD, 13 EC	Sentence writing	Kinematic and spatiotemporal	Impairments in performing tasks involving complex uni-manual upper limb movements are an early sign of PD
Broderick et al., 2009	16 PD, 16 EC, 16 YC	Shape drawing	Kinematic	PD patients show smaller-than-required movement amplitude
Dounskaia et al., 2009	9 PD, 9 EC	Line and circle drawing	Kinematic	PD causes deficits in coordination patterns between wrist and finger motions
Gangadhar et al., 2009	34 PD, 25 EC	No-sense word	Spatiotemporal	PD handwriting exhibits smaller size and larger velocity fluctuations than normal handwriting
Bidet-Ildei et al., 2011	7 PD, 7 EC	No-sense words	Kinematic and spatiotemporal	PD handwriting does not show signs of motor anticipation
Ma et al., 2013	15 PD, 15 EC	A Chinese character	Spatiotemporal	Micrographia is not evident in vertical writing
Broeder et al., 2014	18 PD, 11 EC	Loops while counting	Kinematic and spatiotemporal	PD patients exhibit deficits in handwriting under dual-task conditions
Smits et al., 2014	10 PD, 10 EC	Circle, star, and spiral drawing, a no-sense word and a sentence	Kinematic and spatiotemporal	Handwriting can provide objective measures for bradykinesia, micrographia, and tremor
Senatore and Marcelli, 2019	30 PD, 30 EC, 30 YC	Loop shapes	Kinematic	Parkinsonian handwriting is similar to that produced by beginner writers
Disease monitoring
Eichhorn et al., 1996	29 PD, 40 EC	Circle drawing	Kinematic	Dynamic handwriting analysis is useful for quantifying dopamimetic effects
Contreras-Vidal et al. and Poluha et al., 1998	10 PD	No-sense words	Spatiotemporal	Handwriting measures show significant trends across the levedopa cycle
Siebner et al., 1999	12 PD	Lines, circles, and a no-sense word	Spatiotemporal	subthalamic nucleus stimulation improves handwriting performance in PD patients
Cobbah and Fairhurst, 2000	6 PD	No-sense words	Kinematic	Dynamic handwriting measures are indicative of positive response to dopamine
Boylan et al., 2001	10 PD	Spiral drawing	Temporal	repetitive transcranial magnetic stimulation provides beneficial effects on PD movements
Lange et al., 2006	12 PD, 12 EC	Sentence writing	Kinematic	Participants with an altered dopaminergic neurotransmission shifted from an automatic to a controlled processing of movement execution
Tucha et al., 2006	27 PD, 27 EC	Sentence writing	Kinematic	Dopaminergic medication results in improved dynamics of movement execution
Randhawa et al., 2013	10 PD	No-sense word	Kinematic and spatiotemporal	Fine motor performance benefit from rTMS stimulation
Smits et al., 2015	14 PD	Graphical tasks	Spatiotemporal	Graphical tasks are useful to assess upper limb functioning
Danna et al., 2018	20 PD, 20 EC	Spiral drawing	Kinematic and spatiotemporal	PD handwriting performance may not be impacted by handedness
Disease diagnosis
Ünlü et al., 2006	28 PD, 28 EC	Meanders, sentence, words, and no-sense word writing	Pressure	Pressure features can distinguish between PD patients and controls
Drotár et al., 2013a	PaHaW		Kinematic and spatiotemporal	Good accuracy is obtained by using only on-surface features
Drotár et al., 2013b; 2014			Kinematic and spatiotemporal	Accuracy improved with in-air features
Drotár et al., 2015a; 2015b			Kinematic, spatiotemporal, entropy, and energy	Accuracy improved using entropy and energy-based features
Drotár et al., 2016			Kinematic, spatiotemporal, and pressure	Pressure features can be profitably used
Rosenblum et al., 2013	20 PD, 20 EC	Functional tasks	Kinematic and spatiotemporal	Patients write smaller letters, applying less pressure and requiring more performance time than controls
Pereira et al., 2016; 2018; Afonso et al., 2018	NewHandPD		Deep learning-based	The deep learning approach is promising for the recognition problem at hand
San Luciano et al., 2016	138 PD, 150 EC	Spiral drawing	Kinematic and spatiotemporal	Spiral analysis is a promising quantitative biomarker for the early diagnosis
Kotsavasiloglou et al., 2017	24 PD, 20 EC	Line drawing	Kinematic	High predictive accuracy can be obtained even using very simple tasks
Zham et al., 2017	27 PD, 28 EC	Spiral drawing	Kinematic and pressure	Spiral drawing can be partly used to differentiate among degrees of disease severity
Impedovo et al., 2018	PaHaW		Kinematic, spatiotemporal, entropy, energy, and pressure	Accuracy decreased in distinguishing between controls and patients only at the early stage of disease
Gallicchio et al., 2018	ParkinsonHW		Deep learning-based	Recurrent neural networks are profitably used
Mucha et al., 2018	PaHaW		Fractional derivative-based	Promising results are obtained with fractional derivative-based features
Impedovo, 2019	PaHaW		Velocity-based	New velocity-based features are proposed
Jerkovic et al., 2019	33 PD, 10 EC	Sentence writing	Kinematic	The best results are obtained when combining in-air to on-surface features
Loconsole et al., 2019	4 PD, 7 EC	Word and no-sense word writing	Kinematic	Gyroscope-based features are used with promising results
Rios-Urrego et al., 2019	39 PD, 39 EC, 40 YC	Spiral drawing and sentence writing	Kinematic, geometrical, and nonlinear features	Irregularities of handwriting increase with aging
Diaz et al., 2019	PaHaW		Deep learning-based	A new dynamically enhanced static representation of handwriting is proposed
Ribeiro et al., 2019	NewHandPD		Deep learning-based	Recurrent neural networks can be fruitfully used to capture tremor in time-dependent handwriting signals
Ammour et al., 2020	28 PD, 28 EC	Text copying	Kinematic, pressure, and pen inclination	A new semi-supervised approach is proposed to discriminate among groups

.

3.1.1. Disease Insight

Phillips et al. [57] and Teulings and Stelmach [58] were among the first to use digitizing tablets to assess Parkinsonian handwriting. The work of Phillips et al. was the first showing how dynamic handwriting features (in particular in the velocity domain) can successfully differentiate between patients and healthy controls. Teulings and Stelmach (1991), instead, asked participants to alter their usual handwriting in an attempt to study the extent to which the patients’ motor system can adjust size, force, and speed parameters. Results showed that Parkinsonians, as well as controls, were generally able to modify stroke size, peak accelerations, and stroke duration as they wrote the required patterns. However, a signal-to-noise analysis suggested that the movement deficits were primarily due to an impaired force-amplitude component rather than an impaired stroke-duration component.

Contreras-Vidal and Stelmach [59] were the first to integrate previous experimental data on the anatomy of the basal ganglia to the motor impairments in PD; the aim was to develop a neural model of the basal ganglia useful to explain normal and Parkinsonian movements. The model consists of a model of basal ganglia, in which each nucleus is represented by a single unit, combined with a model capable of learning and generating simple handwriting movements. This model was able to reproduce many aspects of the normal and PD movement control including hypometria, bradykinesia, akinesia, impairments in the coordination of multiple joints, micrographia, effects of levodopa on movement size and speed, and pallidotomy. The simulation data of this model, in fact, were confirmed by the experimental data obtained in some other studies, e.g., References [33,60].

Since proprioceptive, kinaesthetic, and visual feedback are essential for the completion of many movements, impaired utilization of sensory feedback may retard the effective learning of motor programs. Based on this hypothesis, Fucetola and Smith [61] investigated the effects of a distorted visual feedback on the drawing performance of Parkinsonian patients. They observed that patients were less able than controls to adjust the size of their drawing to compensate for distortions in visual feedback. The effect was particularly pronounced when patients were required to draw smaller than normal. Nevertheless, with practice, PD patients showed a similar degree of improvement in size as controls, although they did not match the control group’s level of performance.

Oliveira et al. [62] investigated whether micrographia in individuals with PD is lessened either by giving visual targets or by continually reminding them that they should write in a normal way. In a first trial of free writing, patients showed micrographia, as they reduced their letter size over time within the trial. However, the letter size increased significantly when they were given either visual targets or constant auditory reminders. This improvement persisted when, shortly afterwards, the patients were requested to write freely without external cues.

Teulings et al. [33] investigated whether Parkinsonism reduces coordination of fingers, wrist, and arm in fine motor control. These movement problems contribute to an increase in jerk levels, as jerk represents the rate of change of acceleration over time. In the PD group, back-and-forth strokes involving coordination of fingers and wrist showed larger normalized jerk than strokes performed using either the wrist or the fingers alone. Moreover, wrist flexion showed greater normalized jerk in comparison to wrist extension. The elderly control subjects showed no such effects as a function of coordination complexity. Thus, the authors hypothesized that fine motor control problems in PD patients may be caused by a reduced capability to coordinate the fingers and wrist and by a reduced control of wrist flexion.

Van Gemmert et al. [63] tested the hypothesis that PD patients are more vulnerable to a moderate level of secondary task load than elderly or young controls due to a heightened variability in the motor system. Patients and the two control groups were requested to write a sentence under four load conditions: start writing after they heard the recorded word “start”; ignore auditory presented digits while writing; repeat orally the presented digits; and subtract the number 2 from each presented digit and pronounce the outcome aloud. The results obtained showed that, in contrast to young and elderly controls, PD patients tended to increase movement time and normalized jerk when the secondary task consisted primarily of motor load. Furthermore, it was shown that PD patients did not reduce writing size because of a high level of mental load: this suggested that writing in an automated fashion does not cause micrographia.

Van Gemmert et al. [60] investigated whether PD patients can have difficulty in increasing stroke size, decreasing stroke duration, or both during the execution of noncomplex handwriting tasks. To this end, they designed an experiment comprising simple writing patterns, such as straight lines and circles, requesting participants to vary writing size and speed. Although the different handwriting patterns affected movement time and writing size significantly, patients did not show increasing difficulty in maintaining writing size and/or stroke duration because of a decrease of pattern complexity: thus, it was argued that the complexity of a pattern is likely to not be a relevant factor in handwriting.

In Reference [64], Swinnen et al. addressed the problem of determining whether practice modifies the temporal and spatial features of handwriting in PD patients. The findings obtained showed that PD patients can change their performance thanks to practice, suggesting that practice may help them to partly overcome bradykinesia. Nevertheless, they never reached the performance level obtained by the elderly control group.

Van Gemmert et al. [65] hypothesized that the cause of micrographia in PD patients can be associated with the concurrent processing demands that result from the coordination and control of fingers, wrist, and arm during writing and processing of future words. In their experiments, patients and controls were requested to write four different phrases of various word counts. All phrases started with an llll pattern, and this pattern was repeated later in the phrase. PD patients reduced stroke size when the number of words increased in the phrase, i.e., when the processing demand increased. This finding suggested that the motor system of PD patients anticipates increased processing demands by reducing stroke size rather than increasing stroke duration.

Teulings et al. [66] compared PD patients to elderly people and young adults with respect to their ability to use visual feedback to control handwriting size. Participants wrote sequences of cursive l-shaped loops on a digitizer display, which enabled the authors to distort the visual feedback without the participant’s knowledge by altering the vertical dimension of handwriting. The results showed that controls gradually corrected loop size by enlarging (or reducing) the size of the entire loop sequence. Conversely, PD patients showed an entirely different response: instead of correcting for the distortions, they progressively amplified its effect. This suggested that PD patients do not adapt their visuomotor map in response to the distorted visual feedback of handwriting. Instead, they seem to rely constantly on the visible trace feedback during the ongoing movement. The authors thus hypothesized that they either plan their writing based on the visual feedback of their previous strokes or that they attempt to track the ongoing, distorted handwritten trace.

Van Gemmert et al. [67] evaluated the ability of PD patients to increase stroke size independently of stroke duration for different sizes. Patients and controls were requested to write cursive patterns at different sizes (1, 1.5, 2, 3, and 5 cm). Each target pattern was displayed at its required size on the tablet but disappeared as soon as the pen touched the surface of the screen. In contrast to controls, patients with PD undershot the target size of 2 cm and, when required to write as fast as possible, they even undershot the 1.5 cm target size. These findings support the hypothesis that the range in which stroke size can be manipulated without significant changes in stroke duration is smaller in Parkinsonian handwriting than in the healthy handwriting.

Caligiuri et al. [68] examined the handwriting dynamics of patients with idiopathic PD, schizophrenia, and drug-induced Parkinsonism (SZ) and of healthy control. Participants were instructed to write the word hello twice at three vertical height scales. The (in)ability to scale movement velocity with increasing movement distance was quantified. Four observations were drawn: (1) both SZ patients with drug-induced Parkinsonism and PD patients exhibited impaired movement velocities and velocity scaling; (2) performance on the velocity scaling measure can distinguish drug-induced Parkinsonism from controls with 90% accuracy; (3) SZ, but not PD, participants displayed abnormalities in movement smoothness; and (4) there was a positive correlation between age and magnitude of the velocity scaling deficit in PD participants.

Ponsen et al. [69] were among the first to analyze handwriting in newly diagnosed, untreated PD patients. The results of the study showed that newly diagnosed patients are impaired in performing complex uni-manual upper limb motor tasks in comparison to healthy subjects. They appeared to be particularly impaired in their handwriting, exhibiting reduced sentence length and writing velocity and a decrease in letter height during writing. Therefore, the authors concluded that impairments in performing tasks involving complex uni-manual upper limb movements are an early characteristic of PD; thus, they could be used for the early disease diagnosis.

Broderick et al. [70] considered a drawing task instead of handwriting ones to address the hypothesis that PD patients exhibit deficits in controlling acceleration when the task involves an increase in inertial load, specifically under the requirement to increase movement amplitude and/or speed, and in the weight of the pen. Patients showed significantly lower mean velocity, lower acceleration, higher constant error of stroke length, and higher normalized jerk scores than controls. Nevertheless, these effects were not worsened by adding weight to the pen. The observed smaller-than-required movement amplitude suggested a relationship between hypometria and bradykinesia in drawing and/or handwriting.

In Reference [71], Dounskaia et al. tested the hypothesis that PD affects differently handwriting movements depending on the coordination pattern of wrist and finger motions. To investigate this hypothesis, the groups under study were requested to perform three types of cyclic wrist and finger movements: drawing two lines and a circle. Although both groups deformed the circle during fast movements, the deformation was more pronounced in patients than in controls. A possible reason for this is that PD patients may be unable to properly regulate the influence of biomechanical factors on wrist and finger motion.

In the model of PD handwriting proposed in Reference [59], basal ganglia nuclei is modeled as lumped units, with activity levels represented by rate codes. Basal ganglia dynamics is described in terms of fixed-point behavior; thus, only magnitude-related aspects of handwriting—faster/slower, larger/smaller, etc.—can be captured. Gangadhar et al. [72] presented an alternative model of Parkinsonian handwriting, which produces a stable rhythm in a network of oscillators and resolves the stroke output in a Fourier-style. In the paper, the model predictions were compared to handwriting data obtained by patients and controls. PD handwriting statistically exhibited smaller size and larger velocity fluctuation compared to normal handwriting. These findings were reflected in both experimental data and network predictions.

Bidet-Ildei et al. [35] hypothesized that, if it is true that PD patients produce sequential movements in a more segmented fashion, then they should have difficulties in anticipating the forthcoming letter. Their experimental findings revealed that handwriting in PD patients did not exhibit any sign of motor anticipation: although they could write three letters without pauses, PD patients tended to produce each letter in a more independent manner. In order to explain this, the authors suggested that the difficulty in anticipating the upcoming letters may be the expression of a general difficulty in producing simultaneous actions.

Ma et al. [73] noticed that all the published studies investigating micrographia in PD examined handwriting only in the horizontal direction, as the handwritten samples analyzed were primarily in Western languages. However, several other languages, as those from Eastern Asia, can be written not only horizontally but also vertically, from top to bottom. Since different directions require different joint coordination patterns and writing horizontally requires more wrist extension than writing vertically, the micrographia reported in horizontal writing may not be generalized to characters written vertically. To investigate on this problem, the authors asked patients and controls to write Chinese character Zheng. The main finding was that the PD group had a linear decrease in overall character size and horizontal strokes along the writing sequence in the horizontal direction but not in the vertical direction. This observation confirms that micrographia in PD may be associated with wrist extension.

Broeder et al. [74] obtained results in line with the abovementioned study by Van Gemmert et al. [63]: they observed that PD patients experience more dual-task interference during writing than controls when performing a cognitive tone-counting task and a writing task simultaneously. Dual-task interference refers to the decreased performance experienced during dual tasking, i.e., when two motor tasks with different goals are combined. More specifically, the secondary task consisted in counting high and low tones during writing. The results obtained showed that dual-task performance was affected in PD patients. In particular, they suggested that the control of writing at small amplitudes requires more compensation brain-processing resources in PD than controls.

Smits et al. [36] investigated handwriting tasks that may be helpful to provide a quantitative method to differentiate between PD patients and healthy controls: circle, star, and spiral drawing; elel; and writing a sentence. The drawing and writing tasks were analyzed to evaluate the speed of movement to assess bradykinesia and the size of writing to assess micrographia. In addition, a frequency analysis was carried out to assess rest tremor. The results showed that Parkinson patients tend to be slower than healthy control participants. PD patients also wrote smaller than controls. Furthermore, rest tremor was detected in the group of patients who were clinically assessed as having rest tremor.

In a very recent work [75], Senatore and Marcelli proposed a novel paradigm aimed at emulating the early stage of handwriting learning in proficient writers by asking them to produce a familiar l-shape with a novel, unfamiliar motor plan. In other words, participants were asked to produce the sequence of strokes by using a motor plan different from the one an individual is used to. The authors involved young and elderly healthy participants comparing them with the data of the pathological group of the PaHaW dataset. The authors found that Parkinsonian writing during a familiar movement is characterized by lack of fluency, slowness, and abrupt changes of direction, as the handwriting produced by beginner writers. These results support the hypothesis that the fine tuning of the motor plan parameters involved during the production of handwriting is deteriorated by PD.

3.1.2. Disease Monitoring

Papers falling in this category mainly investigated two kinds of treatment: antiparkinson medication and neurostimulation. Concerning the former, in particular, PD treatment often involves the administration of levedopa to reduce the associated rigidity and bradykinesia. During this treatment, a conversion process occurs in the brain so that levedopa becomes dopamine and the reduced level of the body own’s dopamine is compensated.

Eichhorn et al. [17] used a computational analysis of open-loop handwriting movements to monitor the effect of levodopa and apomorphine in three groups of Parkinson patients: those with untreated probable Parkinson’s disease, those with fluctuating PD, and some other patients with known levedopa unresponsive Parkinsonism. Subjects were instructed to draw fluently concentric circles. After apomorphine injection, the group with untreated probable PD and the group with long-standing PD showed significant improvement of kinematic features. The patients with levedopa unresponsive Parkinsonism did not change significantly in any of the parameters under study. In conclusion of the paper, the authors observed that the improvement of handwriting kinematics by dopamimetic stimulation may be helpful to predict responsiveness to levodopa treatment in Parkinsonian syndromes.

Levedopa levels decay over several hours; thus, every few hours, another dose of levedopa should be taken. In light of this, Contreras-Vidal et al. [76] and Poluha et al. [77] hypothesized that Parkinsonian handwriting would change across the levedopa cycle. The most remarkable finding of these studies was that handwriting up-stroke duration varied significantly across the medication cycle.

In Reference [78], Siebner et al. investigated the effect on handwriting of high-frequency stimulation of the subthalamic nucleus (STN), which is a therapeutic approach in patients with severely disabling PD. During high-frequency STN stimulation, handwriting movements became faster and smoother, indicating a partial restoration of the open-loop automatic performance. In addition, a stimulation-related reduction in micrographia was observed.

Cobbah and Fairhurst [79] investigated the dynamic changes evident in ordinary handwriting under strict dopamimetic challenge test conditions. Patients with Parkinsonism were requested to write handwriting patterns before medication and once again at peak motor performance, after doses of apomorphine or levodopa were administered. The results obtained suggested that a differentiation between on and off states in dopamimetic tests is possible by using ordinary handwritten samples. The effects reflected on kinematic features of handwriting, in fact, suggested improvements in movement efficiency in the on state.

Boylan et al. [80] studied the therapeutic potential of repetitive transcranial magnetic stimulation (rTMS) for PD by delivering stimulation at high intensity and frequency over time. rTMS is a noninvasive technique that allows the cortical excitability to be altered; thus, it can induce a dopamine release in the stratum of people with PD. Among some other tests, assessment included spiral drawing as handwriting task. The major finding of the study was the worsening of motor performance on spiral drawing with active rTMS to the supplementary motor area (SMA) of patients.

Lange et al. [81] carried out a battery of experiments to study the role of dopamine in movement execution during handwriting. The findings of the experiments showed that alterations of the dopamine system adversely affect movement execution during handwriting. All experiments showed that the number of inversions of the direction of velocity is increased in participants with an altered dopaminergic neurotransmission.

A study analogous to the one reported in Reference [81], with the same apparatus and procedure, allowed the authors to reach some other conclusions on the dopaminergic effects on handwriting movements [18]. The main finding was that dopamine medication results in a partial restoration of automatic movement execution: although dopaminergic treatment in PD patients resulted in marked improvements in the handwriting dynamics, patients never reached an undisturbed level of performance.

Analogously to Reference [80], Randhawa et al. [82] investigated whether the delivery of rTMS impacts handwriting performance. The authors found that 5-Hz rTMS over SMA increased the global size of handwriting. Moreover, the stimulation led to a decrease in the amount of pen pressure. These findings suggested that 5-Hz rTMS over SMA can influence key aspects of handwriting including vertical size and axial pressure, at least in the short term.

In Reference [83], Smits et al. evaluated the validity of a battery of graphical tasks useful to assess upper limb functions in individuals with probable PD. The Purdue Pegboard Test (PPT), in which metal pins have to be placed within holes, was used as a reference test. Only PD patients, who were on and off medication, performed the tasks. Moderate correlations between performance on graphical tasks and the PPT test were obtained, suggesting that the set of graphical tasks is a valid tool to assess and monitor upper limb functions in PD. In addition, the study showed that this set can be used to detect subtle changes in performance after medication that are barely visible by only observing the patient.

Considering that handwriting involves linguistic processes that can be influenced by cognitive impairments and sociocultural factors, Danna et al. [84] focused only on drawing tasks, particularly spiral drawing, which have the advantage to involve exclusively motor mechanisms. Different analyses were carried out to evaluate the effectiveness of digitized spiral drawing in distinguishing patients with and without medical treatment. The results obtained confirmed this hypothesis. Surprisingly, the general performance of PD patients was not impacted by handedness, suggesting that the side-dominance of PD symptoms can prevail over handedness.

3.1.3. Disease Diagnosis

In Reference [22], Ünlü et al. focused on approaches for PD diagnosis based on the pressure information provided by the electronic biosensor BiSP pen. It turned out that the most discriminating feature, which achieved an Area Under the ROC Curve (AUC) equal to 0.933, was based on the difference between the controlled writing pressure in the x-y direction and the tilt tremor of the pen. It was observed, in fact, that, for PD patients, the tremor control is better achieved during movements (like handwriting) instead of constant pressure (pen tilt).

A remarkable contribution to the application of machine learning algorithms to the automatic discrimination of PD was provided by Drotár et al. All their studies were carried out on a same dataset, i.e., PaHaW, which the authors made freely available. In Reference [85], by comparing the predictive potential of models built on every task individually and models trained merging all tasks, the authors found that the best classification performance was reached by the combination of all tasks. In Reference [86], the authors investigated the extent to which classification performance can be improved considering not only on-surface but also in-air movement, since the two modalities appear to carry on nonredundant information. They found that in-air features outperform on-surface features. These findings were further improved in Reference [87], where different feature selection strategies were employed. In addition to conventional kinematic handwriting measures, Drotár et al. [37] also computed novel measures based on entropy, signal energy, and empirical mode decomposition of the handwriting signals. These features provided more insight and better understanding of the data. It is worth noting that, in this study, only on-surface movement was considered. In Reference [88], instead, the authors employed these novel features also considering in-air movement. In Reference [38], the authors introduced additional features based on the pressure exerted over the writing surface. The fundamental pressure features were the value of pressure as captured by the tablet during the particular task and the rate at which pressure changes with respect to time. Then, they introduced correlation coefficients to capture the relationship between pressure and kinematic features.

It is worth remarking that, in all of the studies by Drotár et al., the spiral task was undertaken with no significant impact on classification. This may have been due to the use of measures only tailored to handwriting; instead, visual features, for example, those provided by deep learning algorithms [25,89], seem to overcome this issue.

Rosenblum et al. [21] assessed whether simple characteristics of handwriting can provide quantitative measures to accurately differentiate between PD patients and controls. Study participants were requested to write their name and to copy an address. Significant group effects were observed: compared to controls, patients wrote smaller letters, applying less pressure and requiring more performance time. A discriminant function was found for the effective group classification of all participants. Furthermore, the authors highlighted the importance to analyze handwriting not only on-paper but also in-air, as significant differences were observed between these two writing conditions. In fact, as the authors wrote, in-air time is a manifestation of “planning the next movement”, which can reflect cognitive ability and supply information about the writer.

In Reference [23], Pereira et al. proposed NewHandPD, a dataset of signals extracted from the BiSP smart pen comprising spiral and meander drawings. Each sensor of the device outputs the whole signal acquired during the handwriting tasks; thus, it can be subsequently represented as a time series. The authors used CNNs and meta-heuristic-based optimization techniques to fine-tune the network hyper-parameters due to their ability to learn without human intervention. Hence, the main contribution of the work was the application of a deep learning-oriented approach to aid PD diagnosis as well as the design of a signal-based dataset.

The abovementioned work was extended by the authors in References [52,90]. In Reference [90], CNNs were used to learn features directly from time-series-based images. The main hypothesis was that texture-oriented features are able to encode the tremors during handwriting. In Reference [52], the recurrence plot technique was used to map the pen signals into the image domain; then, these images were used to instruct a CNN on how to learn discriminating features. A recurrence plot enables to visualize repeated events of higher dimensions through projections onto low-dimensional representations.

San Luciano et al. [29] assessed the validity of the digitized Archimedes spiral drawing as a biomarker for the early diagnosis of PD. Spatial and temporal variables of handwriting were, in general, significantly different between PD subjects and controls. A model using all features showed high discriminating validity. Therefore, the authors claimed spiral analysis to be a promising quantitative biomarker for the early disease diagnosis.

Kotsavasiloglou et al. [91] asked patients and healthy subjects to draw a horizontal line on the tablet’s surface, keeping the pen’s velocity as constant as possible. The choice of this simple task was made with the expectation that one should be able to detect differences between the groups even in very simple tasks, as the impairment manifests independently of the complexity of the task. Indeed, good accuracy performance were obtained with a Bayesian classifier. It is worth noting that, as an additional contribution, the authors introduced a new metric, termed normalized velocity variability, which quantifies the variability of the pen’s horizontal speed as the line is drawn.

The majority of the studies focused on the binary discrimination healthy vs unhealthy, independently of the degree of disease severity. In other words, the Parkinsonian group is typically considered as a single cluster in which all subjects share the same degree of disease severity. In Reference [92], Zham et al. addressed this issue by investigating the correlation between the speed and pen pressure while sketching a spiral and the severity of disease symptoms. The strongest correlation was found with a combination of these two parameters, which turned out to be useful for the automatic differentiation between the low and high degree of severity. However, this measure was not able to differentiate between low and middle and between middle and high disease severity. In Reference [93], classification accuracy was refined by focusing on angular features and the count of direction inversion during the sketching of the spiral.

In Reference [24], Impedovo et al. also addressed this problem by performing a classification study on only a subset of the PaHaW dataset, focusing on the earlier and mild degree of disease severity. They found that classification performance significantly drops when considering this subset, instead of taking into account the entire dataset including the more severe cases. In this work, the authors also showed how a multi-expert approach based on ensembling the different tasks at disposal can provide better results than combining the features coming from each task into a unique high dimensional feature vector.

Gallicchio et al. [94] further explored the application of deep learning techniques to aid PD diagnosis through handwriting by exploiting recurrent neural networks. These networks were used to obtain automatically significant features without human intervention from the time series data of the ParkinsonHW dataset [43].

Mucha et al. proposed a new methodology for the kinematic feature analysis of PD handwriting based on fractional derivatives of arbitrary order. Promising results using this techniques have been reported in Reference [95].

In Reference [96], the author improved the results obtained on the PaHaW dataset [37] by combining more classic features to new velocity-based features. The extended set of features include parameters obtained from the application of the sigma-lognormal model, the Maxwell–Boltzmann distribution, and the Discrete Fourier Transform to the velocity profile of handwriting.

Confirming the findings reported in Reference [21,87], Jerkovic et al. [97] found that in-air and on-surface movement on the tablet tend to be statistically independent and to carry on nonredundant information. The highest prediction accuracy in discriminating patients with PD and atypical Parkinsonism from controls, in fact, resulted from the combination of both in-air and on-surface parameters.

In Reference [98], Loconsole et al. were among the first to use features based on the gyroscope signal obtained by the tablet. Unfortunately, their classification study was based on a very small sample of participants.

Rios-Urrego et al. [99], in addition to using kinematic features, proposed to use geometrical and nonlinear dynamic features. The latter, in particular, was meant to capture the distortions and irregularities of handwriting, which are assumed to increase as the disease advances.

Diaz et al. [25] recently proposed a “dynamically enhanced” handwriting representation which consists of synthetically generated images obtained by exploiting simultaneously static and dynamic properties of handwriting. Specifically, they proposed a static representation that embeds dynamic information based on drawing the points of the samples, instead of linking them, so as to retain temporal/velocity information, and that adds pen-ups in the same way. The new handwriting representation was able to outperform the results obtained by using static and dynamic handwriting separately on the PaHaW dataset.

Ribeiro et al. [55] focused on the analysis of tremor, being one of the most distinctive characteristics of PD. In particular, they proposed to learn temporal information from time-dependent signals collected from handwriting exams by exploiting bidirectional gated recurrent units along with an attention mechanism. These units are a gated mechanism in recurrent neural network architectures. In addition, the authors also introduced the concept of “bag of samplings” as a compact representation of the signals. Experimental results on the NewHandPD dataset compared favorably with the previous literature.

In Reference [53], Ammour et al. proposed to use a clustering method to analyze several factors (i.e., age, intellectual level, frequency of writing per week, etc.), which can intervene in the characterization of the groups under study. Then, by using a semi-supervised approach, the authors developed a model for distinguishing the aspects of handwriting pertaining those factors from those related to pathological conditions. A balanced cohort of healthy subjects and PD patients were involved, and they were asked to copy a given Arabic text. Interestingly, among the features used, the authors also considered measures of pen inclination based on the azimuth and altitude information provided by the tablet. During data analysis, three clusters were observed: one where the pathological factor appeared to be the only discriminating element of the corresponding subpopulation; another cluster with mostly healthy people; and one characterized by a mixture of elderly controls (ECs) with medium intellectual level and PD patients with high intellectual level and writing frequency. This finding corroborates the hypothesis that education level may act as a resilience mechanism against the deterioration caused by neurodegeneration [16].

3.2. Alzheimer’s Disease

Similarly to the classification of studies on PD made in the previous subsection, papers focusing on handwriting in AD can be grouped in accordance with two main research questions:

• Disease insight: a group of papers examined changes in handwriting of AD and MCI patients to identify patterns of sensorimotor dysfunction associated with the disease;
• Disease diagnosis: another group of works applied dynamic handwriting analysis for the purpose to develop a computer-aided diagnosis system.

A schematic overview is provided in

Table 4. Summary of studies on AD (EC = elderly controls; YC = young controls; DEP = depressed patients).

Reference	Groups	Tasks	Main Features	Main Findings
Disease insight
Slavin et al., 1999	16 AD, 16 EC	No-sense word	Kinematic and spatiotemporal	AD patients exhibit less consistent movement than controls
Schröter et al., 2003	35 AD, 39 MCI, 39 DEP, 40 EC	Circle drawing	Kinematic	AD and MCI patients differ from healthy subjects in automation parameters and regularity of movement sequence
Yan et al., 2008	9 AD, 9 MCI, 10 EC	Handwriting patterns requiring the coordination of finger and/or wrist	Kinematic and spatiotemporal	Patients show slow and uncoordinated movements
Impedovo et al., 2013	ISUNIBA		Kinematic	Handwriting in impaired subjects presents strongly irregular velocity profiles
Faundez-Zanuy et al., 2014	Not specified	House drawing	Kinematic and pressure	Group differences are reflected by cognitive impairments than motor ones
Yu and Chang, 2016	20 AD, 12 MCI, 16 EC	Line and circle drawing	Kinematic	The degree of motor impairment may help identify those at risk for AD
Disease diagnosis
Werner et al., 2006	22 AD, 31 MCI, 41 EC	Functional tasks	Kinematic and spatiotemporal	Handwriting measures, especially those related to in-air movements, are promising for the automatic discrimination
Pirlo et al., 2015	29 AD, 30 EC	Signature	Sigma-lognormal-based	Signature deterioration can be a disease predictor
Garre-Olmo et al., 2017	23 AD, 12 MCI, 17 EC	Sentence copying and writing, figure copying, and CDT	Kinematic and pressure	Higher specificity in distinguishing between normal and impaired condition and higher sensitivity in distinguishing between AD and MCI
Kawa et al., 2017	37 MCI, 37 EC	Single letter and sentence writing	Spatiotemporal	MCI writing is significantly slower than the normal one
Müller et al., 2017a; 2017b	20 AD, 30 MCI, 20 EC	House drawing and CDT	Spatiotemporal	In-air time is a good predictor for disease diagnosis
El-Yacoubi et al., 2018	Different aging conditions	Word and sentence writing	Kinematic and spatiotemporal	Classification based on temporal representations improves
Ghaderyan et al., 2018	15 AD, 13 MCI, 15 EC	Spiral drawing and word writing	Kinematic and spatiotemporal	Individual variability of handwriting can be mitigated by noise-robust methods such as singular value decomposition
Angelillo et al., 2019	36 MCI, 29 EC	Attentional matrices	Kinematic and entropy	Digitized attentional tasks are promising for discriminating cognitively impaired individuals from controls
Impedovo et al., 2019	71 MCI, 34 EC	Standard and nonstandard writing and drawing tasks	Kinematic	An integrated protocol for disease diagnosis based on handwriting is proposed
Ishikawa et al., 2019	10 AD, 25 MCI, 36 EC	Standard writing and drawing tasks	Kinematic	The usefulness of digitizing neuropsychological tests on the tablet is supported

.

It is worth noting that, in contrast to studies focusing on PD, less research effort has been made towards the investigation of AD; moreover, the literature still lacks studies involving the application of dynamic handwriting analysis to support monitoring of disease progression. This is largely due to the absence of effective cures that slow down disease symptoms. However, as MCI patients are at high risk to develop in AD, handwriting changes found in this condition may be used not only for the early disease diagnosis but also to monitor disease progression.

Disease Insight

Slavin et al. [34] were among the first to assess handwriting dynamics in patients with dementia of Alzheimer’s type by making use of a digitizing tablet. Irrespective of medication or disease severity, patients wrote strokes of significantly less consistent length than controls and were disproportionately impaired by a reduction of visual feedback. Moreover, patients’ strokes had a significantly less consistent duration and a significantly less consistent peak velocity than controls. The authors suggested that the more variable performance of patients indicates a degradation of the base motor program and resembles that of Huntington’s disease rather than PD. It may indeed reflect frontal rather than basal ganglia dysfunction; thus, it seems that relative movement duration may be useful to differentiate between subcortical dementias (like PD) and cortical dementias (like AD).

In Reference [28], Schröter et al. adopted dynamic handwriting analysis to quantify differences in fine hand motor function in patients with probable AD and MCI compared to depressed patients and controls. All participants were instructed to perform two tasks. The first one consisted in drawing concentric superimposed circles as fast and fluently as possible with the dominant hand; the second task was identical to the first one but, in addition, participants were requested to simultaneously perform a distraction task (pressing a counting device as often as possible) with the nondominant hand. The results obtained showed that kinematic handwriting parameters were effectively related to cognitive status in elderly patients. Patients with MCI and probable AD exhibited a loss of fine motor performance: especially when compared to control subjects, movements of AD patients were significantly less automated, accurate, and regular.

Yan et al. [100] investigated whether the decline in fine motor control and coordination characterizes sensorimotor deficiencies of cognitively impaired patients with AD or MCI. Their findings supported this hypothesis. Specifically, when performing handwriting tasks, movement slowing was associated to MCI and AD. When performing fine movements, the AD patients also showed more jerky movement than the other groups.

In Reference [44], Impedovo et al. investigated the relationship between the delta-lognormal and the sigma-lognormal models [49] and the early signs and symptoms of AD. The previously mentioned dataset ISUNIBA was collected and used to perform the analysis. By looking at the speed profile along the writing process, it was observed that the maximum speed value was almost always regular in healthy subjects; instead, this regularity was strongly reduced for the patients at the beginning of the disease and completely lost in the patients at advanced stages of the disease.

Faundez-Zanuy et al. [101] compared dynamic characteristics of drawing tasks performed by patients with probable AD and controls. Although some pathological drawings looked “normal” if only considering on-surface movements, in-air patterns and pressure appeared quite entangled. Interestingly, pressure and in-air information were significantly different between the groups even when controls were requested to perform the tasks with the nondominant hand. This suggested that the differences between the groups may not reflect physical problems but cognitive ones.

Yu and Chang [102] explored the motor impairments of individuals with probable AD and amnestic MCI through handwriting analysis. The results showed that slowness and irregularity of movement of AD and MCI patients were not present in all the proposed tasks. For example, impairments were not found when drawing straight lines and cursive-connected loops. Instead, AD and MCI participants had more difficulty than the control group when drawing circles. The study mainly provided evidence that MCI is characterized also by motor dysfunction.

3.3. Disease Diagnosis

The study by Werner et al. [40] was aimed at examining kinematically the handwriting process of individuals with MCI compared with those with mild Alzheimer’s disease and healthy controls; assessing the importance of the kinematic measures for the differentiation of the groups; and assessing characteristics of the handwriting process across different functional tasks. Participants were requested to perform five functional writing tasks, such as copying a phone number and a grocery list. Two underlying assumptions guided the selection of these tasks: they are functional tasks related to the performance of daily activities; moreover, they reflect an increase in difficulty, as they are long and involve cognitive effort. An ANOVA test was used to test group differences across measures (both on-surface and in-air) for each writing task. Furthermore, a discriminant analysis was carried out to determine which features would be the best predictors for classification. The results of the work showed significant differences between the three groups under study in almost all measures, with the MCI group assuming, as expected by the authors, a position between the other two groups. Temporal measures (especially in-air time) were higher in the more cognitively deteriorated groups, while the mean pressure was lower. The results also showed that kinematic measures of the handwriting process, together with cognitive status measures, provide an efficient way to differentiate between the groups, although the classification of MCI was relatively poor. Finally, the writing characteristics of participants in all groups showed that, although measures of velocity and pressure remained stable across the different tasks, the temporal and spatial measures increased as the difficulty of the task increased. Although this finding might be obvious, it is interesting that the increase was reflected mainly in the in-air movements.

In Reference [39], Pirlo et al. investigated the extent to which the analysis of dynamic features extracted from handwritten signatures can be fruitfully used for the binary classification healthy vs. AD. A signature, in fact, is well-known to convey a huge amount of information related not only to the representation of the name and surname of the signer but also to the writing system (hand, arm, etc.) as well as the psychophysical state. The feature extraction phase was accomplished in accordance with the sigma-lognormal model of the Kinematic Theory of Rapid Human Movements. The best classification performance was obtained by using a Bagging CART (classification and regression tree) classifier. It is worth noting that some contrasting result was obtained by Renier et al. [103], who found no significant correlation between signature deterioration and level of cognitive decline.

Garre-Olmo et al. [31] compared the dynamic characteristics of handwriting and drawing between patients with probable AD and MCI and healthy controls. Participants were asked to copy one sentence, to write a dictated sentence and an own sentence, to copy two and-three dimensions drawings, and to execute the Clock Drawing Test. By means of discriminant analyses, the authors explored the value of several kinematic features in order to classify participants depending on their degree of cognitive functioning. The degree of correct classification was dependent on the nature of the groups to be classified and the specific task. Classification performance showed higher specificity values when distinguishing between normal and impaired cognition (MCI and AD) and higher sensitivity when distinguishing between impaired cognition levels (MCI and AD). Interestingly, the results obtained showed that, for the same task, the discriminant parameters differed depending on the type of group to be discriminated, suggesting that they are not the dimensional features of the parameters but rather the qualitative combination of these parameters that are relevant for group discrimination.

Kawa et al. [104] evaluated the usefulness of handwriting features obtained with an electronic pen to distinguish MCI patients from controls. Subjects with confirmed MCI needed more time to complete two out of three writing tasks, as their writing was significantly slower. These results were associated with a longer time to complete a single stroke of written text. The written text was also noticeably larger in the MCI group in all three tasks.

Müller et al. [30] investigated movement kinematics between patients with early dementia due to probable AD, patients with amnestic MCI, and cognitively healthy control individuals while copying a three-dimensional house using a digitizing tablet. Receiver operating characteristic (ROC) curves and logistic regression analyses were conducted to explore whether alterations in movement kinematics could be used to discriminate patients with MCI and AD from controls. In-air time differed significantly between the three groups, showing an excellent sensitivity and a moderate specificity to discriminate MCI subjects from normal elderly and an excellent sensitivity and specificity to discriminate patients affected by mild AD from healthy individuals. On-surface time differed only between controls and patients with AD but not between controls and patients with MCI. Furthermore, the total time (i.e., in-air plus on-surface time) did not differ between patients with MCI and early dementia due to AD.

In Reference [105], Müller et al. reported the results of an experiment analogous to the previous one, employing the same apparatus and the same participants. What differed from the previous study was the task the participants were requested to perform, which consisted in a digitized version of the classic Clock Drawing Test. While the traditional CDT revealed only poor sensitivity but excellent specificity in discriminating MCI patients from healthy individuals, excellent sensitivity and a good specificity were obtained in discriminating these groups when considering the digital version of the test. In Reference [106], this research was extended by comparing the digital Clock Drawing Test with the traditional Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) neuropsychological test score. Digital Clock Drawing Test (dCDT) provided a slightly better diagnostic accuracy for the discrimination of amnestic MCI from controls than using the CERAD score. Instead, in differentiating patients with mild AD from controls, both tests provided excellent accuracy.

El-Yacoubi et al. [107] proposed a novel paradigm for studying handwriting changes due to cognitive decline (or aging) by addressing the major limitations of the state-of-the-art solutions. The first one is the assumption of a unique behavioural trend for each cognitive profile. This restriction was relaxed by allowing, for each profile, the emergence of a multi-modal behavioural pattern reflecting the diversity of behaviours within a given healthy condition. The authors achieved this by using unsupervised or semi-supervised learning algorithms to uncover homogeneous groups of subjects and byanalyzing how much information these clusters carry on about the cognitive profiles. The second main limitation is the encoding of handwriting spatiotemporal dynamics only by using global or semi-global parameters, assumed implicitly to be discriminant. The proposed method is based on a representation learning approach which is suitable for treating sequential data from which temporal feature representations can be uncovered. A key advantage of this temporal representation learning is that it is fully explainable, as it can be visualized and easily understood. The main finding of the work, from a diagnostic perspective, is that MCI individuals tend to form clusters either with controls or AD patients, revealing that MCI individuals have fine motor skills with characteristics from both the two extreme groups.

An approach similar to the previous one was followed by the authors in Reference [108]. In particular, they modeled the velocity trajectory of loop-like movements through a temporal clustering based on dynamic time warping as a dissimilarity measure. For classification, the authors used a Bayesian framework, which aggregates the contributions of the clusters by combining the discriminating power of each oh them probabilistically.

In Reference [109], Ghaderyan et al. pointed out how shape and writing style differences among individuals are sources of undesired variability in the handwriting signals which may affect the recognition performance. In order to mitigate this effect, the authors proposed a noise-robust method based on the singular value decomposition and a sparse nonnegative least-square classifier to make the handwritten patterns less dependent on small individual variations.

In Reference [42], Angelillo et al. proposed a digitized version of the Attentional Matrices Test for selective attention assessment: it is based on three matrices of increasing difficulty, and the subject is asked to the mark target digits assigned. The authors observed how, although a pathological matrix may look “normal” if considering only the on-surface pattern, the information provided by in-air movements reveal a completely deteriorated search strategy of the targets to be marked among the distractors. An ensemble built over three different classifiers trained on the matrices separately provided the best classification performance.

In Reference [32], Impedovo et al. proposed a handwriting-based protocol for screening and follow-up of dementia based on a digitized version of standardized cognitive and functional tests (such as Mini-Cog and MMSE), together with handwriting and drawing tasks currently under investigation by researchers. The proposed protocol achieved good specificity in distinguishing MCI patients and controls. A similar work has been recently presented in Reference [110], where Ishikawa et al. proposed to use a digitized version of neuropsychological tests developed on a digitizing tablet to learn to distinguish between AD, MCI, and EC subjects.

4. Conclusions and Future Directions

The body of evidence on computerized handwriting analysis supports the hypothesis that physical, cognitive, and psychological characteristics of individuals can be captured by handwriting measures. In particular, changes in handwriting seem to be a prominent biomarker for the evaluation of neurodegenerative diseases. Several works, in fact, provided evidence that the automatic discrimination between unhealthy and healthy people can be accomplished on the basis of features obtained through simple and easy-to-perform handwriting tasks. In this view, as the number of devices for data capturing and processing is increasing all over the world, the use of handwriting to detect and monitor health conditions is becoming more and more attractive. In particular, the advent of digitizing tablets and electronic pens allow researchers to investigate not only the static characteristics of handwriting available only after the writing process has occurred but also dynamic characteristics collected when the handwriting task is still in progress. Based on dynamic analyses, several works provided evidence that certain aspects of the handwriting process are more vulnerable than others and may therefore present diagnostic signs.

A handwriting-based decision support system has the potential to assist clinicians at the point of care, providing a novel diagnostic tool while reducing the expenditure of public health care. Moreover, it can be used to quantify aspects of the motor system and its disorders in order to better understand the involved underlying mechanisms, e.g., the difficulties in coordinating the components of a motor sequence movement. Finally, it can help study the effects of medication on handwriting with the aim to monitor the responsiveness of the patient to therapy. More in general, handwriting can provide a simple, user-friendly, and easy-to-use instrument to support the daily clinical trials. Artificial intelligence and machine learning, in fact, are changing the way we think of health care from many perspectives, and the use of computer aided tools within medical practice is continuously increasing. The best perspective of this line of research is the integration of new medical tools which can increase the level of diagnostic accuracy. Doctors can be provided with user-friendly tools in their daily practice, even though they are not necessarily skilled in high-level algorithms. In this sense, a handwriting-based tool is attractive since it not only provides the user with an automatic response within few seconds but also allows the doctor to store useful metadata, e.g., the patient’s information and diagnosis, for later use. Of course, handwriting-based intelligent systems are not expected to replace standard techniques or even doctors but rather to provide additional evidence to further support the clinical assessment. Intelligent systems technology is proving beneficial in a number of health domains, also including neuroimaging [111] and cardiovascular risk assessment [112].

The present paper has been devoted to provide a comprehensive overview of the literature dealing with the application of dynamic handwriting analysis to the assessment of Parkinson’s disease and Alzheimer’s disease. Three well-defined research trends have been identified, ranging from studies aimed at understanding the facets of fine motor control related to the disease to works investigating the application of dynamic handwriting features to disease monitoring and diagnosis. Handwriting features in PD and AD patients can overlap; in particular, handwriting is typically slower, less regular, and less consistent in patients if compared to the healthy counterpart. However, the two diseases also show distinctive characteristics. PD patients tend to exhibit rigidity of movements and unwanted muscle contractions, while preserving, in most cases, their cognitive faculties. AD and MCI patients, on the other hand, tend to exhibit a more pronounced alteration of their visuospatial abilities and executive functions, while preserving their fine motor control. These features can be reflected on the production of handwritten patterns, especially in more complex handwriting tasks.

Encouraging results have been so far obtained; however, there still remain open issues demanding further research. First, interoperability is still a problem, since data are typically obtained from different devices and different handwriting tasks. Some works, e.g., References [29,32,87], provided the community with tentative hand-drawing/handwriting protocols for the assessment of PD and AD. An integrated protocol, as the one proposed in Reference [32], may be useful to the research community to collect different handwritten traits; at the same time, it may be of real use for doctors to support their daily activities.

Another important problem is related to the collection of a statistically significant large amount of samples. The research community still lacks a large benchmark dataset so that different tools, algorithms, and techniques can be effectively evaluated and compared. The few datasets currently available are composed by very few subjects and do not always consider other factors such as the stage of disease, the medical treatment, and so on. Unfortunately, collecting a large benchmark database is a time-consuming and expensive process. Furthermore, as neurodegeneration evolves during time, such a dataset should longitudinally follow a same group over several years.

Senatore et al., in Reference [113], recently raised another important issue: the acceptance of artificial intelligence-based diagnosis systems by physicians may be hampered by the black-box approach implemented by most state-of-art systems. To address this problem, the authors proposed an evolutionary approach based on Cartesian Genetic Programming which allows for the automatic detection of the presence of disease and, simultaneously, provides the explicit classification rules used by the system. This approach can allow physicians to derive guidelines that may be used to define novel testing protocols and intervention strategies. Also classic decision trees, which are capable of providing the decision criteria in terms of both the most relevant features and how their values are used to reach the final decision, can be used for the purpose [114]. Explainable AI can be therefore a prominent research direction within this context.

The lessons learned from the mentioned body of evidence can be profitably used for other health and psychological domains. For example, in Reference [115], handwriting was employed with successful results to the problem of recognizing malingering in health care, i.e., the false information given by patients about their health. Preliminary results suggested that a computerized tool based on handwriting can help detect deception. A similar tool was used in Reference [116] for capturing cognitive load implications during complex figure drawing. The work was then extended in Reference [117], where the contribution of handwriting to classifying cognitive mental workload was assessed. Also, handwriting-based measures recently showed promising results in investigating emotional states, such as stress, depression, and anxiety [118].

It is worth noting that tablet technology enables the implementation of a multi-modal interaction system in which not only the input provided by the electronic pen but also tactile, speech, or visual input can be acquired. Promising results on noninvasive methods based on speech and handwriting analysis for neurological disorder assessment have been obtained, e.g., Reference [119]. Moreover, thanks to such a multimodal interface, the development of a mobile conversational agent appears to be feasible. An example of a mobile conversational agent successfully used in the context of AD has been recently reported in Reference [120]. A purely automatic diagnostic tool paws the way of a quick instrument which enables mass screening of the population or even home training for improving cognitive abilities.

Finally, another open issue is a technological one and is related to the realization of an all-in-one solution specifically devised for the automatic differential diagnosis.

Some other aspects are related specifically to AD assessment. The most obvious observation that can be drawn from the present survey is that less research effort has been done for AD; thus, additional effort is necessary for advancing this line of research. Concerning disease insight, investigating the neural process that underlies handwriting may provide further criteria for selecting the most representative features associated to a writer, i.e., those containing more information about the message the handwriting measures represent and the way to examine them profitably. This goal may be achieved through a multidisciplinary approach, which involves both the analysis/comparison of handwritten data provided by healthy subjects and patients affected by AD, and through the analysis of the behavior of a neural network model that emulates the neural mechanisms occurring in the brain areas involved in handwriting generation and learning. Understanding the neural process involved in learning complex motor behaviors could also provide a meaningful help to the development of devices and important insights in developing more effective treatments for the motor deficits affecting AD patients.

In the context of AD diagnosis, the challenge appears to be to separate from healthy people the mild cognitive impairment subjects, which are likely to evolve in AD. MCI is characterized by slight problems with memory loss, language, or other mental functions; thus, finding deterministic patterns useful to discriminate the impairment from a non-pathological condition is very difficult: this is well-known to the research community working on neuroimaging. Some works, for example, References [40,104], clearly indicated that such difficulty is also reflected on dynamic handwriting analysis.