Next Article in Journal
Introduction to Special Thematic Issue, Part 2 “Microsaccades: Empirical Research and Methodological Advances”
Previous Article in Journal
Two Hours in Hollywood: A Manually Annotated Ground Truth Data Set of Eye Movements During Movie Clip Watching
 
 
Journal of Eye Movement Research is published by MDPI from Volume 18 Issue 1 (2025). Previous articles were published by another publisher in Open Access under a CC-BY (or CC-BY-NC-ND) licence, and they are hosted by MDPI on mdpi.com as a courtesy and upon agreement with Bern Open Publishing (BOP).
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Oculomotor Fatigue and Neuropsychological Assessments Mirror Multiple Sclerosis Fatigue

by
Wolfgang H. Zangemeister
1,2,
Christof Heesen
1,
Dorit Röhr
1 and
Stefan M. Gold
1
1
University Medical Center Hamburg-Eppendorf, Hamburg, Germany
2
MPCH—Medical Prevention Center Hamburg, Hamburg, Germany
J. Eye Mov. Res. 2020, 13(4), 1-19; https://doi.org/10.16910/jemr.13.4.6
Submission received: 30 January 2020 / Published: 14 September 2020

Abstract

:
Fatigue is a major complaint in MS. Up to now no objective assessment tools have been established which hampers any treatment approach. Previous work has indicated an association of fatigue with cognitive measures of attention. Oculomotor tests have been established in healthy individuals as a read-out of fatigue, and to some extent in MS patients. Based on these observations we compared two groups of MS patients, one with fatigue (n=28) and one without fatigue (n=21) and a group of healthy subjects (n=15) with a standardised computerised measure of alertness and an oculomotor stress test. Patients with fatigue showed highly significant changes of their saccade dynamics as defined by the Main Sequence and Phase Plane plots: They showed slowing of saccades, the characteristical fatigue double peak, and an asymmetrical phase plane. Oculomotor tests differentiated significantly between fatigue and fatigabiliy in our MS patients. They also showed significantly worse performance in the alertness test as well as in the oculomotor task. Significantly slower reaction times were observed for tonic alertness in 2 series without a cue (p=.025 and p=.037) but not in phasic alertness with a cue (p=.24 and p=.34). Performance was influenced by disability as well as by affective state. We conclude, when controlling for disability and depression, saccadic stress tests and alertness tests could be used as an objective read-out for fatigability and fatigue in MS patients.

Introduction

Alertness is the state of active attention by high sensory awareness, such as being watchful and prompt to meet danger or emergency, or being quick to perceive and act.
Many studies of vigilance research during the past few decades have shown that, for most or all operators engaged in sustained, attention-intensive and monotonous tasks, retaining a constant level of alertness is rare if not impossible --- leading to alertness deficits and fatigue (Thorndike, 1900). Posner and Petersen (1990) reviewed the breadth of the knowledge about attention in 1990 in his comprehensive examination of the nature and functions of attention and its relationship to broader cognitive processes. An abnormal perception of effort is a salient feature of chronic fatigue. Chronic fatigue is prevalent in a large number of pathological conditions: neurological, autonomic, immunological, hormonal and cardiovascular diseases, and also as a significant side effect of many pharmacological interventions. Although the implications of mental fatigue are well known, measuring mental fatigue—especially in complex, dynamic tasks—is still challenging (Shen et al., 2008; Barnett, 2005). Recent studies investigated mental fatigue using behavioural measures, such as eye movements (Stone et al., 2003; Schleicher et al., 2008), psychophysics, fMRI and EEG.
Fatigue is a major complaint of MS patients affecting activity and participation substantially. Up to 80% of patients report fatigue and half of those describe fatigue as the worst symptom (Fatigue MS Council 1998). Data indicate that fatigue leads to more unemployment than mobility restrictions (Julian et al., 2008). During the disease course fatigue persists and may deteriorate (Johansson et al., 2008). Fatigue may be classified into primary and secondary fatigue with the latter being a consequence of disability and reduced activity, sleep disorders, psychological factors, drugs and other medical diseases (Kos et al., 2008; Bol et al., 2009). Factors discussed in the development of primary fatigue are functional changes in the brain such as abnormal recruitment of cortical and subcortical networks in MS (Filippi & Rocca, 2007), fatigue associated with the disruption of frontal and parietal pathways (Sepulcre et al., 2009), nerve conduction alterations (Liepert et al., 2005), immune and neuroendocrine factors (Krupp et al., 2010), and alterations of muscle metabolism (review in: Kos, 2006). While there seems to be an association of overall lesion load on MRI and fatigue (Cook et al., 2007), studies searching for a pattern of lesion distribution with conventional and nonconventional MRI methods did not lead to conclusive results (Bakshi, 2000, 2003). Worsening of fatigue within the first 2 years of MS (Calabrese et al., 2010; Tedeschi et al., 2007) was a predictor of brain atrophy 6 years later: Yaldizli et al. (2011) have shown correlations of fatigue scores with corpus callosum atrophy in 70 relapsing-remitting MS patients.
Any study on MS fatigue is hampered by the fact that up to now no objective assessment methods have been established. Monitoring of its natural course as well as under experimental treatments necessarily relies on self-report questionnaires. A study in unselected RRMS patients indicated that longer reaction times in a computerized test of alertness, i.e. sustained attention may be associated with MS fatigue (Weinges-Evers et al., 2010). Maclachlan et al. (2017) examined whether current chronic fatigue syndrome (CSF) diagnostic criteria are identifying different disease phenotypes using the DSQ-DePaul Symptom Questionnaire. Self-reported autonomic and cognitive symptoms were significantly greater in CFS subjects compared to controls (Thompson et al., 1997). There were no statistically significant differences in objective autonomic measures between CFS and controls.
Based on the concept that the ascending arousal system and the Locus coeruleus are major determinants of wakefulness and attention (Sturm & Willems, 2001; Heesen et al., 2006) the oculomotor system, located in close proximity to these systems, might be an objective read-out for disturbed brain stem functioning and may therefore serve as a substrate of MS fatigue (Barnes & van Dyne, 2009; Boksem & Tops, 2008; Lorist et al., 2008; Bartlett, 1943). Indeed, reduced velocity of horizontal saccades has been associated with tiredness in sleep deprived healthy individuals (Zils et al., 2005; Schleicher et al., 2008). Recently, Serra et al. (2018) have reviewed the most common eye movement disorders in MS and discussed known pathophysiological correlates for each of them. Study of eye movements in MS is particularly valuable as use of conventional and upcoming non-conventional imaging techniques coupled with eye movement recordings can provide particular insights into the course of the disease.
Fatigue can be subdivided into an effort-independent (trait-fatigue) (Harrison et al., 2017) and an effort-dependent component (fatigability) (Kluger at al., 2013). Spiteri et al. (2019) tried to disentangle activity changes associated with effort-independent “trait-fatigue” from those associated with effort-dependent fatigability in MS patients through behavioral measures and functional magnetic imaging. Their results indicated that effort-independent (fatigue) and effort-dependent fatigue (fatigability) in MS patients have functionally related but fundamentally different neural correlates. Saccadic velocity is not subject to voluntary control (Leigh & Zee, 1999), unlike saccadic amplitude or fixation duration. Thus it represents the underlying neural activity more accurately than other gaze parameters (Bower et al., 2005; Ferreira et al., 2017). Saccadic eye movements and their neurological control signals change significantly as the human subject fatigues. Dodge and Cline (1901) were probably the first to describe the dynamics of saccades and changes of their dynamic due to fatigue with decrease velocities (Dodge, 1917), and later Bahill and Stark (1975b). With increased attention saccade dynamics change to the opposite and very fast saccades can be found indeed (Fischer & Weber, 1993). A different new method for the analysis of eye movements and scanpaths has been proposed by Hein and Zangemeister (2017). They applied the theory of topology for addressing the relationship between spatial features of eye movements and their visual-objects.
Purpose of this study. Based on the above noted earlier observations our study addressed the question whether using an objective, computerized measure of alertness and a horizontal saccadic stress task would allow to differentiate MS patients with and without fatigue and/or fatigability. We compared two groups of MS patients, one with fatigue, one without fatigue and a group of healthy subjects with a standardised computerised measure of alertness and an oculomotor stress test. Our hypothesis was that when controlling for disability and depression, saccadic stress tests and alertness tests could be used as an objective read-out for fatigability and fatigue in MS patients.

Methods

Recruitment and inclusion/exclusion criteria Patients with clinically definite MS according to McDonald criteria (McDonald et al., 2001), age 18-60, Expanded-Disability-Status-Scale Scores (EDSS, Kurtzke, 1983) 0- 6, and the ability to read were allowed to participate. The database of the MS Outpatient clinic in Hamburg was screened for patients with substantial fatigue based on their responses on the fatigue items of the Hamburg Quality of Life Questionnaire for MS (HAQUAMS) (Gold et al., 2001). Patients had to be untreated or on stable disease-modifying therapy for at least 3 months. Furthermore, patients were excluded if they had received steroid treatment within the last 4 weeks. For demographic and clinical characteristics of patient sample see Tab.1 (Röhr, 2015).

Fatigue scoring

Patients were grouped into Fatigue and Non-Fatigue based on the score on the Fatigue Severity Scale (FSS, Krupp et al., 1989, 2010) according to a published cut-off (FSS score ≥4). In addition, patients completed the Modified Fatigue Impact Scale (MFIS, MS Council 1998) and the Beck depression inventory (BDI, Beck et al., 1961).

Clinical measures

Neurological impairment was assessed using the EDSS (Expanded Disability Status Scale). In addition, we obtained clinical disease course and disease duration. Cognitive function was screened with the Symbol-Digit-Modalities-Test (SDMT, Symbol Digit Modalities Test, Smith, 1968), a sensitive screening measure. Hand function was controlled by use of the 9-HPT (Nine-Hole-Peg-Test, Cutter et al., 1999).

Eye movement recordings

All subjects had normal or corrected-to-normal vision. Only non-smokers were recruited and they had to abstain from alcohol 24 h and from caffeine-based drinks 12 h before participating. All participants slept at least 7 h before being evaluated. In order to avoid confounding influence of circadian rhythm (Williamson et al., 2011) or any diurnal variation (Grace et al., 2010), all experimental sessions were conducted between noon and 4p.m. during the day. The study was conducted in conformity with the declaration of Helsinki (WMA, 2008).

Infrared - Oculogram (IR-OG) - Instruments.

We used a 22-inch monitor (74Hz) of 1.4 cd/m2 background luminance. The horizontal visual angle of the stimulus (a white cross of 1 deg diameter and 20.6 cd/m2 luminance) was ± 5, 10, and 15 degree. To prevent head movements, a head fixation device that tightly strapped the head with a circular head holder was attached to the forehead. Participants were seated comfortably with a viewing distance of 57 cm (1° is equivalent to 1cm). Under standardized conditions, the eye movements were recorded with infrared reflection oculography (OBER JazzNovo). We used a high resolution software (Eye Track Project 1.21: Zangemeister et al., 1995); Zangemeister & Stark, 1982; Nagel et al., 2008) to present horizontal stimuli and preanalyze the recordings using Main Sequence characteristics of recorded eye movements (Bahill et al., 1975a; Bahill, et al., 1981)) that were used for later statistical comparisons (duration, peak velocity, peak positive acceleration). Eye blinks, which appeared in the original data were removed by computer editing. Sampling time was at 1000 Hz using the Ober JAZZ NOVO remote eye tracking system with an average spatial accuracy of <0.1°. Saccades and fixations were measured using the saccade detection algorithm supplied by JazzNovoResearch (see Appendix). Saccades were identified by deflections in eye position in excess of 0.1°, with a minimum velocity of 30°/s and a minimum acceleration of 4000°/s2, maintained for at least 4 ms. A nine-point calibration and validation was performed before the start and at the end of each session. Saccades around blinks, as well as fixations shorter than 100ms, and saccades with durations less than 10 ms, were not considered in the analysis.

Experimental protocol, stimuli, stress test

Prior to each session and block, calibration and accuracy were tested. Calibration was repeated up to three times prior to each session. Each subject was measured during a standardized routine that consisted of
(1)
calibration,
(2)
for 10 sec each sinusoidal pursuit eye movement of 0.7 Hz, predictive square wave jerks of ± 10 deg, and random square wave jerks between 2deg and 30deg.
(3)
To induce oculomotor fatigue in our 3 groups, participants had to complete four optokinetic (30°/sec) stress tests of 4 minutes each: alternating between right and left direction.
(4)
for 10 sec each sinusoidal pursuit EM of 0.7 Hz, predictive square wave jerks of ± 10 deg, and random square wave jerks between 2deg and 30deg. Total test duration was about 20 minutes.
The recorded single values were transferred into a table that showed: Latency, Duration, Amplitude, peak velocity, peak acceleration and deceleration, times of these maximum values, and averages of these comparing left- and rightward saccades. These values were displayed by means of the double logarithmic Main Sequence graph (Bahill et al., 1975a) comparing them with normal values of our lab (Zangemeister et al., 1995; Nagel et al., 2008). See appendix for velocity and acceleration analysis formulas.

Phase plane analysis of saccades

When two time trajectories are plotted against each other, phase-plane plots are formed. Phase-plane plots of dependent variables and their derivatives are often used to display behavior without explicit time dependency. With respect to the dynamics of saccades, Cook et al., (1966), and Clarke and Stark (1974) where the first to apply this tool. Later, Zangemeister and Stark (1982), Peng et al. (2004), and Benko et al. (2011) also used it for description of head-eye movement dynamics with respect to the active VOR (vestibular ocular reflex).
These plots provided the clearest and most efficient means for comparing single time trajectories. Large amounts of data from multiple saccades could be presented in a compressed format using the phase-plane plots. The time trajectories of the eye in amplitude and velocity, and the corresponding phase plane plots during multiple saccades before and after the stress test of a MS fatigue patient are presented in Fig. 1. The two uppermost rows, display a response from a MS fatigue patient (during multiple predictive saccades of +-10°). We created a phase plane in which the saccadic velocity trajectories are plotted against the saccadic amplitude trajectories (thick solid lines in the time trajectories shown in the top panels). Fig. 1, lower-most row displays a response from a healthy subject after the stress test, since there was no difference between pre and post stress test; the corresponding phase planes of the time trajectories are shown on right. The log-log Main Sequence graphs are shown in Fig.2. For outcome measures derived from the IR-OG, normality of data distribution was tested using the Kolmogorov-Smirnov Z test. Results indicated no violation of the normal distribution assumption for these parameters. Thus the two tailed students t-test was applied.

TAP – alertness

Alertness refers to the condition of general wakefulness that enables a person to respond quickly and appropriately to any given demand. It is the prerequisite for effective behavior, and is in this respect the basis of every sustained attention performance. This attention test has a low complexity with simple reaction paradigms. In the subtest alertness of the Tests for Attention (TAP) (Zimmermann & Fimm, 1994), reaction time is examined under two conditions. The Test consists of two components, which can be divided into four series with 75-90 sec duration each. A key is pressed when a crosshair (easily discriminated speech-free stimulus) appears in the middle of the screen. The reaction speed is recorded in "ms". The crosshairs disappear when a key on the PC keyboard is pressed. After an unpredictable time period a cross appears again. Each series contains 20 individual evaluations (Kley, 2010).
The testing was carried out according to the ABBA scheme. In the series 1 and 4 (A) the determination of the general reactivity (Tonic Alertness) is done by the appearance of the crosshair on the image surface. In series 2 and 3 (B) the phasic alertness is tested. Here the subject is required to perform a more complex, dual-task. At the same time, two stimulus presentations must be observed, which include the visual task and an acoustic task. In the visual task, the respondent must register a cross appearing on the monitor and in the acoustic task she must practice the perception of a high-pitched tone followed by the appearance of the crosshairs in temporal independence of the signal tone.

Statistical analysis

Clinical and demographic variables between the groups were compared using independent samples t-tests or chi-square tests as appropriate. For outcome measures derived from the TAP, normality of data distribution was tested using the Kolmogorov-Smirnov Z test. Results indicated violation of the normal distribution assumption for most TAP coefficients. Thus, these group comparisons were tested using non-parametric Mann-Whitney U tests for independent samples. Statistical analyses were performed using IBM/ PASW software.

Test-abbriviation

TAP Test battery for attention
MFIS Modified Fatigue Impact Scale
MFIS Cog mfis cognition
MFIS Physe mfis physical
MFIS PsySoz mfis psychsocial
SDMT Symbol Digit Modalities Test
EDSS expanded disability status score
9-HPT Nine-Hole-Peg-Test
PASAT paced-serial addition-task
MACFIMS validity of minimal assessment of cognitive function in MS
BDI Beck-Depression-Inventory
FSS Kurtzke Functional Systems Scores
RRMS Relapsing Remitting Multiple Sclerosis
HAQUAMS Hamburg Quality of Life Questionnaire for MS

Results

Demographic and clinical characteristics

A total of 49 MS patients were included in the study. According to the predefined cut-off (FSS≥4), 28 patients were classified as fatigue, and 21 were classified as non fatigue patients. There were no significant differences in age, disease duration, or disease course between the fatigue and the non-fatigue group (Tab.1). Fatigue patients had marginally higher EDSS scores (p=.06); the fatigue group included a slightly higher percentage of female patients.
Using the ratio of saccadic peakVelocity/ Amplitude, which is the important indicator of the Main Sequence as described by Bahill et al. (1975a) and its variance by Bahill et al. (1981), we used the single straight forward parameter to define dynamics of fatigued saccades.
Table 1. Demographic and clinical characteristics of patient sample.
Table 1. Demographic and clinical characteristics of patient sample.
MS FatigueMS non-FatigueP*
(n=28)(n=21)
Age41.7 +-1.440.05+-2.16n.s.
Gender (F/M)26 / 215 / 60.06
Disease duration6.00+-1.05.62+-1.1n.s.
EDSS3.0+-0.12.3+-0.30.06
SDMT56.4+-1.660.1+-3.1n.s.
9-HPT18.2+-0.518.2+-0.7n.s.
Note. *according to independent samples t-tests.
As expected, significant group differences were observed in fatigue scores for cognitive, physical and psychological fatigue symptoms (Table 2). In addition, fatigue patients showed higher levels of depression as measured by the BDI.
Using the ratio of peakVelocity/ Amplitude, both of which are the main indicators of the Main Sequence as described by Bahill et al. (1975a), we found highly significant differences between the groups: Before and - more enhanced -, after the stress test. It is crucial to reduce the variability of the quantification of saccadic dynamics by i. defining accurately the saccadic amplitude and ii. the peak velocity that belongs to the found amplitude – falling within or outside the normal Main Sequence. In case of non-fatigue compared to fatigue MS patients a greater difference after the stress test would be expected.
The normal healthy group for comparison showed normal values of all parameters pre and post stress as well, (student´s t-test in confirmed Gaussian distribution of IR-OG values). The stress test had no effect in the non-fatigue MS patients, but a significant effect in the fatigue group. The initial -pre stress test- comparison of the non-fatigue with fatigue group did not show a difference; but after the test we found a significant difference within the fatigue group. The initial pre- stress test values of the fatigue group showed a highly significant difference to those after the test. This showed that the fatigue patients´ initial condition was significantly worse than in the non-fatigue group. Using in addition the stress test reveals an even more pronounced difference. Using both discriminatory values gave us a clear and reliable objective measure in our cohort of fatigue MS patients.
Table 3. Statistical results (student´s t-test in confirmed Gaussian distribution of IR-OG values) of infrared oculography in MS patients with and without fatigue before (pre-) and after (post-) the oculomotor stress test.
Table 3. Statistical results (student´s t-test in confirmed Gaussian distribution of IR-OG values) of infrared oculography in MS patients with and without fatigue before (pre-) and after (post-) the oculomotor stress test.
pV/AmplNPre stress testPost stress test
P*P*
NOR15 0.42
NOR/ MS-Non-Fatigue 0.290.05
MS Non-Fatigue21 0.1
MS Non-Fatigue/ MS Fatigue 0.830.01
MS Fatigue28 0.0005
NOR/ MS Fatigue 0.240.0001
Shown in Fig.1 are median curves of 30 samples before [pre] and after [post] stress test (upper two rows). They demonstrate the saccadic amplitude reduction (arrows 1a&1b), and also the significant reduction of peak velocity after the stress test (arrow 2b), compared to before the test (arrow 2a). Saccades with double velocity peaks (arrow 2b) demonstrate the dynamical fatigue effect on saccade dynamics. These are double saccades as seen also in asymmetrical (3a) and double (3b) phase-plane graphs. They lie below the two sigma main sequence limits of healthy subjects aged 20 to 60 as shown in figure 2.
The left vertical line in Fig.1 depicts the time when final saccadic amplitude has been reached: normally this is after 35ms (lowermost), but only after 60ms and 50ms respectively in case of stress-induced additional fatigue in the patient (middle and uppermost).
In comparison, post stress saccade dynamics of the heatlthy subject show correct amplitude (1c), same velocity as pre test (2c), and symmetrica , non-double peaked phase planes (3c).
Figure 1. Characteristics of amplitude, velocity, velocity/amplitude phase-plane of MS fatigue patient SC. Before [pre] and after [post] stress test (upper two rows). Median curves of 30 samples. For comparison, lowermost row shows corresponding graphs of a healthy subject (KF) after stress test, with normal dynamic and phase-plane graphs, even after the stress test. Note the saccadic amplitude reduction (arrow 1b) and significant reduction of peak velocity (2b) after the stress test, compared to before stress test (1a and 2a); falling below the two sigma main sequence limits of healthy subjects aged 20 to 60 in our lab. Note also the typical dynamical fatigue effects of the patient´s saccades with double velocity peaks (2b) [already slightly indicated before stress test (2a)]. Double saccades (3b) and asymmetrical (3a) phase-plane graphs demonstrate this result as well.
Axes. Ordinates: Left: position, deg. Middle and right: velocity, deg/s. Abszissae: Time, 0 to 100 ms. Left vertical line depicts time when final saccadic amplitude has been reached: after 35ms (lowermost: normal), uppermost and middle (fatigue) after 50ms and 60ms respectively.
Figure 1. Characteristics of amplitude, velocity, velocity/amplitude phase-plane of MS fatigue patient SC. Before [pre] and after [post] stress test (upper two rows). Median curves of 30 samples. For comparison, lowermost row shows corresponding graphs of a healthy subject (KF) after stress test, with normal dynamic and phase-plane graphs, even after the stress test. Note the saccadic amplitude reduction (arrow 1b) and significant reduction of peak velocity (2b) after the stress test, compared to before stress test (1a and 2a); falling below the two sigma main sequence limits of healthy subjects aged 20 to 60 in our lab. Note also the typical dynamical fatigue effects of the patient´s saccades with double velocity peaks (2b) [already slightly indicated before stress test (2a)]. Double saccades (3b) and asymmetrical (3a) phase-plane graphs demonstrate this result as well.
Axes. Ordinates: Left: position, deg. Middle and right: velocity, deg/s. Abszissae: Time, 0 to 100 ms. Left vertical line depicts time when final saccadic amplitude has been reached: after 35ms (lowermost: normal), uppermost and middle (fatigue) after 50ms and 60ms respectively.
Jemr 13 00033 g001
Main Sequence (log-log) graphs of MS fatigue patient SC (Fig.2). The results depicted in Fig.1 show up also in the double log Main Sequence graphs of Fig.2. Before stress test (uppermost) already slow peak velocity values are seen in the Main Sequence of a patient. A significantly increased effect shows up after the stress test (middle) with lowered peak velocity values.
Similarly, normal duration values are seen in the fatigue patient before (uppermost) the stress test that increase significantly (middle) after the stress test.
In the healthy subject (lowermost) normal saccade dynamics are seen, even after the stress test for the peak velocity values. The same is true for the duration values.
Figure 2. Characteristics of Main Sequence (log-log) graphs of MS fatigue patient SC before [pre] and after [post] stress test (upper two graphs), and a normal healthy subject for comparison (lowest graph) [post] stress test. Each panel shows peak velocity in deg/s upper, and duration (sec) lower as functions of amplitude (deg). Note the saccadic amplitude reduction and significant reduction of peak velocity after the stress test; falling below the two sigma main sequence limits of healthy subjects aged 20 to 60 in our lab.
Figure 2. Characteristics of Main Sequence (log-log) graphs of MS fatigue patient SC before [pre] and after [post] stress test (upper two graphs), and a normal healthy subject for comparison (lowest graph) [post] stress test. Each panel shows peak velocity in deg/s upper, and duration (sec) lower as functions of amplitude (deg). Note the saccadic amplitude reduction and significant reduction of peak velocity after the stress test; falling below the two sigma main sequence limits of healthy subjects aged 20 to 60 in our lab.
Jemr 13 00033 g002

Influence of disability and affective state

During the TAP alertness paradigm we observed significantly longer reaction times in patients with fatigue (Table 4). Significantly slower reaction times were observed for tonic alertness (series 1 and 4), but not for phasic alertness (series 2 and 3).
Since there were slight differences in EDSS and gender and significant differences in depression scores (BDI) between the fatigued and the non-fatigued patients, we next explored the associations between these variables and the TAP results in order to rule out potentially confounding effects. No significant differences were found in the TAP performance between female and male patients (all values p >.20), indicating that differences in gender distribution are unlikely to confound the reported results in TAP scores. But there were significant correlations between EDSS and TAP reaction times (Spearman Rank coefficients Tab.5, second row). After statistically controlling for EDSS, group differences between fatigued and non-fatigued MS patients remained significant for TAP series 1 (p=.032) but not TAP series 4 (p=.113).
BDI total score significantly correlated with some of the TAP reaction times (Spearm.Rho: series1 r=.41,p=.004; series4 r=.34, p=.02). Since the BDI contains numerous items covering vegetative symptoms of depression, which overlap with symptoms of fatigue, we next examined the impact of depression after removing those items. The affective subscore of BDI adressing six purely depressive symptoms (items A, B, E, F, I , J) showed no significant associations with TAP reaction times (Tab.5 third row). This indicates that pure affective symptomatology is unlikely to confound group differences in TAP scores between fatigued and non-fatigued MS patients.
Table 5. Correlations ( Spearman Rho) TAP and SDMT, EDSS, BDIaff, 9-HPT.
Table 5. Correlations ( Spearman Rho) TAP and SDMT, EDSS, BDIaff, 9-HPT.
Jemr 13 00033 i002
Both FSS and nine-HPT showed high correlations with the TAP test.
Table 6. Correlations (Spearman Rho) between IR-OG results after stress test with MFIS Cog, 9-HPT.
Table 6. Correlations (Spearman Rho) between IR-OG results after stress test with MFIS Cog, 9-HPT.
Jemr 13 00033 i003
Table 6 depicts the correlations between MFIS Cog and Nine-HPT with the oculomotor parameters after the stress test. The MFIS Cog (10 items) is a modified subtest version of the Fatigue Impact Scale, designed as a self-report measure to rate fatigue in MS. As expected correlation with saccade dynamics is very high, particularly for peak positive acceleration. This is also true for the Nine HPT as a motor performance test. The table demonstrates that correlation of the dynamic parameters i.e. peak positive acceleration and peak positive velocity with both neuropsychological tests is highly significant, compared to amplitude and duration that are less valuable in this respect.

Discussion

This study assessed two functional brainstem tests, saccade dynamics and a measure of alertness as possible correlates of MS fatigue. Based on earlier observations we compared two groups of MS patients, one with fatigue and one without fatigue with a group of healthy subjects using standardised computerised measure of alertness and an oculomotor stress test. Patients with fatigue showed highly significant changes of their saccade dynamics as defined by the Main Sequence and Phase Plane plots. They showed slowing of saccades, the characteristical fatigue double peak, and an asymmetrical phase plane. The oculomotor tests differentiated significantly between fatigue and fatigabiliy in our MS patients. They also showed a high correlation between the alertness tests and the oculomotor task. Significantly slower reaction times were observed for tonic alertness in two series without a cue, but not in phasic alertness with a cue. Performance was influenced by disability as well as by affective state. In the second part of the discussion section you should compare your results to past studies, particularly studies discussed in the introduction. If the results are not the same, discuss possible reasons for the difference.
The oculomotor system has been a focus of fatigue research. Unlike saccadic amplitude or fixation duration, saccadic velocity is not subject to voluntary control (Leigh & Zee, 1999), and therefore represents thus it may represent the underlying neural activity more accurately than other gaze parameters (Bower et al., 2005). Saccadic eye movements and their neurological control signals change significantly as the human fatigues: Dodge (1901, 1917). High resolution recordings of anomalous looking saccadic eye movements that occurred as the subject's physiological state changed to fatigue were analyzed by Bahill et al. (1975a,b). They described the neuromotor control signals for differentiating saccades during fatigue: 1. Overlapping saccades in which the high-frequency saccadic motoneural bursts showed large pauses; 2. glissades in which the high-frequency motoneural bursts were much shorter than appropriate for the size of the intended saccades; 3. and low-velocity, long-duration, non-Main Sequence saccades in which the motoneuronal bursts were of lower frequency and longer duration than normal. As few as 30 saccades of 50 deg magnitude or a longer sequence of small saccades of 10 deg could produce these aberrant eye movements. These effects of fatigue explain most of the variations between published data of velocity to amplitude rate of human saccadic eye movements – the Main Sequence (Bahill et al., 1975a). The fatigue noted in these saccades was probably manifested at several levels. In the typical fatigue sequence glissades -slow corrective movements of the eye- were the first signs of fatigue to appear. In these eye movements the control signal pulse duration did not carry the eye as far as normal. This could have been due to fatigue of the extraocular muscles or their motoneurons. Later, in order to compensate, the CNS might have changed its stratagem to produce double saccades or low-velocity, long-duration, non-Main Sequence saccades (Bahill & Stark, 1975b). Dieter Schmidt et al. (1979) described these effects in mental and muscular fatigue upon saccade velocity.

Decrease of peak saccadic velocity

Injections of diazepam, which increases presynaptic inhibition in the spinal cord, and affects the limbic system and cerebellar structures, produced double saccades and low velocity saccades (Aschoff, 1968). Also, alcohol produced similar decrements in the speed of saccades (Dodge, 1917). Drugs that affect the GABA/benzodiazepine receptor influence also attention and cognitive activity as well as saccade velocity (Barker & Nussbaum, 2011). Clonidine led to peak velocity decreases. Grace et al. (2010) reported that morphine-induced sedation and sleep deprivation lowered peak velocity. Reduced arousal due to task features (Di Stasi et al., 2010, 2012) and sleepiness (Hirvonen et al., 2010) lead to decreased saccadic velocity. In Parkinson´s disease attention may be decreased and in conjunction saccades slowed (Zangemeister et al., 1996; Chaudhuri & Behan, 2000; Buhmann et al., 2015; Zangemeister et al., 2009; Nilsson et al., 1997).
Already in the early years of eye movement research, decreased saccadic speed had been consistently found in healthy controls during high continuous mental load and sleep deprivation (Miles, 1929). Matta et al. (2009) studied fatigability of horizontal saccades in 9 multiple sclerosis patients with clinically proven internuclear ophthalmoplegia (INO). INO patients with mild affection showed worsened conjugacy during the 10 minute fatigue test, while more severe INO patients showed improved conjugacy perhaps due to adapative mechanisms as the authors speculated. However, chronic fatigue patients with INO do not appear to be a good model to give a specific measure of both chronic fatigue and /or fatigability. The Matta et al. (2009) article reports on MS patients who suffer already from an INO that is it originates from a non- peripheral, central paresis that is defined by a distinct lesion of the internuclear brainstem connection with the result of slowed nasal saccade(s). In fatigue of MS patients however, we have not a distinct lesion but a more general state of fatigue that can involve all saccadic movements and their dynamics; i.e. not just one particular connection as in INO. Also, this general state involves more than the brainstem connectivity and arousal system, as the bulk of reports about fatigue in MS patients show. So, we feel that a distinct localized brainstem lesion like the INO would interfere with a general state of fatigue: Fatigue signs in saccades then would have to be disentangled from velocity changes due to zentral pareses caused by an INO.
As a fatigue-test our suggested method had the advantage that it records both trait-fatigue (before the stress test) and fatigability (after the stress test). If one would find already at the test start an INO, the general aspect of traitfatigue could not properly be disentangled. Therefore patients with INO or other special pareses of saccades should be excluded from this test – what we did.
Di Stasi et al. (2013a) showed that microsaccades and drift dynamics reflect mental fatigue; they showed that saccadic and microsaccadic velocity decreased with time-on-task whereas drift velocity increased, suggesting that ocular instability increases with mental fatigue. Task difficulty did not influence eye movements despite affecting reaction times, performance errors and subjective complexity ratings. They concluded that changes in fixational and saccadic dynamics can indicate mental fatigue due to time-on-task, irrespective of task complexity.

Increase of saccadic peak velocity

Idazoxan, which generally increases arousal, led to saccadic peak velocity increases (Cohen & Fisher, 1989; Glue et al., 1991). Also, Increased arousal due to drug use (Gijsman et al., 2002), increased motivation (i.e. due to reward) (Takikawa et al., 2002), and perhaps increased effort (Galley, 1989; Di Stasi et al., 2013) can raise saccadic velocity.
These effects of arousal on saccadic peak velocity could arise at the level of the excitatory burst neurons, whose firing rates encode the velocity signal of saccades (Edelman & Goldberg, 2002; Sparks, 1990, 2002; Zils et al., 2005). Changes in attentional processing --for instance, due to variations in arousal-- can affect the strength of the excitatory connections from the frontal cortex to the brainstem reticular formation (Munoz & Everling, 2004), thereby changing the characteristics of the main sequence. Arousal may affect peak velocity via the inhibitory connections between the sleep-regulating centers --- nucleus raphe magnus, nucleus raphe dorsalis, and locus coeruleus--- and the superior colliculus on the reticular formation and cerebellum.

Pupillary oscillations

Pupillary oscillations as an indicator of central autonomic nervous system tone have been associated with tiredness (Wilhelm et al., 1996). However, in a study comparing MS patients (with minor disability) and healthy volunteers, Egg et al. (2002) could not show a correlation of increased pupillary oscillations with fatigue scales (Chalder et al., 1993). Matta et al. (2009) has studied horizontal saccades in MS as a model for central fatigability. However, subjectively perceived fatigue was not assessed and all their patients did show an internuclear ophthalmoplegia, INO, which our patients did not. Therefore, central nerve conduction changes or subclinical pareses might have accounted for the findings in their study.

Level of fatigue symptoms

Our results confirmed our prediction that the level of fatigue symptoms significantly reduced the characteristic Main Sequence ratio of peak velocity to amplitude due to saccadic fatigue as described by Bahill et al. (1975a, b). Neurophysiological evidence indicates that ocular motor fatigue is of premotor and brainstem origin, rather than of muscular origin. These premotor processes likely reflect altered cortical or cerebellar influences that might result in a decreasing alertness, i.e. the ability to sustain attention (Prsa et al., 2010; Bares, 2007; Takagi et al., 1996). The functional paradigm used in our study is likely to capture pre-motor changes in the oculomotor system (see also Finke et al., 2012) and requires top-down control of the eye movements (Fielding et al. 2009; Walker et al., 2012).

Variability of peak velocity

As seen in Table 3, slowed peak velocities in MS patients with fatigue were also found in patients with nonfatigue in our cohort. This points to the variability of saccade dynamics particularly when we do not look at the lead parameter of the main sequence which is the relation between amplitude and peak velocity, i.e. the ratio of these two. Defining saccades through their exact amplitudes permitted us to link the correct peak velocity to each saccadic amplitude. with low variability of the pV/Amplitude ratio (Bahill et al., 1981). Mental fatigue. For the clinician, the difference before and after the stress test is of importance. Our findings explain the divergence and variability of reports about saccadic dynamics of MS patients with and without fatigue. In our view only by using a stress test similar to the one that we have described, a useful result could be obtained; i.e. a significant and reproducible difference between saccade amplitude velocity relationship and fatigue versus non-fatigue MS patients.
In monkeys, Straube et al. (1997) described that after 2000 to 7000 saccades in the dark, peak eye velocity on the average decreased by 20%, and showed increased variability. In contrast, when testing was done in dim light, there was little to no change in average saccadic metrics and latency. These changes in saccadic metrics and dynamics in the dark did not reflect a change of the ocular plant but reflected a change in the cortical or cerebellar influences on the brain stem burst generator linked to the monkeys' attentional state. They showed that this slowing of saccades depended not on fatigue of the extraocular muscles but on mental fatigue and concluded that saccades provide an objective measure of mental fatigue which central processes mediating saccades are responsible for.

Fatigue and deficits of alertness in sustained attention

An association between fatigue and deficits of alertness/sustained attention has been described by Flachenecker and Meissner (2008). They reported cases of severely impaired sustained attention performance, especially in the alertness subdomain in the TAP during a MS relapse perceived as severely increased fatigue which resolved after treatment. They concluded that measures of alertness, i.e. sustained attention might be objective correlates of MS fatigue. Di Stasi et al. (2010, 2013a) showed that slowing of saccadic velocity is a reliable indicator of the subjective fatigue of health care and other professionals during prolonged time-on-duty. Time-on-duty decreased saccadic velocity and increased subjective fatigue but did not affect surgical performance. These results supported the hypothesis that saccadic indices reflect graded changes in fatigue.

Chronotype influence

Cazzoli et al. (2014) demonstrated that mean saccadic speed significantly decreased throughout the duration of a fatiguing task, but was not influenced by the optimal or non-optimal time of the day for both chronotypes, i.e. a ‘morning type’ or an ‘evening type’. The results suggested that different oculomotor parameters are discriminative for fatigue due to different sources. A decrease in saccadic speed may have reflected fatigue due to time spent on task. An increase in mean fixation duration may have shown a lack of synchronicity between chronotype and time of the day.
Many neurocognitive correlates of subjectively perceived fatigue have been studied in the past. The largest cross-sectional and longitudinal cohort compared fatigue scores and the MACFIMS cognitive battery in an unselected sample of n=465 patients (Morrow et al., 2009; Zifko, 2004). No specific measures of alertness were applied. In this study, only the SDMT showed differences between MS patients with fatigue and without fatigue. Walker et al. (2012) postulated a correlation of PASAT performance and subjectively perceived fatigue measured by the Fatigue Impact Scale but showed only modest correlations of up to 0.30.

Fatigue and Depression

Weinges-Evers et al. (2010) were the first to show FSS scores as predictors of alertness performance although with a low standardized beta coeffcient of 0.298 in a cohort not selected based on fatigue complaints. However, their study also showed substantial correlations of fatigue scores with disability and depression ratings. While in our cohort depression was more prevalent both cohorts showed that depression and disability impact on fatigue ratings. Melancholic and non-melancholic depression (Siegert & Abernethy, 2005) are subtypes of major depressive disorder, each having distinct cognitive and motor impairments (Winograd-Gurvich et al., 2006). They found abnormal main sequences in patients with melancholic depression (decreased saccadic velocity), and relatively normal main sequences in patients with non-melancholic depression. We demonstrated that performance in the attention test was different in MS patients with fatigue compared to a control cohort without fatigue; MS patients with fatigue showed impaired tonic alertness in the applied tests. However, disability and depressive symptomatology also impacted on TAP results, underlining that this objective functional test only reflects partial effects of MS fatigue.
The correlation of fatigue with depression and disability has repeatedly been studied (Erbaugh, 1961; for review see Bol, 2009). Already in 1908 Diefendorf and Dodge (1908) studied saccadic velocities in mentally ill patients, anticipating that saccadic metrics might help to diagnose psychiatric disease. They found abnormal unusually fast saccades in manic patients, but slow saccades in extremely depressed patients. More recently Gooding and Basso (2008) performed saccadic research in psychiatric populations. E.g. in normal healthy subjects, execution of visually guided saccades improved the accuracy of corrective saccades made after the first memory-guided saccades to drive the eyes closer to the target, and this improvement was independent from the number of the visually guided saccades Colnaghi et al. (2008). In patients with depression this capability is significantly reduced. Visually guided saccades therefore provide a template that improves the capability of corrective saccades to compensate for the residual position error at the end of the first saccade to the memorized goal.
Due to the overlap between fatigue and vegetative symptoms of depression, it is difficult to dissociate these two closely linked symptom domains (Penner et al., 2007). However, through screening of large cohorts of patients one might be able to define a subgroup with high levels of fatigue but very low levels of depression. On the other hand such an approach by definition reduces the generalizability of the findings for MS-associated fatigue.

Conclusion

As fatigue is always a self-reported symptom, while depression can be identified and diagnosed by an external observer, i.e. an objective sign, it suggests that one part of fatigue is a perceptual, inference chronic phenomenon (Kuppuswamy, 2017). Our measures assessed facets of perceived fatigue that are substantially influenced by disability and affective state. On the other hand, we demonstrate the central neuro-motor part of fatigue which can be objectively measured, as the sense of the oculomotor activity is different from general skeletal muscular sense. When controlling for disability and depression, saccadic stress tests and alertness tests could be used as an objective read-out for both fatigability and chronic fatigue in MS patients. Oculomotor tests differentiated significantly between fatigue and fatigabiliy in our MS patients. Patients with fatigue showed highly significant changes of their saccade dynamics as defined by the Main Sequence and Phase Plane plots. They showed slowing of saccades, the characteristical fatigue double peak, and an asymmetrical phase plane. Further work should study these outcomes in longitudinal studies as well in clinical trials of pharmacological or behavioral interventions.

Ethics and Conflict of Interest

The author(s) declare(s) that the contents of the article are in agreement with the ethics described in http://biblio.unibe.ch/portale/elibrary/BOP/jemr/ethics.html and that there is no conflict of interest regarding the publication of this paper.

Appendix A

Jazz Manager (version 3.13) uses the following
eye velocity calculation formula:
Ev[n] = ( 300 Ep[n-5] - 294 Ep[n-4] - 532 Ep[n-3] - 503 Ep[n-2] +296Ep[n-1] + 296Ep[n+1] +503Ep[n+2] + 523 Ep[n+3] + 294 Ep[n+4] - 300 Ep[n+5]) / (5140 * h)
where: Ev[n] - eye velocity sample at moment n. Ep[n] - eye position sample at moment n. And h - sampling interval (0.001 s for 1kHz sampling).
the eye acceleration calculation formula is:
Ea[n] = ( 2 Ep[n-3] - 27 Ep[n-2] +270Ep[n-1] - 490 Ep[n] + 270Ep[n+1] - 27 Ep[n+2] + 2 Ep[n+3] ) / (180 * h^2)
where: Ea[n] - eye acceleration sample at moment n. Ep[n] - eye position sample at moment n. And h - sampling interval (0.001 s for 1kHz sampling).

Inside IR-OG data

The standard deviations of our IR-OG data were small because we plotted the parameters as functions of the size of the actual saccade, not as functions of the size of the target movement and the software calculated the parameters from the velocity trace, derived with a zero-phase digital filter. We defined the size of the saccade as the size of the initial, dynamic saccade. Eye movements with long glissades or drifts were discarded and the magnitude of the dynamic saccade was carefully calculated. For saccades with overshoot, the saccadic magnitude was defined to be the foot-to-peak angle, i.e., the angle between zero velocity at the start of the saccade and zero velocity at the end of the saccade. Less variability was provided by this definition for the size of the saccade than when the size of the saccade was defined as the angle between the two steady state fixation points. Saccades with dynamic overshoot were treated as two saccades: the main saccade and a small return saccade. We then used the ratio of peakVelocity/ Amplitude, which is the main indicator of the Main Sequence as described by Bahill et al. (1975a), for specifically analyzing and comparing the the group data. This permitted to narrow the otherwise relatively high variability of the single dynamic saccade parameters (see Bahill et al. 1981 for an overview of the causes of dynamic variance of saccades).

The standard deviations of our data were small

The standard deviations of our data were small, because we plotted the parameters as functions of the size of the actual saccade, not as functions of the size of the target movement and the software calculated the parameters from the velocity trace, derived with a zero-phase digital filter. We defined the size of the saccade as the size of the initial, primary saccade. Either the eye movements with long glissades or drifts were discarded or else the magnitude of the dynamic saccade was carefully calculated. For saccades with overshoot, the saccadic magnitude was defined to be the foot-to-peak angle, i.e., the angle between zero velocity at the start of the saccade and zero velocity at the end of the saccade. Less variability was provided by this definition for the size of the saccade than when the size of the saccade was defined as the angle between the two steady state fixation points. Saccades with dynamic overshoot were treated as two saccades: the main saccade and a small return saccade. We then used the ratio of peakVelocity/ Amplitude, which is the main indicator of the Main Sequence as described by Bahill and Stark (1975), for specifically analyzing and comparing the the group data. This permitted to narrow the otherwise relatively high variability of the single dynamic saccade parameters (see Bahill et al. 1981 for an overview of cause of dynamic variance of saccades.

Appendix B

Complete Table 5. Correlations ( Spearman Rho) TAP and SDMT and EDSS
Jemr 13 00033 i004

References

  1. Aschoff, J. C. 1968. Veränderungen rascher Blickbewegungen (Saccaden) beim Menschen unter Diazepam (Valium®). Archiv für Psychiatrie und Nervenkrankheiten 211, 4: 325–332. [Google Scholar] [CrossRef] [PubMed]
  2. Bahill, A. T., M. R. Clark, and L. Stark. 1975a. The main sequence, a tool for studying human eye movements. Mathematical biosciences 24, 3-4: 191–204. [Google Scholar] [CrossRef]
  3. Bahill, A. T., and L. Stark. 1975b. Overlapping saccades and glissades are produced by fatigue in the saccadic eye movement system. Experimental neurology 48, 1: 95–106. [Google Scholar] [CrossRef] [PubMed]
  4. Bahill, A. T., A. Brockenbrough, and B. T. Troost. 1981. Variability and development of a normative data base for saccadic eye movements. Investigative ophthalmology & visual science 21, 1: 116–125. [Google Scholar]
  5. Bakshi, R., Z. A. Shaikh, R. S. Miletich, D. Czarnecki, J. Dmochowski, K. Henschel, and P. R. Kinkel. 2000. Fatigue in multiple sclerosis and its relationship to depression and neurologic disability. Multiple Sclerosis Journal 6, 3: 181–185. [Google Scholar] [CrossRef]
  6. Bakshi, R. 2003. Fatigue associated with multiple sclerosis: diagnosis, impact and management. Multiple Sclerosis Journal 9, 3: 219–227. [Google Scholar] [CrossRef]
  7. Bares, M., O. Lungu, T. Liu, T. Waechter, C. M. Gomez, and J. Ashe. 2007. Impaired predictive motor timing in patients with cerebellar disorders. Experimental brain research 180, 2: 355–365. [Google Scholar] [CrossRef]
  8. Barker, L. M., and M. A. Nussbaum. 2011. The effects of fatigue on performance in simulated nursing work. Ergonomics 54, 9: 815–829. [Google Scholar] [CrossRef]
  9. Barnes, C.M., and Linn Van Dyne. 2009. I'm tired': Differential effects of physical and emotional fatigue on workload management strategies. Human Relations 62, 1: 59–92. [Google Scholar] [CrossRef]
  10. Bartlett, F. C. 1943. Ferrier Lecture-Fatigue following highly skilled work. Proceedings of the Royal Society of London. Series B-Biological Sciences 131, 864: 247–257. [Google Scholar] [CrossRef]
  11. Beck, A., C. Ward, M. Mendelsohn, J. Mock, and J. Erbaugh. 1961. An inventory for measuring depression. Archives of General Psychiatry 4: 561–571. [Google Scholar] [CrossRef]
  12. Benko, W., M. Ries, E. A. Wiggs, R. O. Brady, R. Schiffmann, and E. J. FitzGibbon. 2011. The saccadic and neurological deficits in type 3 Gaucher disease. PloS one 6, 7: e22410. [Google Scholar] [CrossRef] [PubMed]
  13. Boksem, M. A., and M. Tops. 2008. Mental fatigue: costs and benefits. Brain Research Reviews 59, 1: 125–139. [Google Scholar] [CrossRef] [PubMed]
  14. Bol, Y., A. A. Duits, R. M. Hupperts, J. W. Vlaeyen, and F. R. Verhey. 2009. The psychology of fatigue in patients with multiple sclerosis: a review. Journal of Psychosomatic Research 66, 1: 3–11. [Google Scholar] [CrossRef] [PubMed]
  15. Bower, J. E., P. A. Ganz, and N. Aziz. 2005. Altered cortisol response to psychologic stress in breast cancer survivors with persistent fatigue. Psychosomatic Medicine 67, 2: 277–80. [Google Scholar] [CrossRef]
  16. Buhmann, C., and et al. 2015. Visual attention and saccadic oculomotor control in parkinson's disease. European Neurology 73, 5-6: 283–293. [Google Scholar] [CrossRef]
  17. Calabrese, M., F. Rinaldi, P. Grossi, I. Mattisi, V. Bernardi, A. Favaretto, and P. Gallo. 2010. Basal ganglia and frontal/parietal cortical atrophy is associated with fatigue in relapsing—remitting multiple sclerosis. Multiple Sclerosis Journal 16, 10: 1220–1228. [Google Scholar] [CrossRef]
  18. Cazzoli, D., C. A. Antoniades, C. Kennard, T. Nyffeler, C. L. Bassetti, and R. M. Müri. 2014. Eye movements discriminate fatigue due to chronotypical factors and time spent on task–a double dissociation. PloS one 9, 1: e87146. [Google Scholar] [CrossRef]
  19. Chalder, T., G. Berelowitz, T. Pawlikowska, L. Watts, S. Wessely, D. Wright, and E. P. Wallace. 1993. Development of a fatigue scale. Journal of psychosomatic research 37, 2: 147–153. [Google Scholar] [CrossRef]
  20. Chaudhuri, A., and P.O Behan. 2000. Fatigue and basalganglia. Journal of the Neurological Sciences 179: 34–42. [Google Scholar] [CrossRef]
  21. Clark, M. R., and L. Stark. 1974. Control of human eye movements: III. Dynamic characteristics of the eye tracking mechanism. Mathematical Biosciences 20, 3-4: 239–265. [Google Scholar] [CrossRef]
  22. Cook, G., L. Stark, and B. L. Zuber. 1966. Horizontal eye movements studied with the online computer. Archives of Ophthalmology 76, 4: 589–595. [Google Scholar] [CrossRef]
  23. Cook, D. B., P. J. O’Connor, G. Lange, and J. Steffener. 2007. Functional neuroimaging correlates of mental fatigue induced by cognition among chronic fatigue syndrome patients and controls. Neuroimage 36, 1: 108–122. [Google Scholar] [CrossRef] [PubMed]
  24. Cohen, R. A., and M. Fisher. 1989. Amantadine treatment of fatigue associated with multiple sclerosis. Archives of neurology 46, 6: 676–680. [Google Scholar] [CrossRef]
  25. Colnaghi, S., G. Beltrami, A. Cortese, W. H. Zangemeister, V. Cosi, and M. Versino. 2008. Multiple memory-guided saccades: movement memory improves the accuracy of memory-guided saccades. In Progress in Brain Research. Elsevier: Vol. 171, pp. 425–427. [Google Scholar] [CrossRef]
  26. Cutter, G. R., M. L. Baier, R. A. Rudick, D. L. Cookfair, J. S. Fischer, J. Petkau, and G. W. Ellison. 1999. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 122, 5: 871–882. [Google Scholar] [CrossRef] [PubMed]
  27. Diefendorf, A. R., and R. Dodge. 1908. An experimental study of the ocular reactions of the insane from photographic records. Brain 31, 3: 451–489. [Google Scholar] [CrossRef]
  28. Di Stasi, L. L., R. Renner, A. Catena, J. J. Cañas, B. M. Velichkovsky, and S. Pannasch. 2012. Towards a driver fatigue test based on the saccadic main sequence: A partial validation by subjective report data. Transportation research part C: emerging technologies 21, 1: 122–133. [Google Scholar] [CrossRef]
  29. Di Stasi, L. L., R. Renner, P. Staehr, J. R. Helmert, B. M. Velichkovsky, J. J. Cañas, and S. Pannasch. 2010. Saccadic peak velocity sensitivity to variations in mental workload. Aviation, space, and environmental medicine 81, 4: 413–417. [Google Scholar] [CrossRef]
  30. Di Stasi, L. L., M. B. McCamy, A. Catena, S. L. Macknik, J. J. Canas, and S. Martinez-Conde. 2013a. Microsaccade and drift dynamics reflect mental fatigue. European Journal of Neuroscience 38, 3: 2389–2398. [Google Scholar] [CrossRef]
  31. Di Stasi, L. L., A. Catena, J. J. Canas, S. L. Macknik, and S. Martinez-Conde. 2013b. Saccadic velocity as an arousal index in naturalistic tasks. Neuroscience & Biobehavioral Reviews 37, 5: 968–975. [Google Scholar] [CrossRef]
  32. Dodge, R. 1917. The laws of relative fatigue. Psychological Review 24, 2: 89. [Google Scholar] [CrossRef]
  33. Dodge, R., and T. S. Cline. 1901. The angle velocity of eye movements. Psychological Review 8, 2: 145. [Google Scholar] [CrossRef]
  34. Edelman, J. A., and M. E. Goldberg. 2002. Effect of short-term saccadic adaptation on saccades evoked by electrical stimulation in the primate superior colliculus. Journal of Neurophysiology 87, 4: 1915–1923. [Google Scholar] [CrossRef]
  35. Egg, R., B. Högl, S. Glatzl, R. Beer, and T. Berger. 2002. Autonomic instability, as measured by pupillary unrest, is not associated with multiple sclerosis fatigue severity. Multiple Sclerosis Journal 8, 3: 256–260. [Google Scholar] [CrossRef] [PubMed]
  36. Erbaugh, J. 1961. An Inventory for Measuring Depression. Archives of General Psychiatry (4): 561–71. [Google Scholar]
  37. Guidelines, M. S. C. P. 1998. Fatigue and multiple sclerosis: evidence-based management strategies for fatigue in multiple sclerosis. Paralyzed Veterans of America. [Google Scholar]
  38. Ferreira, M., P. A. Pereira, M. Parreira, I. Sousa, J. Figueiredo, J. J. Cerqueira, and A. F. Macedo. 2017. Using endogenous saccades to characterize fatigue in multiple sclerosis. Multiple sclerosis and related disorders 14: 16–22. [Google Scholar] [CrossRef]
  39. Fielding, J., T. Kilpatrick, L. Millist, and O. White. 2009. Control of visually guided saccades in multiple sclerosis: disruption to higherorder processes. Neuropsychologia 47, 7: 1647–1653. [Google Scholar] [CrossRef]
  40. Filippi, M., and M. A. Rocca. 2007. Toward a definition of structural and functional MRI substrates of fatigue in multiple sclerosis. Journal of the Neurological Sciences 263, 1-2: 1–2. [Google Scholar] [CrossRef]
  41. Finke, C., L. M. Pech, C. Sömmer, J. Schlichting, S. Stricker, M. Endres, and F. Paul. 2012. Dynamics of saccade parameters in multiple sclerosis patients with fatigue. Journal of neurology 259, 12: 2656–2663. [Google Scholar] [CrossRef]
  42. Fischer, B., and H. Weber. 1993. Express saccades and visual attention. Behavioral and Brain Sciences 16, 3: 553–567. [Google Scholar] [CrossRef]
  43. Flachenecker, P., and H. Meissner. 2008. Fatigue in multiple sclerosis presenting as acute relapse: subjective and objective assessment. Multiple Sclerosis Journal 14, 2: 274–277. [Google Scholar] [CrossRef]
  44. Galley, N. 1989. Saccadic eye movement velocity as an indicator of (de)activation: a review and some speculations. Journal of Psychophysiology (3): 229–244. [Google Scholar]
  45. Gijsman, H. J., J. M. van Gerven, R. J. Verkes, R. C. Schoemaker, M. S. Pieters, E. J. Pennings, and A. F. Cohen. 2002. Saccadic peak velocity and EEG as end-points for a serotonergic challenge test. Human Psychopharmacology: Clinical and Experimental 17, 2: 83–89. [Google Scholar] [CrossRef]
  46. Gold, S. M., C. Heesen, H. Schulz, U. Guder, A. Mönch, J. Gbadamosi, and K. H. Schulz. 2001. Disease specific quality of life instruments in multiple sclerosis: validation of the Hamburg Quality of Life Questionnaire in Multiple Sclerosis (HAQUAMS). Multiple Sclerosis Journal 7, 2: 119–130. [Google Scholar] [CrossRef] [PubMed]
  47. Gooding, D. C., and M. A. Basso. 2008. The tell-tale tasks: a review of saccadic research in psychiatric patient populations. Brain and cognition 68, 3: 371–390. [Google Scholar] [CrossRef] [PubMed]
  48. Grace, P. M., T. Stanford, M. Gentgall, and P. E. Rolan. 2010. Utility of saccadic eye movement analysis as an objective biomarker to detect the sedative interaction between opioids and sleep deprivation in opioidnaive and opioidtolerant populations. Journal of Psychopharmacology 24, 11: 1631–1640. [Google Scholar] [CrossRef]
  49. Guidelines, M. S. C. P. 1998. Fatigue and multiple sclerosis: evidence-based management strategies for fatigue in multiple sclerosis. Paralyzed Veterans of America. [Google Scholar]
  50. Harrison, A. M., R. das Nair, and R. Moss-Morris. 2017. Operationalising cognitive fatigability in multiple sclerosis: a Gordian knot that can be cut? Multiple Sclerosis Journal 23, 13: 1682–1696. [Google Scholar] [CrossRef]
  51. Heesen, C., L. Nawrath, C. Reich, N. Bauer, K. H. Schulz, and S. M. Gold. 2006. Fatigue in multiple sclerosis: an example of cytokine mediated sickness behaviour? Journal of Neurology, Neurosurgery & Psychiatry 77, 1: 34–39. [Google Scholar] [CrossRef]
  52. Hein, O., and W. H. Zangemeister. 2017. Topology for gaze analyses-Raw data segmentation. Journal of Eye Movement Research 10, 1: 1–25. [Google Scholar] [CrossRef]
  53. Hirvonen, K., S. Puttonen, K. Gould, J. Korpela, V. F. Koefoed, and K. Müller. 2010. Improving the saccade peak velocity measurement for detecting fatigue. Journal of neuroscience methods 187, 2: 199–206. [Google Scholar] [CrossRef] [PubMed]
  54. Julian, L. J., L. Vella, T. Vollmer, O. Hadjimichael, and D. C. Mohr. 2008. Employment in multiple sclerosis. Journal of neurology 255, 9: 1354–1360. [Google Scholar] [CrossRef]
  55. Johansson, S., C. Ytterberg, J. Hillert, L. W. Holmqvist, and L. Von Koch. 2008. A longitudinal study of variations in and predictors of fatigue in multiple sclerosis. Journal of Neurology, Neurosurgery & Psychiatry 79, 4: 454–457. [Google Scholar] [CrossRef]
  56. Kley, F. 2010. Evaluierung der Testbatterie zur Aufmerksamkeitsprüfung (TAP) als kognitiver Endophänotyp der Schizophrenie. Doctoral dissertation. [Google Scholar] [CrossRef]
  57. Kluger, B. M., L. B. Krupp, and R. M. Enoka. 2013. Fatigue and fatigability in neurologic illnesses: proposal for a unified taxonomy. Neurology 80, 4: 409–416. [Google Scholar] [CrossRef] [PubMed]
  58. Kos, D., E. Kerckhofs, G. Nagels, M. B. D'hooghe, and S. Ilsbroukx. 2008. Origin of fatigue in multiple sclerosis: review of the literature. Neurorehabilitation and neural repair 22, 1: 91–100. [Google Scholar] [CrossRef]
  59. Kuppuswamy, A. 2017. The fatigue conundrum. Brain 140, 8: 2240–2245. [Google Scholar] [CrossRef]
  60. Krupp, L. B., D. J. Serafin, and C. Christodoulou. 2010. Multiple sclerosis-associated fatigue. Expert review of neurotherapeutics 10, 9: 1437–1447. [Google Scholar] [CrossRef]
  61. Krupp, L. B., N. G. LaRocca, J. Muir-Nash, and A. D. Steinberg. 1989. The fatigue severity scale: application to patients with multiple sclerosis and systemic lupus erythematosus. Archives of neurology 46, 10: 1121–1123. [Google Scholar] [CrossRef]
  62. Kurtzke, J. F. 1983. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale. Neurology 33, 11: 1444–1444. [Google Scholar] [CrossRef]
  63. Leigh, J.R., and D.S. Zee. 1999. The neurology of eye movements. F.A. Davis Company. [Google Scholar]
  64. Liepert, J., D. Mingers, C. Heesen, T. Bäumer, and C. Weiller. 2005. Motor cortex excitability and fatigue in multiple sclerosis: a transcranial magnetic stimulation study. Multiple Sclerosis Journal 11, 3: 316–321. [Google Scholar] [CrossRef]
  65. Lorist, M. M., M. A. Boksem, and K. R. Ridderinkhof. 2005. Impaired cognitive control and reduced cingulate activity during mental fatigue. Cognitive Brain Research 24, 2: 199–205. [Google Scholar] [CrossRef] [PubMed]
  66. Maclachlan, L., S. Watson, P. Gallagher, A. Finkelmeyer, L. A. Jason, M. Sunnquist, and J. L. Newton. 2017. Are current chronic fatigue syndrome criteria diagnosing different disease phenotypes? PloS one 12, 10: e0186885. [Google Scholar] [CrossRef]
  67. Matta, M., R. J. Leigh, M. Pugliatti, I. Aiello, and Serra. 2009. Using fast eye movements to study fatigue in multiple sclerosis. Neurology 73, 10: 798–804. [Google Scholar] [CrossRef]
  68. McDonald, W. I., A. Compston, G. Edan, D. Goodkin, H. P. Hartung, F. D. Lublin, and M. Sandberg-Wollheim. 2001. Recommended diagnostic criteria for multiple sclerosis: guidelines from the International Panel on the diagnosis of multiple sclerosis. Annals of Neurology: Official Journal of the American Neurological Association and the Child Neurology Society 50, 1: 121–127. [Google Scholar] [CrossRef]
  69. Miles, W. R. 1929. Horizontal eye movements at the onset of sleep. Psychological Review 36, 2: 122. [Google Scholar] [CrossRef]
  70. Morrow, S. A., B. Weinstock-Guttman, F. E. Munschauer, D. Hojnacki, and R. H. B. Benedict. 2009. Subjective fatigue is not associated with cognitive impairment in multiple sclerosis: cross-sectional and longitudinal analysis. Multiple Sclerosis Journal 15, 8: 998–1005. [Google Scholar] [CrossRef]
  71. Munoz, D. P., and S. Everling. 2004. Look away: the anti-saccade task and the voluntary control of eye movement. Nature Reviews Neuroscience 5, 3: 218–228. [Google Scholar] [CrossRef] [PubMed]
  72. Nagel, M., H. Behrmann, and W. H. Zangemeister. 2008. Disturbance of predictive response initiation of eye and head movements in cerebellar patients. European Neurology 60, 4: 179–185. [Google Scholar] [CrossRef] [PubMed]
  73. Nilsson, T., T. M. Nelson, and D. Carlson. 1997. Development of fatigue symptoms during simulated driving. Accident Analysis & Prevention 29, 4: 479–488. [Google Scholar] [CrossRef]
  74. Peng, G. C., D. S. Zee, and L. B. Minor. 2004. Phase-plane analysis of gaze stabilization to high acceleration head thrusts: a continuum across normal subjects and patients with loss of vestibular function. Journal of Neurophysiology 91, 4: 1763–1781. [Google Scholar] [CrossRef]
  75. Penner, I. K., N. Bechtel, C. Raselli, M. Stöcklin, K. Opwis, L. Kappos, and P. Calabrese. 2007. Fatigue in multiple sclerosis: relation to depression, physical impairment, personality and action control. Multiple Sclerosis Journal 13, 9: 1161–1167. [Google Scholar] [CrossRef] [PubMed]
  76. Posner, M. I., and S. E. Petersen. 1990. The attention system of the human brain. Annual Review of Neuroscience 13, 1: 25–42. [Google Scholar] [CrossRef]
  77. Prsa, M., P. W. Dicke, and P. Thier. 2010. The absence of eye muscle fatigue indicates that the nervous system compensates for non-motor disturbances of oculomotor function. Journal of Neuroscience 30, 47: 15834–15842. [Google Scholar] [CrossRef]
  78. Röhr, D. 2015. Okulomotorik-und Aufmerksamkeitsprüfung als objektive Diagnoseverfahren für Fatigue bei Multipler Sklerose. [Doctoral dissertation, University of Hamburg], Staats- und Universitätsbibliothek Hamburg Archive. Available online: https://ediss.sub.uni-hamburg.de/volltexte/2015/7539/pdf/Dissertation.pdf.
  79. Schleicher, R., N. Galley, S. Briest, and L. Galley. 2008. Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 51, 7: 982–1010. [Google Scholar] [CrossRef] [PubMed]
  80. Schmidt, D., L. A. Abel, L. F. DellOsso, and R. Daroff. 1979. Saccadic velocity characteristics-Intrinsic variability and fatigue. Aviation, Space, and Environmental Medicine 50, 4: 393–395. [Google Scholar]
  81. Sepulcre, J., J. C. Masdeu, J. Goni, G. Arrondo, N. Vélez de Mendizábal, B. Bejarano, and P. Villoslada. 2009. Fatigue in multiple sclerosis is associated with the disruption of frontal and parietal pathways. Multiple Sclerosis Journal 15, 3: 337–344. [Google Scholar] [CrossRef]
  82. Serra, A., C. G. Chisari, and M. Matta. 2018. Eye movement abnormalities in multiple sclerosis: pathogenesis, modeling, and treatment. Frontiers in Neurology 9: 31. [Google Scholar] [CrossRef]
  83. Shen, K. Q., X. P. Li, C. J. Ong, S. Y. Shao, and E. P. Wilder-Smith. 2008. EEG-based mental fatigue measurement using multiclass support vector machines with confidence estimate. Clinical neurophysiology 119, 7: 1524–1533. [Google Scholar] [CrossRef]
  84. Siegert, R. J., and D. A. Abernethy. 2005. Depression in multiple sclerosis: a review. Journal of Neurology, Neurosurgery & Psychiatry 76, 4: 469–475. [Google Scholar] [CrossRef]
  85. Smith, A. 1968. The symbol-digit modalities test: a neuropsychological test of learning and other cerebral disorders. Learning Disorders, 83–91. [Google Scholar]
  86. Sparks, D. L., and L. E. Mays. 1990. Signal transformations required for the generation of saccadic eye movements. Annual Review of Neuroscience 13, 1: 309–336. [Google Scholar] [CrossRef] [PubMed]
  87. Sparks, D. L. 2002. The brainstem control of saccadic eye movements. Nature Reviews Neuroscience 3, 12: 952–964. [Google Scholar] [CrossRef]
  88. Spiteri, S., T. Hassa, D. Claros-Salinas, C. Dettmers, and M. A. Schoenfeld. 2019. Neural correlates of effort-dependent and effort-independent cognitive fatigue components in patients with multiple sclerosis. Multiple sclerosis journal 25, 2: 256–266. [Google Scholar] [CrossRef]
  89. Straube, A., F. R. Robinson, and A. F. Fuchs. 1997. Decrease in saccadic performance after many visually guided saccadic eye movements in monkeys. Investigative ophthalmology & visual science 38, 13: 2810–2816. [Google Scholar]
  90. Takagi, M., D. S. Zee, and R. J. Tamargo. 1996. Effect of dorsal cerebellar lesions on saccades and pursuit in monkeys. In Society of Neuroscience Abstracts 22: 1458. [Google Scholar]
  91. Takikawa, Y., R. Kawagoe, H. Itoh, H. Nakahara, and O. Hikosaka. 2002. Modulation of saccadic eye movements by predicted reward outcome. Experimental Brain Research 142, 2: 284–291. [Google Scholar] [CrossRef]
  92. Tedeschi, G., D. Dinacci, L. Lavorgna, A. Prinster, G. Savettieri, A. Quattrone, and V. Bresciamorra. 2007. Correlation between fatigue and brain atrophy and lesion load in multiple sclerosis patients independent of disability. Journal of the Neurological Sciences 263, 1-2: 15–19. [Google Scholar] [CrossRef]
  93. Thompson, A. J., C. H. Polman, D. H. Miller, W. I. McDonald, B. Brochet, X. Filippi M Montalban, and J. De Sa. 1997. Primary progressive multiple sclerosis. Brain: a Journal of Neurology 120, 6: 1085–1096. [Google Scholar] [CrossRef]
  94. Thorndike, E. 1900. Mental fatigue. Psychological Review 7: 466–482. [Google Scholar] [CrossRef]
  95. Walker, L. A., J. A. Berard, L. I. Berrigan, L. M. Rees, and M. S. Freedman. 2012. Detecting cognitive fatigue in multiple sclerosis: method matters. Journal of the Neurological Sciences 316, 1-2: 86–92. [Google Scholar] [CrossRef]
  96. Weinges-Evers, N., A. U. Brandt, M. Bock, C. F. Pfueller, J. Dörr, J. Bellmann-Strobl, and F. Zipp. 2010. Correlation of self-assessed fatigue and alertness in multiple sclerosis. Multiple Sclerosis Journal 16, 9: 1134–1140. [Google Scholar] [CrossRef]
  97. Wilhelm, B., H. Wilhelm, H. Lüdtke, M. Adler, and P. Streicher. 1996. Pupillography for objective vigilance assessment. Methodological problems and possible solutions. Der Ophthalmologe: Zeitschrift der Deutschen Ophthalmologischen Gesellschaft 93, 4: 446. [Google Scholar] [PubMed]
  98. Williamson, A., D. A. Lombardi, S. Folkard, J. Stutts, T. K. Courtney, and J. L. Connor. 2011. The link between fatigue and safety. Accident Analysis & Prevention 43, 2: 498–515. [Google Scholar] [CrossRef]
  99. Winograd-Gurvich, C., N. Georgiou-Karistianis, P. B. Fitzgerald, L. Millist, and O. B. White. 2006. Selfpaced and reprogrammed saccades: differences between melancholic and non-melancholic depression. Neuroscience Research 56, 3: 253–260. [Google Scholar] [CrossRef]
  100. Yaldizli, Ö., S. Glassl, D. Sturm, A. Papadopoulou, A. Gass, B. Tettenborn, and N. Putzki. 2011. Fatigue and progression of corpus callosum atrophy in multiple sclerosis. Journal of neurology 258, 12: 2199–2205. [Google Scholar] [CrossRef] [PubMed]
  101. Zangemeister, W. H., and L. Stark. 1982. Gaze latency: variable interactions of head and eye latency. Experimental Neurology 75, 2: 389–406. [Google Scholar] [CrossRef] [PubMed]
  102. Zangemeister, W. H., U. Oechsner, and C. Freksa. 1995. Short-term adaptation of eye movements in patients with visual hemifield defects indicates high level control of human scanpath. Optometry and vision science: official publication of the American Academy of Optometry 72, 7: 467. [Google Scholar] [CrossRef]
  103. Zangemeister, W. H., C. Buhmann, L. Maintz, and J Hierling. 2009. Influence of STN-stimulation on Parkinson patient's simulator driving. NCM-Neural Control of Movement 14, 42–43. [Google Scholar]
  104. Zangemeister, W. H., H. S. Stiehl, and C. Freksa, eds. 1996. Visual attention and cognition. In Advances in psychology. Elsevier. [Google Scholar]
  105. Zifko, U. A. 2004. Management of fatigue in patients with multiple sclerosis. Drugs 64, 12: 1295–1304. [Google Scholar] [CrossRef]
  106. Zils, E., A. Sprenger, W. Heide, J. Born, and S. Gais. 2005. Differential effects of sleep deprivation on saccadic eye movements. Sleep 28, 9: 1109–1115. [Google Scholar] [CrossRef]
  107. Zimmermann, P., and B. Fimm. 1994. Testbatterie zur Aufmerksamkeitsprüfung (Version 1.02c). Psytest. [Google Scholar]
Table 2. Self-report measures of fatigue and depression.
Table 2. Self-report measures of fatigue and depression.
MS FatigueMS non-FatigueP*
(n=28)(n=21)
FSS5.60+-0.151.73 +-0.24p< .001
MFIS phys23.25+-1.186.00+-1.28p< .001
MFIS cog25.68+-1.306.95+-1.68p< .001
MFIS psych4.39+-0.350.86+-0.27p< .001
BDI14.17+-1.254.86+-0.96p< .001
Note. *according to independent samples t-tests.
Table 4. TAP alertness in MS patients with and without fatigue
Table 4. TAP alertness in MS patients with and without fatigue
Jemr 13 00033 i001

Share and Cite

MDPI and ACS Style

Zangemeister, W.H.; Heesen, C.; Röhr, D.; Gold, S.M. Oculomotor Fatigue and Neuropsychological Assessments Mirror Multiple Sclerosis Fatigue. J. Eye Mov. Res. 2020, 13, 1-19. https://doi.org/10.16910/jemr.13.4.6

AMA Style

Zangemeister WH, Heesen C, Röhr D, Gold SM. Oculomotor Fatigue and Neuropsychological Assessments Mirror Multiple Sclerosis Fatigue. Journal of Eye Movement Research. 2020; 13(4):1-19. https://doi.org/10.16910/jemr.13.4.6

Chicago/Turabian Style

Zangemeister, Wolfgang H., Christof Heesen, Dorit Röhr, and Stefan M. Gold. 2020. "Oculomotor Fatigue and Neuropsychological Assessments Mirror Multiple Sclerosis Fatigue" Journal of Eye Movement Research 13, no. 4: 1-19. https://doi.org/10.16910/jemr.13.4.6

APA Style

Zangemeister, W. H., Heesen, C., Röhr, D., & Gold, S. M. (2020). Oculomotor Fatigue and Neuropsychological Assessments Mirror Multiple Sclerosis Fatigue. Journal of Eye Movement Research, 13(4), 1-19. https://doi.org/10.16910/jemr.13.4.6

Article Metrics

Back to TopTop