3.1.1. Clinical Setting
Eight of the 12 quantitative studies were conducted in clinical settings. All eight were evaluation studies [32
] with data gathered from epileptic inpatients and outpatients; none were two-arm or controlled studies with healthy participants. Most studies compared recorded device data with other clinical reference recordings, including EEG, video EEG (vEEG), electromyography (EMG), and electrocardiogram (ECG). The studies are summarised in Table 3
in terms of the devices used, the numbers of participants, the numbers of seizures detected, and, where specified, the study duration. As shown in the summary in Table 3
, four of the studies used Empatica E3, E4, and Embrace devices, three used Smart Monitor’s evolving SmartWatch devices, and one used the Epi-Care free. The numbers of patient participants varied from 3 to 135. A study [33
] with three participants selected 1 h recorded segments rather than continuous recordings. Otherwise, observation durations varied within studies [33
] as well as between studies from 17 h to 487 days, and two studies [35
] did not report durations. The total number of seizures detected in studies varied from 7 and 55 and, across all studies, a total of 226 seizures were reported as detected. Only one study [33
] did not report the number of detected seizures.
summarises the performance assessments of the studies. The reporting of performance metrics was variable and sparse across most of the studies. For example, false alarm rates for only three studies could be identified. The studies using the Empatica E3 and E4 implemented machine learning detection methods (kNN: k-nearest neighbour; RF: random forest; NB: naïve Bayes; SVM: support vector machine). Regalia et al., 2019 [35
] made brief reference to previously unpublished assessments with 135 patients and 22 seizures with 100% sensitivity and an FAR of 0.42 per day for a “fixed and frozen” algorithm. No methodology, sensitivity, or other assessment information was provided, and the paper largely focused on compiling and comparing other Empatica wristband performance indicators. Heldberg et al., 2015 [32
] reported the sensitivity and specificity for two different classifiers. Vandecasteele et al. [34
] compared the performance of SVM classifiers on hospital ECG with wearable ECG and E4 PPG recordings. PPG motion artefacts (which would have been largely induced by the seizures themselves) made more than half of the seizures undetectable via this approach and resulted in a poor sensitivity of 32%. The studies encompassed different seizure types but with TCS and “motor” seizures often included. Dramatically different performance results were observed. For example, sensitivities of 100% and 16% were reported by the authors of [35
] and [37
], respectively. Notably, the latter paper [37
] comprised a large number of (undetected) nonmotor seizures. The levels of patient activity and any movement constraints were not generally explicitly reported and, in any case, are difficult to convey. However, in the clinical setting, worn sensors usually benefit from reduced interference from activities of daily living. For example, the good wearable performance for the small study in [33
] was achieved from recordings taken simultaneously with EEGs, i.e., when one would expect patients to be inactive.
Smart Monitor’s SmartWatch was used in three of the eight clinical assessments. Patterson et al. [37
] reported the lowest sensitivity (16% overall: 31% for general tonic-clonic (GTC) and 0% for MS) in a study of 41 patients aged 5–41 years. Citing Lockman et al. [36
], the authors did not record false positives “because these are well known”. Lockman et al. [36
] did report 204 false alarm occurrences in their SmartWatch study with 40 patients between “March 2009 and June 2010” but did not specify an FAR or confirm the duration of actual usage within the study period. Velez et al. [38
] referred to 81 false alarms but also did not specify an FAR (and one cannot be estimated because of the varying durations of 1–9 days). Beniczky et al. [39
] reported a sensitivity of 90% and an FAR of 0.2 per day in a study with 73 participants with GTC seizures who were monitored for 17–171 hours. An average detection latency of 55 s was reported.