Next Article in Journal
Analysis and Correction of Measurement Error of Spherical Capacitive Sensor Caused by Assembly Error of the Inner Frame in the Aeronautical Optoelectronic Pod
Next Article in Special Issue
Deep SE-BiLSTM with IFPOA Fine-Tuning for Human Activity Recognition Using Mobile and Wearable Sensors
Previous Article in Journal
Surface Functionalization Strategies of Polystyrene for the Development Peptide-Based Toxin Recognition
Previous Article in Special Issue
Matched Filter Interpretation of CNN Classifiers with Application to HAR
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 22
Issue 23
10.3390/s22239540
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Validity of Two Consumer Multisport Activity Tracker and One Accelerometer against Polysomnography for Measuring Sleep Parameters and Vital Data in a Laboratory Setting in Sleep Patients

by
Mario Budig
1,
Riccardo Stoohs
2 and
Michael Keiner
1,*
1
Department of Sports Science, German University of Health & Sport, 85737 Ismaning, Germany
2
Somnolab, 44263 Dortmund, Germany
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(23), 9540; https://doi.org/10.3390/s22239540
Submission received: 24 October 2022 / Revised: 25 November 2022 / Accepted: 1 December 2022 / Published: 6 December 2022
(This article belongs to the Special Issue Human Activity Recognition Using Sensors and Machine Learning)
Browse Figures
Versions Notes

Abstract

:
Two commercial multisport activity trackers (Garmin Forerunner 945 and Polar Ignite) and the accelerometer ActiGraph GT9X were evaluated in measuring vital data, sleep stages and sleep/wake patterns against polysomnography (PSG). Forty-nine adult patients with suspected sleep disorders (30 males/19 females) completed a one-night PSG sleep examination followed by a multiple sleep latency test (MSLT). Sleep parameters, time in bed (TIB), total sleep time (TST), wake after sleep onset (WASO), sleep onset latency (SOL), awake time (WASO + SOL), sleep stages (light, deep, REM sleep) and the number of sleep cycles were compared. Both commercial trackers showed high accuracy in measuring vital data (HR, HRV, SpO2, respiratory rate), r > 0.92. For TIB and TST, all three trackers showed medium to high correlation, r > 0.42. Garmin had significant overestimation of TST, with MAE of 84.63 min and MAPE of 25.32%. Polar also had an overestimation of TST, with MAE of 45.08 min and MAPE of 13.80%. ActiGraph GT9X results were inconspicuous. The trackers significantly underestimated awake times (WASO + SOL) with weak correlation, r = 0.11–0.57. The highest MAE was 50.35 min and the highest MAPE was 83.02% for WASO for Garmin and ActiGraph GT9X; Polar had the highest MAE of 21.17 min and the highest MAPE of 141.61% for SOL. Garmin showed significant deviations for sleep stages (p < 0.045), while Polar only showed significant deviations for sleep cycle (p = 0.000), r < 0.50. Garmin and Polar overestimated light sleep and underestimated deep sleep, Garmin significantly, with MAE up to 64.94 min and MAPE up to 116.50%. Both commercial trackers Garmin and Polar did not detect any daytime sleep at all during the MSLT test. The use of the multisport activity trackers for sleep analysis can only be recommended for general daily use and for research purposes. If precise data on sleep stages and parameters are required, their use is limited. The accuracy of the vital data measurement was adequate. Further studies are needed to evaluate their use for medical purposes, inside and outside of the sleep laboratory. The accelerometer ActiGraph GT9X showed overall suitable accuracy in detecting sleep/wake patterns.

1. Introduction

Options for monitoring daily human activity have been continuously developed by using activity trackers to determine almost all vital and movement parameters during the day and at night. These analyses became possible with relatively few and very small sensors integrated into wristwatch-sized components, such as photodiodes, triaxial accelerometers, Global Navigation Satellite System (GNSS) sensors, and temperature and barometric pressure sensors. Activities, sporting movements, heart rate, rest and even stress can be recognized, measured and analyzed [1,2]. This integration and the use of algorithms for calculation and evaluation developed quickly with cloud-based data evaluation systems. The steadily increasing number of commercial wearable devices [3] and the resulting increase in available and relevant measurement data are having a lasting and increasingly rapid impact on accuracy [4]. Unfortunately, the scientific study of activity trackers has some limitations, as the commercial systems do not fully disclose all analysis methods. Most devices are essentially a “black box” in relation to data acquisition and processing using proprietary algorithms and artificial intelligence [5]. Nevertheless, the ability to record and assess rest periods, such as night sleep, has been greatly improved in recent years by wearables and activity trackers [6].
Human sleep as a reversible physiological state is essential for health and performance in everyday life [7]. Its functions are not fully understood, despite extensive studies on its influence on energy homeostasis, immune function, cognitive performance and behavior, and the influence on various clinical pictures [8,9,10,11]. Sleep can be measured in multiple dimensions such as quantity, continuity, timing and quality [12,13,14,15]. Sensors have been used to study sleep for decades since the early 1950s. In laboratory settings, polysomnography (PSG) paired with clinical evaluation has been the gold standard to study sleep as well as to diagnose a variety of sleep disorders [16]. In recent years, industry and academia have invested heavily in the development of smaller, less obtrusive and more portable devices for the continuous monitoring of sleep, such as accelerometers or different multi-sensor devices such as activity trackers with heart rate and movement sensors [17]. The goal is to enable data collection in larger groups over longer periods of time and in a more natural environment outside the lab, with lower costs and more readily available participants. However, challenges remain with respect to data acquisition and long-term data evaluation, which may mislead the estimations of health markers due to possible interruptions and missing periods [16]. Accelerometers and commercial activity trackers for monitoring sleep have received increased scientific attention in recent years. Accelerometers for actigraphy use have been developed to such an extent that the accuracy of measuring sleep epochs was 87–99% in relation to the gold standard PSG, but with a specification of sleep of only 28% and 67% [18,19,20,21]. The actigraphy tends to overestimate TST and SE and underestimate SOL and WASO in sleep disorders [22,23,24]. It should also be noted that actigraphy can generally only be used to determine sleep/wake patterns. The detection and further analysis of sleep stages (light, deep or REM sleep) is not possible [25,26].
The scientific investigation of commercial low-cost activity trackers started to increase exponentially about 10 years ago [27]. Fitbit Ultra and Jawbone UP were among the first wrist-worn fitness trackers to be scientifically studied, with moderate accuracy scores compared to the PSG measurement [28,29,30]. Various wearable sleep trackers are currently on the market, such as bracelets, smartwatches and wrist-worn activity trackers, headbands, rings, sensor clips and others [27]. These devices are easy to use and ready to purchase in the consumer market. Most of the consumer wrist-worn activity trackers rely on a similar mechanism of clinical actigraphy that infers wake and sleep patterns from limb movement [31,32]. The newly launched models also incorporate, besides the triaxial accelerometer, other streams of bio signals such as photoplethysmographic (PPG) sensors to measure heart rate (HR) and heart rate variability (HRV) for further vital data (respiratory rate and SpO2) and sleep stage analysis [33]. Normal resting HRV is an indication of cardiovascular and autonomic health as well as general fitness. Greater HRV at night has been linked with better sleep quality and reacts to sleep phases [34,35,36]. HRV is also a biofeedback tool for improved relaxation—when a person is under physical or mental stress, parasympathetic activity decreases and sympathetic activity increases, resulting in HR increase and HRV decrease [37,38]. This systematic fact is the basis for determining the respiratory rate, which can be calculated by algorithms depending on the changes in the autonomic nervous system (ANS) using HRV and HF data [39,40,41]. This phenomenon is called respiratory sinus arrhythmia (RSA) [42]. Thus, the respiratory rate is another building block for the assessment of sleep and sleep quality [43]. More recently, it is possible to visualize a sleep hypnogram of a whole night via the evaluation app of the activity tracker manufacturer and to analyze the aggregated sleep parameters such as the total sleep time (TST), wake times and the ratio of the individual sleep stages [44]. The latest validity studies of wrist-worn activity trackers showed clear improvements in the accuracy of the sleep analyses compared to the first studies from 2015 [29,30,44]. However, differences in the degree of accuracy can be seen depending on the brand, the sensors and algorithms used [2,45,46], with accuracies to the gold standard between moderate and good in measuring sleep phases such as TST, WASO, SOL and SE [47].
The analysis of sleep in stages of NREM (light, NREM stages 1 and 2; deep sleep, NREM stage 3) as well as REM sleep is rather underrepresented in the current study situation. Most of the studies show problems with the fragmentation and discrimination of the individual sleep stages during sleep. Liang and Chapa Martell [12] and Liang and Martell [31] showed difficulties of the trackers in the transition phase between the stages of light to REM and deep to awake with a bias of 0–60%, as well as significant deviations in the total time of the measured stages REM and light sleep. It has been found that trackers for sleep patients with sleep disorders are not yet sufficiently developed. Another study showed that the accuracy of determining deep sleep with the tracker was limited, with a medium accuracy of 0.49 [47]. Moreno-Pino et al. [48] found that the tracker recorded significant differences in all sleep stages except in REM. In most studies, light sleep was overestimated and deep sleep was underestimated [12,31,47,48]. Results in the literature show a variable accuracy (moderate to good) of the wrist-worn activity trackers. Most of the research has been carried out on healthy young subjects [6,48,49]. A study of middle-aged to older sleep patients, patients with an average BMI in the obesity range, and patients with sleep disorders or sleep apnea and the isolated usage of nocturnal breathing aids, such as Continuous Positive Airway Pressure (CPAP) devices, can provide an extended view of the accuracy under these conditions.
Only a few sleep validation studies have been conducted with the latest Garmin Forerunner or Fenix models and Polar models with moderate to high accuracy in measuring sleep stages and sleep parameters, with r = 0.60–0.88 for Garmin [2,45,46,50] and r = 0.54–0.90 for Polar [51,52,53,54]. The accelerometer ActiGraph GT9X has been validated several times in the recent past for sleep wake/pattern measurement against PSG with high accuracy of r = 0.85–0.89 [55,56,57,58,59,60], and it is listed as a Food and Drug Administration (FDA)-approved device for clinical use with an AASM recommendation for clinical use in sleep disorders and circadian rhythm sleep–wake disorders [55,56,57,58,59,60].
The aim of this study was to validate sleep stages and parameters as well as vital data measurements of two multisport activity trackers (Garmin Forerunner 945 and Polar Ignite) and one accelerometer (ActiGraph GT9X) against PSG in a laboratory setting. Forty-nine adult sleep patients (30 males/19 females) completed a standardized one-night sleep examination followed by a multiple sleep latency test (MSLT) the following day.

2. Materials and Methods

2.1. Participants

Forty-nine (30 males/19 females) adult sleep laboratory patients volunteered to participate in the study at the sleep laboratory, Somnolab Dortmund (Dortmund, Germany), between 23 April 2020 and 10 December 2020. The participants were consecutively recruited based on the examination combination of one night’s sleep followed by the MSLT the following day. Descriptive characteristics as well as additional information about the skin and activity scales are shown in Table 1. Twelve participants (9 males/3 females) wore a CPAP during their nocturnal sleep examination. The study population was entirely Caucasian. All participants provided written informed consent to participate in this study.
The study was approved by the Medical Association of Westphalia-Lippe, which is connected to the Westfälische-Wilhelms-University Münster, Germany (file number 2019-611-f-S). Furthermore, the Ethics Committee of the German University for Health and Sport (DHGS) checked and approved the study design (reference number 08/2019.1). In addition, the study was included as a medical trial in the World Health Organization (WHO) Primary Register via the German Clinical Trials Register (DRKS) with the reference number DRKS00021701 and conducted according to the guidelines of the Declaration of Helsinki.

2.2. Design and Laboratory Procedures

The study was designed as a laboratory study with various patient complaints relating to their sleep. All participants received a standardized polysomnography to measure sleep over one night and a standardized test assessing daytime sleep propensity, the MSLT, the following day [65,66,67].
On the recording night, participants reported to the laboratory around 2.5 to 3.0 h prior to their typical lights-off time. They underwent in-processing with some vital data measurements such as reaction times, weight, blood pressure and HR, later followed by the application of the PSG electrodes and sensors by the sleep technician. Both multisport activity trackers and the accelerometer were randomly fitted tightly, alternately on the left and right forearm behind the processus styloideus ulnae, according to each device company’s guidelines [68,69]. The beginning and the end of the PSG and activity tracker device data collection periods coincided with the lights-off time and the end of the MSLT. To ensure time synchronization, all devices were connected to the computer or a cellphone app once a day during data upload, using the Network Time Protocol (NTP), which continuously synchronizes to the atomic time of the Physikalisch-Technische Bundesanstalt (Braunschweig, Germany). All recordings were performed in sound-attenuated and temperature-controlled sleeping rooms.

2.3. Devices and Sleep Parameters

Laboratory PSG assessments were performed using Embla Somnologica—PSG unit (Embla Somnologica System, Amsterdam, Netherlands; Resmed Inc., Martinsried, Germany), with data sampled and stored at 256 Hz. Electroencephalographic (EEG) (including F3-M2, F4-M1, C3-M2, C4-M1, O1-M2, O2-M1), electrooculographic (EOG), electromyographic (chin EMG) and two-lead electrocardiographic (ECG) measurements were performed. PSG sleep stages were scored in 30 s epochs by an experienced medical technician according to the standards of the American Academy of Sleep Medicine (AASM) rules [70]. Further vital measurements of oxygen saturation (SpO2), respiration (respiratory rate) and leg movements were collected to recognize potential sleep disorders according to the guidelines of the AASM [70]. The following PSG measurements were calculated: time in bed (TIB, minutes), total sleep time (TST, minutes), sleep efficiency TST/TIB*100 (SE, %), sleep onset latency (SOL, minutes), wake after sleep onset (WASO, minutes) and time spent in N1, N2, N3 and REM sleep (all in min) according to the AASM [70]. The definitions of the sleep parameters, sleep stages and the vital parameter measurements are summarized in Table 2.

2.3.1. Accelerometer ActiGraph GT9X

The accelerometer ActiGraph GT9X (ActiGraph, Pensacola, FL, USA) (serial number TAS1F28170667) is a wrist-worn triaxial accelerometer on a research-grade level. Depending on its version, it can also be worn on the waist or ankle. Accelerometers are able to detect movement by acceleration of the human body using internal sensors such as an inertial measurement unit (IMU) and wear time sensors [68,69]. The measured raw data were downloaded in the lab using ActiLife software (version 6.13.4, ActiGraph, Pensacola, FL, USA). ActiLife software uses the Cole–Kripke sleep algorithm to calculate sleep parameters. In and out of bed times, TIB, TST, WASO, SOL, SE, Awake time and number of awakenings were calculated [68,69]. Due to the lack of appropriate internal sensors, HR, HRV, SpO2 oxygen saturation and respiratory rate could not be determined.

2.3.2. Garmin Forerunner 945

The multisport activity tracker Garmin Forerunner 945 (Garmin Ltd., 2022; Olathe, KS, USA) (ID 3996687672. firmware 5.50, e6bbb98) is able to detect activity and sleep as well as vital data with several sensors, including a triaxial accelerometer, a Global Navigation Satellite System (GNSS) sensor for GPS or GLONASS and a photodiode sensor for photoplethysmographical measurements of HR, HRV, SpO2 and respiratory rate. The Garmin device allows tracking of sleep stages (light, deep and REM sleep time) in addition to the sleep parameters (TIB, TST, Awake, SOL and WASO) [71]. The measured raw data were transmitted to the Garmin Connect cloud via Bluetooth and the Internet and analyzed by Garmin using a proprietary sleep assessment algorithm. The scientific investigator could not influence this process and was blind to it. The sleep stages and sleep parameters as well as a hypnogram were displayed in the Garmin Connect app and stored in the Garmin data cloud. In addition, SpO2 and respiratory rate data were provided. Respiratory rate can be calculated depending on the changes in the autonomic nervous system (ANS) using HRV and HF data [39,40,41]. Unfortunately, HRV and beat to beat data were not directly provided in the spreadsheet report by the Garmin app [71].

2.3.3. Polar Ignite

The Polar Ignite, the second multisport activity tracker (Polar Electro Oy, 2022; Kempele, Finland) (ID 5D935A29, firmware 2.0.25), uses similar sensors to the Garmin device. The tracker is also able to detect sleep stages such as awake time and light, deep and REM sleep time, as well as the normal sleep/wake patterns TIB, TST, Awake, SOL and WASO. For further detailed sleep analysis, the raw data were automatically transferred to the Polar data cloud. Polar uses the “Nightly Recharge System” as a proprietary algorithm. All analyzed data were displayed in the Polar Flow app. In addition to the hypnogram, three further diagrams were displayed, HRV (RMSSD), beat to beat measurement and the respiratory rate data. The respiratory rate was calculated according to the same principle as the Garmin device, explained in Section 1. The measurement detection was restricted to the first four consecutive hours of sleep detection during sleep. In addition, no SpO2 data were calculated or reported in the spreadsheet report of the Polar Flow app [72].

2.3.4. Data Extraction and Scoring

After the MSLT measurement, both multisport activity trackers Garmin and Polar were synchronized with the respective companies’ data cloud app via computer and Internet. After the calculation of sleep parameters, sleep stages and vital data by proprietary algorithms, these data were available as a report for further scientific analysis via the apps Garmin Connect (Garmin Ltd., Olathe, KS, USA, version: 4.37.2.0) and Polar Flow (Polar Electro Oy, Kempele, Finland, version 4.8.0). Garmin and Polar did not provide any raw data due to their company restrictions. It was not possible to extract data directly from the activity trackers. The accelerometer ActiGraph GT9X data were uploaded to the local lab computer software ActiLife with the respective sleep parameter calculation function. ActiGraph GT9X and PSG laboratory software (Somnologica version 3.1), in combination with scoring of an experienced sleep laboratory technician, provided a detailed sleep-related written data report, standardized in accordance with AASM guidelines [72]. All data were extracted from each spreadsheet report and summarized in MS Excel sheets (Microsoft, Redmond, WA, USA, version 2016) for further statistical analysis.

2.4. Statistical Analysis

All statistical analyses were performed using MS Excel 2016 and IBM SPSS (version 26.0 Armonk, NY, USA). The significance level was set at p < 0.05. Descriptive data analyses of each subject’s physical data were performed, and the normal distribution of all data was assessed using the Kolmogorov–Smirnov test. After all t-test analyses, false discovery rate (FDR) corrections were performed to counteract the alpha error summation by multiple testing [73]. The following statistical tests were based on the recommendations for wearable and actigraphy monitoring, validation and assessment selected by [74,75,76,77,78,79]. They were also used in other previous wearable validation studies [28,80,81].
Correlations of sleep parameters and vital data measurements between the gold standard PSG and Garmin, Polar and the accelerometer ActiGraph GT9X results were calculated (two-way random, absolute agreement). Agreement was considered high >0.79, moderate 0.40–0.79 or low <0.40. Difference analyses for sleep parameters and vital data measurements were calculated using the mean absolute error (MAE) and mean absolute percentage error (MAPE) (|[mean difference activity tracker − criterion measurement] × mean criterion measurement −1| × 100). The total deviation from the PSG measurement was calculated and expressed in a boxplot diagram. T-tests followed by FDR correction, one-way ANOVA followed by the Scheffé post hoc test, and effect size calculations in accordance with Cohen [82] were calculated. The interpretation of the effect size was based on Cohen’s d classification: there is a small effect from a value of d = 0.20, a medium effect from d = 0.50 and a large effect from d = 0.80, and Eta2 (ή2) with >0.01 is a small effect, >0.06 a medium effect and >0.14 a large effect [82]. The level of agreement (LoA) was calculated for sleep parameter measurements between the wearable devices’ results and PSG measurements. LoA was assessed as described by Bland–Altman and was expressed using a Bland–Altman diagram [83].
Statistical power (1-ß error probability) was determined post hoc using G*Power (University Düsseldorf, Düsseldorf, Germany) for correlation, t-test and ANOVA analysis [84,85].

3. Results

3.1. Vital Data Measurement

HR measurements (HR, HRV and beat to beat) showed strong correlations with r > 0.92 (p = 0.000) in general between each multisport activity tracker and the PSG-derived two-lead ECG. The difference analysis of all HR measurements showed a small MAE <12.67 beats per minute/MAPE <1.93%. HRV and beat-to-beat measurements could only be calculated for Polar Ignite; Garmin did not present these data even though it is used to calculate sleep data. The Garmin oxygen saturation measurement showed low to moderate correlations with r < 0.52; the SpO2 minimum analysis was particularly noticeable with r = 0.27 (p = 0.057) and MAE of 4.58/MAPE of 5.38%. Statistical power for correlation analysis was determined post hoc with 47–99%. The respiratory rate measurements of both devices were unobtrusive. Paired t-test analysis showed for all vital data measurements no significant differences (p = 0.051–0.992). Further boxplot analysis of all vital data confirmed the calculated results.

3.2. Sleep Parameters

The measurement of the sleep parameters showed different results than the vital data analysis. Overall, the calculated correlations were low to moderate with r = 0.11–0.63 (p = 0.000–0.569). The TST and the awake phases (Awake, WASO and SOL) were particularly noticeable. The accelerometer calculation showed the highest values with r > 0.52 (p < 0.009). The difference analysis of all sleep parameters showed, especially for the awake phases for both activity trackers and the TST for Garmin, high values of MAE up to 84.63 min/MAPE up to 141.61%. The accelerometer again showed lower values for all measurements (Table 3).
The calculated ANOVA values showed a similar picture. Garmin and Polar showed significant differences in all awake phases (p = 0.000) with medium to large effect up to ή2 = 0.429 except SOL on the Polar watch. Garmin also showed significant differences in TST (p = 0.000) with large effect ή2 = 0.281. ActiGraph GT9X was unremarkable in all calculations, with the exception of SOL (p = 0.000) with medium effect ή2 = 0.131. The differences between the multisport activity trackers in these measurements were also significant for TIB, TST, Awake and SOL (p = 0.000) with medium to large effect (Table 4). Statistical power for ANOVA calculations was determined post hoc with 49–100%.
The Bland–Altman and boxplot diagrams show the greater fluctuation range of activity trackers Garmin and Polar for TST, awake parameters and SE (Figure 1 and Figure 2). In particular, the LoA differences between the diagrams showed deviations up to 190 min for Garmin and Polar. The accelerometer analysis was comparatively lower for all measurements. This supports the previous calculations.

3.3. Sleep Stages

The analysis of the sleep stages yielded logical results based on the sleep measurement analysis. Due to device and sensor limitations, the accelerometer ActiGraph GT9X did not calculate any sleep stage. Overall, the differences in sleep stages between the two activity trackers and the PSG are slightly lower than in the sleep parameter analysis. With the exception of deep sleep, higher values were calculated here, with MAE up to 47.33 min/MAPE up to 116.50% for both (Table 5).
The correlation calculation shows similar results, especially with the deep sleep values of r = 0.11–037 (p = 0.008–0.432). With the paired t-test analysis, Garmin showed for all stages significant values (p < 0.045) with medium to large effect sizes up to d = 1.840, except for REM sleep, against the PSG. Polar showed significant results in sleep cycle only (p = 0.000) with large effect size, d = 1.036 (Table 6). Statistical power for t-test calculations was determined post hoc with 42–100%.
The Bland–Altman diagrams and the boxplot analysis show differences between the two activity trackers Garmin and Polar, as well as the larger differences measured in light and deep sleep with Garmin (Figure 3 and Figure 4a,b).

3.4. Hypnogram Analysis

Both commercial activity trackers have problems recognizing the exact distribution in time of the sleep stages. Two hypnogram triads (Garmin, PSG and Polar) were put together as an example (Figure 5a,b). The diagrams show that neither could adequately represent the transitions from light to deep sleep or light sleep to REM sleep and back to the awake state. The exact lengths and actual times of the start and end of the individual stages were mostly not reproduced correctly. Garmin had bigger problems than Polar, especially in detecting and distinguishing between light sleep and awake, as well as REM sleep and light sleep. Both had significant problems recognizing deep sleep. Comparing the rest of the hypnograms, similar results were seen, regardless of the use of CPAP. However, the more restless the patient’s sleep with regard to movement in bed, the number of arousals and the frequent change from awake to light sleep, the more mismatches were recognizable (Figure 5b).

3.5. MSLT Measurement

Unfortunately, the statistical evaluation of the MSLT measurement was not possible because the activity trackers Garmin and Polar did not calculate any data during this time period. Only a visual, schematic, imprecise analysis of the Garmin hypnogram would be possible, since Garmin provides additional visual data via the manual adjustment of the wake-up time function. With Polar, this was not yet possible at the time of the study; currently, it is possible due to further software updates. Due to the lack of data from the multisport activity trackers, it was decided not to further investigate the existing ActiGraph data. The main result is therefore that the multisport activity trackers Garmin and Polar currently do not indicate daytime sleep, although the sensors for it are available.

4. Discussion

The aim of this study was to assess and validate the accuracy of two multisport activity trackers (Garmin and Polar) and one accelerometer (ActiGraph GT9X) in measuring sleep stages, sleep-related measurements and vital parameters of sleep patients in a laboratory setting. As the main findings, the study showed accurate measurement of the vital data, such as HR, HRV, respiratory rate and SpO2, with the exception of the SpO2 minimum measurement by the Garmin device. In contrast, the sleep-related measurements were far less accurate for the two multisport activity trackers Garmin and Polar. Both trackers tending to overestimate TST and underestimate awake (SOL and WASO). In the further analysis of the sleep stages, this resulted in larger deviations of both trackers in light and deep sleep, mainly in the overestimation of light sleep and underestimation of deep sleep, less in REM sleep. Garmin presented a larger and more significant deviation than Polar. The accuracy of the accelerometer to detect sleep/wake patterns was adequate with the only restrictions of the significant deviation and underestimation in SOL.
Sleep stages differ from each other in physiological terms, such as breathing, autonomic nervous system (ANS) reaction and body movement. These behavioral differences and their physiological responses are driven by a coupling between central nervous system activity and ANS activity that can provide the theoretical framework for sleep calculations without EEG-based systems [86]. The examination of the vital data measurement was the logical prerequisite for further validation of the sleep data calculations. Unfortunately, only HRV data were provided by Polar and SpO2 data by Garmin. The measured vital data HR, HRV and the respiratory rate showed high agreement with the PSG for both activity trackers, r > 0.92. The differentiation calculation analysis showed similar results with MAE of 5.82 and MAPE up to 8.72% of Polar HRV, but with a small mean of −1.93 min. The boxplot analysis supports these results and shows the greatest variance in HRV and beat to beat, which could have a negative impact on the calculation of the sleep-related measurements and the calculation of the sequence algorithm in the sleep stages. The less accurate agreement r = 0.27 (p = 0.057) of the SpO2 minimum measurement of the Garmin device has no direct influence on the sleep stage calculation. Recent studies have shown mostly similar results with a high level of agreement between the measured vital data (HR, HRV and respiratory rate) against the PSG and ECG measurements [1,37,87,88,89].
The TIB and TST measurements showed medium to high correlation values with r > 0.42 (p > 0.003). TIB results showed no significant differences, with the highest values of MAE 34.00 min and MAPE of 7.53% for Polar. Garmin generally overestimated the time by 17.27 min, and Polar underestimated it by −25.84 min. In the TST measurement, the differences between the Garmin device and the PSG were significantly noticeable (p = 0.000; d = 0.3). The more detailed difference calculation yielded MAE 84.63 min with MAPE of 25.32% for Garmin, MAE 45.08 min and MAPE of 13.80% for Polar, and MAE of 31.39 min and MAPE of 9.33% for the accelerometer (ActiGraph GT9X). The Bland–Altman and boxplot analysis showed clear overestimations by Garmin with 83.03 min and Polar with 24.03 min, and the accelerometer (ActiGraph GT9X) was negligible. This is in line with the findings of studies with an overestimation of up to 37 min for TST [2,22,23,47,90,91,92].
For the Awake measurements, all devices showed weaker correlation values r = 0.11–0.57 (p = 0.000–0.098). Garmin and the accelerometer (ActiGraph GT9X) showed the highest deviations for the WASO with MAE of 50.35 min and MAPE of up to 83.02%. Polar showed the highest deviations for SOL with MAE of 21.71 min and MAPE of up to 141.61%. Garmin had the highest differences and the accelerometer (ActiGraph GT9X) had differences in SOL only. All results were significantly different (p < 0.010) and with medium to large effect sizes of up to ή2 = 0.429, except the accelerometer (ActiGraph GT9X) for WASO and Polar for SOL. All trackers mostly underestimated the awake times. The large effect size underlines the underestimation of the mentioned awake phases and supports the MAE/MAPE difference analysis. The results for the activity trackers are in agreement with the current study situation with significant underestimation of WASO and SOL [2,20,22,58,90,91,92,93]. The current accelerometer studies differ only slightly in the SOL measurements; the deviations were significantly less than in this study [22,23]. This led to longer TST times with the two multisport activity trackers, since the TIB did not show any major deviations. Furthermore, this affected sleep efficiency, since less awake time leads to higher SE results. However, in this study, the differences were mostly significant and the mean deviations were significantly larger, especially in awake time measurements [91,92].
The sleep phase calculation showed significant differences for the Garmin measurement (p < 0.045) with medium to large effect sizes, d = 0.658–1.840. Polar was not significantly different here. The correlation values for all sleep stages (light, deep, REM sleep) and the sleep cycles were low to moderate (r = 0.02–0.50) for both activity trackers. Garmin had the highest deviation in all three sleep stages, with MAE up to 64.94 min and MAPE up to 116.50%; Polar showed smaller values, with MAE up to 40.14 min and MAPE up to 87% for light and deep sleep. Light sleep was overestimated and deep sleep underestimated by both trackers. The large effect size results for Garmin underline these significant difference results. The results are in line with existing studies by [12,27,33]. Specifically, [48] showed congruent results, with significant differences and overestimation of light sleep and underestimation of deep sleep. Garmin had problems in acquiring and calculating the exact sleep stages with significant deviations. The deviations from Polar, on the other hand, were not quite as large. The overestimation of light sleep is directly related to Garmin’s measured earlier time of falling asleep in combination with the underestimation of awake times. A possible pre-rest phase could have been rated as sleep and thus led to higher TST and light sleep times. Polar showed a slightly larger error range for all stages, which, however, did not lead to tendencies in one direction due to the distribution. The relatively small mean deviations from the PSG can therefore easily be explained by the arithmetic mean.
Neither multisport activity trackers recorded the multiple sleep latency test (MSLT) on the following day after night sleep measurement. Neither tracker recognized daytime sleep times after rise time from bed. The night sleep time window had to be set for both trackers (Garmin and Polar) before use, usually between 10:00 p.m. and 08:00 a.m. Sleep times on the following day past 08:00 a.m. or after wake-up time were not included in the sleep calculation by the tracker. During the study, it was possible to adjust the sleep onset and wake-up time after the night sleep with the Garmin device only. This lengthened the hypnogram and added to the night sleep calculation. There was no defined calculation, delineation or interpretation of sleep stages for the MSLT test at all. No distinction was made in the calculation between night and day sleep, so it was not useable. Polar did not provide any data at all. The accelerometer (ActiGraph GT9X) recorded MSLT data, which were not further statistically evaluated. So far, no current studies on MSLT testing with commercial activity trackers are known.
Possible influence on measurement or calculation inaccuracies can arise from the composition of the sample. Mostly older normal sleep patients (55.01 +/− 10.19 years) with and without sleep disorders and sleep apnea (including twelve CPAP patients) were chosen. The high BMI (30.57 +/− 6.32 kg*m−2) was particularly noticeable. The quantity and quality of sleep can change profoundly across the lifespan [49,94]. Older people commonly have difficulty falling asleep and staying asleep. The sleep architecture and depth may change with ageing. Behavioral changes and daytime napping can impact nocturnal sleep. Older people spend more time in stages N1 and N2 (light sleep). This leads to multiple awakenings during the night, which is described as sleep fragmentation with ageing. With increasing age, the percentage of REM sleep decreases in elderly women; in contrast, the slow-wave sleep significantly decreases in elderly men [95,96,97]. The fragmentation of sleep in the elderly and the longer periods of wakefulness may affect the calculation, since studies involving healthy young subjects had much smaller deviations [33,47] than this study. The assumption is that the sleep algorithm has possible problems with the calculation. It should also be noted that self-learning functions within the sleep algorithms could achieve better measurement results in the long term for vital data such as HR and HRV and also for sleep parameters and sleep stages. Because of the usable number of devices during the study, only two trackers were used once per sleep patient. If each participant uses their own device and wears it for a longer time, a self-learning effect could occur and could lead to measurement improvements [16,98,99].
As a limitation of this study, only two multisport activity trackers and one accelerometer were included. Therefore, a general statement about all wearable activity trackers and accelerometers for actigraphy use is not intended. The sleep data used were extracted directly from the spreadsheet reports of the manufacturers’ apps (Garmin Connect and Polar Flow). Even when asked, Garmin and Polar were not able to provide the raw data. Therefore, it was not known exactly which formulas and algorithms or artificial intelligence were used in the calculation. Furthermore, the epoch-by-epoch analysis of the sleep measurements could possibly vary, since changes in the manufacturers’ cloud-based calculations are not communicated directly to the user. It should be noted that only one night’s sleep with MSLT was conducted in a laboratory setting. With multiple-night studies including also non-laboratory settings, a larger dataset could contribute to an even more accurate validation analysis. Finally, an ad hoc sample was analyzed in this study. Therefore, an a priori sample size estimation via G*Power was not possible. Therefore, the results must be considered with a certain degree of caution. Still, based on the post hoc G*Power analysis, the sample size is sufficient to allow a solid (Power [1-β err prob] 42–100%) evaluation; additionally, comparable studies have used a similar number of subjects (n = 26–56) [22,47,50,56,90].

5. Conclusions

In summary, this study showed high accuracy in measurements of vital data (HR, HRV, SpO2 and respiratory rate) during one night of sleep examination. When measuring the sleep-relevant parameters (TIB, TST, Awake (SOL and WASO) and SE), the multisport activity trackers showed significant differences from the PSG measurement with medium to large effect sizes, with the exception of the TIB measurement. TST was overestimated and awake times underestimated; Garmin deviated significantly more than Polar. The accelerometer ActiGraph GT9X showed significant deviations in the SOL measurement only with medium effect size. In the sleep stage measurement (light, deep, REM sleep), the detection and measurement of time spent in individual sleep stages and states were particularly problematic for both commercial trackers. Again, Garmin deviated more than Polar, with significant differences and medium to large effect sizes by underestimating deep sleep und overestimating light sleep. Neither multisport activity tracker detected or calculated the multiple sleep latency test (MSLT). The accelerometer ActiGraph GT9X confirmed the known accuracies to determine sleep/wake patterns and could be used as a possible reference for further scientific studies under free-living conditions.
The use of the multisport activity trackers (Garmin Forerunner 945 and Polar Ignite) for sleep analysis can only be recommended for everyday use and general purposes to give important feedback to active populations and for research purposes where large sample sizes are needed. If precise data on sleep stages and parameters are required, their use is limited. Further studies are needed to evaluate their use for medical purposes, both inside and outside of the sleep laboratory. Follow-up studies are recommended with the newest devices, including elderly sleep patients with sleep disorders and other physical limitations such as obesity BMI status.

Author Contributions

Conceptualization, M.B. and M.K.; methodology, M.B. and M.K.; validation, M.B. and R.S.; formal analysis, M.B.; investigation, M.B. and R.S.; data curation, M.B.; writing—original draft preparation, M.B.; writing—review and editing, M.K. and R.S.; visualization, M.B.; supervision, R.S.; project administration, M.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of the Medical Association of Westphalia-Lippe and the Westfälische Wilhelms-Universität, Münster, Germany (protocol code: 2019-611-f-S; date: 9 March 2020). In addition, the study was registered in the World Health Organization (WHO) Primary Registry via the German Clinical Trials Registry (DRKS) as a medical trial (reference number: DRKS00021701).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data are not yet publicly available.

Acknowledgments

The authors would like to thank the participants for their participation.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Düking, P.; Giessing, L.; Frenkel, M.O.; Koehler, K.; Holmberg, H.-C.; Sperlich, B. Wrist-Worn Wearables for Monitoring Heart Rate and Energy Expenditure While Sitting or Performing Light-to-Vigorous Physical Activity: Validation Study. JMIR Mhealth Uhealth 2020, 8, e16716. [Google Scholar] [CrossRef]
  2. Miller, D.J.; Roach, G.D.; Lastella, M.; Scanlan, A.T.; Bellenger, C.R.; Halson, S.L.; Sargent, C. A Validation Study of a Commercial Wearable Device to Automatically Detect and Estimate Sleep. Biosensors 2021, 11, 185. [Google Scholar] [CrossRef]
  3. Laricchia, F. Fitness Trackers—Statistics & Facts. Available online: https://www.statista.com/topics/4393/fitness-and-activity-tracker/#topicHeader__wrapper (accessed on 17 November 2022).
  4. Li, X.; Zhang, Y.; Jiang, F.; Zhao, H. A novel machine learning unsupervised algorithm for sleep/wake identification using actigraphy. Chronobiol. Int. 2020, 37, 1002–1015. [Google Scholar] [CrossRef]
  5. Topalidis, P.; Florea, C.; Eigl, E.-S.; Kurapov, A.; Leon, C.A.B.; Schabus, M. Evaluation of a Low-Cost Commercial Actigraph and Its Potential Use in Detecting Cultural Variations in Physical Activity and Sleep. Sensors 2021, 21, 3774. [Google Scholar] [CrossRef] [PubMed]
  6. Ibáñez, V.; Silva, J.; Navarro, E.; Cauli, O. Sleep assessment devices: Types, market analysis, and a critical view on accuracy and validation. Expert Rev. Med. Devices 2019, 16, 1041–1052. [Google Scholar] [CrossRef]
  7. Perez-Pozuelo, I.; Posa, M.; Spathis, D.; Westgate, K.; Wareham, N.; Mascolo, C.; Brage, S.; Palotti, J. Detecting sleep outside the clinic using wearable heart rate devices. Sci. Rep. 2022, 12, 7956. [Google Scholar] [CrossRef]
  8. Benington, J.H.; Craig Heller, H. Restoration of brain energy metabolism as the function of sleep. Prog. Neurobiol. 1995, 45, 347–360. [Google Scholar] [CrossRef] [PubMed]
  9. Schwartz, J.R.L.; Roth, T. Neurophysiology of sleep and wakefulness: Basic science and clinical implications. Curr. Neuropharmacol. 2008, 6, 367–378. [Google Scholar] [CrossRef] [Green Version]
  10. Imeri, L.; Opp, M.R. How (and why) the immune system makes us sleep. Nat. Rev. Neurosci. 2009, 10, 199–210. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. St-Onge, M.-P.; Grandner, M.A.; Brown, D.; Conroy, M.B.; Jean-Louis, G.; Coons, M.; Bhatt, D.L.; American Heart Association Obesity, Behavior Change, Diabetes, and Nutrition Committees of the Council on Lifestyle and Cardiometabolic Health; Council on Cardiovascular Disease in the Young; Council on Clinical Cardiology; et al. Sleep Duration and Quality: Impact on Lifestyle Behaviors and Cardiometabolic Health: A Scientific Statement From the American Heart Association. Circulation 2016, 134, e367–e386. [Google Scholar] [CrossRef]
  12. Liang, Z.; Chapa Martell, M.A. Validity of Consumer Activity Wristbands and Wearable EEG for Measuring Overall Sleep Parameters and Sleep Structure in Free-Living Conditions. J. Healthc. Inf. Res. 2018, 2, 152–178. [Google Scholar] [CrossRef] [PubMed]
  13. Natale, V.; Léger, D.; Martoni, M.; Bayon, V.; Erbacci, A. The role of actigraphy in the assessment of primary insomnia: A retrospective study. Sleep Med. 2014, 15, 111–115. [Google Scholar] [CrossRef] [PubMed]
  14. Buysse, D.J. Sleep health: Can we define it? Does it matter? Sleep 2014, 37, 9–17. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Carskadon, M.A.; Dement, W.C. Normal Human Sleep. Princ. Pract. Sleep Med. 2011, 4, 16–26. [Google Scholar] [CrossRef]
  16. Perez-Pozuelo, I.; Zhai, B.; Palotti, J.; Mall, R.; Aupetit, M.; Garcia-Gomez, J.M.; Taheri, S.; Guan, Y.; Fernandez-Luque, L. The future of sleep health: A data-driven revolution in sleep science and medicine. NPJ Digit. Med. 2020, 3, 42. [Google Scholar] [CrossRef] [Green Version]
  17. Hao, Y.; Foster, R. Wireless body sensor networks for health-monitoring applications. Physiol. Meas. 2008, 29, R27–R56. [Google Scholar] [CrossRef] [PubMed]
  18. Latshang, T.D.; Mueller, D.J.; Lo Cascio, C.M.; Stöwhas, A.-C.; Stadelmann, K.; Tesler, N.; Achermann, P.; Huber, R.; Kohler, M.; Bloch, K.E. Actigraphy of Wrist and Ankle for Measuring Sleep Duration in Altitude Travelers. High Alt. Med. Amp. Biol. 2016, 17, 194–202. [Google Scholar] [CrossRef] [PubMed]
  19. Sargent, C.; Lastella, M.; Halson, S.L.; Roach, G.D. The validity of activity monitors for measuring sleep in elite athletes. J. Sci. Med. Sport 2016, 19, 848–853. [Google Scholar] [CrossRef]
  20. Haghayegh, S.; Khoshnevis, S.; Smolensky, M.H.; Diller, K.R.; Castriotta, R.J. Performance assessment of new-generation Fitbit technology in deriving sleep parameters and stages. Chronobiol. Int. 2019, 37, 47–59. [Google Scholar] [CrossRef]
  21. Van De Water, A.T.M.; Holmes, A.; Hurley, D.A. Objective measurements of sleep for non-laboratory settings as alternatives to polysomnography—A systematic review. J. Sleep Res. 2011, 20, 183–200. [Google Scholar] [CrossRef]
  22. Hamill, K.; Jumabhoy, R.; Kahawage, P.; Zambotti, M.; Walters, E.M.; Drummond, S.P.A. Validity, potential clinical utility and comparison of a consumer activity tracker and a research—Grade activity tracker in insomnia disorder II: Outside the laboratory. J. Sleep Res. 2019, 29, e12944. [Google Scholar] [CrossRef] [PubMed]
  23. Marino, M.; Li, Y.; Rueschman, M.N.; Winkelman, J.W.; Ellenbogen, J.M.; Solet, J.M.; Dulin, H.; Berkman, L.F.; Buxton, O.M. Measuring sleep: Accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep 2013, 36, 1747–1755. [Google Scholar] [CrossRef] [PubMed]
  24. Montgomery-Downs, H.E.; Insana, S.P.; Bond, J.A. Movement toward a novel activity monitoring device. Sleep Breath. 2011, 16, 913–917. [Google Scholar] [CrossRef] [PubMed]
  25. Beattie, Z.; Oyang, Y.; Statan, A.; Ghoreyshi, A.; Pantelopoulos, A.; Russell, A.; Heneghan, C. Estimation of sleep stages in a healthy adult population from optical plethysmography and accelerometer signals. Physiol. Meas. 2017, 38, 1968–1979. [Google Scholar] [CrossRef]
  26. Ancoli-Israel, S.; Cole, R.; Alessi, C.; Chambers, M.; Moorcroft, W.; Pollak, C.P. The Role of Actigraphy in the Study of Sleep and Circadian Rhythms. Sleep 2003, 26, 342–392. [Google Scholar] [CrossRef] [Green Version]
  27. de Zambotti, M.; Cellini, N.; Goldstone, A.; Colrain, I.M.; Baker, F.C. Wearable Sleep Technology in Clinical and Research Settings. Med. Sci. Sport Exerc. 2019, 51, 1538–1557. [Google Scholar] [CrossRef]
  28. de Zambotti, M.; Baker, F.C.; Colrain, I.M. Validation of Sleep-Tracking Technology Compared with Polysomnography in Adolescents. Sleep 2015, 38, 1461–1468. [Google Scholar] [CrossRef]
  29. de Zambotti, M.; Claudatos, S.; Inkelis, S.; Colrain, I.M.; Baker, F.C. Evaluation of a consumer fitness-tracking device to assess sleep in adults. Chronobiol. Int. 2015, 32, 1024–1028. [Google Scholar] [CrossRef] [Green Version]
  30. Meltzer, L.J.; Hiruma, L.S.; Avis, K.; Montgomery-Downs, H.; Valentin, J. Comparison of a Commercial Accelerometer with Polysomnography and Actigraphy in Children and Adolescents. Sleep 2015, 38, 1323–1330. [Google Scholar] [CrossRef] [Green Version]
  31. Liang, Z.; Martell, M.A.C. Measurement Accuracy of Consumer Sleep Tracking Wristbands is Associated to User-Specific Factors. J. Phys. Fit. Sports Med. Misc. 2019, 8, 388–394. [Google Scholar] [CrossRef]
  32. Kolla, B.P.; Mansukhani, S.; Mansukhani, M.P. Consumer sleep tracking devices: A review of mechanisms, validity and utility. Expert Rev. Med. Devices 2016, 13, 497–506. [Google Scholar] [CrossRef] [PubMed]
  33. Liang, Z.; Chapa-Martell, M.A. Accuracy of Fitbit Wristbands in Measuring Sleep Stage Transitions and the Effect of User-Specific Factors. JMIR Mhealth Uhealth 2019, 7, e13384. [Google Scholar] [CrossRef] [PubMed]
  34. Stein, P.K.; Pu, Y. Heart rate variability, sleep and sleep disorders. Sleep Med. Rev. 2012, 16, 47–66. [Google Scholar] [CrossRef] [PubMed]
  35. Xiao, M.; Yan, H.; Song, J.; Yang, Y.; Yang, X. Sleep stages classification based on heart rate variability and random forest. Biomed. Signal Process. Control 2013, 8, 624–633. [Google Scholar] [CrossRef]
  36. de Zambotti, M.; Willoughby, A.R.; Franzen, P.L.; Clark, D.B.; Baker, F.C.; Colrain, I.M. K-Complexes: Interaction between the Central and Autonomic Nervous Systems during Sleep. Sleep 2016, 39, 1129–1137. [Google Scholar] [CrossRef] [Green Version]
  37. Kinnunen, H.; Rantanen, A.; Kenttä, T.; Koskimäki, H. Feasible assessment of recovery and cardiovascular health: Accuracy of nocturnal HR and HRV assessed via ring PPG in comparison to medical grade ECG. Physiol. Meas. 2020, 41, 4NT01. [Google Scholar] [CrossRef]
  38. Kiviniemi, A.M.; Hautala, A.J.; Kinnunen, H.; NissilÄ, J.; Virtanen, P.; Karjalainen, J.; Tulppo, M.P. Daily Exercise Prescription on the Basis of HR Variability among Men and Women. Med. Amp. Sci. Sport Amp. Exerc. 2010, 42, 1355–1363. [Google Scholar] [CrossRef]
  39. Badra, L.J.; Cooke, W.H.; Hoag, J.B.; Crossman, A.A.; Kuusela, T.A.; Tahvanainen, K.U.O.; Eckberg, D.L. Respiratory modulation of human autonomic rhythms. Am. J. Physiol.-Heart Circ. Physiol. 2001, 280, H2674–H2688. [Google Scholar] [CrossRef] [Green Version]
  40. Eckberg, D.L. The human respiratory gate. J. Physiol. 2003, 548, 339–352. [Google Scholar] [CrossRef]
  41. Berryhill, S.; Morton, C.J.; Dean, A.; Berryhill, A.; Provencio-Dean, N.; Patel, S.I.; Estep, L.; Combs, D.; Mashaqi, S.; Gerald, L.B.; et al. Effect of wearables on sleep in healthy individuals: A randomized crossover trial and validation study. J. Clin. Sleep Med. 2020, 16, 775–783. [Google Scholar] [CrossRef]
  42. Yasuma, F.; Hayano, J.-I. Respiratory Sinus Arrhythmia: Why Does the Heartbeat Synchronize With Respiratory Rhythm? Chest 2004, 125, 683–690. [Google Scholar] [CrossRef] [PubMed]
  43. Douglas, N.J.; White, D.P.; Pickett, C.K.; Weil, J.V.; Zwillich, C.W. Respiration during sleep in normal man. Thorax 1982, 37, 840–844. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Duncan, M.; Murawski, B.; Short, C.E.; Rebar, A.L.; Schoeppe, S.; Alley, S.; Vandelanotte, C.; Kirwan, M. Activity Trackers Implement Different Behavior Change Techniques for Activity, Sleep, and Sedentary Behaviors. Interact. J. Med. Res. 2017, 6, e13. [Google Scholar] [CrossRef] [PubMed]
  45. Chinoy, E.D.; Cuellar, J.A.; Huwa, K.E.; Jameson, J.T.; Watson, C.H.; Bessman, S.C.; Hirsch, D.A.; Cooper, A.D.; Drummond, S.P.A.; Markwald, R.R. Performance of seven consumer sleep-tracking devices compared with polysomnography. Sleep 2021, 44, zsaa291. [Google Scholar] [CrossRef] [PubMed]
  46. Tedesco, S.; Sica, M.; Ancillao, A.; Timmons, S.; Barton, J.; O’Flynn, B. Validity Evaluation of the Fitbit Charge2 and the Garmin vivosmart HR+ in Free-Living Environments in an Older Adult Cohort. JMIR Mhealth Uhealth 2019, 7, e13084. [Google Scholar] [CrossRef] [PubMed]
  47. de Zambotti, M.; Goldstone, A.; Claudatos, S.; Colrain, I.M.; Baker, F.C. A validation study of Fitbit Charge 2™ compared with polysomnography in adults. Chronobiol. Int. 2018, 35, 465–476. [Google Scholar] [CrossRef]
  48. Moreno-Pino, F.; Porras-Segovia, A.; López-Esteban, P.; Artés, A.; Baca-García, E. Validation of Fitbit Charge 2 and Fitbit Alta HR Against Polysomnography for Assessing Sleep in Adults With Obstructive Sleep Apnea. J. Clin. Sleep Med. 2019, 15, 1645–1653. [Google Scholar] [CrossRef]
  49. Giurgiu, M.; Timm, I.; Becker, M.; Schmidt, S.; Wunsch, K.; Nissen, R.; Davidovski, D.; Bussmann, J.B.J.; Nigg, C.R.; Reichert, M.; et al. Quality Evaluation of Free-living Validation Studies for the Assessment of 24-Hour Physical Behavior in Adults via Wearables: Systematic Review. JMIR Mhealth Uhealth 2022, 10, e36377. [Google Scholar] [CrossRef]
  50. Miller, D.J.; Sargent, C.; Roach, G.D. A Validation of Six Wearable Devices for Estimating Sleep, Heart Rate and Heart Rate Variability in Healthy Adults. Sensors 2022, 22, 6317. [Google Scholar] [CrossRef]
  51. Klier, K.; Wagner, M. Agreement of Sleep Measures-A Comparison between a Sleep Diary and Three Consumer Wearable Devices. Sensors 2022, 22, 6189. [Google Scholar] [CrossRef]
  52. Henriksen, A.; Svartdal, F.; Grimsgaard, S.; Hartvigsen, G.; Hopstock, L.A. Polar Vantage and Oura Physical Activity and Sleep Trackers: Validation and Comparison Study. JMIR Form. Res. 2022, 6, e27248. [Google Scholar] [CrossRef] [PubMed]
  53. Kubala, A.G.; Barone Gibbs, B.; Buysse, D.J.; Patel, S.R.; Hall, M.H.; Kline, C.E. Field-based Measurement of Sleep: Agreement between Six Commercial Activity Monitors and a Validated Accelerometer. Behav. Sleep Med. 2020, 18, 637–652. [Google Scholar] [CrossRef] [PubMed]
  54. Pesonen, A.-K.; Kuula, L. The Validity of a New Consumer-Targeted Wrist Device in Sleep Measurement: An Overnight Comparison Against Polysomnography in Children and Adolescents. J. Clin. Sleep Med. 2018, 14, 585–591. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  55. Cole, R.J.; Kripke, D.F.; Gruen, W.; Mullaney, D.J.; Gillin, J.C. Automatic Sleep/Wake Identification From Wrist Activity. Sleep 1992, 15, 461–469. [Google Scholar] [CrossRef]
  56. Burkart, S.; Beets, M.W.; Armstrong, B.; Hunt, E.T.; Dugger, R.; von Klinggraeff, L.; Jones, A.; Brown, D.E.; Weaver, R.G. Comparison of multichannel and single-channel wrist-based devices with polysomnography to measure sleep in children and adolescents. J. Clin. Sleep Med. 2021, 17, 645–652. [Google Scholar] [CrossRef] [PubMed]
  57. Smith, M.T.; McCrae, C.S.; Cheung, J.; Martin, J.L.; Harrod, C.G.; Heald, J.L.; Carden, K.A. Use of Actigraphy for the Evaluation of Sleep Disorders and Circadian Rhythm Sleep-Wake Disorders: An American Academy of Sleep Medicine Clinical Practice Guideline. J. Clin. Sleep Med. 2018, 14, 1231–1237. [Google Scholar] [CrossRef] [Green Version]
  58. Lee, J.-M.; Byun, W.; Keill, A.; Dinkel, D.; Seo, Y. Comparison of Wearable Trackers’ Ability to Estimate Sleep. Int. J. Environ. Res. Public Health 2018, 15, 1265. [Google Scholar] [CrossRef] [Green Version]
  59. Full, K.M.; Kerr, J.; Grandner, M.A.; Malhotra, A.; Moran, K.; Godoble, S.; Natarajan, L.; Soler, X. Validation of a physical activity accelerometer device worn on the hip and wrist against polysomnography. Sleep Health 2018, 4, 209–216. [Google Scholar] [CrossRef]
  60. Wan, T.; Luo, Q. Literature Review of the Application of Wearable Device GT3X in Monitoring Physical Activity. Adv. Intell. Syst. Comput. 2019, 1018, 569–575. [Google Scholar] [CrossRef]
  61. WHO Expert Consultation. Appropriate body-mass index for Asian populations and its implications for policy and intervention strategies. Lancet 2004, 363, 157–163. [Google Scholar] [CrossRef]
  62. Fitzpatrick, T.B. The validity and practicality of sun-reactive skin types I through VI. Arch. Dermatol. 1988, 124, 869–871. [Google Scholar] [CrossRef] [PubMed]
  63. Ross, R.M.; Jackson, A.S. Exercise Concepts, Calculations, and Computer Applications; Benchmark Press: Carmel, CA, USA, 1990. [Google Scholar]
  64. Wier, L.T.; Ayers, G.W.; Jackson, A.S.; Rossum, A.C.; Carlos Poston, W.S.; Foreyt, J.P. Determining the amount of physical activity needed for long-term weight control. Int. J. Obes. 2001, 25, 613–621. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  65. Littner, M.R.; Kushida, C.; Wise, M.; Davila, D.G.; Morgenthaler, T.; Lee-Chiong, T.; Hirshkowitz, M.; Loube, D.L.; Bailey, D.; Berry, R.B.; et al. Practice Parameters for Clinical Use of the Multiple Sleep Latency Test and the Maintenance of Wakefulness Test. Sleep 2005, 28, 113–121. [Google Scholar] [CrossRef] [Green Version]
  66. Stuck, B.A.; Maurer, J.T.; Schlarb, A.A.; Schredl, M.; Weeß, H.-G. Praxis der Schlafmedizin; Springer: Berlin/Heidelberg, Germany, 2018. [Google Scholar] [CrossRef]
  67. Peter, H.; Penzel, T.; Peter, J.H. Enzyklopädie der Schlafmedizin; Springer: Berlin/Heidelberg, Germany, 2007. [Google Scholar] [CrossRef]
  68. ActiGraph, L. User Guide, ActiGraph GT9X Link + ActiLife. Available online: https://s3.amazonaws.com/actigraphcorp.com/wp-content/uploads/2020/03/05155628/ActiGraph_Link_UserGuide_E.200.6001_Revision6_FINAL.pdf (accessed on 20 September 2022).
  69. Madrid-Navarro, C.J.; Puertas Cuesta, F.J.; Escamilla-Sevilla, F.; Campos, M.; Ruiz Abellán, F.; Rol, M.A.; Madrid, J.A. Validation of a Device for the Ambulatory Monitoring of Sleep Patterns: A Pilot Study on Parkinson’s Disease. Front. Neurol. 2019, 10, 356. [Google Scholar] [CrossRef] [PubMed]
  70. Iber, C.; Ancoli-Israel, S.; Chesson, A.L.; Quan, S. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications; American Academy of Sleep Medicine: Westchester, NY, USA, 2007. [Google Scholar]
  71. Garmin. Forerunner 945 Owner’s Manual. Available online: https://www8.garmin.com/manuals/webhelp/forerunner945/EN-US/Forerunner_945M_OM (accessed on 20 September 2022).
  72. Polar, E.O. Polar Ignite User Manual. Available online: https://support.polar.com/e_manuals/ignite/polar-ignite-user-manual-english/manual.pdf (accessed on 20 September 2022).
  73. Benjamini, Y.; Hochberg, Y. Controlling the False Discovery Rate—A Practical and Powerful Approach to Multiple Testing. J. Royal Stat. Soc. Series B (Methodol.) 1995, 57, 289–300. [Google Scholar] [CrossRef]
  74. Ancoli-Israel, S.; Martin, J.L.; Blackwell, T.; Buenaver, L.; Liu, L.; Meltzer, L.J.; Sadeh, A.; Spira, A.P.; Taylor, D.J. The SBSM Guide to Actigraphy Monitoring: Clinical and Research Applications. Behav. Sleep Med. 2015, 13, S4–S38. [Google Scholar] [CrossRef]
  75. Menghini, L.; Cellini, N.; Goldstone, A.; Baker, F.C.; de Zambotti, M. A standardized framework for testing the performance of sleep-tracking technology: Step-by-step guidelines and open-source code. Sleep 2021, 44, zsaa170. [Google Scholar] [CrossRef] [PubMed]
  76. Grandner, M.A.; Rosenberger, M.E. Chapter 12-Actigraphic sleep tracking and wearables: Historical context, scientific applications and guidelines, limitations, and considerations for commercial sleep devices. In Sleep and Health; Grandner, M.A., Ed.; Academic Press: Cambridge, MA, USA, 2019; pp. 147–157. [Google Scholar] [CrossRef]
  77. Mühlen, J.M.; Stang, J.; Lykke Skovgaard, E.; Judice, P.B.; Molina-Garcia, P.; Johnston, W.; Sardinha, L.B.; Ortega, F.B.; Caulfield, B.; Bloch, W.; et al. Recommendations for determining the validity of consumer wearable heart rate devices: Expert statement and checklist of the Interlive Network. Br. J. Sport Med. 2021, 55, 767–779. [Google Scholar] [CrossRef]
  78. Düking, P.; Fuss, F.K.; Holmberg, H.-C.; Sperlich, B. Recommendations for Assessment of the Reliability, Sensitivity, and Validity of Data Provided by Wearable Sensors Designed for Monitoring Physical Activity. JMIR Mhealth Uhealth 2018, 6, e102. [Google Scholar] [CrossRef]
  79. Nelson, B.W.; Low, C.A.; Jacobson, N.; Areán, P.; Torous, J.; Allen, N.B. Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research. NPJ Digit. Med. 2020, 3, 90. [Google Scholar] [CrossRef]
  80. Shcherbina, A.; Mattsson, C.M.; Waggott, D.; Salisbury, H.; Christle, J.W.; Hastie, T.; Wheeler, M.T.; Ashley, E.A. Accuracy in Wrist-Worn, Sensor-Based Measurements of Heart Rate and Energy Expenditure in a Diverse Cohort. J. Pers. Med. 2017, 7, 3. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  81. Gilgen-Ammann, R.; Schweizer, T.; Wyss, T. RR interval signal quality of a heart rate monitor and an ECG Holter at rest and during exercise. Eur. J. Appl. Physiol. 2019, 119, 1525–1532. [Google Scholar] [CrossRef] [PubMed]
  82. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Erlbaum Associates: Hillsdale, Australia, 1988; p. 567. [Google Scholar]
  83. Bland, J.M.; Altman, D.G. Measuring agreement in method comparison studies. Stat. Methods Med. Res. 1999, 8, 135–160. [Google Scholar] [CrossRef] [PubMed]
  84. Faul, F.; Erdfelder, E.; Buchner, A.; Lang, A.-G. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods 2009, 41, 1149–1160. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  85. Faul, F.; Erdfelder, E.; Lang, A.-G.; Buchner, A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 2007, 39, 175–191. [Google Scholar] [CrossRef]
  86. Altini, M.; Kinnunen, H. The Promise of Sleep: A Multi-Sensor Approach for Accurate Sleep Stage Detection Using the Oura Ring. Sensors 2021, 21, 4302. [Google Scholar] [CrossRef]
  87. Nuuttila, O.-P.; Korhonen, E.; Laukkanen, J.; Kyröläinen, H. Validity of the Wrist-Worn Polar Vantage V2 to Measure Heart Rate and Heart Rate Variability at Rest. Sensors 2021, 22, 137. [Google Scholar] [CrossRef]
  88. Budig, M.; Keiner, M.; Stoohs, R.; Hoffmeister, M.; Höltke, V. Heart Rate and Distance Measurement of Two Multisport Activity Trackers and a Cellphone App in Different Sports: A Cross-Sectional Validation and Comparison Field Study. Sensors 2021, 22, 180. [Google Scholar] [CrossRef]
  89. Charlton, P.H.; Birrenkott, D.A.; Bonnici, T.; Pimentel, M.A.F.; Johnson, A.E.W.; Alastruey, J.; Tarassenko, L.; Watkinson, P.J.; Beale, R.; Clifton, D.A. Breathing Rate Estimation From the Electrocardiogram and Photoplethysmogram: A Review. IEEE Rev. Biomed. Eng. 2018, 11, 2–20. [Google Scholar] [CrossRef] [Green Version]
  90. Cheung, J.; Leary, E.B.; Lu, H.; Zeitzer, J.M.; Mignot, E. PSG Validation of minute-to-minute scoring for sleep and wake periods in a consumer wearable device. PLoS ONE 2020, 15, e0238464. [Google Scholar] [CrossRef]
  91. Svensson, T.; Chung, U.-I.; Tokuno, S.; Nakamura, M.; Svensson, A.K. A validation study of a consumer wearable sleep tracker compared to a portable EEG system in naturalistic conditions. J. Psychosom. Res. 2019, 126, 109822. [Google Scholar] [CrossRef] [PubMed]
  92. Roomkham, S.; Hittle, M.; Cheung, J.; Lovell, D.; Mignot, E.; Perrin, D. Sleep monitoring with the Apple Watch: Comparison to a clinically validated actigraph. F1000Research 2019, 8, 754. [Google Scholar] [CrossRef] [Green Version]
  93. Ameen, M.S.; Cheung, L.M.; Hauser, T.; Hahn, M.A.; Schabus, M. About the Accuracy and Problems of Consumer Devices in the Assessment of Sleep. Sensors 2019, 19, 4160. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  94. Gulia, K.K.; Kumar, V.M. Sleep disorders in the elderly: A growing challenge. Psychogeriatrics 2018, 18, 155–165. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  95. Redline, S.; Kirchner, H.L.; Quan, S.F.; Gottlieb, D.J.; Kapur, V.; Newman, A. The Effects of Age, Sex, Ethnicity, and Sleep-Disordered Breathing on Sleep Architecture. Arch. Intern. Med. 2004, 164, 406. [Google Scholar] [CrossRef]
  96. Cooke, J.R.; Ancoli-Israel, S. Normal and abnormal sleep in the elderly. Handb. Clin. Neurol. 2011, 98, 653–665. [Google Scholar] [CrossRef] [Green Version]
  97. Moraes, W.; Piovezan, R.; Poyares, D.; Bittencourt, L.R.; Santos-Silva, R.; Tufik, S. Effects of aging on sleep structure throughout adulthood: A population-based study. Sleep Med. 2014, 15, 401–409. [Google Scholar] [CrossRef]
  98. Liang, Z.; Martell, M.A.C. Considering interpersonal differences in validating wearable sleep-tracking technologies. In Proceedings of the 2017 Tenth International Conference on Mobile Computing and Ubiquitous Network (ICMU), Toyama, Japan, 3–5 October 2017. [Google Scholar]
  99. Jeon, S.; Park, T.; Paul, A.; Lee, Y.-S.; Son, S.H. A Wearable Sleep Position Tracking System Based on Dynamic State Transition Framework. IEEE Access 2019, 7, 135742–135756. [Google Scholar] [CrossRef]
Sensors 22 09540 g001 550
Figure 1. Bland–Altman plots of sleep parameters (n = 49). TIB = time in bed, TST = total sleep time, Awake = awake time (WASO + SOL), WASO = wake after sleep onset, SOL = sleep onset latency (all expressed in minutes), SE = sleep efficiency (in %). x-axis represents the mean values of the device and PSG; y-axis represents the differences between the PSG and the device; dashed black line represents the upper and lower limit of agreement (mean +/− 1.96 SD); solid red line represents the mean value of difference; solid blue line represents the trend; shaded green area represents 95% CI (confidence interval) of mean difference.
Figure 1. Bland–Altman plots of sleep parameters (n = 49). TIB = time in bed, TST = total sleep time, Awake = awake time (WASO + SOL), WASO = wake after sleep onset, SOL = sleep onset latency (all expressed in minutes), SE = sleep efficiency (in %). x-axis represents the mean values of the device and PSG; y-axis represents the differences between the PSG and the device; dashed black line represents the upper and lower limit of agreement (mean +/− 1.96 SD); solid red line represents the mean value of difference; solid blue line represents the trend; shaded green area represents 95% CI (confidence interval) of mean difference.
Sensors 22 09540 g001
Sensors 22 09540 g002 550
Figure 2. Boxplot analysis of calculated deviation in sleep parameters, Garmin, Polar and ActiGraph GT9X against the gold standard PSG (n = 49), TIB = time in bed, TST = total sleep time, Awake = awake time (WASO + SOL), WASO = wake after sleep onset, SOL = sleep onset latency (all expressed in minutes); the x represents the mean value of deviation.
Figure 2. Boxplot analysis of calculated deviation in sleep parameters, Garmin, Polar and ActiGraph GT9X against the gold standard PSG (n = 49), TIB = time in bed, TST = total sleep time, Awake = awake time (WASO + SOL), WASO = wake after sleep onset, SOL = sleep onset latency (all expressed in minutes); the x represents the mean value of deviation.
Sensors 22 09540 g002
Sensors 22 09540 g003 550
Figure 3. Bland–Altman plots of sleep stages (n = 49). NREM, light sleep (NREM1 + NREM2), deep sleep (SWS, NREM3) and REM sleep = rapid eye movement sleep (all expressed in minutes). x-axis represents the mean values of the device and PSG; y-axis represents the differences between the PSG and the device; dashed black line represents the upper and lower limit of agreement (mean +/− 1.96 SD); solid red line represents the mean value of difference; solid blue line represents the trend line; shaded green area represents 95% CI (confidence interval) of mean difference.
Figure 3. Bland–Altman plots of sleep stages (n = 49). NREM, light sleep (NREM1 + NREM2), deep sleep (SWS, NREM3) and REM sleep = rapid eye movement sleep (all expressed in minutes). x-axis represents the mean values of the device and PSG; y-axis represents the differences between the PSG and the device; dashed black line represents the upper and lower limit of agreement (mean +/− 1.96 SD); solid red line represents the mean value of difference; solid blue line represents the trend line; shaded green area represents 95% CI (confidence interval) of mean difference.
Sensors 22 09540 g003
Sensors 22 09540 g004 550
Figure 4. Boxplot analysis of calculated deviation (a) in sleep stages, Garmin and Polar against the gold standard PSG (n = 49), Light = light sleep, Deep = slow-wave sleep (SWS), REM = rapid eye movement sleep (all expressed in minutes) and (b) in sleep onset time = start of sleep and wake-up time = end of sleep (deviation expressed in minutes), Garmin, Polar and accelerometer (ActiGraph GT9X) against the gold standard PSG (n = 49); the x represents the mean value of deviation.
Figure 4. Boxplot analysis of calculated deviation (a) in sleep stages, Garmin and Polar against the gold standard PSG (n = 49), Light = light sleep, Deep = slow-wave sleep (SWS), REM = rapid eye movement sleep (all expressed in minutes) and (b) in sleep onset time = start of sleep and wake-up time = end of sleep (deviation expressed in minutes), Garmin, Polar and accelerometer (ActiGraph GT9X) against the gold standard PSG (n = 49); the x represents the mean value of deviation.
Sensors 22 09540 g004
Sensors 22 09540 g005 550
Figure 5. Night sleep hypnograms (Garmin, PSG, Polar); (a) CPAP patient (male); (b) sleep patient (female). Arousal = partial, temporary or complete wake-up reaction with sleep-disrupting effect [66]; MT = movement time; Wake = awake time; REM = rapid eye movement sleep; S1–S3 represent NREM1–NREM3 (N1 + 2 = light sleep; N3 = deep sleep [70]). x-axis represents the time in hours; y-axis represents the respective sleep stage.
Figure 5. Night sleep hypnograms (Garmin, PSG, Polar); (a) CPAP patient (male); (b) sleep patient (female). Arousal = partial, temporary or complete wake-up reaction with sleep-disrupting effect [66]; MT = movement time; Wake = awake time; REM = rapid eye movement sleep; S1–S3 represent NREM1–NREM3 (N1 + 2 = light sleep; N3 = deep sleep [70]). x-axis represents the time in hours; y-axis represents the respective sleep stage.
Sensors 22 09540 g005
Table
Table 1. Descriptive characteristics of participants (n = 49).
Table 1. Descriptive characteristics of participants (n = 49).
Descriptive Characteristic Male (n = 30)Female (n = 19)Total (n = 49)
Age (year)Mean +/− SD52.10 +/− 9.6359.58 +/− 9.5655.01 +/− 10.19
Range27–6637–7427–74
Height (meter)Mean +/− SD182.7 +/− 6.38165.8 +/− 5.50176.2 +/− 10.24
Range171–198158–178158–198
Weight (Kg)Mean +/− SD98.10 +/− 13.7789.37 +/− 25.4094.71 +/− 19.36
Range75.20–125.0060.60–150.4060.60–150.40
BMI (kg × m−2)Mean +/− SD29.43 +/− 4.2332.37 +/− 8.4930.57 +/− 6.32
Range21.63–41.0422.61–52.0421.36–52.04
Underweight (BMI < 18.5) 000
Normal
(BMI 18.5–24.9)		 448
Pre-obesity
(BMI 25.0–29,9)		 11516
Obesity
(BMI > 29.9)		 151025
Skin color typeMean +/− SD2.60 +/− 0.622.37 +/− 0.762.51 +/− 0.68
Range1.0–4.01.0–4.01.0–4.0
Activity levelMean +/− SD4.9 +/− 2.594.63 +/− 2.714.80 +/− 2.61
Range0.0–9.01.0–9.00.0–9.0
BMI = Body Mass Index, scale acc. [61], skin color type = Fitzpatrick scale with 6 stages [62], activity level = classification according to Ross and Jackson extended by the NASA scale with 10 stages [63,64].
Table
Table 2. Definitions of sleep parameters, sleep stages and vital parameters.
Table 2. Definitions of sleep parameters, sleep stages and vital parameters.
Sleep ParametersDefinition
TIBTime in bed in minutes from lights off to lights on.
TSTTotal sleep time in minutes of NREM (N1, N2 and N3) + REM from lights off to lights on in the morning.
AwakeTotal time in minutes in absence of any NREM stage and REM from lights off to lights on (WASO + SOL).
WASOTime in minutes of wakefulness after sleep onset.
SOLTime in minutes from lights off to the first epoch of sleep scored.
SEPercentage of sleep time from lights off to lights on (TST/TIB × 100).
Sleep cycleOne sleep cycle consists of an individual sequence of NREM and REM sleep, expressed in total numbers.
Sleep StagesDefinition
Light sleepSleep stages NREM1 + NREM2, expressed in minutes.
Deep sleepSleep stage NREM3 (also termed slow-wave sleep; SWS), expressed in minutes.
REM sleepCharacterized by rapid eye movements and EMG decrement, expressed in minutes.
Vital ParametersDefinition
HRHeart rate, measured in beats per minute.
HRVHeart rate variability, RMSSD (root mean square of successive differences) measured in 5 min intervals, expressed in milliseconds.
Beat to beatAverage delta of R–R intervals in milliseconds.
SpO2Measurement of arterial oxygen saturation in percent %.
Respiratory rateDefined as respiratory cycles per minute.
Table
Table 3. Sleep parameters deviation calculation of mean absolute error (MAE) and mean absolute percentage error (MAPE) (n = 49).
Table 3. Sleep parameters deviation calculation of mean absolute error (MAE) and mean absolute percentage error (MAPE) (n = 49).
Sleep Parameters MAEMAPE %MinMaxSD +/−
TIB
(in minutes)		Garmin vs. PSG26.946.320.00110.0029.67
Polar vs. PSG34.007.530.00201.0045.39
ActiGraph GT9X vs. PSG0.920.210.0018.002.70
Garmin vs. Polar49.2213.480.00197.0051.82
TST
(in minutes)		Garmin vs. PSG84.6325.320.00252.0052.28
Polar vs. PSG45.0813.801.00172.0045.60
ActiGraph GT9X vs. PSG31.399.331.00139.0029.17
Garmin vs. Polar60.2917.432.00200.0053.29
Awake
(in minutes)		Garmin vs. PSG60.5376.033.00176.0047.55
Polar vs. PSG50.0441.0614.00188.0039.77
ActiGraph GT9X vs. PSG31.2464.890.00140.0029.43
Garmin vs. Polar37.8470.811.00177.0037.76
WASO
(in minutes)		Garmin vs. PSG50.3583.025.00154.0041.94
Polar vs. PSG38.2462.380.00145.0036.43
ActiGraph GT9X vs. PSG28.5184.790.00130.0026.47
Garmin vs. Polar18.1874.832.0044.0010.38
SOL
(in minutes)		Garmin vs. PSG12.1672.090.0050.0011.41
Polar vs. PSG21.71141.611.00161.0028.40
ActiGraph GT9X vs. PSG11.6770.410.0053.0011.67
Garmin vs. Polar24.02105.310.00163.0034.62
SE
(in %)		Garmin vs. PSG11.6715.300.1540.0610.05
Polar vs. PSG8.6811.560.2237.818.60
ActiGraph GT9X vs. PSG6.998.650.0332.225.94
Garmin vs. Polar4.444.780.1211.262.61
MAE = mean absolute error (respective unit), MAPE = mean absolute percentage error (in %), min = minimum (absolute in respective unit), max = maximum (absolute in respective unit), SD = standard deviation (respective unit), TIB = time in bed, TST = total sleep time, Awake = awake time (WASO + SOL), WASO = wake after sleep onset, SOL = sleep onset latency (all expressed in minutes), SE = sleep efficiency (in %).
Table
Table 4. Sleep parameters of Garmin Forerunner 945, Polar Ignite and ActiGraph GT9X and their comparison with the gold standard PSG using one-way ANOVA (n = 49).
Table 4. Sleep parameters of Garmin Forerunner 945, Polar Ignite and ActiGraph GT9X and their comparison with the gold standard PSG using one-way ANOVA (n = 49).
Sleep ParametersF-Value
(p-Value)/
ή2-Value		Devices Mean
Diff.		SD+/−p-Value95% CI
LowerUpper
TIB
(in minutes)		F = 6.758
(p = 0.000)/
ή2 = 0.096		Garmin vs. PSG17.2736.290.367−10.0444.58
Polar vs. PSG−25.8450.580.072−53.151.47
ActiGraph GT9X vs. PSG0.672.771.000−26.6427.98
Garmin vs. Polar43.1157.120.00015.7970.41
TST
(in minutes)		F = 25.021
(p = 0.000)/
ή2 = 0.281		Garmin vs. PSG8354.760.00053.49112.67
Polar vs. PSG24.4359.540.147−5.1654.02
ActiGraph GT9X vs. PSG10.4141.780.805−19.1840.00
Garmin vs. Polar5955.120.00029.0688.24
Awake
(in minutes)		F = 48.024
(p = 0.000)/
ή2 = 0.429		Garmin vs. PSG−60.2547.920.000−81.73−38.76
Polar vs. PSG−26.0452.570.010−47.53−4.55
ActiGraph GT9X vs. PSG−11.0041.700.556−32.4910.49
Garmin vs. Polar−34.2041.140.000−55.69−12.72
WASO
(in minutes)		F = 36.138
(p = 0.000)/
ή2 = 0.361		Garmin vs. PSG−49.8642.530.000−66.50−33.22
Polar vs. PSG−34.1640.360.000−50.80−17.52
ActiGraph GT9X vs. PSG−0.0239.121.000−16.6616.62
Garmin vs. Polar−15.6913.920.073−32.340.95
SOL
(in minutes)		F = 9.612
(p = 0.000)/
ή2 = 0.131		Garmin vs. PSG−10.3913.100.000−14.15−6.63
Polar vs. PSG8.1234.810.292−3.6919.93
ActiGraph GT9X vs. PSG−10.9812.340.000−14.52−7.43
Garmin vs. Polar−18.5137.850.000−30.32−6.70
SE
(in %)		F = 39.407
(p = 0.000)/
ή2 = 0.381		Garmin vs. PSG14.8610.050.00010.8118.91
Polar vs. PSG10.839.560.0006.7814.88
ActiGraph GT9X vs. PSG2.319.190.461−1.746.36
Garmin vs. Polar4.033.210.052−0.028.08
TIB = time in bed, TST = total sleep time, Awake = awake time (WASO + SOL), WASO = wake after sleep onset, SOL = sleep onset latency (all expressed in minutes), SE = sleep efficiency (in %), mean Diff. = mean difference, SD = standard deviation, 95% CI = confidence interval with lower and upper limit.
Table
Table 5. Sleep stage deviation calculation of mean absolute error (MAE) and mean absolute percentage error (MAPE) (n = 49).
Table 5. Sleep stage deviation calculation of mean absolute error (MAE) and mean absolute percentage error (MAPE) (n = 49).
Sleep Stages MAEMAPE %MinMaxSD +/−
Light sleep
(in minutes)		Garmin vs. PSG64.9430.080.00197.0042.01
Polar vs. PSG40.1418.241.00116.0027.69
Garmin vs. Polar57.9826.751.00239.0049.89
Deep sleep
(in minutes)		Garmin vs. PSG47.33116.500.00179.0038.55
Polar vs. PSG28.0687.362.0078.0019.09
Garmin vs. Polar54.0491.661.00189.0040.89
REM sleep
(in minutes)		Garmin vs. PSG53.1097.544.00180.0035.29
Polar vs. PSG24.8049.130.0088.0021.37
Garmin vs. Polar45.6961.641.00145.0033.33
Sleep cycle
(total number)		Garmin vs. PSG1.1432.660.004.001.00
Polar vs. PSG1.1233.250.004.000.99
Garmin vs. Polar1.0422.410.003.000.93
MAE = mean absolute error (respective unit), MAPE = mean absolute percentage error (in %), min = minimum (absolute in respective unit), max = maximum (absolute in respective unit), SD = standard deviation (respective unit), Light = light sleep, Deep = slow-wave sleep (SWS), REM = rapid eye movement sleep (all expressed in minutes), sleep cycle = total number of sleep cycles.
Table
Table 6. Sleep stages of Garmin Forerunner 945, Polar Ignite and ActiGraph GT9X and their comparison with the gold standard PSG (n = 49).
Table 6. Sleep stages of Garmin Forerunner 945, Polar Ignite and ActiGraph GT9X and their comparison with the gold standard PSG (n = 49).
Sleep Stages Mean Diff.SD+/−t-Value
z-Value		p-Valued-Value95% CI
LowerUpper
Light sleep
(in minutes)		Garmin vs. PSG52.2057.346.3740.0001.84026.9177.50
Polar vs. PSG9.7348.121.4160.6370.409−15.5635.03
Garmin vs. Polar42.4763.874.6540.0001.34317.1867.76
Deep sleep
(in minutes)		Garmin vs. PSG17.2958.882.0550.0450.5930.3734.20
Polar vs. PSG7.9233.261.6670.6320.481−12.5128.34
Garmin vs. Polar9.3767.570.9700.3370.280−10.0428.78
REM sleep
(in minutes)		Garmin vs. PSG11.8463.101.3130.3510.379−8.3131.98
Polar vs. PSG4.7132.581.0130.8460.292−15.4324.86
Garmin vs. Polar7.1256.480.8830.6830.255−13.0227.27
Sleep cycle
(total numbers)		Garmin vs. PSG0.761.63−2.981z0.0030.6580.201.31
Polar vs. PSG1.101.37−4.572z0.0001.0360.551.65
Garmin vs. Polar−0.351.36−1.797z0.0720.318−0.900.20
Light = light sleep (NREM1 + NREM2), Deep = slow-wave sleep (SWS, NREM3), REM = rapid eye movement sleep (all expressed in minutes), sleep cycle = total number of sleep cycle, z represents the z-value according to Wilcoxon test, mean Diff. = mean difference, SD = standard deviation, 95% CI = confidence interval with lower and upper limit.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Budig, M.; Stoohs, R.; Keiner, M. Validity of Two Consumer Multisport Activity Tracker and One Accelerometer against Polysomnography for Measuring Sleep Parameters and Vital Data in a Laboratory Setting in Sleep Patients. Sensors 2022, 22, 9540. https://doi.org/10.3390/s22239540

AMA Style

Budig M, Stoohs R, Keiner M. Validity of Two Consumer Multisport Activity Tracker and One Accelerometer against Polysomnography for Measuring Sleep Parameters and Vital Data in a Laboratory Setting in Sleep Patients. Sensors. 2022; 22(23):9540. https://doi.org/10.3390/s22239540

Chicago/Turabian Style

Budig, Mario, Riccardo Stoohs, and Michael Keiner. 2022. "Validity of Two Consumer Multisport Activity Tracker and One Accelerometer against Polysomnography for Measuring Sleep Parameters and Vital Data in a Laboratory Setting in Sleep Patients" Sensors 22, no. 23: 9540. https://doi.org/10.3390/s22239540

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop