Accuracy of Three Commercial Wearable Devices for Sleep Tracking in Healthy Adults
Abstract
:1. Introduction
2. Method
2.1. Participants
2.2. Protocol
2.3. Polysomnography
2.4. Data
2.5. Wake Interpolation
2.6. Statistical Analysis
3. Results
3.1. Sleep–Wake Agreement
3.2. Sleep Stage Agreement
3.2.1. Sensitivity
3.2.2. Precision
3.3. Agreement with Nightly Summary Estimates
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Troester, M.; Quan, S.F.; Berry, R.; American Academy of Sleep Medicine. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications, version 3.0; AASM: Darien, IL, USA, 2023. Available online: www.aasmnet.org (accessed on 10 June 2024).
- Hirshkowitz, M.; Whiton, K.; Albert, S.M.; Alessi, C.; Bruni, O.; DonCarlos, L.; Hazen, N.; Herman, J.; Katz, E.S.; Kheirandish-Gozal, L.; et al. National Sleep Foundation’s sleep time duration recommendations: Methodology and results summary. Sleep Health J. Natl. Sleep Found. 2015, 1, 40–43. [Google Scholar] [CrossRef] [PubMed]
- Watson, N.F.; Badr, M.S.; Belenky, G.; Bliwise, D.L.; Buxton, O.M.; Buysse, D.; Dinges, D.F.; Gangwisch, J.; Grandner, M.A.; Kushida, C.; et al. Recommended Amount of Sleep for a Healthy Adult: A Joint Consensus Statement of the American Academy of Sleep Medicine and Sleep Research Society. J. Clin. Sleep Med. 2015, 11, 591–592. [Google Scholar] [CrossRef] [PubMed]
- Zheng, N.S.; Annis, J.; Master, H.; Han, L.; Gleichauf, K.; Ching, J.H.; Nasser, M.; Coleman, P.; Desine, S.; Ruderfer, D.M.; et al. Sleep patterns and risk of chronic disease as measured by long-term monitoring with commercial wearable devices in the All of Us Research Program. Nat. Med. 2024, 30, 2648–2656. [Google Scholar] [CrossRef] [PubMed]
- Centers for Disease Control and Prevention (CDC). Perceived insufficient rest or sleep-four states, 2006. MMWR Morb. Mortal. Wkly. Rep. 2008, 57, 200–203. [Google Scholar]
- Robbins, R.; Quan, S.F.; Buysse, D.J.; Weaver, M.D.; Walker, M.P.; Drake, C.L.; Monten, K.; Barger, L.K.; Rajaratnam, S.M.W.; Roth, T.; et al. A Nationally Representative Survey Assessing Restorative Sleep in US Adults. Front. Sleep 2022, 1, 935228. Available online: https://www.frontiersin.org/articles/10.3389/frsle.2022.935228 (accessed on 14 August 2022). [CrossRef]
- Robbins, R.; Krebs, P.; Rapoport, D.M.; Jean-Louis, G.; Duncan, D.T. Examining Use of Mobile Phones for Sleep Tracking among a National Sample in the USA. Health Commun. 2019, 34, 545–551. [Google Scholar] [CrossRef]
- Leung, W.; Case, L.; Sung, M.-C.; Jung, J. A meta-analysis of Fitbit devices: Same company, different models, different validity evidence. J. Med. Eng. Technol. 2022, 46, 102–115. [Google Scholar] [CrossRef]
- Yoon, H.; Choi, S.H. Technologies for sleep monitoring at home: Wearables and nearables. Biomed. Eng. Lett. 2023, 13, 313–327. [Google Scholar] [CrossRef]
- Imtiaz, S.A. A Systematic Review of Sensing Technologies for Wearable Sleep Staging. Sensors 2021, 21, 1562. [Google Scholar] [CrossRef]
- Baron, K.G.; Abbott, S.; Jao, N.; Manalo, N.; Mullen, R. Orthosomnia: Are Some Patients Taking the Quantified Self Too Far? J. Clin. Sleep Med. JCSM Off. Publ. Am. Acad. Sleep Med. 2017, 13, 351–354. [Google Scholar] [CrossRef]
- Younes, M.; Hanly, P.J. Minimizing Interrater Variability in Staging Sleep by Use of Computer-Derived Features. J. Clin. Sleep Med. 2016, 12, 1347–1356. [Google Scholar] [CrossRef] [PubMed]
- Ancoli-Israel, S.; Cole, R.; Alessi, C.; Chambers, M.; Moorcroft, W.; Pollak, C.P. The role of actigraphy in the study of sleep and circadian rhythms. Sleep 2003, 26, 342–392. [Google Scholar] [CrossRef] [PubMed]
- Van De Water, A.T.M.; Holmes, A.; Hurley, D.A. Objective measurements of sleep for non-laboratory settings as alternatives to polysomnography—A systematic review. J. Sleep Res. 2011, 20 Pt 2, 183–200. [Google Scholar] [CrossRef] [PubMed]
- Statista. Most Used eHealth Tracker/Smart Watches by Brand in the U.S. (n.d.). 2024. Available online: https://www.statista.com/forecasts/997195/most-used-ehealth-tracker-smart-watches-by-brand-in-the-us (accessed on 5 September 2024).
- The Best Smart Ring I’ve Tested Isn’t Made by Samsung or Ultrahuman. (n.d.). ZDNET. Available online: https://www.zdnet.com/article/the-best-smart-ring-ive-tested-isnt-made-by-samsung-or-ultrahuman/ (accessed on 24 September 2024).
- Altini, M.; Kinnunen, H. The Promise of Sleep: A Multi-Sensor Approach for Accurate Sleep Stage Detection Using the Oura Ring. Sensors 2021, 21, 4302. [Google Scholar] [CrossRef] [PubMed]
- Ong, J.L.; Golkashani, H.A.; Ghorbani, S.; Wong, K.F.; Chee, N.I.Y.N.; Willoughby, A.R.; Chee, M.W.L. Selecting a sleep tracker from EEG-based, iteratively improved, low-cost multisensor, and actigraphy-only devices. Sleep Health 2024, 10, 9–23. [Google Scholar] [CrossRef]
- Svensson, T.; Madhawa, K.; Nt, H.; Chung, U.; Svensson, A.K. Validity and reliability of the Oura Ring Generation 3 (Gen3) with Oura sleep staging algorithm 2.0 (OSSA 2.0) when compared to multi-night ambulatory polysomnography: A validation study of 96 participants and 421,045 epochs. Sleep Med. 2024, 115, 251–263. [Google Scholar] [CrossRef]
- Fitzpatrick, T.B. The Validity and Practicality of Sun-Reactive Skin Types I Through VI. Arch. Dermatol. 1988, 124, 869–871. [Google Scholar] [CrossRef]
- Cohen, J. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 1988, 20, 37–46. [Google Scholar] [CrossRef]
- Liljequist, D.; Elfving, B.; Skavberg Roaldsen, K. Intraclass correlation—A discussion and demonstration of basic features. PLoS ONE 2019, 14, e0219854. [Google Scholar] [CrossRef]
- Cicchetti, D.V. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol. Assess. 1994, 6, 284. [Google Scholar] [CrossRef]
- Grandner, M.A.; Bromberg, Z.; Hadley, A.; Morrell, Z.; Graf, A.; Hutchison, S.; Freckleton, D. Performance of a multisensor smart ring to evaluate sleep: In-lab and home-based evaluation of generalized and personalized algorithms. Sleep 2023, 46, zsac152. [Google Scholar] [CrossRef]
- Chinoy, E.D.; Cuellar, J.A.; Jameson, J.T.; Markwald, R.R. Performance of Four Commercial Wearable Sleep-Tracking Devices Tested Under Unrestricted Conditions at Home in Healthy Young Adults. Nat. Sci. Sleep 2022, 14, 493–516. [Google Scholar] [CrossRef]
- de Zambotti, M.; Rosas, L.; Colrain, I.M.; Baker, F.C. The Sleep of the Ring: Comparison of the ŌURA Sleep Tracker Against Polysomnography. Behav. Sleep Med. 2019, 17, 124–136. [Google Scholar] [CrossRef]
- de Zambotti, M.; Goldstein, C.; Cook, J.; Menghini, L.; Altini, M.; Cheng, P.; Robillard, R. State of the science and recommendations for using wearable technology in sleep and circadian research. Sleep 2023, 47, zsad325. [Google Scholar] [CrossRef]
- Fawzy, A.; Wu, T.D.; Wang, K.; Robinson, M.L.; Farha, J.; Bradke, A.; Golden, S.H.; Xu, Y.; Garibaldi, B.T. Racial and Ethnic Discrepancy in Pulse Oximetry and Delayed Identification of Treatment Eligibility Among Patients With COVID-19. JAMA Intern. Med. 2022, 182, 730–738. [Google Scholar] [CrossRef]
- Bandura, A. Social cognitive theory of self-regulation. Organ. Behav. Hum. Decis. Process. 1991, 50, 248–287. [Google Scholar] [CrossRef]
- Chinoy, E.D.; Cuellar, J.A.; Huwa, K.E.; Jameson, J.T.; Watson, C.H.; Bessman, S.C.; Hirsch, D.A.; Cooper, A.D.; Drummond, S.P.A.; Markwald, R.R. Performance of seven consumer sleep-tracking devices compared with polysomnography. Sleep 2021, 44, zsaa291. [Google Scholar] [CrossRef]
- Nguyen, Q.N.T.; Le, T.; Huynh, Q.B.T.; Setty, A.; Vo, T.V.; Le, T.Q. Validation Framework for Sleep Stage Scoring in Wearable Sleep Trackers and Monitors with Polysomnography Ground Truth. Clocks Sleep 2021, 3, 274–288. [Google Scholar] [CrossRef]
- Chee, N.I.; Ghorbani, S.; Golkashani, H.A.; Leong, R.L.; Ong, J.L.; Chee, M.W. Multi-night validation of a sleep tracking ring in adolescents compared with a research actigraph and polysomnography. Nat. Sci. Sleep 2021, 13, 177–190. [Google Scholar] [CrossRef]
Characteristic | n | % | |
---|---|---|---|
Age | 20 to 29 years of age | 14 | 40.0% |
30 to 39 years of age | 13 | 37.1% | |
40 to 50 years of age | 8 | 22.9% | |
Sex | Female | 20 | 57.1% |
Male | 15 | 42.9% | |
Race/Ethnicity | White, non-Hispanic | 20 | 57.1% |
Asian | 8 | 22.9% | |
Black or African American | 3 | 8.6% | |
More than one race | 1 | 2.9% | |
Preferred not to answer | 3 | 8.6% | |
Skin Type | Type 1 | 1 | 2.9% |
Type 2 | 1 | 2.9% | |
Type 3 | 12 | 34.3% | |
Type 4 | 16 | 45.7% | |
Type 5 | 5 | 14.2% |
Oura Assignment | PSG Assignment | |||
---|---|---|---|---|
Wake | Light | Deep | REM | |
Wake | 68.6% (18.4%) | 7.4% (4.6%) | 0.1% (0.2%) | 5.8% (7.3%) |
Light | 19.0% (11.6%) | 78.2% (7.6%) | 19.9% (14.1%) | 17.8% (13.1%) |
Deep | 2.6% (3.6%) | 8.5% (6.5%) | 79.5% (14.5%) | 0.4% (0.9%) |
REM | 9.7% (10.9%) | 6.0% (5.0%) | 0.5% (1.6%) | 76.0% (15.2%) |
Fitbit Assignment | Wake | Light | Deep | REM |
Wake | 67.7% (23.1%) | 6.8% (4.2%) | 1.5% (1.6%) | 6.1% (4.5%) |
Light | 23.3% (18.8%) | 78.0% (9.0%) | 34.8% (20.4%) | 24.5% (20.9%) |
Deep | 0.8% (2.9%) | 7.5% (5.8%) | 61.7% (21.1%) | 1.8% (5.3%) |
REM | 8.2% (15.3%) | 7.7% (6.6%) | 2.1% (6.9%) | 67.6% (23.3%) |
Apple Assignment | Wake | Light | Deep | REM |
Wake | 52.4% (21.5%) | 4.2% (3.5%) | 0.3% (0.7%) | 1.4% (2.0%) |
Light | 37.2% (18.2%) | 86.1% (6.2%) | 48.3% (20.0%) | 15.8% (14.4%) |
Deep | 1.0% (1.9%) | 2.1% (3.2%) | 50.5% (20.6%) | 0.2% (1.1%) |
REM | 9.4% (12.8%) | 7.6% (4.7%) | 0.9% (2.0%) | 82.6% (14.8%) |
Oura Assignment | PSG Assignment | |||
---|---|---|---|---|
Wake | Light | Deep | REM | |
Wake | 53.5% (24.4%) | 35.4% (21.1%) | 0.4% (1.0%) | 10.7% (11.8%) |
Light | 3.7% (3.9%) | 79.5% (8.6%) | 9.1% (8.6%) | 7.6% (5.7%) |
Deep | 0.8% (0.7%) | 21.5% (17.0%) | 77.0% (17.4%) | 0.5% (0.9%) |
REM | 5.9% (8.6%) | 14.5% (11.2%) | 0.5% (1.4%) | 79.1% (13.1%) |
Fitbit Assignment | Wake | Light | Deep | REM |
Wake | 51.0% (21.7%) | 33.9% (18.0%) | 3.1% (3.3%) | 12.0% (10.1%) |
Light | 5.6% (7.9%) | 72.8% (13.4%) | 12.8% (9.8%) | 8.8% (6.5%) |
Deep | 1.3% (5.9%) | 23.3% (18.6%) | 73.2% (21.5%) | 1.8% (5.1%) |
REM | 4.5% (9.5%) | 20.3% (18.3%) | 2.1% (6.5%) | 73.1% (21.9%) |
Apple Assignment | Wake | Light | Deep | REM |
Wake | 61.9% (23.0%) | 33.4% (22.8%) | 0.8% (2.4%) | 3.9% (4.8%) |
Light | 5.4% (5.5%) | 72.7% (10.3%) | 16.5% (11.1%) | 5.5% (5.2%) |
Deep | 0.6% (0.9%) | 10.4% (17.3%) | 87.8% (19.7%) | 0.6% (2.8%) |
REM | 3.6% (5.3%) | 17.8% (10.5%) | 0.9% (2.2%) | 77.7% (12.9%) |
Oura n = 35 Mean (SD) | PSG n = 35 Mean (SD) | Fitbit n = 33 Mean (SD) | PSG n = 33 Mean (SD) | Apple Watch n = 29 Mean (SD) | PSG n = 29 Mean (SD) | |
---|---|---|---|---|---|---|
Total Sleep Time (min) | 421 (34) | 430 (41) | 428 (23) | 431 (40) | 442 (35) | 434 (40) |
Wake (min) | 59 (34) | 50 (41) | 49 (26) | 49 (40) | 39 (35) * | 46 (40) |
Light Sleep (min) | 233 (28) | 239 (41) | 258 (37) * | 240 (41) | 289 (24) * | 244 (38) |
Deep Sleep (min) | 95 (21) | 95 (35) | 79 (27) * | 94 (36) | 51 (18) * | 94 (38) |
REM (min) | 93 (25) | 96 (21) | 90 (28) | 97 (21) | 102 (22) | 96 (21) |
Sleep Latency (min) | 18 (25) * | 13 (22) | 13 (21) | 13 (22) | 17 (26) | 15 (23) |
WASO (min) | 42 (26) | 38 (37) | 40 (15) | 36 (35) | 22 (23) * | 32 (33) |
Sleep Efficiency (%) | 88% (7%) | 90% (9%) | 89% (5%) | 90% (8%) | 92% (7%) | 90% (8%) |
Oura n = 35 | Fitbit n = 33 | Apple Watch n = 29 | |
---|---|---|---|
ICC (95% CI) | ICC (95% CI) | ICC (95% CI) | |
Total Sleep Time (min) | 0.74 (0.54–0.86) | 0.56 (0.28–0.76) | 0.85 (0.70–0.93) |
Light Sleep (min) | 0.40 (0.08–0.64) | 0.52 (0.22–0.73) | 0.37 (0.00–0.64) |
Deep Sleep (min) | 0.32 (−0.01–0.59) | 0.36 (0.02–0.62) | 0.13 (−0.24–0.47) |
REM (min) | 0.27 (−0.06–0.55) | 0.13 (−0.22–0.45) | 0.37 (0.01–0.64) |
Sleep Latency (min) | 0.95 (0.90–0.97) | 0.95 (0.89–0.97) | 0.94 (0.87–0.97) |
WASO (min) | 0.63 (0.38–0.80) | 0.41 (0.08–0.66) | 0.72 (0.48–0.86) |
Sleep Efficiency (%) | 0.74 (0.55–0.86) | 0.56 (0.28–0.76) | 0.85 (0.71–0.93) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Robbins, R.; Weaver, M.D.; Sullivan, J.P.; Quan, S.F.; Gilmore, K.; Shaw, S.; Benz, A.; Qadri, S.; Barger, L.K.; Czeisler, C.A.; et al. Accuracy of Three Commercial Wearable Devices for Sleep Tracking in Healthy Adults. Sensors 2024, 24, 6532. https://doi.org/10.3390/s24206532
Robbins R, Weaver MD, Sullivan JP, Quan SF, Gilmore K, Shaw S, Benz A, Qadri S, Barger LK, Czeisler CA, et al. Accuracy of Three Commercial Wearable Devices for Sleep Tracking in Healthy Adults. Sensors. 2024; 24(20):6532. https://doi.org/10.3390/s24206532
Chicago/Turabian StyleRobbins, Rebecca, Matthew D. Weaver, Jason P. Sullivan, Stuart F. Quan, Katherine Gilmore, Samantha Shaw, Abigail Benz, Salim Qadri, Laura K. Barger, Charles A. Czeisler, and et al. 2024. "Accuracy of Three Commercial Wearable Devices for Sleep Tracking in Healthy Adults" Sensors 24, no. 20: 6532. https://doi.org/10.3390/s24206532
APA StyleRobbins, R., Weaver, M. D., Sullivan, J. P., Quan, S. F., Gilmore, K., Shaw, S., Benz, A., Qadri, S., Barger, L. K., Czeisler, C. A., & Duffy, J. F. (2024). Accuracy of Three Commercial Wearable Devices for Sleep Tracking in Healthy Adults. Sensors, 24(20), 6532. https://doi.org/10.3390/s24206532