Validation of a Neurophysiological-Based Wearable Device (Somfit) for the Assessment of Sleep in Athletes
Abstract
:1. Introduction
2. Materials and Methods
2.1. Participants
2.2. Equipment
2.2.1. Polysomnography (Gold Standard)
2.2.2. Somfit (Wearable Device)
2.3. Procedure
2.4. Measures
2.4.1. Epoch Scoring—Polysomnography (Gold Standard)
2.4.2. Epoch Scoring—Somfit (Wearable Device)
2.4.3. Summary Sleep Variables—Polysomnography and Somfit
2.5. Data Analysis
2.5.1. Epoch-by-Epoch Comparisons
- Sensitivity for sleep (%) = TS/(TS + FW) × 100 (i.e., the percentage of PSG sleep epochs correctly scored as sleep by Somfit).
- Sensitivity for wake (%) = TW/(TW + FS) × 100 (i.e., the percentage of PSG wake epochs correctly scored as wake by Somfit [sometimes referred to as specificity]).
- Agreement (%) = (TS + TW)/(TS + TW + FS + FW) × 100 (i.e., the percentage of all PSG epochs correctly scored as sleep or wake by Somfit).
- Sensitivity for N1 (%) = TN1/(TN1 + FWN1 + FN2N1 + FN3N1+ FRN1) × 100(i.e., the percentage of PSG N1 epochs correctly scored as N1 by Somfit).
- Sensitivity for N2 (%) = TN2/(TN2 + FWN2+ FN1N2+ FN3N2 + FRN2) × 100(i.e., the percentage of PSG N2 epochs correctly scored as N2 by Somfit).
- Sensitivity for N3 (%) = TN3/(TN3 + FWN3 + FN1N3 + FN2N3 + FRN3) × 100(i.e., the percentage of PSG N3 epochs correctly scored as N3 by Somfit).
- Sensitivity for REM (%) = TR/(TR + FWR + FN1R + FN2R + FN3R) × 100(i.e., the percentage of PSG REM epochs correctly scored as REM by Somfit).
- Sensitivity for wake (%) = TW/(TW + FN1w + FN2w + FN3w + FRw) × 100(i.e., the percentage of PSG wake epochs correctly scored as wake by Somfit [sometimes referred to as specificity]).
- Agreement (%) = (TW + TN1 + TN2 + TN3 + TR)/(TW + TN1 + TN2 + TN3 + TR + FWN1 + FWN2 + FWN3 + FWR + FN1W + FN1N2 + FN1N3 + FN1R + FN2W + FN2N1 + FN2N3 + FN2R + FN3W + FN3N1 + FN3N2 + FN3R + FRW + FRN1 + FRN2 + FRN3) × 100(i.e., the percentage of all PSG epochs correctly scored as N1, N2, N3, REM or wake, by Somfit).
2.5.2. Summary Sleep Variables (Bland–Altman Analyses, Means Comparisons)
3. Results
3.1. Data Capture and Data Quality
- Unfiltered subset (n = 26): contains Somfit records (and their matching PSG records) from all participants.
- Good-capture subset (n = 15): contains Somfit records (and their matching PSG records) from participants for whom > 80% of the 30-s epochs were captured/scored by Compumedics’ Profusion Nexus360 system.
- Excellent-capture subset (n = 7): contains Somfit records (and their matching PSG records) from participants for whom > 99.9% of the 30-s epochs were captured/scored by Compumedics’ Profusion Nexus360 system.
3.2. Epoch-by-Epoch Comparisons
3.3. Summary Variables (Bland–Altman Analyses, Bias, Absolute Bias)
4. Discussion
4.1. Comparison of Somfit for Assessing the Sleep of Athletes and Non-Athletes
4.2. Validity of Somfit for Assessing Athletes’ Sleep
4.3. Maximise the Likelihood of Capturing Excellent Somfit Data
4.4. Types of Use of Somfit to Assess Athletes’ Sleep
4.5. Use of Somfit to Capture Metrics Related to the Circadian System
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Keenan, S.; Hirshkowitz, M. Sleep stage scoring. In Principles and Practice of Sleep Medicine, 7th ed.; Kryger, M.H., Roth, T., Goldstein, C.A., Eds.; Elsevier: Amsterdam, The Netherlands, 2022; pp. 1839–1847. [Google Scholar]
- Miller, D.J.; Sargent, C.; Roach, G.D. A validation of six wearable devices for estimating sleep, heart rate and heart rate variability in healthy adults. Sensors 2022, 22, 6317. [Google Scholar] [CrossRef] [PubMed]
- Miller, D.J.; Lastella, M.; Scanlan, A.T.; Bellenger, C.; Halson, S.L.; Roach, G.D.; Sargent, C. A validation study of the WHOOP strap against polysomnography to assess sleep. J. Sports Sci. 2020, 38, 2631–2636. [Google Scholar] [CrossRef] [PubMed]
- Budig, M.; Stoohs, R.; Keiner, M. Validity of two consumer multisport activity tracker and one accelerometer against polysomnography for measuring sleep parameters and vital data in a laboratory setting in sleep patients. Sensors 2022, 22, 9540. [Google Scholar] [CrossRef]
- McMahon, M.; Goldin, J.; Kealy, E.S.; Wicks, D.J.; Zilberg, E.; Freeman, W.; Aliahmad, B. Performance investigation of Somfit sleep staging algorithm. Nat. Sci. Sleep 2024, 16, 1027–1043. [Google Scholar] [CrossRef]
- Robbins, R.; Weaver, M.D.; Sullivan, J.P.; Quan, S.F.; Gilmore, K.; Shaw, S.; Benz, A.; Qadri, S.; Barger, L.K.; Czeisler, C.A.; et al. Accuracy of three commercial wearable devices for sleep tracking in healthy adults. Sensors 2024, 24, 6532. [Google Scholar] [CrossRef]
- Leeder, J.; Glaister, M.; Pizzoferro, K.; Dawson, J.; Pedlar, C. Sleep duration and quality in elite athletes measured using wristwatch actigraphy. J. Sports Sci. 2012, 30, 541–545. [Google Scholar] [CrossRef]
- Driller, M.W.; Dixon, Z.T.; Clark, M.I. Accelerometer-based sleep behavior and activity levels in student athletes in comparison to student non-athletes. Sport Sci. Health 2017, 13, 411–418. [Google Scholar] [CrossRef]
- Cook, J.D.; Eftekari, S.C.; Dallmann, E.; Sippy, M.; Plante, D.T. Ability of the Fitbit Alta HR to quantify and classify sleep in patients with suspected central disorders of hypersomnolence: A comparison against polysomnography. J. Sleep Res. 2019, 28, e12789. [Google Scholar] [CrossRef]
- Moreno-Pino, F.; Porras-Segovia, A.; López-Esteban, P.; Artés, A.; Baca-García, E. Validation of Fitbit Charge 2 and Fitbit Alta HR against polysomnography for assessing sleep in adults with obstructive sleep apnea. J. Clin. Sleep Med. 2019, 15, 1645–1653. [Google Scholar] [CrossRef]
- Kahawage, P.; Jumabhoy, R.; Hamill, K.; de Zambotti, M.; Drummond, S.P.A. Validity, potential clinical utility, and comparison of consumer- and research-grade activity trackers in Insomnia Disorder I: In-lab validation against polysomnography. J. Sleep Res. 2020, 29, e12931. [Google Scholar] [CrossRef]
- Sargent, C.; Lastella, M.; Halson, S.L.; Roach, G.D. The validity of activity monitors for measuring sleep in elite athletes. J. Sci. Med. Sport 2016, 19, 848–853. [Google Scholar] [CrossRef] [PubMed]
- Fuller, K.L.; Juliff, L.; Gore, C.J.; Peiffer, J.J.; Halson, S.L. Software thresholds alter the bias of actigraphy for monitoring sleep in team-sport athletes. J. Sci. Med. Sport 2017, 20, 756–760. [Google Scholar] [CrossRef]
- Sargent, C.; Lastella, M.; Romyn, G.; Versey, N.; Miller, D.J.; Roach, G.D. How well does a commercially available wearable device measure sleep in young athletes? Chronobiol. Int. 2018, 35, 754–758. [Google Scholar] [CrossRef]
- Kawasaki, Y.; Kasai, T.; Sakurama, Y.; Sekiguchi, A.; Kitamura, E.; Midorikawa, I.; Shiroshita, N.; Kawana, F.; Ogasawara, E.; Kitade, M.; et al. Evaluation of sleep parameters and sleep staging (slow wave sleep) in athletes by Fitbit Alta HR, a consumer sleep tracking device. Nat. Sci. Sleep 2022, 14, 819–827. [Google Scholar] [CrossRef] [PubMed]
- Jeukendrup, A.E.; Craig, N.P.; Hawley, J.A. The bioenergetics of world class cycling. J. Sci. Med. Sport 1999, 3, 414–433. [Google Scholar] [CrossRef]
- Berry, R.B.; Quan, S.F.; Abreu, A.R.; Bibbs, M.L.; Del Rosso, L.; Harding, S.M. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications, version 2.6; American Academy of Sleep Medicine: Darien, IL, USA, 2020. [Google Scholar]
- Nielsen, T. A review of mentation in REM and NREM sleep: “Covert” REM sleep as a possible reconciliation of two models. Behav. Brain Sci. 2000, 23, 851–866. [Google Scholar] [CrossRef]
- Menghini, L.; Cellini, N.; Goldstone, A.; Baker, F.C.; de Zambotti, M.A. Standardized framework for testing the performance of sleep-tracking technology: Step-by-step guidelines and open-source code. Sleep 2021, 44, zsaa170. [Google Scholar] [CrossRef]
- Landis, J.R.; Koch, G.G. The measurement of observer agreement for categorical data. Biometrics 1977, 33, 159–174. [Google Scholar] [CrossRef]
- Bland, J.M.; Altman, D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986, 327, 307–310. [Google Scholar] [CrossRef]
- de Zambotti, M.; Cellini, N.; Goldstone, A.; Colrain, I.M.; Baker, F.C. Wearable sleep technology in clinical and research settings. Med. Sci. Sports Exerc. 2019, 51, 1538–1557. [Google Scholar] [CrossRef]
- Bernert, R.A.; Hom, M.A.; Iwata, N.G.; Joiner, T.E. Objectively assessed sleep variability as an acute warning sign of suicidal ideation in a longitudinal evaluation of young adults at high suicide risk. J. Clin. Psychiatry 2017, 78, e678–e687. [Google Scholar] [CrossRef] [PubMed]
- Kuula, L.; Pesonen, A.-K.; Heinonen, K.; Kajantie, E.; Eriksson, J.G.; Andersson, S.; Lano, A.; Lahti, J.; Wolke, D.; Räikkönen, K. Naturally occurring circadian rhythm and sleep duration are related to executive functions in early adulthood. J. Sleep Res. 2018, 27, 113–119. [Google Scholar] [CrossRef] [PubMed]
- Windred, D.P.; Burns, A.C.; Lane, J.M.; Saxena, R.; Rutter, M.K.; Cain, S.W.; Phillips, A.J.K. Sleep regularity is a stronger predictor of mortality risk than sleep duration: A prospective cohort study. Sleep 2024, 47, zsad253. [Google Scholar] [CrossRef] [PubMed]
- Levandovski, R.; Dantas, G.; Fernandes, L.C.; Caumo, W.; Torres, I.; Roenneberg, T.; Hidalgo, M.P.L.; Allebrandt, K.V. Depression scores associate with chronotype and social jetlag in a rural population. Chronobiol. Int. 2011, 28, 771–778. [Google Scholar] [CrossRef]
- Roenneberg, T.; Allebrandt, K.V.; Merrow, M.; Vetter, C. Social jetlag and obesity. Curr. Biol. 2012, 22, 939–943. [Google Scholar] [CrossRef]
- Haraszti, R.Á.; Ella, K.; Gyöngyösi, N.; Roenneberg, T.; Káldi, K. Social jetlag negatively correlates with academic performance in undergraduates. Chronobiol. Int. 2014, 31, 603–612. [Google Scholar] [CrossRef]
Somfit | |||
---|---|---|---|
Wake | Sleep | ||
PSG | Wake | True Wake (TW) | False Sleep (FS) |
Sleep | False Wake (FW) | True Sleep (TS) |
Somfit | ||||||
---|---|---|---|---|---|---|
Wake | N1 | N2 | N3 | REM | ||
PSG | Wake | True Wake (TW) | False N1 (FN1W) | False N2 (FN2W) | False N3 (FN3W) | False REM (FRW) |
N1 | False Wake (FWN1) | True N1 (TN1) | False N2 (FN2N1) | False N3 (FN3N1) | False REM (FRN1) | |
N2 | False Wake (FWN2) | False N1 (FN1N2) | True N2 (TN2) | False N3 (FN3N2) | False REM (FRN2) | |
N3 | False Wake (FWN3) | False N1 (FN1N3) | False N2 (FN2N3) | True N3 (TN3) | False REM (FRN3) | |
REM | False Wake (FWR) | False N1 (FN1R) | False N2 (FN2R) | False N3 (FN3R) | True REM (TR) |
Variable | Unfiltered Subset (n = 26) | Good-Capture Subset (n = 15) | Excellent-Capture Subset (n = 7) | |
---|---|---|---|---|
2-state Categorisation of TIB | ||||
Somfit v. PSG | ||||
Sleep Sensitivity (%) | 88.7 ± 8.3 | 93.1 ± 5.0 | 97.0 ± 1.7 | |
Wake Sensitivity (%) | 59.8 ± 21.3 | 58.3 ± 19.1 | 65.8 ± 14.1 | |
Agreement (%) | 83.6 ± 11.1 | 88.7 ± 6.1 | 93.8 ± 2.2 | |
Cohen’s Kappa (κ) | 0.45 ± 0.27 | 0.51 ± 0.23 | 0.64 ± 0.12 | |
Scorer v. Scorer (PSG) | ||||
A v. B Agreement (%) | 96.4 ± 2.6 | 96.7 ± 3.0 | 97.1 ± 3.5 | |
A v. C Agreement (%) | 92.9 ± 3.6 | 93.7 ± 2.6 | 94.3 ± 2.5 | |
B v. C Agreement (%) | 94.7 ± 3.5 | 95.6 ± 2.6 | 96.2 ± 1.7 | |
5-state Categorisation of TIB | ||||
Somfit v. PSG | ||||
N1 Sensitivity (%) | 18.5 ± 9.2 | 18.9 ± 9.1 | 24.8 ± 8.8 | |
N2 Sensitivity (%) | 69.1 ± 18.6 | 72.9 ± 18.7 | 84.4 ± 10.1 | |
N3 Sensitivity (%) | 60.8 ± 33.3 | 67.6 ± 29.5 | 84.3 ± 12.5 | |
REM Sensitivity (%) | 53.3 ± 37.7 | 63.7 ± 31.4 | 80.2 ± 28.5 | |
Wake Sensitivity (%) | 59.8 ± 21.3 | 58.3 ± 19.1 | 65.8 ± 14.1 | |
Agreement (%) | 62.6 ± 19.5 | 66.2 ± 19.5 | 79.4 ± 10.4 | |
Cohen’s Kappa (κ) | 0.47 ± 0.27 | 0.52 ± 0.28 | 0.70 ± 0.16 | |
Scorer v. Scorer (PSG) | ||||
A v. B Agreement (%) | 86.8 ± 4.6 | 86.9 ± 5.3 | 87.0 ± 6.4 | |
A v. C Agreement (%) | 77.4 ± 5.8 | 77.7 ± 5.2 | 78.0 ± 5.5 | |
B v. C Agreement (%) | 81.2 ± 5.8 | 81.7 ± 4.8 | 82.1 ± 3.3 |
Somfit (Unfiltered Subset) | |||||||
---|---|---|---|---|---|---|---|
Wake | N1 | N2 | N3 | REM | Sum | ||
PSG | Wake | 60 | 7 | 17 | 5 | 10 | 100 |
N1 | 28 | 19 | 36 | 4 | 13 | 100 | |
N2 | 10 | 5 | 69 | 9 | 6 | 100 | |
N3 | 10 | 2 | 22 | 61 | 5 | 100 | |
REM | 17 | 2 | 22 | 5 | 53 | 100 |
Somfit (Good-Capture Subset) | |||||||
---|---|---|---|---|---|---|---|
Wake | N1 | N2 | N3 | REM | Sum | ||
PSG | Wake | 58 | 7 | 20 | 3 | 12 | 100 |
N1 | 23 | 19 | 41 | 4 | 13 | 100 | |
N2 | 5 | 4 | 73 | 11 | 8 | 100 | |
N3 | 5 | 3 | 22 | 68 | 3 | 100 | |
REM | 8 | 2 | 23 | 4 | 63 | 100 |
Somfit (Excellent-Capture Subset) | |||||||
---|---|---|---|---|---|---|---|
Wake | N1 | N2 | N3 | REM | Sum | ||
PSG | Wake | 66 | 9 | 11 | 2 | 13 | 100 |
N1 | 24 | 25 | 40 | 3 | 8 | 100 | |
N2 | 1 | 2 | 84 | 10 | 2 | 100 | |
N3 | 0 | 0 | 15 | 84 | 0 | 100 | |
REM | 3 | 1 | 14 | 2 | 80 | 100 |
Variable | PSG M ± SD | Somfit (UF) M ± SD | Mean Bias M ± SD | Absolute Bias M ± SD | Paired t-Test t, df, p | Effect Size d, Low, High |
---|---|---|---|---|---|---|
Total sleep time (min) | 455.4 ± 37.9 | 311.7 ± 156.6 | −143.7 ± 155.0 | 152.0 ± 146.6 | −4.7, 25, <.001 | −0.93, −1.38, −0.46 |
N1 sleep (min) | 31.2 ± 13.0 | 18.7 ± 11.3 | −12.5 ± 13.0 | 13.9 ± 11.4 | −4.8, 25, <.001 | −0.96, −1.41, −0.48 |
N2 sleep (min) | 229.8 ± 32.6 | 161.4 ± 91.9 | −68.4 ± 87.0 | 80.3 ± 75.6 | −4.0, 25, <.001 | −0.79, −1.22, −0.34 |
N3 sleep (min) | 94.0 ± 28.0 | 67.7 ± 48.0 | −26.3 ± 46.9 | 42.4 ± 32.4 | −2.9, 25, .008 | −0.56, −0.97, −0.14 |
REM sleep (min) | 100.4 ± 16.4 | 63.8 ± 46.1 | −36.6 ± 43.5 | 43.4 ± 36.5 | −4.3, 25, <.001 | −0.84, −1.28, −0.39 |
Wake (min) | 74.9 ± 36.8 | 64.1 ± 34.2 | −10.8 ± 26.9 | 21.8 ± 18.8 | −2.0, 25, .052 | −0.40, −0.80, 0.00 |
Variable | PSG M ± SD | Somfit (GC) M ± SD | Mean Bias M ± SD | Absolute Bias M ± SD | Paired t-Test t, df, p | Effect Size d, Low, High |
---|---|---|---|---|---|---|
Total sleep time (min) | 456.1 ± 41.3 | 431.0 ± 69.8 | −25.2 ± 48.1 | 39.5 ± 36.4 | −2.0, 14, .062 | −0.52, −1.06, 0.03 |
N1 sleep (min) | 27.7 ± 10.4 | 22.0 ± 9.9 | −5.7 ± 9.5 | 8.1 ± 7.3 | −2.3, 14, .037 | −0.60, −1.14, −0.04 |
N2 sleep (min) | 232.5 ± 36.4 | 225.3 ± 56.0 | −7.1 ± 34.6 | 27.9 ± 20.5 | −0.8, 14, .438 | −0.21, −0.71, 0.31 |
N3 sleep (min) | 92.3 ± 30.5 | 90.9 ± 48.2 | −1.4 ± 39.1 | 29.4 ± 24.7 | −0.1, 14, .894 | −0.04, −0.54, 0.47 |
REM sleep (min) | 103.7 ± 16.8 | 92.7 ± 34.8 | −11.0 ± 31.3 | 22.7 ± 23.5 | −1.4, 14, .194 | −0.34, −0.85, 0.17 |
Wake (min) | 75.8 ± 40.8 | 72.6 ± 39.2 | −3.1 ± 18.6 | 14.1 ± 11.9 | −0.6, 14, .524 | −0.17, −0.68, 0.34 |
Variable | PSG M ± SD | Somfit (EC) M ± SD | Mean Bias M ± SD | Absolute Bias M ± SD | Paired t-Test t, df, p | Effect Size d, Low, High |
---|---|---|---|---|---|---|
Total sleep time (min) | 478.6 ± 16.7 | 489.1 ± 17.9 | 10.4 ± 14.8 | 14.1 ± 10.6 | 1.9, 6, .112 | 0.70, −0.16, 1.51 |
N1 sleep (min) | 23.7 ± 9.6 | 18.9 ± 11.2 | −4.8 ± 7.2 | 7.1 ± 4.5 | −1.8, 6, .129 | −0.67, −1.47, 0.18 |
N2 sleep (min) | 246.9 ± 29.6 | 253.7 ± 43.0 | 6.8 ± 19.6 | 18.8 ± 4.9 | 0.9, 6, .394 | 0.35, −0.43, 1.10 |
N3 sleep (min) | 95.3 ± 30.4 | 110.6 ± 53.9 | 15.4 ± 46.7 | 33.6 ± 33.7 | 0.9, 6, .417 | 0.33, −0.45, 1.08 |
REM sleep (min) | 112.7 ± 17.4 | 105.8 ± 45.0 | −6.9 ± 39.3 | 23.4 ± 31.0 | −0.5, 6, .657 | −0.18, −0.92, 0.58 |
Wake (min) | 55.5 ± 17.0 | 50.9 ± 18.0 | −4.6 ± 15.9 | 14.1 ± 6.8 | −0.8, 6, .468 | −0.29, −1.04, 0.48 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Roach, G.D.; Miller, D.J.; Shell, S.J.; Miles, K.H.; Sargent, C. Validation of a Neurophysiological-Based Wearable Device (Somfit) for the Assessment of Sleep in Athletes. Sensors 2025, 25, 2123. https://doi.org/10.3390/s25072123
Roach GD, Miller DJ, Shell SJ, Miles KH, Sargent C. Validation of a Neurophysiological-Based Wearable Device (Somfit) for the Assessment of Sleep in Athletes. Sensors. 2025; 25(7):2123. https://doi.org/10.3390/s25072123
Chicago/Turabian StyleRoach, Gregory D., Dean J. Miller, Stephanie J. Shell, Kathleen H. Miles, and Charli Sargent. 2025. "Validation of a Neurophysiological-Based Wearable Device (Somfit) for the Assessment of Sleep in Athletes" Sensors 25, no. 7: 2123. https://doi.org/10.3390/s25072123
APA StyleRoach, G. D., Miller, D. J., Shell, S. J., Miles, K. H., & Sargent, C. (2025). Validation of a Neurophysiological-Based Wearable Device (Somfit) for the Assessment of Sleep in Athletes. Sensors, 25(7), 2123. https://doi.org/10.3390/s25072123