Next Article in Journal
An Energy-Efficient Sensing Matrix for Wireless Multimedia Sensor Networks
Next Article in Special Issue
A Customized Deep Sleep Recommender System Using Hybrid Deep Learning
Previous Article in Journal
Infrared Evanescent Wave Sensing Based on a Ge10As30Se40Te20 Fiber for Alcohol Detection
Previous Article in Special Issue
Validating Force Sensitive Resistor Strip Sensors for Cardiorespiratory Measurement during Sleep: A Preliminary Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 23
Issue 10
10.3390/s23104842
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Evaluation of the Accuracy of Contactless Consumer Sleep-Tracking Devices Application in Human Experiment: A Systematic Review and Meta-Analysis

by
Huifang Zhai
1,2,
Yonghong Yan
1,2,*,
Siqi He
3,
Pinyong Zhao
4 and
Bohan Zhang
5
1
Faculty of Architecture and Urban Planning, Chongqing University, Chongqing 400044, China
2
Key Laboratory of New Technology for Construction of Cities in Mountain Area, Chongqing University, Chongqing 400044, China
3
College of Landscape Architecture, Nanjing Forestry University, Nanjing 210037, China
4
College of Mathematics and Statistics, Chongqing University, Chongqing 400044, China
5
Faculty of Engineering, The University of Sydney, Camperdown, NSW 2006, Australia
*
Author to whom correspondence should be addressed.
Sensors 2023, 23(10), 4842; https://doi.org/10.3390/s23104842
Submission received: 28 January 2023 / Revised: 12 May 2023 / Accepted: 15 May 2023 / Published: 17 May 2023
(This article belongs to the Special Issue Sensors in Sleep Monitoring)
Browse Figures
Versions Notes

Abstract

:
Compared with the gold standard, polysomnography (PSG), and silver standard, actigraphy, contactless consumer sleep-tracking devices (CCSTDs) are more advantageous for implementing large-sample and long-period experiments in the field and out of the laboratory due to their low price, convenience, and unobtrusiveness. This review aimed to examine the effectiveness of CCSTDs application in human experiments. A systematic review and meta-analysis (PRISMA) of their performance in monitoring sleep parameters were conducted (PROSPERO: CRD42022342378). PubMed, EMBASE, Cochrane CENTRALE, and Web of Science were searched, and 26 articles were qualified for systematic review, of which 22 provided quantitative data for meta-analysis. The findings show that CCSTDs had a better accuracy in the experimental group of healthy participants who wore mattress-based devices with piezoelectric sensors. CCSTDs’ performance in distinguishing waking from sleeping epochs is as good as that of actigraphy. Moreover, CCSTDs provide data on sleep stages that are not available when actigraphy is used. Therefore, CCSTDs could be an effective alternative tool to PSG and actigraphy in human experiments.

1. Introduction

Sleep is an important indicator when analyzing fatigue [1,2], working performance [3,4], mood [5,6], circadian entrainment [7,8], etc. Some human experiments om large samples have been conducted in the field, with sleep monitoring performed at home [6,9,10]. Although the gold standard for measuring sleep, polysomnography (PSG), is the most precise device used to evaluate sleep quality, it is expensive and requires the assistance of specialists [11]. In addition, the discomfort caused by electrodes attached to the human body can interfere with normal sleep behavior, making it impossible to monitor long-term sleep in home environments [12]. Currently, actigraphy has become a widely used instrument for assessing sleep in human experiments [1,13,14,15]. Compared with PSG, actigraphy provides a convenient and feasible solution for long-term sleep monitoring in the home environment. However, actigraphy is still costly to use in sleep experiments with a large sample size as it requires simultaneity. Moreover, as a wearable device, some people feel uncomfortable and easily forget to wear them before sleep [16].
Contactless consumer sleep-tracking devices (hereinafter referred to as ‘CCSTDs’) have a low cost and are simple to operate. Whether it is a bedside device using technologies, such as radiofrequency identification, an infrared camera, or a mattress device that employs pressure and piezoelectric sensors, they make minimal or zero contact with the human body, minimizing interference with the user’s natural sleep behavior [17]. Studies on the accuracy, validity, and reliability of different brands of CCSTDs have been extensively conducted on different groups, including psychiatric patients [18], burn patients, rehabilitation patients [19], ICU patients [20], autistic children [21], dementia patients [22], elderly patients [22,23,24], and infants [25]. The findings suggest that CCSTDs have great advantages and potential for application in large-sample and long-period sleep experiments. The objective of this study is to evaluate whether CCSTDs can be widely used as a valid alternative tool to PSG and actigraphy in human experiments by assessing the accuracy and performance of CCSTDs with a systematic review of relevant publications and a meta-analysis of their reported data.

2. Method

This systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement www.prisma-statement.org (accessed on 2 April 2022). The protocol for this review was registered in the International Prospective Register of Systematic Reviews (PROSPERO: CRD42022342378, which is available at: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022342378, accessed on 2 April 2022).

2.1. Search Strategy

An online comprehensive search of the following databases was performed between April and August 2022: PubMed, EMBASE, Cochrane CENTRALE, and Web of Science.
The search was designed to find studies of CCSTDs that involved the validation of sleep outcomes using the gold standard, PSG. The following terms (including synonyms and medical subject headings terms) were used for the search: “sleep” and “contactless” or “brand name (e.g., Emfit, Beddit, ResMed, etc.)” and “polysomnography” (Mesh). The search was performed without date, language, or publication status restrictions. The search strategies for each database can be found in the Supplementary Materials Table S1.

2.2. Eligibility Criteria

Literature eligible for inclusion in the meta-analysis was as follows: (1) contactless sleep monitoring products were commercially available and of consumer grade. (2) One or more of the following outcomes were reported: TST, total sleep time; WASO, wake after sleep onset; SOL, sleep onset latency; SE, sleep efficiency; light sleep (N1 + N2); deep sleep (N3); REM, rapid eye movement; sensitivity, specificity, and accuracy in detecting sleep epochs; (3) sleep data from PSG or professional sleep monitoring instruments with equivalent validity were used as the reference standard; (4) the experimental subjects were human.
Exclusion criteria included (1) fewer than five subjects, (2) conference papers, case series, and case reports, (3) absent or inappropriate statistical analysis, and (4) sleep tracking apps on smartphones.

2.3. Data Extraction

Characteristics for each study were extracted and collated in a standardized table containing the first author, year of publication, sample characteristics (sample size, age, health status, and country), sleep device characteristics (name, sensor, and placement location), reference, sleep site, duration of the assessment, and bedtime. The study outcomes relative to the denoted reference standard, total sleep time (TST), sleep onset latency (SOL), wake after sleep onset (WASO), sleep efficiency (SE), light sleep, deep sleep, and rapid eye movement (REM), as well as the sensitivity, specificity, and accuracy of detecting sleep epochs, were extracted. Endpoints reported as median and range were transformed into mean and standard deviation using the method of Hozo et al. [26].

2.4. Bias Assessment

The studies’ quality was assessed (unblinded) using the standards of the quality assessment of diagnostic accuracy studies, version 2 (QUADAS-2) [27] (see Supplementary Materials (Figures S1 and S2)).

2.5. Statistical Analysis

For studies that compared CCSTDs with PSG and provided quantitative data, the mean, standard deviation, and sample size were used to produce a pooled estimated mean effect size and 95% confidence intervals (CI) using a random effects model. Heterogeneity among studies was determined by calculating I2 statistics. I2 values in the order of 25%, 50%, and 75% were considered as having small, medium, and large heterogenicity, respectively [28]. A threshold probability of 5% (p = 0.05) was selected as the basis for rejecting the null hypothesis. As recommended, the threshold probability of 10% (p = 0.10) was the basis for testing the significance of heterogenicity, and also, for determining the statistical significance of subgroup comparisons [29]. Meta-analyses were performed using Stata version 14, with metan, metafunnel, and metabias packages. In addition, Review Manager 5.4.1 and R program were used for analysis in this study.

3. Results

3.1. Study Selection

Figure 1 presents a summary of the selection and qualification of articles that were reviewed. A total of 1744 publications were retrieved through searching the databases. An additional publication was identified in the reference list of identified articles. After removing duplicate publications found in multiple databases, 1165 articles remained for screening. The examination of individual titles and abstracts yielded 137 publications for the full-text appraisal. However, after scrutinizing each of them according to a priori inclusion and exclusion criteria, only 26 articles were qualified for systematic review [17,20,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53]. Out of the 26 articles, one did not evaluate PSG and CCSTD simultaneously [30], which likely led to a bias; three did not include data of meta-analysis [30,39,41,46]. Finally, 22 articles qualified for meta-analysis [17,20,31,32,33,34,35,36,37,38,40,42,43,44,45,47,48,49,50,51,52,53].

3.2. Main Characteristics of the Studies Included in the Review

The descriptive characteristics of 26 included articles are summarized in Table 1, and the results from qualified publications on CCSTDs and actigraphy versus PSG can be found in the Supplementary Materials Table S2. These articles described 32 studies of CCSTDs. Among the studies that used PSG as the standard test, five studies compared CCSTDs and actigraphy using the same cohort [34,35,42,49,53]. Their performance was tested against that of PSG by making participants wear both CCSTD and an actigraphy device. CCSTDs use two types of device, i.e., mattress-based devices and bedside devices. Out of 26 articles, 11 (42%) studied mattress-based devices, 12 (46%) studied bedside devices, 2 (8%) studied both mattress-based and bedside devices [49,51], and 1 (4%) studied bedside an automatic video device [36]. The most common device sensors used in CCSTDs were piezoelectric sensors (n = 9), pressure sensors (n = 5), radiofrequency sensors (n = 17), and infrared cameras (n = 1). Figure 2 shows the different types of sensors and their sleep monitoring mechanism. Piezoelectric or pressure sensors are used in mattress-based devices, and radiofrequency sensors and infrared cameras are used in bedside devices. These sensors could monitor two or four physiological signals in real time, including heart rate, respitation rate, body movement, and sleep posture, and data were analyzed to determine the sleep status via built-in algorithms. These studies were performed in 11 countries. The number of participants ranged from 5 to 198, and they aged from 16 to 84 years. The participants included normal sleepers and those diagnosed with sleep (PLMS), obstructive sleep apnea (OSA), sleep-disordered breathing (SDB), central disorders of hypersomnolence (CDH), insomnia, diabetes, hypertension, arthritis, perioperative, and septic shock, according to periodic limb movements. Feng et al. [45] was included in the patient group in the meta-analysis since only one of the subjects did not have OSA. Among twenty-six articles, experiments in twenty-one articles (81%) were conducted in sleep laboratories, one (4%) was conducted in the home environment, three (11%) were conducted either at home or in a sleep laboratory, and one was conducted (4%) in an ICU [20].

3.3. Publication Bias

According to the Cochrane handbook, a publication bias test was performed on the outcome indicators included in the meta-analysis for more than ten studies. The Funnel plot showed no publication bias (see Supplementary Materials Figure S3). Egger’s test (PTST = 0.608, PSOL = 0.595, PSE = 0.458, PWASO = 0.637, Plightsleep = 0.172, Pdeepsleep = 0.997, and PREM = 0.487) and Begg’s test (PTST = 0.733, PSOL = 0.434, PSE = 0.707, PWASO = 0.620, Plightsleep = 0.371, Pdeepsleep = 0.721, and PREM = 0.592) indicated no publication bias.

3.4. Meta-Analysis on Accuracy of Sleep Parameters Assessed Using CCSTDs Versus PSG

3.4.1. Principal Analysis

Out of the 26 articles, 22 articles were included in the meta-analysis. Of the four articles excluded, three articles reported their results without raw data, and in one article, the CCSTD was not monitored at the same time as the gold standard, PSG, in the experiment [30], which likely led to a bias. The articles included described thirty-six studies in total, with six articles each reporting from two to five studies of CCSTDs. A summary of the values for each outcome is provided in Table 2. The pooled estimate of effect size reveals the following results on CCSTDs relative to PSG: significant overestimations of TST (N = 35 studies; pooled mean = 19.55 min, 95% CI from 12.22 to 26.88; p < 0.001), SE (N = 28 studies; pooled mean = 2.88%, 95% CI from 1.58% to 4.17%; p < 0.01), deep sleep (N = 10 studies; pooled mean = 11.07 min, 95% CI from 0.37 to 21.76; p < 0.05); significant underestimation of SOL (N = 17 studies; pooled mean = −4.61 min, 95% CI from −6.56 to −2.66; p < 0.01), WASO (N = 16 studies; pooled mean = −12.07 min, 95% CI from −18.75 to −5.38; p < 0.01); non-significant differences in the estimation of light sleep (N = 10 studies; pooled mean = 5.62 min, 95% CI from −12.81 to 24.06; p = 0.550) and REM (N = 10 studies; pooled mean = 3.44 min, 95% CI from −14.81, 21.68; p = 0.823). All outcome indexes showed medium and large degrees of heterogeneity, except SE (I2 = 34.2%, p = 0.041).

3.4.2. Subgroup Analyses

The results of subgroup analyses of sensors, device type, participant type, and brands compared with those of PSG are presented in the Supplementary Materials Tables S3–S6 and Figures S4–S31.
The subgroup analyses of sensors reveal that there were no significant differences for the piezoelectric sensor in the estimation of TST, SOL, WASO, and REM; for the pressure sensor in the estimation of SOL, SE, and deep sleep; for the radiofrequency sensor in the estimation of sleep stages; for infrared camera in the estimation of TST. In the subgroup of device types, there were no significant differences for the mattress-based devices in the estimation of SOL, WASO, and sleep stages or for the bedside devices in the estimation of sleep stages. In the subgroup of participant types, there were no significant differences among healthy participants in assessing sleep stages; for patient who participated in REM studies; for healthy and patient who participated in WASO and deep sleep studies. The results of the subgroup analyses for the brand of devices showed no significant differences for the use of ResMed S+ in terms of all the sleep parameters; for the use of Beddit in terms of SOL, WASO, and SE; for the use of Sonomat in terms of TST; for the use of other brands of mattress devices in terms of SOL, SE, and sleep stages; for the use of other brands of bedside devices in terms of TST, SOL, SE, light sleep, and REM.

3.5. Accuracy, Sensitivity, and Specificity in Detecting Sleep Epochs by CCSTDs Versus PSG

3.5.1. Sleep and Wake Epoch Identification

A total of twelve studies (n = 1200 samples) involved an epoch-by-epoch (EBE) investigation of CCSTDs, taking PSG as a reference for sleeping and awake state determination [20,31,32,34,35,39,40,41,42,43,48,49]. Out of these twelve studies, four consisted of two or five different samples, thereby increasing the total number of evaluations to twenty-four. Across those trials, CCSTDs versus PSG analyses identified sleep epochs with accuracy values between 0.68 and 0.91 (0.81 ± 0.07) (N = 22 studies), sensitivity values between 0.75 and 0.97 (0.90 ± 0.06) (N = 23 studies), and specificity values between 0.37 and 0.80 (0.51 ± 0.12) (N = 23 studies). Figure 3 displays error bars of the EBE agreement for sleeping and awake states in terms of sensors, the device type, the health conditions of the participants, and the brand of device. As only one study [20] provided EBE data given by the pressure sensor, it was not included in the error bar of the sensors subgroup. The analysis of four subgroups showed a high degree of consistency in identifying asleep and awake states, which related to high sensitivity and low specificity, respectively. Compared to the mean values in the subgroups, the results of the EBE agreement for asleep and awake states in terms of accuracy, sensitivity, and specificity are as follows: the piezoelectric sensor (accuracy: N = 2, 0.90 ± 0.01; sensitivity: N = 3, 0.92 ± 0.05; specificity: N = 3, 0.56 ± 0.21) was better than the radiofrequency sensor was (accuracy: N = 19, 0.81 ± 0.06; sensitivity: N = 19, 0.90 ± 0.07; specificity: N = 19, 0.51 ± 0.11); the mattress-based one (accuracy: N = 3, 0.83 ± 0.13; sensitivity: N = 4, 0.91 ± 0.04; specificity: N = 4, 0.52 ± 0.19) was slightly better than the bedside one was (accuracy: N = 19, 0.81 ± 0.06; sensitivity: N = 19, 0.90 ± 0.07; specificity: N = 19, 0.51 ± 0.11); healthy patients (N = 13, 0.94 ± 0.03) had the best sensitivity results; the healthy + patient group (N = 5, 0.53 ± 0.16) was slightly better than the healthy group was (N = 13, 0.52 ± 0.12) in terms of specificity; the healthy (N = 13, 0.84 ± 0.06) and healthy + patient groups (N = 4, 0.82 ± 0.06) generally had consistent accuracy, and the patient group had the worst scores of all; the others group (B) (N = 3, 0.96 ± 0.02) scored the best in terms of sensitivity, and ResMed S had a higher accuracy (N = 4, 0.84 ± 0.09) and specificity (N = 4, 0.59 ± 0.15) than the other brands did. The coefficients of variation (CV) of the piezoelectric sensor, mattress-based devices, the healthy group, and the others group (B) had the lowest sensitivity scores, while the CVs of the radiofrequency sensor, bedside devices, the patient group, and SleepMinder had the lowest specificity results.

3.5.2. Sleep Stage Identification

A total of eight studies (n = 390 samples) appraised the performance of CCSTDs using sleep staging functions in identifying sleep stages via EBE analysis [17,39,40,41,42,43,48,49]. Out of these eight studies, two consisted of two or three different samples, thereby increasing the total number of evaluations to eleven. Figure 4 displays the error bar of the EBE agreement between sleep stages. Compared with PSG, CCSTDs had the highest sensitivity (N = 4, 0.64 ± 0.05) and the lowest specificity (N = 4, 0.60 ± 0.08) and accuracy (N = 9, 0.63 ± 0.06) in detecting light sleep. REM showed the opposite result: the highest specificity (N = 6, 0.91 ± 0.05) and accuracy (N = 9, 0.75 ± 0.13), but the lowest sensitivity (N = 6, 0.49 ± 0.20). Deepsleep (CV = 0.06) had the smallest coefficient of variation in specificity, but a high level of specificity (N = 5, 0.88 ± 0.03). Light sleep had the smallest coefficient of variation in terms of sensitivity (CV = 0.08) and accuracy (CV = 0.09).

3.6. Comparison of Sleep Parameters Assessed via CCSTDs Versus Actigraphy

Five studies investigated the accuracy of CCSTDs relative to that of actigraphy [34,35,42,49,53]. As reported in Table 1 and Table S2, the participants in one study were patients undergoing an assessment for obstructive sleep apnoea syndrome [35], and participants in the other four studies were healthy people [34,42,49,53]. Two of those studies compared an early or newer version of ResMed S+ and Beddit and employed actigraphy [42,53]. ResMed S + V1 and ResMed S + V2 demonstrated a similar degree of sensitivity (ResMed S + V1 = 0.948; ResMed S + V2 = 0.938; actigraphy = 0.966), a higher degree of specificity (ResMed S + V1 = 0.695; ResMed S + V2 = 0.731; actigraphy = 0.476), and a similar degree of accuracy (ResMed S + V1 = 0.875; ResMed S + V2 = 0.876; actigraphy = 0.851) [42]. Beddit 3.5 showed a higher degree of agreement with PSG in terms of TST (ICC = 0.998), SE (ICC = 0.98), SOL (ICC = 0.92), and WASO (ICC = 0.92) than it did with actigraphy (TST (ICC = 0.96), SE (ICC = 0.81), SOL (ICC = 0.74), and WASO (ICC = 0.44)), while Beddit 3.0 showed a low degree of agreement with PSG, and its TST (ICC = 0.13), SE (ICC = 0.26), SOL (ICC = 0.02), and WASO (ICC = −0.02) were lower than those of actigraphy [53].
Three articles compared five different brands of CCSTDs using actigraphy [34,35,49]. One article compared actigraphy with SleepMinder. The results showed that actigraphy overestimated TST (57 min) and SE (8.8%), and SleepMinder overestimated TST (10 min) and SE (0.8%). Compared to actigraphy, SleepMinder demonstrated a similar degree of accuracy for sleep/wake determination (SleepMinder = 0.773; actigraphy = 0.765), a lower degree of sensitivity (SleepMinder = 0.864; actigraphy = 0.938), and a higher degree of specificity (SleepMinder = 0.518; actigraphy = 0.341) [35]. Another article compared two brands of devices with actigraphy and reported similar results [34]. SleepMinder (0.856), HSL-101 (0.856), and actigraphy (0.855) had similar degrees of accuracy for sleep/wake determination. SleepMinde and HSL-101 showed less sensitivity (SleepMinder = 0.953; HSL-101 = 0.964; actigraphy = 0.973), more specificity (SleepMinder = 0.389; HSL-101 = 0.358; actigraphy = 0.339), as well as slightly improved estimates of TST, SOL, WASO, and SE than actigraphy did. The other article compared three brands of CCSTDs with actigraphy. TST, SE, SOL, and WASO estimations using EarlySense Live, ResMed S+, and SleepScore Max were superior to those of actigraphy. The degree of sensitivity of EarlySense Live (0.96), ResMed S+ (0.93), and SleepScore Max (0.94) was slightly lower than that of actigraphy (0.97). However, the specificities of EarlySense Live (0.47), ResMed S+ (0.51), and SleepScore Max (0.50) were much higher than those of actigraphy (0.39), and the accuracies of EarlySense Live (0.90), ResMed S+ (0.88), and SleepScore Max (0.88) were the same as that of actigraphy (0.89) [49]. In summary, the findings in these three articles are consistent. Compared to actigraphy, CCSTDs have similar degrees of accuracy, less sensitivity, but more specificity.

4. Discussion

The aim of the study was to assess the feasibility of CCSTDs as an alternative to PSG and actigraphy using large sample size or via the long-term monitoring of humans through a comprehensive analysis of the effectiveness of CCSTDs. The findings are based on a review of relevant published articles and a meta-analysis.
According to the qualified publications, all CCSTDs can track sleep and wake, among which ResMed S+, EarlySense Live, and Somnofy can also monitor sleep stages. Compared to PSG, CCSTDs overestimated TST, SE and deep sleep, and underestimated SOL and WASO, but showed non-significant difference for light sleep only. The EBE analysis of asleep and awake participants showed that CCSTDs had a high degree of sensitivity (0.90 ± 0.06), but a relatively low degree of specificity (0.51 ± 0.12), indicating a tendency for the devices to accurately detect sleep epochs, but less accurately detect awake epoch. Moreover, there is a wide range of values for accuracy (i.e., between 0.68 and 0.91) of sleep and wake epoch identification. In terms of EBE agreement for sleep stages, the degree of sensitivity was relatively low for light sleep, deep sleep, and REM. The degree of specificity was relatively high, with a narrower range of values for deep sleep, and REM, but low values for light sleep. This indicates an overall poorer and inconsistent ability of CCSTDs to correctly detect sleep stage epochs [17,49]. The results of the analysis of the accuracy for EBE agreement for sleep stages confirm this. In summary, the accuracy of CCSTDs varied widely, which could be attributed to the varied health states of participants and different sensors and algorithms used in CCSTDs.
The health state of participants is one of the most important factors affecting the accuracy of CCSTD sleep monitoring. For healthy participants with normal sleep patterns and behaviors, the CCSTDs results did not significantly differ in assessing sleep stages compared to those of PSG, but for the patient participants, CCSTDs did not show significant difference in terms of REM only. In the cases when healthy participants were involved, CCSTDs demonstrated much higher degrees of accuracy, sensitivity, and specificity than when patients were involved in the EBE analysis of asleep and awake epochs. The same conclusion was confirmed in previous studies that used the same device to study healthy participants and patients [20,31,35,55]. This is because patients who suffer from disturbed sleep are prone to repeated arousal during sleep [56]. The more wakefulness occurs, the more erroneous data there are, leading to sleep overestimation [57]. Hence, sleep/wake identification becomes more inaccurate for patients.
The sensor is the most important factor affecting the accuracy of CCSTDs in sleep monitoring. CCSTDs are used to sense the sleep state by detecting one or more physiological signals, such as chest and abdominal breathing movements, heart movements, and body movements using sensors, such as radiofrequency, infrared light, pressure, and piezoelectricity ones, and microphones [58]. The subgroup analysis of sensors showed that both piezoelectric and radiofrequency sensors performed well in sleep monitoring. However, the piezoelectric sensor had high degree of accuracy in terms of both wakefulness and sleep, while the radiofrequency sensor had high degree of accuracy in terms of sleep only. This is consistent with the results of the EBE analysis, where the piezoelectric sensor had excellent accuracy and sensitivity, and the radiofrequency sensor excelled only in terms of sensitivity. The reason for this is that the piezoelectric sensor collects data on heartbeat rate and respiratory rate, body movements, and sleep postures during sleep, while the radiofrequency sensor obtains sleep data by detecting breathing and body movements [59]. In addition, it is difficult to detect small movements within the body, such as heartbeat or pulse, using current radiofrequency technology, and it cannot monitor multiple people simultaneously [58]. Therefore, the radiofrequency sensor is slightly worse than the piezoelectric sensor in terms of sleep monitoring performance. This could also explain the results of the meta-analysis in which mattress-based devices performed slightly better than bedside devices did in terms of the mean values of sensitivity and specificity for monitoring the sleep period. The other advantages of mattress-based devices over bedside devices are that they not only allow small movements within the body to be more easily detected, but they also allow multiple people in the same room to be monitored simultaneously [16].
It is worth noting that some bedside instruments with better sensor performances and better algorithms show as accuracy values as high as those of mattress instruments, such as ResMed S+, EarlySense, and Somnofy [42,48,49], indicating a significant difference among different brands. The results of the subgroup meta-analysis by brand confirm this. For example, ResMed S+ performed as well PSG did in measuring the sleep parameters in this review.
Therefore, CCSTDs, especially, mattress-based devices with a built-in piezoelectric sensor, demonstrated excellent performances in the sleep monitoring of healthy populations. They can be used as an alternative tool to PSG in monitoring overall sleep conditions in long-period and large-sample healthy population experiments, rather than experiments that require precise data because CCSTDs overestimation or underestimation some sleep parameters to some degree [43,47,49,53].
Actigraphy devices have been extensively validated [34,60], and found to be highly sensitive (0.965) and accurate (0.863), but poorly specific (0.329) to sleep [61]. Actigraphy is used to estimate asleep/awake epochs by measuring body movements using accelerometers. The definition of sleep is a lack of movement. Thus, the brief awake periods that produce a small amount of motion and physical stillness that precede sleep onset are often classified incorrectly as sleep, leading to low specificity, overestimated TST and SE, and underestimated SOL and WASO, especially among individuals with sleep disturbances [61,62,63,64]. The specificity for studies that have included healthy participants ranged from 0.269 to 0.77 [56,65,66,67,68,69,70,71,72,73], whereas others that included a variety of patient groups report specificity values that range from 0.325 to 0.80 [61,74,75,76]. A low degree of specificity is the main limitation of sleep monitoring via actigraphy. Further limitations concern the inability to discriminate between sleep stages. Studies that compared CCSTDs with actigraphy [34,35,42,49,53] revealed that the CCSTDs demonstrated a similar degree of accuracy, a higher degree of specificity, and a lower degree of sensitivity than actigraphy did for sleep/wake epoch identification. TST, SE, SOL, and WASO estimation using CCSTDs were superior to those of actigraphy. The accuracies of actigraphy systems varies widely and are typically dependent on the observed population [77]. The accuracy of actigraphy in determining asleep and awake epochs is reasonably high in the normal subjects [78,79,80]. The American Academy of Sleep Medicine Practice Guidelines indicates that actigraphy represents a reliable method of measuring sleep in the normal healthy adult population [81]. However, the differences in accuracy are large in the presence of sleep disorders, such as sleep-disordered breathing, insomnia, and periodic limb movements [82,83]. For example, in sleep-disordered breathing patients, a per-epoch accuracy of 0.80–0.86 has been reported, and the accuracy tends to be less in severe apnea cases (0.861 in the normal group versus 0.799 in the severe OSA group) [77]. This difference is consistent with the results of EBE agreement for asleep/awake epochs in the subgroup of participant type in this study. In summary, this study shows that CCSTDs can be a valid alternative to the silver standard of actigraphy for sleep monitoring. In addition, CCSTDs have the advantage of detecting sleep stages.
However, the included studies have certain limitations. First, the participants were mostly monitored for one night. Second, there is a lack of published information, such as that about the algorithm and the reliability of hardware, to explain the different performances of various CCSTDs. Third, most of the included studies analyzed only one or two sleep indicators using the device, which may have led to a bias in the final results. In addition, a wide range of CCSTDs have appeared on the market. They may have the problems of unstandardized, undisclosed, and unvalidated data and algorithms [54,84]. Hence, in the face of the constant introduction of new devices and algorithms, it is essential to refer to the gold standard, PSG, and perform validation evaluations for specific populations before using them in human experiments.

5. Conclusions

This systematic review and meta-analysis have shown that CCSTDs, especially, mattress-based devices with a piezoelectric sensor, have a better accuracy when they are used in experiments involving healthy participants. They can be used as an alternative tool to PSG for long-period and large-sample human experiments that do not require precise data. Furthermore, CCSTDs demonstrate accuracy values as high as those for actigraphy and can provide data on sleep stages that are not available with the use of actigraphy. Therefore, CCSTDs can be an effective alternative tool to actigraphy for monitoring sleep.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/s23104842/s1.

Author Contributions

Conceptualization, Y.Y. and H.Z.; methodology, H.Z.; software, H.Z. and P.Z.; validation, H.Z. and P.Z.; formal analysis, H.Z.; investigation, H.Z. and S.H.; resources, H.Z. and S.H.; data curation, H.Z.; writing—original draft preparation, H.Z.; writing—review and editing, H.Z., Y.Y., S.H. and B.Z.; visualization, H.Z. and P.Z.; supervision, Y.Y.; project administration, H.Z.; funding acquisition, Y.Y. and H.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the National Natural Science Foundation of China (Grant No. 51978097) and graduate research and innovation foundation of Chongqing, China (Grant No. CYB20035).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank Ning Yan from University-Town Hospital of Chongqing Medical University and Baojie He from Chongqing University for advice on this review.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Maki, K.A.; Fink, A.M.; Weaver, T.E. Sleep, time, and space-fatigue and performance deficits in pilots, commercial truck drivers, and astronauts. Sleep Adv. A J. Sleep Res. Soc. 2022, 3, zpac033. [Google Scholar] [CrossRef] [PubMed]
  2. Hong, R.M.; Hsu, C.Y.; Hu, C.J. Exploring relationships between health-related lifestyle habits and fatigue among flight attendants and trainees. Work 2022, 74, 1361–1369. [Google Scholar] [CrossRef] [PubMed]
  3. Uji, M.; Tamaki, M. Sleep, learning, and memory in human research using noninvasive neuroimaging techniques. Neurosci. Res. 2022, 189, 66–74. [Google Scholar] [CrossRef]
  4. Drozdowska, A.; Falkenstein, M.; Lücke, T.; Kersting, M.; Jendrusch, G.; Platen, P.; Sinningen, K. Unfavorable Behaviors in Children Run in Packs! Dietary and Non-Dietary Modulators of Attentional Capacity. Nutrients 2022, 14, 5264. [Google Scholar] [CrossRef]
  5. David, M.; Vieira, G.R.; de Lima Leoncio, L.M.; Dos Santos Neves, L.; Bezerra, C.G.; de Mattos, M.S.B.; Dos Santos, N.F.; Antunes, R.B.; Silva, D.M.; Araujo, J.F.; et al. Different sleep patterns during the COVID-19 pandemic: Association with mood, exercise and light exposure. Psychol. Health Med. 2022, 28, 1–13. [Google Scholar] [CrossRef] [PubMed]
  6. Menghini, L.; Yuksel, D.; Prouty, D.; Baker, F.C.; King, C.; de Zambotti, M. Wearable and mobile technology to characterize daily patterns of sleep, stress, presleep worry, and mood in adolescent insomnia. Sleep Health 2022, 9, 108–116. [Google Scholar] [CrossRef] [PubMed]
  7. Knufinke, M.; Nieuwenhuys, A.; Geurts, S.A.E.; Most, E.I.S.; Moen, M.H.; Maase, K.; Coenen, A.M.L.; Gordijn, M.C.M.; Kompier, M.A.J. Dim light, sleep tight, and wake up bright—Sleep optimization in athletes by means of light regulation. Eur. J. Sport Sci. 2021, 21, 7–15. [Google Scholar] [CrossRef] [PubMed]
  8. Sharkey, K.M.; Carskadon, M.A.; Figueiro, M.G.; Zhu, Y.; Rea, M.S. Effects of an advanced sleep schedule and morning short wavelength light exposure on circadian phase in young adults with late sleep schedules. Sleep Med. 2011, 12, 685–692. [Google Scholar] [CrossRef]
  9. He, S.; Yan, Y. Impact of advance light exposure on assembly-line workers’ subjective work alertness and sleep quality. Light. Res. Technol. 2022, 55, 105–128. [Google Scholar] [CrossRef]
  10. Morrison, S.; Haszard, J.J.; Galland, B.C.; Jackson, R.; Meredith-Jones, K.A.; Elder, D.E.; Taylor, R.W. Where does the time go when children don’t sleep? A randomized crossover study. Obesity 2022, 31, 625–634. [Google Scholar] [CrossRef]
  11. Pandi-Perumal, S.R.; Spence, D.W.; BaHammam, A.S. Polysomnography: An Overview. In Primary Care Sleep Medicine; Springer: Berlin/Heidelberg, Germany, 2014; pp. 29–42. [Google Scholar] [CrossRef]
  12. Ibanez, V.; Silva, J.; Navarro, E.; Cauli, O. Sleep assessment devices: Types, market analysis, and a critical view on accuracy and validation. Expert Rev. Med. Devices 2019, 16, 1041–1052. [Google Scholar] [CrossRef] [PubMed]
  13. Lujan, M.R.; Perez-Pozuelo, I.; Grandner, M.A. Past, Present, and Future of Multisensory Wearable Technology to Monitor Sleep and Circadian Rhythms. Front Digit Health 2021, 3, 721919. [Google Scholar] [CrossRef] [PubMed]
  14. Miles, K.H.; Clark, B.; Fowler, P.M.; Miller, J.; Pumpa, K.L. What are the sleep characteristics of elite female athletes? A systematic review with meta-analysis. Biol. Sport 2022, 39, 751–763. [Google Scholar] [CrossRef]
  15. de Zambotti, M.; Cellini, N.; Menghini, L.; Sarlo, M.; Baker, F.C. Sensors Capabilities, Performance, and Use of Consumer Sleep Technology. Sleep Med. Clin. 2020, 15, 1–30. [Google Scholar] [CrossRef] [PubMed]
  16. Li, F.Y.; Valero, M.; Clemente, J.; Tse, Z.; Song, W.Z. Smart Sleep Monitoring System via Passively Sensing Human Vibration Signals. IEEE Sens. J. 2021, 21, 14466–14473. [Google Scholar] [CrossRef]
  17. Kholghi, M.; Szollosi, I.; Hollamby, M.; Bradford, D.; Zhang, Q. A validation study of a ballistocardiograph sleep tracker against polysomnography. J. Clin. Sleep Med. 2022, 18, 1203–1210. [Google Scholar] [CrossRef] [PubMed]
  18. Heglum, H.S.A.; Kallestad, H.; Vethe, D.; Langsrud, K.; Sand, T.; Engstrøm, M. Distinguishing sleep from wake with a radar sensor: A contact-free real-time sleep monitor. Sleep 2021, 44, zsab060. [Google Scholar] [CrossRef]
  19. Hendriks, M.M.S.; van Lotringen, J.H.; Vos-van der Hulst, M.; Keijsers, N.L.W. Bed Sensor Technology for Objective Sleep Monitoring Within the Clinical Rehabilitation Setting: Observational Feasibility Study. JMIR Mhealth Uhealth 2021, 9, e24339. [Google Scholar] [CrossRef]
  20. Nagatomo, K.; Masuyama, T.; Iizuka, Y.; Makino, J.; Shiotsuka, J.; Sanui, M. Validity of an under-mattress sensor for objective sleep measurement in critically ill patients: A prospective observational study. J. Intensive Care 2020, 8, 16. [Google Scholar] [CrossRef]
  21. Adkins, K.W.; Goldman, S.E.; Fawkes, D.; Surdyka, K.; Wang, L.L.; Song, Y.N.; Malow, B.A. A Pilot Study of Shoulder Placement for Actigraphy in Children. Behav. Sleep Med. 2012, 10, 138–147. [Google Scholar] [CrossRef]
  22. Ravindran, K.K.G.; Monica, C.D.; Atzori, G.; Enshaeifar, S.; Mahvash-Mohammadi, S.; Dijk, D.J.; Revell, V. Validation of technology to monitor sleep and bed occupancy in older men and women. Alzheimer’s Dement. J. Alzheimer’s Assoc. 2021, 17, e056018. [Google Scholar] [CrossRef]
  23. Schütz, N.; Saner, H.; Botros, A.; Pais, B.; Santschi, V.; Buluschek, P.; Gatica-Perez, D.; Urwyler, P.; Müri, R.M.; Nef, T. Contactless Sleep Monitoring for Early Detection of Health Deteriorations in Community-Dwelling Older Adults: Exploratory Study. JMIR Mhealth Uhealth 2021, 9, e24666. [Google Scholar] [CrossRef] [PubMed]
  24. Lach, H.W.; Lorenz, R.A.; Palmer, J.L.; Koedbangkham, J.; Noimontree, W. Home Monitoring to Track Activity and Sleep Patterns Among Older Adults: A Feasibility Study. Comput. Inform. Nurs. CIN 2019, 37, 628–637. [Google Scholar] [CrossRef] [PubMed]
  25. Zhang, Z.; Liu, X. Effect of the sleep nursing intervention on sleep status and gross motor development of infants. Chin. J. Behav. Med. Brain Sci. 2019, 28, 881–886. [Google Scholar]
  26. Hozo, S.P.; Djulbegovic, B.; Hozo, I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med. Res. Methodol. 2005, 5, 13. [Google Scholar] [CrossRef]
  27. Whiting, P.F.; Rutjes, A.W.S.; Westwood, M.E.; Mallett, S.; Deeks, J.J.; Reitsma, J.B.; Leeflang, M.M.G.; Sterne, J.A.C.; Bossuyt, P.M.M.; Grp, Q. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann. Intern. Med. 2011, 155, 529–536. [Google Scholar] [CrossRef]
  28. Higgins, J.P.T.; Thompson, S.G.; Deeks, J.J.; Altman, D.G. Measuring inconsistency in meta-analyses. Br. Med. J. 2003, 327, 557–560. [Google Scholar] [CrossRef]
  29. Richardson, M.; Garner, P.; Donegan, S. Interpretation of subgroup analyses in systematic reviews: A tutorial. Clin. Epidemiol. Glob. Health 2019, 7, 192–198. [Google Scholar] [CrossRef]
  30. Zhang, Q.-F.; Tong, Y.-F.; She, C.-P.; Zhang, X.-R.; Song, W.; Wang, L.-M.; Cheng, C.-J. Comparative analysis of polysomnography and micro-sensitive mattress-sleep monitor used for obstructive sleep apnea hypopnea syndrome. Chin. J. Otorhinolaryngol. Head Neck Surg. 2010, 45, 1014–1019. [Google Scholar]
  31. De Chazal, P.; Fox, N.; O’Hare, E.; Heneghan, C.; Zaffaroni, A.; Boyle, P.; Smith, S.; O’Connell, C.; McNicholas, W.T. Sleep/wake measurement using a non-contact biomotion sensor. J. Sleep Res. 2011, 20, 356–366. [Google Scholar] [CrossRef]
  32. Hashizaki, M.; Nakajima, H.; Tsutsumi, M.; Shiga, T.; Chiba, S.; Yagi, T.; Ojima, Y.; Ikegami, A.; Kawabata, M.; Kume, K. Accuracy validation of sleep measurements by a contactless biomotion sensor on subjects with suspected sleep apnea. Sleep Biol. Rhythm. 2014, 12, 106–115. [Google Scholar] [CrossRef]
  33. Norman, M.B.; Middleton, S.; Erskine, O.; Middleton, P.G.; Wheatley, J.R.; Sullivan, C.E. Validation of the Sonomat: A contactless monitoring system used for the diagnosis of sleep disordered breathing. Sleep 2014, 37, 1477–1487. [Google Scholar] [CrossRef] [PubMed]
  34. O’Hare, E.; Flanagan, D.; Penzel, T.; Garcia, C.; Frohberg, D.; Heneghan, C. A comparison of radio-frequency biomotion sensors and actigraphy versus polysomnography for the assessment of sleep in normal subjects. Sleep Breath. 2014, 19, 1–8. [Google Scholar] [CrossRef] [PubMed]
  35. Pallin, M.; O’Hare, E.; Zaffaroni, A.; Boyle, P.; Fagan, C.; Kent, B.; Heneghan, C.; de Chazal, P.; McNicholas, W.T. Comparison of a novel non-contact biomotion sensor with wrist actigraphy in estimating sleep quality in patients with obstructive sleep apnoea. J. Sleep Res. 2014, 23, 475–484. [Google Scholar] [CrossRef] [PubMed]
  36. Abad, J.; Munoz-Ferrer, A.; Cervantes, M.A.; Esquinas, C.; Marin, A.; Martinez, C.; Morera, J.; Ruiz, J. Automatic Video Analysis for Obstructive Sleep Apnea Diagnosis. Sleep 2016, 39, 1507–1515. [Google Scholar] [CrossRef] [PubMed]
  37. Terjung, S.; Wang, Y.; Werther, S.; Zaffaroni, A.; Teschler, H.; Weinreich, G. Validation of SleepMinder® for sleep quality evaluation in patients with OSAS. Somnologie 2016, 20, 54–60. [Google Scholar] [CrossRef]
  38. Norman, M.B.; Pithers, S.M.; Teng, A.Y.; Waters, K.A.; Sullivan, C.E. Validation of the sonomat against PSG and quantitative measurement of partial upper airway obstruction in children with sleep-disordered breathing. Sleep 2017, 40, zsx017. [Google Scholar] [CrossRef]
  39. Tal, A.; Shinar, Z.; Shaki, D.; Codish, S.; Goldbart, A. Validation of contact-free sleep monitoring device with comparison to polysomnography. J. Clin. Sleep Med. 2017, 13, 517–522. [Google Scholar] [CrossRef]
  40. Zaffaroni, A.; Doheny, E.P.; Gahan, L.; Ivanov, Y.; Kilroy, H.; O’Mahony, N.; O’Rourke, D. Non-Contact Estimation of Sleep Staging. In Joint Conference of the European Medical and Biological Engineering Conference (EMBEC), Proceedings of the Nordic-Baltic Conference on Biomedical Engineering and Medical Physics (NBC), Tampere, Finland, 11–15 June 2018; Springer: Berlin/Heidelberg, Germany, 2018; pp. 77–80. [Google Scholar]
  41. Chung, K.-Y.; Song, K.; Cho, S.H.; Chang, J.-H. Noncontact Sleep Study Based on an Ensemble of Deep Neural Network and Random Forests. IEEE Sens. J. 2018, 18, 7315–7324. [Google Scholar] [CrossRef]
  42. Schade, M.M.; Bauer, C.E.; Murray, B.R.; Gahan, L.; Doheny, E.P.; Kilroy, H.; Zaffaroni, A.; Montgomery-Downs, H.E. Sleep validity of a non-contact bedside movement and respiration-sensing device. J. Clin. Sleep Med. 2019, 15, 1051–1061. [Google Scholar] [CrossRef]
  43. Tuominen, J.; Peltola, K.; Saaresranta, T.; Valli, K. Sleep parameter assessment accuracy of a consumer home sleep monitoring ballistocardiograph beddit sleep tracker: A validation study. J. Clin. Sleep Med. 2019, 15, 483–487. [Google Scholar] [CrossRef] [PubMed]
  44. Zaffaroni, A.; Coffey, S.; Dodd, S.; Kilroy, H.; Lyon, G.; O’Rourke, D.; Lederer, K.; Fietze, I.; Penzel, T. Sleep Staging Monitoring Based on Sonar Smartphone Technology. In Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany, 23–27 July 2019; Volume 2019, pp. 2230–2233. [Google Scholar]
  45. Feng, C.; Zhang, H.; Han, Y.; Jin, X.; Yang, F.; Zou, J.; Wang, Y.; Li, Y. Application of impulse-radio ultra-wideband radar as a non-contact portable monitoring device for the diagnosis of obstructive sleep apnea. J. Clin. Otorhinolaryngol. Head Neck Surg. 2020, 34, 634–638. [Google Scholar] [CrossRef]
  46. Miyata, S.; Otake, H.; Ando, M.; Okuda, M.; Fujishiro, H.; Iwamoto, K.; Noda, A.; Sone, M.; Ozaki, N. Patient characteristics affecting accurate detection of sleep apnea using a bed sheet-type portable monitor. Sleep Breath. 2020, 24, 783–790. [Google Scholar] [CrossRef] [PubMed]
  47. Stone, J.D.; Rentz, L.E.; Forsey, J.; Ramadan, J.; Markwald, R.R.; Finomore, V.S., Jr.; Galster, S.M.; Rezai, A.; Hagen, J.A. Evaluations of Commercial Sleep Technologies for Objective Monitoring During Routine Sleeping Conditions. Nat. Sci. Sleep 2020, 12, 821–842. [Google Scholar] [CrossRef]
  48. Toften, S.; Pallesen, S.; Hrozanova, M.; Moen, F.; Grønli, J. Validation of sleep stage classification using non-contact radar technology and machine learning (Somnofy®). Sleep Med. 2020, 75, 54–61. [Google Scholar] [CrossRef]
  49. Chinoy, E.D.; Cuellar, J.A.; Huwa, K.E.; Jameson, J.T.; Watson, C.H.; Bessman, S.C.; Hirsch, D.A.; Cooper, A.D.; Drummond, S.P.A.; Markwald, R.R. Performance of seven consumer sleep-tracking devices compared with polysomnography. Sleep 2021, 44, zsaa291. [Google Scholar] [CrossRef]
  50. Edouard, P.; Campo, D.; Bartet, P.; Yang, R.Y.; Bruyneel, M.; Roisman, G.; Escourrou, P. Validation of the Withings Sleep Analyzer, an under-the-mattress device for the detection of moderate-severe sleep apnea syndrome. J. Clin. Sleep Med. 2021, 17, 1217–1227. [Google Scholar] [CrossRef]
  51. Ellender, C.M.; Zahir, S.F.; Meaklim, H.; Joyce, R.; Cunnington, D.; Swieca, J. Prospective cohort study to evaluate the accuracy of sleep measurement by consumer-grade smart devices compared with polysomnography in a sleep disorders population. BMJ Open 2021, 11, e044015. [Google Scholar] [CrossRef]
  52. Xue, J.B.; Zhao, R.; Li, J.; Zhao, L.; Zhou, B.; Dong, X.S.; Han, F. Validation of a contact-free sleep apnea monitor in adults with obstructive sleep apnea. Chin. J. Tuberc. Respir. Dis. 2021, 44, 880–885. [Google Scholar]
  53. Hsiou, D.A.; Gao, C.; Matlock, R.C.; Scullin, M.K. Validation of a nonwearable device in healthy adults with normal and short sleep durations. J. Clin. Sleep Med. 2022, 18, 751–757. [Google Scholar] [CrossRef]
  54. Menghini, L.; Cellini, N.; Goldstone, A.; Baker, F.C.; de Zambotti, M. A standardized framework for testing the performance of sleep-tracking technology: Step-by-step guidelines and open-source code. Sleep 2021, 44, zsaa170. [Google Scholar] [CrossRef] [PubMed]
  55. Kogure, T.; Shirakawa, S.; Shimokawa, M.; Hosokawa, Y. Automatic sleep/wake scoring from body motion in bed: Validation of a newly developed sensor placed under a mattress. J. Physiol. Anthropol. 2011, 30, 103–109. [Google Scholar] [CrossRef]
  56. Paquet, J.; Kawinska, A.; Carrier, J. Wake detection capacity of actigraphy during sleep. Sleep 2007, 30, 1362–1369. [Google Scholar] [CrossRef]
  57. Sivertsen, B.; Omvik, S.; Havik, O.E.; Pallesen, S.; Bjorvatn, B.; Nielsen, G.H.; Straume, S.; Nordhus, I.H. A comparison of actigraphy and polysomnography in older adults treated for chronic primary insomnia. Sleep 2006, 29, 1353–1358. [Google Scholar] [CrossRef]
  58. Zhang, Y. Sensing, computing and intervention for sleep health. Chin. Sci. Bull. 2021, 67, 27–36. [Google Scholar] [CrossRef]
  59. Lai, J.L.; Yang, Z.C.; Guo, B.N. A Two-Stage Low-Complexity Human Sleep Motion Classification Method Using IR-UWB. IEEE Sens. J. 2021, 21, 20740–20749. [Google Scholar] [CrossRef]
  60. Montgomery-Downs, H.E.; Insana, S.P.; Bond, J.A. Movement toward a novel activity monitoring device. Sleep Breath. 2012, 16, 913–917. [Google Scholar] [CrossRef] [PubMed]
  61. Marino, M.; Li, Y.; Rueschman, M.N.; Winkelman, J.W.; Ellenbogen, J.M.; Solet, J.M.; Dulin, H.; Berkman, L.F.; Buxton, O.M. Measuring sleep: Accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep 2013, 36, 1747–1755. [Google Scholar] [CrossRef]
  62. Sadeh, A. The role and validity of actigraphy in sleep medicine: An update. Sleep Med. Rev. 2011, 15, 259–267. [Google Scholar] [CrossRef]
  63. Tryon, W.W. Issues of validity in actigraphic sleep assessment. Sleep 2004, 27, 158–165. [Google Scholar] [CrossRef]
  64. Martin, J.L.; Hakim, A.D. Wrist actigraphy. Chest 2011, 139, 1514–1527. [Google Scholar] [CrossRef] [PubMed]
  65. Bélanger, M.; Bernier, A.; Paquet, J.; Simard, V.; Carrier, J. Validating actigraphy as a measure of sleep for preschool children. J. Clin. Sleep Med. 2013, 9, 701–706. [Google Scholar] [CrossRef] [PubMed]
  66. de Souza, L.; Benedito-Silva, A.A.; Pires, M.L.; Poyares, D.; Tufik, S.; Calil, H.M. Further validation of actigraphy for sleep studies. Sleep 2003, 26, 81–85. [Google Scholar] [CrossRef] [PubMed]
  67. Jean-Louis, G.; Kripke, D.F.; Cole, R.J.; Assmus, J.D.; Langer, R.D. Sleep detection with an accelerometer actigraph: Comparisons with polysomnography. Physiol. Behav. 2001, 72, 21–28. [Google Scholar] [CrossRef] [PubMed]
  68. Jean-Louis, G.; Kripke, D.F.; Mason, W.J.; Elliott, J.A.; Youngstedt, S.D. Sleep estimation from wrist movement quantified by different actigraphic modalities. J. Neurosci. Methods 2001, 105, 185–191. [Google Scholar] [CrossRef]
  69. Meltzer, L.J.; Walsh, C.M.; Traylor, J.; Westin, A.M. Direct comparison of two new actigraphs and polysomnography in children and adolescents. Sleep 2012, 35, 159–166. [Google Scholar] [CrossRef]
  70. Meltzer, L.J.; Wong, P.; Biggs, S.N.; Traylor, J.; Kim, J.Y.; Bhattacharjee, R.; Narang, I.; Marcus, C.L. Validation of Actigraphy in Middle Childhood. Sleep 2016, 39, 1219–1224. [Google Scholar] [CrossRef]
  71. Pollak, C.P.; Tryon, W.W.; Nagaraja, H.; Dzwonczyk, R. How accurately does wrist actigraphy identify the states of sleep and wakefulness? Sleep 2001, 24, 957–965. [Google Scholar] [CrossRef]
  72. Quante, M.; Kaplan, E.R.; Cailler, M.; Rueschman, M.; Wang, R.; Weng, J.; Taveras, E.M.; Redline, S. Actigraphy-based sleep estimation in adolescents and adults: A comparison with polysomnography using two scoring algorithms. Nat. Sci. Sleep 2018, 10, 13–20. [Google Scholar] [CrossRef]
  73. Shin, M.; Swan, P.; Chow, C.M. The validity of Actiwatch2 and SenseWear armband compared against polysomnography at different ambient temperature conditions. Sleep Sci. 2015, 8, 9–15. [Google Scholar] [CrossRef]
  74. Kushida, C.A.; Chang, A.; Gadkary, C.; Guilleminault, C.; Carrillo, O.; Dement, W.C. Comparison of actigraphic, polysomnographic, and subjective assessment of sleep parameters in sleep-disordered patients. Sleep Med. 2001, 2, 389–396. [Google Scholar] [CrossRef] [PubMed]
  75. Ward, T.M.; Lentz, M.; Kieckhefer, G.M.; Landis, C.A. Polysomnography and actigraphy concordance in juvenile idiopathic arthritis, asthma and healthy children. J. Sleep Res. 2012, 21, 113–121. [Google Scholar] [CrossRef] [PubMed]
  76. de Zambotti, M.; Cellini, N.; Goldstone, A.; Colrain, I.M.; Baker, F.C. Wearable Sleep Technology in Clinical and Research Settings. Med. Sci. Sport. Exerc. 2019, 51, 1538–1557. [Google Scholar] [CrossRef] [PubMed]
  77. Hedner, J.; Pillar, G.; Pittman, S.D.; Zou, D.; Grote, L.; White, D.P. A Novel Adaptive Wrist Actigraphy Algorithm for Sleep-Wake Assessment in Sleep Apnea Patients. Sleep 2004, 27, 1560–1566. [Google Scholar] [CrossRef] [PubMed]
  78. Blood, M.L.; Sack, R.L.; Percy, D.C.; Pen, J.C. A comparison of sleep detection by wrist actigraphy, behavioral response, and polysomnography. Sleep 1997, 20, 388–395. [Google Scholar] [PubMed]
  79. Jean-Louis, G.; von Gizycki, H.; Zizi, F.; Fookson, J.; Spielman, A.; Nunes, J.; Fullilove, R.; Taub, H. Determination of sleep and wakefulness with the actigraph data analysis software (ADAS). Sleep 1996, 19, 739–743. [Google Scholar] [PubMed]
  80. Cole, R.J.; Kripke, D.F.; Gruen, W.; Mullaney, D.J.; Gillin, J.C. Automatic sleep/wake identification from wrist activity. Sleep 1992, 15, 461–469. [Google Scholar] [CrossRef]
  81. Morgenthaler, T.; Alessi, C.; Friedman, L.; Owens, J.; Kapur, V.; Boehlecke, B.; Brown, T.; Chesson, A., Jr.; Coleman, J.; Lee-Chiong, T.; et al. Practice Parameters for the Use of Actigraphy in the Assessment of Sleep and Sleep Disorders: An Update for 2007. Sleep 2007, 30, 519–529. [Google Scholar] [CrossRef]
  82. Hauri, P.J.; Wisbey, J. Wrist actigraphy in insomnia. Sleep 1992, 15, 293–301. [Google Scholar] [CrossRef]
  83. Smith, M.T.; McCrae, C.S.; Cheung, J.; Martin, J.L.; Harrod, C.G.; Heald, J.L.; Carden, K.A. Use of Actigraphy for the Evaluation of Sleep Disorders and Circadian Rhythm Sleep-Wake Disorders: An American Academy of Sleep Medicine Clinical Practice Guideline. J. Clin. Sleep Med. 2018, 14, 1231–1237. [Google Scholar] [CrossRef]
  84. Khosla, S.; Deak, M.C.; Gault, D.; Goldstein, C.A.; Hwang, D.; Kwon, Y.; O’Hearn, D.; Schutte-Rodin, S.; Yurcheshen, M.; Rosen, I.M.; et al. Consumer Sleep Technology: An American Academy of Sleep Medicine Position Statement. J. Clin. Sleep Med. 2018, 14, 877–880. [Google Scholar] [CrossRef] [PubMed]
Sensors 23 04842 g001 550
Figure 1. PRISMA flow diagram.
Figure 1. PRISMA flow diagram.
Sensors 23 04842 g001
Sensors 23 04842 g002 550
Figure 2. Types of CCSTDs and contactless sleep monitoring mechanism.
Figure 2. Types of CCSTDs and contactless sleep monitoring mechanism.
Sensors 23 04842 g002
Sensors 23 04842 g003a 550Sensors 23 04842 g003b 550
Figure 3. Comparison of the epoch-by-epoch (EBE) agreement for asleep/awake subgroups using error bars. (a) Sensors, (b) device types, (c) participants, and (d) brands (the number of samples of the same brand greater than or equal to two will be included in the same group, and the other samples are divided into (1) other brands of bedside device, and (2) other brands of mattress device). N, number of studies; CV, coefficient of variation; bedside, bedside devices; mattress-based, mattress-based devices; healthy, healthy participants; healthy + patient, both healthy and patient participants; others(M), other brands of mattress device; others(B), other brands of bedside device; accuracy, proportion of correctly classified sleep and wake epochs; sensitivity, proportion of correctly classified sleep epochs; specificity, proportion of correctly classified wake epochs [54].
Figure 3. Comparison of the epoch-by-epoch (EBE) agreement for asleep/awake subgroups using error bars. (a) Sensors, (b) device types, (c) participants, and (d) brands (the number of samples of the same brand greater than or equal to two will be included in the same group, and the other samples are divided into (1) other brands of bedside device, and (2) other brands of mattress device). N, number of studies; CV, coefficient of variation; bedside, bedside devices; mattress-based, mattress-based devices; healthy, healthy participants; healthy + patient, both healthy and patient participants; others(M), other brands of mattress device; others(B), other brands of bedside device; accuracy, proportion of correctly classified sleep and wake epochs; sensitivity, proportion of correctly classified sleep epochs; specificity, proportion of correctly classified wake epochs [54].
Sensors 23 04842 g003aSensors 23 04842 g003b
Sensors 23 04842 g004 550
Figure 4. Comparison of the epoch-by-epoch (EBE) agreement for sleep stages in the error bars. N, number of studies; CV, coefficient of variation. Accuracy, proportion of correctly classified epochs for each stage over the total number of epochs; sensitivity, proportion of epochs classified as each stage; specificity, proportion of epochs not classified as each stage [54].
Figure 4. Comparison of the epoch-by-epoch (EBE) agreement for sleep stages in the error bars. N, number of studies; CV, coefficient of variation. Accuracy, proportion of correctly classified epochs for each stage over the total number of epochs; sensitivity, proportion of epochs classified as each stage; specificity, proportion of epochs not classified as each stage [54].
Sensors 23 04842 g004
Table
Table 1. A detailed summary of qualified publications.
Table 1. A detailed summary of qualified publications.
The First Author (Year)CountryDeviceSensorsReference and ComparisonParticipantsInvestigative Details
Total Number
(Female Number)		Age (Years),
Mean (SD),
and/or Range		Type(Duration)
Study Site		Tracker PlacementBed-Time
Zhang (2010) [30]ChinaMMSM (RS-611)Pressure PSG40 (5)29–71OSAHS 1(2 nights) Sleep labUnder-mattressHabitual
De Chazal (2011) [31]IrelandSleepMinderRadiofrequencyPSG113 (19)53 (13)Healthy, sleep apnoea(1 night) Sleep labBedside tableHabitual 2
50 (3)50 (14)AHI ≤ 15
63 (16)57 (12)AHI > 15
Hashizaki (2014) [32]JapanSleepMinderRadiofrequencyPSG148 (35)43.7 (12.1) ≥18AHI ≤ 15 & AHI > 15(1 night) Sleep labBedside tableHabitual 2
49 (14)37.7 (11.6) ≥18AHI ≤ 15
99 (21)46.7 (11.2) ≥18AHI > 15
Norman (2014) [33]AustraliaSonomatPiezoelectricPSG62 (25)26 (16)
≥18		OSA and Healthy(1 night) Sleep lab/homeUnder-mattressHabitual 2
O’Hare (2014) [34]IrelandSleepMinderRadiofrequencyPSG20 (9)30 (6)
≥18		Healthy(1 night) Sleep labBedside table22:30–6:30
SleepDesign (HSL-101)Radiofrequency
Pallin (2014) [35]IrelandSleepMinderRadiofrequencyPSG & actigraphy103 (19)55 (14)OSA 3, non-OSA(1 night) Sleep labBedside tableHabitual 2
Abad (2016) [36]SpainSleepWiseInfrared cameraPSG5053.1 (14.0)
>18		Healthy, Mild OSA, Moderate OSA, Severe OSA(1 night) Sleep labBedside (60 cm above thorax)Habitual 2
Terjung (2016) [37]GermanySleepMinderRadiofrequencyPSG57 (11)57.1 (13.4); 32–84Healthy, OSA and PLM 4(1 night) Sleep labBedside tableHabitual 2
Norman (2017) [38]AustraliaSonomatPiezoelectric PSG76 (30)5.8 (2.8)
2–17		Suspected SDB 5(1 night) Sleep labUnder-mattressHabitual 2
Tal (2017) [39]IsraelEarlySensePiezoelectric PSG43 (9)45.9 (14.4) 17–72Suspected SDB(1 night) Sleep labUnder-mattressHabitual 2
7 (4)31.2 (10.8) 24–65Healthy(2–3 nights) Home
13 (5)40.0 (10)
29–59		Healthy(2–3 nights) Home
Zaffaroni (2017) [40]IrelandResMed S+RadiofrequencyPSG18 (10)32.6 (10.9)
>18		AHI ≤ 15(1 night) Sleep labBedside tableHabitual 2
Chung (2018) [41]KoreaResMed S+RadiofrequencyPSG8 (1)45.0 (14.6)Suspected OSA(1 night) Sleep labBedside table23:00–5:00
Schade (2019) [42]USAResMed S + V1RadiofrequencyPSG & actigraphy27 (11) (V1 = 27, V2 = 22)29.1 (11.7)
≥18		Healthy(1 night) Sleep lab Bedside tableHabitual–6:00
ResMed S + V2Radiofrequency
Tuominen (2019) [43]FinlandBeddit Sleep Tracker Piezoelectric PSG10 (5)24.5 (2.51)
18–30		Healthy(2 nights)
Sleep lab 		Under-mattressHabitual 2
Zaffaroni (2019) [44]IrelandResMed S+RadiofrequencyPSG62(31)46.9(15.9)
≥18		AHI < 5, PLM < 30/h(1 night) Sleep labBedside tableHabitual 2
Feng (2020) [45]ChinaIR-UWBRadiofrequencyPSG40 (11) 38.3 (9.67)
16~60		OSA, non-OSA(1 night) Sleep labBedside table——
Miyata (2020) [46]JapanSD102 HSAT sleep recorderPressurePSG189 (56)56.1 (18.3)
≥20		OSA, PLMs, Central sleep apnea(1 night) Sleep labUnder-mattressHabitual 2
Nagatomo (2020) [20]JapanNemuri SCANPressure PSG11 (3)71 (2.82)
≥20		Perioperative, septic shock(1 day)
ICU		Under-mattress21:00–05:99 6
Stone (2020) [47]USABeddit Sleep Monitor 3.0Piezoelectric PSG5 (3)27.8 (7.7)
22–41		Healthy(average 19.6 nights) HomeUnder-mattressHabitual 2
Toften (2020) [48]NorwaySomnofyRadiofrequencyPSG71 (43)28.9 (9.7)
19–61		Healthy(1 night) Clinic bedrooms and home Nightstand and wallHabitual 2
Chinoy (2021) [49]USAEarlySense LivePiezoelectric PSG & actigraphy34 (22)28.1 (3.9)
18–35		Healthy(3 nights) Sleep lab Under-mattressHabitual
ResMed S+ RadiofrequencyBedside table
SleepScore MaxRadiofrequency
Edouard (2021) [50]FranceWithings Sleep Analyzers (WSA)Pressure PSG118 (67)49.3 (12.1)
18–70		Healthy, OSA, comorbidities, epilepsy(1 night)
Sleep lab 		Under the mattress——
Ellender
(2021) [51] 		AustraliaBedditPiezoelectric PSG54 (31)
(42 Beddit, 29 ResMed S+)		48.09 (18.05)
>18		OSA, insomnia, central hypersomnolence disorder(1 night)
Sleep lab 		Under-mattress——
ResMed S+RadiofrequencyBedside ——
Xue (2021) [52]ChinaUWB Radar Sleep Monitoring SystemRadiofrequency PSG198 (78)45.5 (4.05)
≥18		Healthy, OSA(1 night)
Sleep lab 		Bedside table22:00–6:00
Hsiou (2022) [53]USABeddit 3.0PiezoelectricPSG & actigraphy35 (27) 18.97 (0.95)
≥18		Healthy(1 night)
Sleep lab 		Under the bedsheet22:30–7:00 h;
1:30–7:00 h
Beddit 3.5Piezoelectric
Kholghi (2022) [17]AustraliaEMFIT Quantified Sleep (QS)Pressure PSG33 (15)53.7 (16.5)
18–80		Healthy; OSA, PLM(1 night)
Sleep lab 		Under the mattress or
mattress topper		Habitual 2
1 OSAHS: obstructive sleep apnea–hypopnea syndrome. 2 Exact bedtime is not reported, but investigative protocol infers habitual bedtime. 3 OSA: obstructive sleep apnea. 4 PLMS: periodic limb movement in sleep. 5 SDB: sleep-disordered breathing. 6 Inclusion of nighttime sleep data only.
Table
Table 2. Meta-analyses results.
Table 2. Meta-analyses results.
OutcomeNnPooled Mean95% CII2 (p)Z (p)
TST (min)35187319.5512.22, 26.8888.2% (0.000)5.23 (0.000)
SOL (min)17796−4.61−6.56, −2.6651.1% (0.008)−4.63 (0.000)
WASO (min)16846−12.07−18.75, −5.3856.1% (0.003)−3.54 (0.000)
SE (%)2813912.881.58, 4.1734.2% (0.041)4.36 (0.000)
Light Sleep (min)104175.62−12.81, 24.0686.8% (0.000)0.60 (0.550)
Deep Sleep (min)1041711.070.37, 21.7675.3% (0.000)2.03 (0.043)
REM (min)104173.44−14.81, 21.6895.3% (0.000)0.37 (0.712)
N: number of studies; n: number of samples; CI, confidence interval; I2: heterogeneity; Z: test of overall effect; TST, total sleep time; SOL, sleep onset latency; WASO, wake after sleep onset; SE, sleep efficiency; REM, rapid eye movement.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhai, H.; Yan, Y.; He, S.; Zhao, P.; Zhang, B. Evaluation of the Accuracy of Contactless Consumer Sleep-Tracking Devices Application in Human Experiment: A Systematic Review and Meta-Analysis. Sensors 2023, 23, 4842. https://doi.org/10.3390/s23104842

AMA Style

Zhai H, Yan Y, He S, Zhao P, Zhang B. Evaluation of the Accuracy of Contactless Consumer Sleep-Tracking Devices Application in Human Experiment: A Systematic Review and Meta-Analysis. Sensors. 2023; 23(10):4842. https://doi.org/10.3390/s23104842

Chicago/Turabian Style

Zhai, Huifang, Yonghong Yan, Siqi He, Pinyong Zhao, and Bohan Zhang. 2023. "Evaluation of the Accuracy of Contactless Consumer Sleep-Tracking Devices Application in Human Experiment: A Systematic Review and Meta-Analysis" Sensors 23, no. 10: 4842. https://doi.org/10.3390/s23104842

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop