1. Introduction
In the era of ubiquitous digital connection, Remote Patient Monitoring (RPM) is an expanding and developing area of healthcare research and delivery improvement [
1,
2,
3]. RPM technologies automatically monitor and report on patients’ activity-related vital signs [
4,
5], oftentimes with chronic conditions [
3,
6]. In the midst of the COVID-19 pandemic, RPM technologies that enable contactless monitoring of patients are integral for minimising the spread of the novel coronavirus [
7,
8,
9] while accommodating remote clinical research [
10,
11,
12].
For human activity monitoring, physical behaviours that occur throughout a full 24 h day are categorised into physical activity (PA), sedentary behaviour (SB) and sleep [
13]. These three behaviours are significant for research and health considerations due to their verifiable impact on health [
14,
15,
16], both independently and symbiotically [
17,
18,
19]. Time spent in one behaviour in a 24 h period will directly influence at least one of the other behaviours [
19]. Higher sleep quality increases energy and reduces fatigue levels [
20,
21]. Reciprocally, greater PA ameliorates sleep quality [
20,
22]. Moreover, the optimal combination between time spent sleeping and in active behaviours (both light and moderate to vigorous physical activities (MVPA)) is associated with lower cardiovascular risk [
23]. It is therefore advisable to target all behaviours together [
24] in free-living observations of a 24 h day to better comprehend the individual and combined impacts of these activity-related parameters [
25].
Accelerometer-based monitors that balance cost and feasibility have emerged as valid tools to directly quantify movement [
26] that results from PA [
27], SB [
28,
29] and sleep [
30,
31]. Accelerometers offer low-cost continuous substitution for polysomnography (PSG) [
32] and indirect calorimetry [
33,
34,
35], which are the gold standards for sleep and PA monitoring. To date, studies that measured waking movement behaviour and sleep typically utilised two separate accelerometer models [
36,
37]. Given that waking activity behaviour and sleep can be directly assessed with similar approaches for body movement acceleration detection [
38], a logical development for convenience and cost-effectiveness would be to utilise one single accelerometer that can measure PA and sleep over the full 24 h spectrum [
22]. Ref. [
39] reported the existence of only one recently developed research-specific device that fulfils such requirements: the Actigraph Link (ActiGraph, LLC), therefore advocating for more monitors that can objectively and simultaneously measure waking movement and sleep, and minimise the burden on research cohorts to wear distinct devices that measure behaviours independently [
25,
40].
The Actiwatch 2 accelerometer (Philips Respironics, Eindhoven, The Netherlands) is a commonly utilised wrist-worn sleep-monitor, that has been validated and widely used for detection of sleep duration and sleep quality [
41]. The Actiwatch 2 also facilitates measurement of PA in proprietary activity counts per time unit. A study by [
36] developed PA thresholds to segment sedentary, light and MVPA levels of activity for the Actiwatch 2 by comparing activity counts to indirect calorimetry using a portable metabolic cart and an actigraph device [
42]. Another study [
43] validated the Actiwatch 2 for PA by examination of activity level against energy expenditure measured using indirect calorimetry with the Actiwatch 2. Results were strongly correlated to a widely validated PA device, the ActiGraph wGT3X-BT, thus making the Actiwatch 2 a valid device for both PA and sleep monitoring [
36]. This establishes the Actiwatch 2 as a device for the full spectrum of 24 h activity monitoring, which is desirable in clinical research involving participants where participant burden is a pertinent issue for both sleep and PA. This is supported by aforementioned findings based on ActiGraph wGT3X-BT, and other previous studies on its validity in sleep monitoring [
43,
44] and thus, a suitable device for validation of novel sensors such as that presented in this research.
The Actiwatch 2 is relatively expensive (approximately US
$1500), and produces summative information with a requirement for manual data upload. A viable alternative research-grade accelerometer that can measure waking movement behaviour and sleep, that is more cost-effective, with access to raw sensor data and a long battery life with automated data upload to a secure cloud server, would be beneficial for long-term activity and sleep measurement and assessment. Recently, the use of the accelerometers that provide raw acceleration data in place of a proprietary filtered data units has increased [
45,
46], with a desired criterion being the production of temporal raw data, as is normally outputted from research-grade monitors [
47]. Long-lasting battery life and memory storage is an important consideration to professionals who require high-resolution outputs. However, a necessary equilibrium is the production of detailed data without compromising other practical considerations such as sensor dimensions and burden on participants to wear the device [
48].
This study implements Verisense, a novel wrist-worn inertial measurement unit (IMU) sensor designed for clinical trials, and developed by Shimmer Research Ltd. (Dublin, Ireland). Verisense accommodates continuous RPM through integration of their wearable sensor, base-station, and cloud platform for automatic data upload. Verisense outputs raw IMU sensor data on waking movement behaviour and sleep and has up to six months of battery life with no recharging. As discussed by [
49], these functionalities fit the desirable requirements for sensor systems measuring healthcare parameters in that they uninterruptedly measure and wirelessly report all health-related information after one initial setup, placing minimal restrictions on participants for interaction or maintenance. While Verisense accommodates these requirements, Actiwatch 2 does not, demonstrating a need for Verisense to be validated for future studies. Other alternatives were considered; however, any sensors of similar specifications were either more expensive or lacked in at least one key feature that Verisense offered [
50,
51]. Furthermore, sensors that differed in body placement location such as shoe-worn devices were deemed unrealistic for the purposes of sleep monitoring as necessitated by the research [
52,
53], due to unfeasibility of wearing footwear while asleep. Wrist-worn placements were chosen for this study in keeping with the findings from a systematic review and practical considerations of device placement in [
54], and the superiority for sleep quality-metrics from wrist-worn sensors as reported in [
55]. Actiwatch 2 was selected for its reliability as a single device capable of 24 h activity and sleep monitoring and Verisense was chosen to investigate the potential match for that reliability while meeting the additional desirable functions for battery life, open-source algorithms and automatic data upload that is needed for further 24 h RPM in PA and sleep studies.
There are four aims of this study: (1) compare temporally matched PA measured via Verisense and Actiwatch 2 over the data collection period; (2) compare PA cut-points of sedentary, light and MVPA measured via Actiwatch 2 and Verisense over the data collection period; (3) evaluate the ability of Verisense to determine sleep time, wake time and total sleep time (TST) compared to sleep metrics measured by Actiwatch 2 and (4) evaluate the objective and subjective comparisons between sensor data and the participant diaries.
3. Results
For the FL study, 15 participants were included (11 males, mean age (±SD) 23 (±3.4) years, mean BMI (±SD) 23.9 (±2.6) kg/m2, and 4 females, mean age (±SD) 29 (±12.6), mean BMI 22.6 (±1.3) kg/m2). For the SP study, 12 participants were included (11 males, mean age (±SD) 23 (±3.4) years, mean BMI (±SD) 23.9 (±2.6) kg/m2, and 1 female, mean age (±SD) 22 (±0), mean BMI 22.8 (±0) kg/m2). Everyone who participated in the SP study also participated in the FL study. All participants were third-level students. Activities performed by the participants during the test days included sitting (e.g., at lectures), standing (e.g., practical classes) and walking. A number of participants were highly active (e.g., did workouts) whereas others were mainly sedentary during the test days.
The study compared epoch-by-epoch data obtained from both the Actiwatch 2 and Verisense devices over the 48 h FL study period from all 15 participants. The overall patterns observed between the Actiwatch 2 and Verisense visually appear to be quite similar for the 48 h FL study (
Figure 2 and
Figure 3). Movement data from 48 h FL absolute activity for the Actiwatch 2 and Verisense sensors were highly correlated (r = 0.85 ± 0.04, range: 0.77–0.92, n = 15; Spearman correlation).
Epoch-by-epoch data obtained from both the Actiwatch 2 and Verisense sensors were compared over the gym-based SP from 12 participants. The overall patterns observed between the Actiwatch 2 and Verisense appear to be visually quite similar for the gym-based SP study (
Figure 4 and
Figure 5). Within participants, gym-based SP activity for the Actiwatch 2 and Verisense sensors were also highly correlated (r = 0.78 ± 0.05, range: 0.72–0.88, n = 12; Spearman correlation).
Epoch-by-epoch level data were segmented into PA levels of sedentary, light and MVPA activity using previously published cut-points from [
36] for Actiwatch 2. These cut-points were defined as sedentary < 145, light <= 274, moderate > 274 and vigorous >= 597. However, limitations of sample size and female-only study participants is a notable limitation in the validity of application to these cut-points in other populations. For this reason, Actiwatch 2 cut-points could not be used as a gold-standard from which to base corresponding Verisense cut-points using Receiver Operating Characteristic (ROC) curves. All Verisense epoch-by-epoch level data were segmented using GGIR and processed in R following mean cut-points utilised by two similar specification accelerometers [
27,
28] of sedentary < 45, light <= 97, moderate > 97 and vigorous >= 423. To date, no cut-point validation has occurred for Verisense, hence these cut-points should be noted as experimental only.
Using the aforementioned cut-point values, sensitivity, specificity and accuracy of the imputed PA levels were examined, as determined by the Actiwatch 2. For the most part, there was moderate correspondence in the determination of cut-points by Verisense and Actiwatch 2. The observed concordance between Verisense and Actiwatch 2 is presented in
Table 2.
Within FL participants, cut-point generated activity for the Actiwatch 2 and Verisense sensors were highly correlated for sedentary, low correlated for light and moderately correlated for MVPA. Within SP participants, cut-point generated activity for the Actiwatch 2 and Verisense sensors were low correlated for sedentary, negligibly correlated for light and low correlated for MVPA as presented in
Table 2.
Epoch-by-epoch sleep metrics were calculated from the two sensors using both the sleep/wake classification proprietary algorithm for Actiwatch 2 and GGIR for Verisense. Using both algorithms, TST (min), sleep and wake times were reported from both sensors. TST are the epochs scored as sleep within the reported time span between sleep and wake. The agreement of the three sleep indicators (TST, sleep time and wake time) between participant diaries, Actiwatch 2, Verisense guided by participant diary and Verisense unguided was tested using Spearman correlation coefficients and Bland– Altman plots. To facilitate comparison, the sleep start time and wake time were converted from time to numerical values for statistical comparison. values were calculated for time duration from 18:00 until sleep start time, and time values from 00:00 until wake time. All times were converted to mins. No day sleepers were included in the study.
Table 3,
Table 4 and
Table 5 shows the Spearman’s correlation coefficient of sleep start times, wake times and TST for participant diaries, Actiwatch 2, Verisense guided by participant diary and Verisense unguided. First, the total average sleep start times of all the participants were 402.5 ± 92.8 min, 417.4 ± 149.5 min, 390.7.5 ± 127.4 min and 398.9 ± 114.5 min, respectively. Next, the total average wake times of all the participants were 555.2 ± 96.7 min, 575.97 ± 115.8 min, 550.5 ± 118.4 min and 576.9 ± 118.8 min, respectively. Finally, the total average sleep times of all the participants were 515.1 ± 107.6 min, 524.8 ± 131.8 min, 526.6 ± 121.5 min and 543.1 ± 102.4 min, respectively.
To examine the possibility of systematic bias in overall sleep parameter scoring, Bland–Altman plots were generated to visually inspect the level of agreement between Verisense and Actiwatch 2 results (
Figure 6,
Figure 7 and
Figure 8). For sleep time, wake time and TST, the spread of the differences visually appears to be even, with no bias in overestimation or underestimation of sleep, wake or TST.
4. Discussion
To our knowledge, this is the first validation study where the Verisense IMU was compared to an actigraph for activity and sleep monitoring. In comparing the accuracy of Verisense, a novel research-grade wearable sensor, against a clinical/research-grade actigraphy device, Actiwatch 2, we find that the former device performs similarly in the estimation of epoch-by-epoch activity scoring and sleep parameters, although future studies on PA level classifications need further examination.
There are notable differences between the Verisense and the Actiwatch 2. While present on the Actiwatch 2, the Verisense lacks a light sensor, a feature often useful in identifying bed and wake times. The Actiwatch 2 stores data at a lower average resolution (e.g., 15 s and 30 s epochs at 32 Hz) in comparison to the Verisense which is capable of raw data monitoring and storage ranging up to 1600 Hz, facilitating higher resolution data with potential for greater accuracy. Verisense devices also remotely uploads all data to a secure cloud portal, eliminating the need for participants to attend a research facility to have data from the device downloaded, which is necessary with the Actiwatch 2. Significantly, Verisense provides access to raw accelerometer data which in place of a proprietary unit such as activity counts and filtered data which is now a desired part of research and increasing in adoption [
45]. Verisense also has a long-lasting battery life of up to 6 months with no need for recharging, while most devices designed for clinical trials require frequent recharging and manual data upload. For longer duration longitudinal studies, these attributes of Verisense could be of significant benefit.
The adoption of wearable technology in healthcare and clinical trials continues to increase, however the paucity of standards for sensor algorithms can hamper their utilisation in research [
79]. To address this, a healthcare industry open-wearables initiative (OWEAR) has been established [
80]. The initiative seeks to develop open source algorithms and software for wearable sensor data analysis available to all medical device and pharmaceutical companies in a pre-competitive environment as a service to the industry [
79,
80,
81].
One key area of debate in accelerometry activity monitoring is most appropriate wear-site for maximum accuracy [
82,
83]. Both devices presented in the study were wrist-worn, however many previous studies have utilised sensors worn at the waist [
84,
85]. It is therefore imperative to understand whether wrist-worn devices are an acceptable alternative compared to the waist for PA monitoring. In 2011, the U.S. National Health and Nutrition Examination Survey began using wrist-worn accelerometers to estimate PA [
86]. Ref. [
48] reported that a waist-worn GENEA triaxial accelerometer produced an almost identical correlation with energy expenditure as the same model worn on the wrist. However, Ref. [
87] reported that a uniaxial accelerometer worn on the wrist and hip of participants during lifestyle activities produced a discrepancy in variance in energy expenditure with the waist-worn accelerometer accounting for 31.7% of the variance and the wrist-worn accelerometer explaining only 3.3% of the variance. This suggests that a triaxial accelerometer such as Verisense is suitable as a wrist-worn wearable device for PA.
Future studies will compare Verisense to indirect caliometry for further PA classification validation, as this is the true, current gold standard in determination of PA cut-points [
33,
34,
35]. The current results do, however, support the potential use of Verisense as an actigraphy device for the purpose of activity and sleep monitoring.