Next Article in Journal
Grant-Free NOMA: A Low-Complexity Power Control through User Clustering
Previous Article in Journal
A Self-Supervised Model Based on CutPaste-Mix for Ductile Cast Iron Pipe Surface Defect Classification
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

PhysioKit: An Open-Source, Low-Cost Physiological Computing Toolkit for Single- and Multi-User Studies

Department of Computer Science, University College London, London NW1 2AE, UK
*
Author to whom correspondence should be addressed.
Current address: UCLIC and Computer Science, 169 Euston Road, Room 403, London NW1 2AE, UK.
Sensors 2023, 23(19), 8244; https://doi.org/10.3390/s23198244
Submission received: 3 August 2023 / Revised: 12 September 2023 / Accepted: 29 September 2023 / Published: 4 October 2023
(This article belongs to the Topic Machine Learning and Biomedical Sensors)

Abstract

:
The proliferation of physiological sensors opens new opportunities to explore interactions, conduct experiments and evaluate the user experience with continuous monitoring of bodily functions. Commercial devices, however, can be costly or limit access to raw waveform data, while low-cost sensors are efforts-intensive to setup. To address these challenges, we introduce PhysioKit, an open-source, low-cost physiological computing toolkit. PhysioKit provides a one-stop pipeline consisting of (i) a sensing and data acquisition layer that can be configured in a modular manner per research needs, and (ii) a software application layer that enables data acquisition, real-time visualization and machine learning (ML)-enabled signal quality assessment. This also supports basic visual biofeedback configurations and synchronized acquisition for co-located or remote multi-user settings. In a validation study with 16 participants, PhysioKit shows strong agreement with research-grade sensors on measuring heart rate and heart rate variability metrics data. Furthermore, we report usability survey results from 10 small-project teams (44 individual members in total) who used PhysioKit for 4–6 weeks, providing insights into its use cases and research benefits. Lastly, we discuss the extensibility and potential impact of the toolkit on the research community.

1. Introduction

Physiological signals have been actively explored in a wide range of research fields, given their usefulness in tracking health, as well as physical and psychological states. Particularly in human–computer interaction (HCI), much attention has recently been paid to physiological computing research, where it focuses on how to evaluate user responses to interventions, enhance user interactions with technology, or investigate how to support interpersonal interactions [1]. Physiological computing involves detecting, acquiring, and processing various physiological signals, such as cardiac rhythm or heart rate (HR), skin conductance, blood-oxygen saturation, body temperature, blood glucose levels, muscle activity, and neural activity. Sensors commonly used for acquisition of physiological signals include: electrocardiogram (ECG), photoplethysmography (PPG), respiratory (RSP), electrodermal activity (EDA), electromyography (EMG) and electroencephalogram (EEG) [2]. The functions of physiological computing systems include acquisition, interpretation, and facilitation of different schemes of interactions, as well as personalized interventions [3,4,5].
While the proliferation of low-cost consumer-grade physiological sensing solutions and fitness trackers (e.g., Apple Watch, https://www.apple.com/uk/watch/, accessed on 28 September 2023, Fitbit, https://www.fitbit.com/, accessed on 28 September 2023, Garmin, https://www.garmin.com/, accessed on 28 September 2023) has boosted the interest in physiological sensing forward, there is a need to pay attention to the challenges of using such devices, including limitations in robustness associated with sensor misplacement, body movements and ambient noise, reduced flexibility in adapting an acquisition interface to the study protocol, and relatively high costs of hardware or software [6,7,8,9]. The majority of these consumer-grade sensors provide limited access to continuous physiological signals and metrics (e.g., inter-beat interval time-series), which can yield considerable insight into our bodily functions and psychological states [8,10,11,12]. Data acquired from these devices is processed using the manufacturer’s proprietary algorithms and exported directly to remote company servers for commercial purposes [13,14], raising ethical concerns about data privacy. Furthermore, the rapid release cycles of new commercial wearable sensor models pose additional research challenges. While the general public may assume that newer models can perform better, it is difficult to conduct and publish validation studies on the same timescale of of new technology releases.
Research-grade physiological sensing devices (e.g., Procomp Infiniti System (https://thoughttechnology.com/procomp-infiniti-system-w-biograph-infiniti-software-t7500m/ (accessed on 28 September 2023)) from Thought Technology, BIOPAC (https://www.biopac.com/(accessed on 28 September 2023)), Empatica (https://www.empatica.com/(accessed on 28 September 2023)), Biofourmis (https://biofourmis.com/(accessed on 28 September 2023)) are often expensive, leading to wide adoption of affordable physiological sensing solutions (e.g., BITalino (https://www.pluxbiosignals.com/collections/bitalino(accessed on 28 September 2023)), OpenBCI (https://openbci.com/(accessed on 28 September 2023)) by the research community. Owing to the access to raw signal data, the research community has demonstrated several application use-cases of research-grade devices and toolkits, including OpenBCI, https://openbci.com/citations (accessed on 28 September 2023), Empatica E4, https://www.empatica.com/research/publications/ (accessed on 28 September 2023), BITalino, https://scholar.google.com/scholar?as_sdt=0,5&q=BITalino&hl=en&as_ylo=2000&as_yhi=2021 (accessed on 28 September 2023) and Movisens GmbH, https://www.movisens.com/en/resources/publications/ (accessed on 28 September 2023), and has contributed towards their validation. Researchers have also explored open-source prototyping platforms (e.g., Arduino with physiological sensing nodes (https://www.arduino.cc/ (accessed on 28 September 2023))) in different contexts [15,16,17,18]. Physiological computing with such platforms often requires a range of technical and computational skills and considerable setup time. Additionally, though few open-source platforms and affordable toolkits are sufficiently validated across the entire healthy-range of physiological variations [19], the usability study of the interface for physiological signal acquisition is often neglected.
Signal quality is another concerning factor both for consumer-grade as well as research-grade devices, which is affected by environmental and experimental factors. Motion artifact noise, as well as participant non-compliance from discomfort while wearing such sensors in different scenarios or physical impairments, may result in a reduced quality of acquired signals [20,21]. While these challenges cannot be fully addressed, one of the potential ways to circumvent these issues can be to enable flexibility in sensor placement (e.g., PPG sensors can be placed on either ear-lobes or fingers). This flexibility also enhances the accessibility of the sensors to suit users with different physical sensitivities or requirements. The complementary measure can be a provision for real-time assessment of signal quality to flag noisy acquisition fragments of signals [22], which would enable researchers to take appropriate actions during data acquisition or processing. Physiological sensing toolkits often neglect such a feature. Lastly, due to social distancing needs in recent years, studies have been conducted in remote settings [23]. While several existing toolkits can support synchronized acquisition from multiple co-located users, provision for synchronized acquisition of physiological signals for remote multi-user scenarios is yet not addressed.
It is therefore crucial to design a toolkit that considers the above mentioned design objectives and existing research gaps. These include access to raw data, provision for synchronized data acquisition in remotely located multi-user scenarios, support for real-time analysis of signal for physiological metrics, as well as signal quality assessment, provision for configuring experimental protocols, and a provision to transmit real-time analysis data for adaptive interactions as well as customized interventions. To our best knowledge, there is no open-source platform for physiological computing, which encompasses all the design objectives on a single platform. To address this, we propose PhysioKit, a new open-source physiological computing toolkit for HCI researchers, hobbyists and practitioners. While being cost-effective, PhysioKit can facilitate flexible experiment configuration, data collection, and support real-time analyses of physiological data. Our contribution is two-fold:
  • A novel open-source physiological computing toolkit (GitHub repo link, https://github.com/PhysiologicAILab/PhysioKit, accessed on 28 September 2023) that offers a one-stop physiological computing pipeline spanning from data collection and processing to a wide range of analysis functions, including a new machine learning module for the physiological signal quality assessment;
  • A report on validation study results, as well as user reports on the PhysioKit’s usability and examples of use cases demonstrating its applicability in diverse applications.
We begin this paper with an overview of related work and identify key challenges. We then introduce PhysioKit, describing its sensing and software application layers, and how they are designed to enable high-quality data acquisition, while considering the flexibility in selecting physiological sensors. We present results from a validation study, an overview of applications in which PhysioKit is used and outcomes from a usability survey. Finally, we conclude with discussions on the implications and benefits of PhysioKit for different contexts and user communities.

2. Related Work

In this section, we first discuss prior work on applications of physiological computing in HCI research and then review existing physiological computing devices and toolkits related to the contribution we make in this work.

2.1. Physiological Computing and HCI Applications

The applications of physiological computing in HCI contexts can be broadly categorized into two themes, based on the role physiological computing plays in the interaction: interventional and passive. This categorization serves as one of the key design considerations for the development of physiological computing toolkits. For interventional studies, real-time computing of physiological metrics is required, which can further be mapped to biofeedback design or to adapt the interaction with the self, with others, or with technology. On the other hand, for passive physiological computing studies, the provision of data acquisition meets the research need, as these studies do not require adapting any interaction aspects based on real-time computing of physiological metrics from the acquired signals.
Interventional studies have examined how real-time physiological computing can be used in several contexts, including health monitoring (e.g., stress [24], diabetes [25]), training healthful practices (e.g., respiration [26]), educating children in anatomy [27], communicating affective states between people during chats [28] or VR gameplay [29], sensing passenger comfort in smart cars [30,31], and personalizing content through adaptive narratives (e.g., interactive storytelling [32], adjustable cultural heritage experiences [33], synchronized content between multiple users [34]). Illustrative studies with passive use of physiological computing include assessing user’s mental states (e.g., stress, workload and attention) [5,10,35,36], exploring user experiences [37,38,39,40], or objective comparisons with subjective reports [41]. The illustrated HCI studies are not exhaustive and are mentioned to emphasize the two distinct ways in which researchers use physiological computing. While most commercial research-grade and low-cost devices support data acquisition for passive studies, they are often not designed considering different needs of interventional studies.

2.2. Physiological Computing Sensors, Devices and Toolkits

PPG, EDA and RSP are among the most prominent physiological sensing channels used in HCI research [1,42,43]. PPG and EDA signals, for instance, have been explored to capture physiological and emotional states [44], engagement levels [45,46], for communicating emotional states [47,48], as well for evoking empathy through shared biofeedback [49]. Respiration cues have been used to help people understand and manage their stress [50,51], or to feel connected to others through sharing breathing signals [52]. To acquire these signals, researchers generally consider consumer-grade devices acceptable to use and prioritize the use of devices based on design and familiarity with the brand, rather than reliability, as well as comfort or ease of use [53]. Here, we present a brief overview of the data acquisition devices as well as data analysis toolkits.

2.2.1. Sensing and Data Acquisition Devices

Support for Passive and Interventional HCI Studies

For passive HCI studies (Section 2.1), researchers can choose from broader spectrum of devices ranging from expensive research-grade physiological sensing devices, including Procomp Infiniti, https://thoughttechnology.com/procomp-infiniti-system-w-biograph-infiniti-software-t7500m/ (accessed on 28 September 2023), BIOPAC, https://www.biopac.com/ (accessed on 28 September 2023), Shimmer, https://shimmersensing.com/product/consensys-bundle-development-kit/ (accessed on 28 September 2023), Empatica, https://www.empatica.com/ (accessed on 28 September 2023), and Biofourmis, https://biofourmis.com/ (accessed on 28 September 2023), to more affordable devices including BITalino, https://www.pluxbiosignals.com/collections/bitalino (accessed on 28 September 2023), OpenBCI, https://openbci.com/ (accessed on 28 September 2023) and Movisens GmbH, https://www.movisens.com/en/resources/publications/ (accessed on 28 September 2023), among several others. However, it can be observed that for interventional studies (Section 2.1), researchers often combine open-source or affordable sensing hardware with their custom developed software [52,54]. While affordable sensing toolkits offer greater flexibility for using their sensing hardware, they insufficiently support configuration for experiments, as well as several types of biofeedback modalities with real-time signal analysis. This requires researchers to spend significant efforts towards customizing the acquisition pipeline with real-time analysis and bio-feedback presentation.

Real-Time Signal Quality Assessment

The signal quality of contact-based physiological sensors get affected in presence of relative motion between sensor and skin surface. PPG sensors, for instance, are susceptible to motion that results in interference from natural and artificial light [55], as well as to varying pressure at the sensor site caused by the activities of daily living affecting the blood flow [56]. Factors leading to artifacts cannot be controlled, though it is often possible to perform signal quality assessment and eliminate the noisy segments from the analysis. While signal quality assessment for physiological signals is an active research field [22,57,58,59,60,61], existing physiological sensing solutions do not offer provision of assessing signal quality, which can immensely increase validity of the analysis. Widely used methods for signal quality assessment include signal-to-noise ratio (SNR), template matching, and relative power signal quality index (pSQI) [61], along with recent machine learning-based approaches based on SVM classifier [60], LSTM [59], and 1D-CNN [57]. The state-of-the-art (SOTA) performance has been demonstrated by 1D-CNN classifier approach [57] achieving 0.978 accuracy on the MIMIC III PPG waveform database. It is noteworthy that stringent signal quality assessment may lower the signal retention, thereby decreasing the usable segments of signal for deriving physiological metrics [62] and, therefore, it is crucial to develop an optimal signal quality measure with an objective to minimize for false positives and false negatives.

Support for Remote Multi-User Studies

Furthermore, existing toolkits offer limited support for multi-user studies. For scenarios in which multi-user studies are conducted with remotely located users, researchers have either used time-stamping information [63] or have deployed manual approaches [64] for synchronizing the time-series data. These approaches of synchronizing the data acquisition are not suitable for interventional studies and may result in a varied amount of time-lag between the physiological signals of different users. One very recent work proposed an open-source toolkit [19] for synchronized acquisition of multiple physiological signals using their sensor fusion unit (SFU). However, it remains unclear if signal acquisition can be synchronized for multiple SFUs that are located at remote locations.

Validation Studies

Research-grade devices undergo rigorous performance tests, as they are required to adhere to the regulatory standards. In contrast, owing to their intended use, consumer-grade devices are not required to comply with medical regulatory standards. However, researchers have contributed to the validation of consumer-grade as well as open-source devices. We present a non-exhaustive scoping review with few of the these devices, along with the comparative analysis based on the published validation studies in Appendix A. For effectively validating the sensing devices, it is essential to induce sufficient variations in psychophysiological states of participants, such as the deployment of Stroop test [65] for validating open-source toolkit [19] as well the deployment of light-to-vigorous physical activity for validating wrist-worn wearables [66,67].

2.2.2. Data Analysis Toolkits

Data analysis software are either bundled with the research-grade physiological sensing solutions, or these are available as add-on packages which are required to purchase separately for analyzing the physiological data [68,69,70,71]. Except for the open-source toolkits such as [19,70], commercial data analysis toolkits implement proprietary algorithms and offer limited flexibility in choosing the algorithms for computing metrics from physiological signals. In spite of advancing research towards analysis of physiological signals, the limited flexibility restricts the use of the state-of-the-art algorithms for computing physiological metrics. In past few years, open-source toolkits have emerged enabling researchers to process raw sensor data in more customized manner. A few notable examples among these toolkits are NeuroKit2 [72], HeartPy [73], FLIRT [74], and PyPhysio [75]. HeartPy [73] focuses primarily on PPG signals, while FLIRT [74], NeuroKit2 [72] and PyPhysio [75] support analyzing multiple physiological signals. Specifically, FLIRT [74] supports analyzing ECG, EDA and accelerometer signals, PyPhysio [75] adds support for PPG, EEG, EMG and RSP, while NeuroKit2 [72] further adds support for analyzing EOG, with each latter toolkit supporting the analysis of signals supported by former toolkits. With growing adoption of the Python programming language, all above-mentioned data analysis toolkits are available as Python libraries. Owing to the low computational complexity of the algorithms implemented by the toolkits, and the increasing availability of computational resources personal computers, these toolkits can be leveraged both for post-processing of acquired signals as well as for real-time computing of physiological metrics. Thus, an integrated solution that can be compatible with existing data analysis packages can immensely benefit interventional studies, providing simultaneous data acquisition and real-time analysis, which we address in this work.

3. The Proposed Physiological Computing Toolkit: PhysioKit

Figure 1 details the system architecture of PhysioKit. The toolkit consists of two layers: (i) the sensing and signal acquisition layer, and (ii) the software application layer. The former layer enables a modular setup for a wider range of physiological sensors which can be connected with a widely available micro-controller, thereby offering flexibility in configuring physiological sensors depending on the research needs. The software application layer includes a data collection module, a real-time streaming module, signal quality assessment module and a data analysis module. Our software application is built with Python, which can support different operating systems (e.g., Windows, Linux).

3.1. Sensing and Signal Acquisition Layer

This layer is designed to facilitate the flexibility of sensor placement, and provide options to configure the toolkit for various single and multi-user study settings. The sensing and signal acquisition layer of PhysioKit supports multiple inexpensive physiological sensors (e.g., PPG channels) that are compatible with a micro-controller (i.e., Arduino board as a default). These sensors can be connected to analog inputs of the board (e.g., A0–A3 pins in Arduino). Users can easily configure acquisition parameters (e.g., sampling rate and analog-to-digital conversion resolution). The layer transmits the collected sensor data to a connected computing device (e.g., laptop) via wired (USB) or wireless (Bluetooth) communication.
Table 1 lists some of the key parameters of the sensing and signal acquisition layer of PhysioKit, as well as it also mentions quick installation step and links to package contents. The Arduino board governs the hardware specifications, which can be selected according to research needs. For instance, some of the most cost-effective microcontroller boards (e.g., Arduino Uno, Arduino Nano) can support up to six physiological sensors and a sampling rate of up to 512 samples per second, whereas Arduino Due and Arduino Mega can support 12 and 16 channels, respectively.
To support flexible placement of physiological sensors, we designed an example template of a 3D-printable CAD model for a mountable PPG sensor wristband, as shown in Figure 2. This lets users acquire PPG signals from different body parts simultaneously, which can possibly help handle motion artifacts. The template model is available in the PhysioKit repository.
In this work, we focused on demonstrating the functionality and validation of the toolkit using two PPG sensors (https://pulsesensor.com/ (accessed on 28 September 2023)) connected to Arduino Due, which is presented in Section 4.1. There, we focus on validating HR and HRV measurements from PPG signal, as it is the most widely used physiological sensing channel. However, the interface and toolkit can be easily extended to other sensing channels, including RSP, EDA, ECG, EMG and EEG. To demonstrate the readiness of extending other sensors, we also integrated a RSP sensor (https://www.pluxbiosignals.com/products/respiration-pzt (accessed on 28 September 2023)) and an EDA sensor (https://seeeddoc.github.io/Grove-GSR_Sensor/(accessed on 28 September 2023)). Though the measurements of these alternative sensing channels were not validated in the current work, we discuss future plans to implement these in Figure 1.

3.2. Software Application Layer

The software application layer includes the user interface (UI), as depicted in Figure 3A, a real-time streaming and signal quality assessment module, as well as a data analysis module. The challenges concerning access to raw, unfiltered physiological signals and flexible configurations for passive and interventional studies involving both co-located and remote multi-user settings were carefully considered when developing the UI. The resulting interface includes features to facilitate data acquisition, signal visualization, and signal quality assessment, as well as bio-feedback visualization. The data analysis module builds upon existing data analysis toolkits [72,73] to further streamline analysis of physiological signals, as per the experimental protocol. The software module is implemented using the Python programming language, as it is a multi-platform language.

3.2.1. User Interface

To optimize acquisition, plotting and user controls, the PhysioKit UI is implemented with a multi-threaded design. Figure 3A shows a PhysioKit interface, which was designed using Qt design tools (https://www.qt.io/product/ui-design-tools (accessed on 28 September 2023)), with controls to configure an experimental study alongside display of real-time signal visualization. Acquired raw data are stored locally and appropriate signal conditioning is applied for plotting each physiological signal, while the quality of the acquired signals is assessed in real-time using a novel 1D-CNN-based signal quality assessment module (Section 3.2.2). In addition, the UI enables options for different visual presentations of biofeedback (Section 3.2.4).

3.2.2. Signal Quality Assessment Module for PPG

The signal quality assessment (SQA-Phys) module of PhysioKit extends 1D-CNN approach [57] that demonstrates the use of the 1D-CNN network for classifying the signal quality of PPG waveforms as “Good” and “Bad”. Contrary to classification, SQA-Phys introduces a novel task in which the ML model is trained to infer signal quality metrics for the entire length of the PPG signal segment. For this, we implement encoder–decoder architecture with 1D-CNN layers that generate high temporal resolution signal quality vector. Figure 4 compares the commonly used machine learning architecture with the proposed architecture, and highlights how the inferred outcome of SQA-Phys differs from the classification task as implemented by existing methods.
To train the model, we used in-house collected data acquired using the Infiniti Procomp and Empatica E4 wristband PPG sensors. This dataset, which includes 170 recordings of PPG signals (5 min) from 17 participants, was manually labeled for the signal quality. Signal quality labels for training were marked for 0.5 s of segment, without overlap. The temporal resolution of SQA-Phys inference was maintained as the same as the training labels (i.e., 0.5 s), as it is an optimal balance between dense temporal resolution (i.e., per sample inference) and classification. The optimal temporal resolution of 0.5 s can significantly reduce the computational complexity in comparison to the per sample semantic segmentation approach as proposed in a recent work [76].
We used a signal segment of 8 s as an input to the model and trained it with the batch size of 256 along with an Adam optimizer. The training and validation split was made based on participant IDs, ensuring that the signals of same individual were not represented in both the training and validation sets. Our validation on a novel signal quality assessment task shows 96% classification accuracy with an inference vector having 0.5 s temporal resolution, whereas the SOTA approach yielded 83% classification accuracy. The SQA-Phys is integrated with the PhysioKit to present signal quality assessment with optimal temporal resolution. The same is also stored alongside the data to provide signal quality annotation, thereby indicating the clean and noisy segments of the signals to researchers. Though SQA-Phys is currently implemented for PPG signals, it can be easily extended to other physiological signals.

3.2.3. Configuring the UI for Experimental Studies

Using experiment and software configuration files (Figure 3B,C), the UI adapts to the sensor configuration as well as researcher’s data collection protocol. The experiment configuration file (Figure 3B) enables users to set up their study protocol by defining the experimental conditions, the acquisition duration for each condition, the required physiological sensors, and the directory path where the acquired data will be saved locally. In contrast to storing data on cloud servers, as most commercial devices do, storing data locally allows researchers to have complete control over the data. In addition, the experiment configuration file allows the number of channels to be selected for real-time plotting of acquired physiological signals. While the maximum number of channels is limited by the number of analog channels on the microcontroller board, a maximum of four channels can be selected for real-time plotting. The acquisition duration for each experimental condition is defined by “max_time_seconds” (see Figure 3B) when “timed_acquisition” is set to “true”. However, when the “timed_acquisition” field is set to “false”, the UI ignores “max_time_seconds”, allowing data acquisition to continue until the user manually stops it.
The software configuration file (Figure 3C) allows users to configure acquisition parameters, such as sampling rate and baudrate for serial data transfer from the micro-controller to the computer. In addition, users can acquire physiological signals simultaneously from multiple users, with each user connected through respective sensing and signal acquisition unit. Here, different computers running the UI would communicate using TCP/IP messaging in order to synchronize the acquisition from the different sensing and signal acquisition unit. This setup requires each computer to be accessible with an IP address, either on a local intranet or remotely using a virtual private network (VPN).
To enable multi-user synchronization, one computer is configured as a TCP server while the others are TCP clients. Server and client roles are specified using the software configuration file. While the server is configured to accept requests from any IP address, the configuration file at the client end is required to specify the server IP address. In these settings, the live acquisition is initiated on all nodes. The client UIs remain idle until the server triggers the synchronized recording by broadcasting a TCP message. The TCP/IP-based messaging can be further extended to the stimulus presentation software (not part of PhysioKit) for synchronized delivery of the study intervention. Figure 5 provides an overview of the multi-user setup.
Lastly, HCI studies oftentimes require capturing asynchronous events during experiments for qualitative and quantitative analyses. To address this need, we designed a simple way for users to mark asynchronous events directly in the PhysioKit UI by activating and deactivating the marking function. This function is associated with an event code to enable marking for different types of events during data acquisition. The acquired data, along with the signal quality assessment and event markings, are stored in a comma-separated value format (CSV) for easy access and further analysis.

3.2.4. Support for Interventional Studies and Biofeedback

PhysioKit supports real-time computing of physiological metrics, which allows adapting interaction for interventional studies. One of the most widely researched interventional study types involves using biofeedback [4,77,78]. Using the experimental configuration file, researchers can specify a physiological metrics to be used for dynamic biofeedback visualization. Physiological metrics are computed in real-time with a provision for researchers to set the window length and step interval (see Figure 3B). PhysioKit currently provides options for basic biofeedback visualization using geometric shapes that vary in size and color. However, the mapping implementation can be easily adjusted to include different biofeedback modalities, such as auditory or haptic, according to the study requirements.

3.2.5. Data Analysis Helper

The software layer also facilitates visualization and analysis of the data acquired using the PhysioKit by integrating NeuroKit2 [72] library. Jupyter notebooks are provided with the repository to illustrate loading, pre-processing and analysis of the acquired PPG, EDA and RSP signals. PhysioKit further supports batch processing of entire dataset acquired from multiple participants for a specific study. To offer flexibility for this analysis, a separate configuration file is provided to specify the key analysis parameters; these include the sampling rate, window length (seconds), overlap (seconds) and a list of physiological metrics to compute for PPG, EDA and RSP signals. Batch processing generates a spreadsheet as well as a NumPy [79] dictionary consisting of computed physiological metrics for all participants, and as per the specified analysis parameters. While both the spreadsheet and the NumPy dictionary comprise the identical analysis data, the former provides an easily accessible format for researchers with less familiarity with programming, and the latter immensely benefits researchers who would like to perform customized analysis in Python environment. The data are organized with participant ID and experimental conditions, as well as participant groups (if specified). The computed metrics are validated as per the normal healthy range for respective metrics. However, this can be adjusted by researchers using the configuration file. Lastly, if the signals were acquired from multiple sensors (e.g., PPG signal from finger and ear) to mitigate noise artifacts, the analysis can be configured to inspect and compare the signal quality for each window segment and ultimately select the one with the highest signal quality.

4. Evaluation

4.1. Study 1: Performance Evaluation

The first study focuses on validating the performance of PhysioKit in extracting heart rate and heart rate variability (HRV) data from two PPGs on different body parts (finger and ear), given our primary focus on the most widely available channel. Our data collection protocol was designed to ensure high variances in physiological patterns for fairer validation (e.g., a narrowed range of HRV values in a dataset tends to lead to high accuracy). We used the Procomp Infiniti System (https://thoughttechnology.com/procomp-infiniti-system-w-biograph-infiniti-software-t7500m/ (accessed on 28 September 2023)) as a reference system (at 256 Hz), as it has been widely used by researchers in both clinical and non-clinical studies. In order to assess level of movements that can affect the signal quality, we also video-recorded each session (at 30 fps).

4.1.1. Data Collection Protocol

The study followed a methodological assessment protocol with the objective of inducing variations in physiological states, as well as moderating movement. Each participant experienced four conditions, including (a) a controlled, slow breathing task, (b) an easy math task, (c) a difficult math task, and (d) a guided head movement task, as depicted in Figure 6. While a higher agreement between the test device and the reference device could potentially be achieved in the absence of these variances, it could lead to misleading results.
Cognitively challenging math tasks with varying degrees of difficulty levels were chosen, as these have been reported to alter the physiological responses [80,81]. Furthermore, as wearable sensors are less reliable under significant motion conditions [82], we added an experimental condition that involved guided head movement. The PPG sensor on the ear remained relatively stable under all conditions, except during head movement, which provided us with the opportunity to investigate the impact of movement on signal quality at different sensor sites under varying motion conditions. Each condition lasted for 3 min, with 1 min of rest after each condition. To randomize the sequence of conditions, we inter-changed “A” with “D” and “B” with “C”. The study protocol was approved by the ethics committee of University College London Interaction Centre.

4.1.2. Participants and Study Preparation

Three physiological signals (PPG, EDA, and RSP) were collected from 16 participants recruited through an online recruitment platform for research. All participants reported having no known health conditions, provided informed consent ahead of the study, and were compensated for their time following the study. After being welcomed and briefed, participants were asked to remove any bulky clothing (e.g., winter coats, jackets) and seated comfortably in front of a 65 by 37 inch screen, where they were fitted with both Infiniti and Physiokit sensors. Respiration belts from both systems were additionally worn just below the diaphragm, one above the other without overlapping. One PhysioKit PPG sensor was attached to participants’ left ear with a metal clip, and the second was strapped around the participant’s middle finger of their non-dominant hand with a velcro strap along with the Infiniti PPG. Extra EDA sensors from both PhysioKit and Infiniti systems were placed on the index and ring fingers of the same hand without overlapping. Of the 16 participants, one was excluded from the analysis due to the incorrect fit of a PPG sensor.

4.1.3. Data Analysis

From 15 participants and four different experimental sessions, 60 pairs of PPG signals from each system (PhysioKit and Infiniti) were prepared to evaluate the system performance in extracting heart rate and heart rate variability data. Data analysis was performed using the data analysis module of PhysioKit, as described in Section 3.2.5. Pre-processing steps and signal quality analysis were uniformly executed for PPG signals acquired from PhysioKit and the reference device. A band-pass filter (0.7–4.0 Hz) of the third order was applied to PPG signals which were then processed to derive HR and HRV metrics. We applied windowing with a window size of 30 s [83] and a step interval of 10 s to calculate HR. As the HRV extracted from the PPG signal is referred to as pulse-rate variability (PRV) [10], in the following text, we mention it as PRV. For extracting PRV metrics, the window size was set to 120 s [84], with a step-interval of 10 s. In this work, our goal was to validate PRV metrics with the reference device, rather than validate PRV with HRV metrics, which are typically derived from the ECG signal. Among different PRV metrics, we selected pNN50, which provides a proportion of the successive heartbeat intervals exceeding 50 ms.
For fair evaluation, we used existing relative power signal quality index (pSQI) as described in [61]. pSQI was computed for each windowed segment from raw signals. Evaluation was conducted on PPG signal segments not affected by artifacts. For this, a threshold of 40% pSQI was applied to the PPG signals acquired using the reference device. For the segments of PPG signals of the reference device with less than 40% pSQI, corresponding segments from the PhysioKit were also discarded, with the assumption that the PPG signals from both devices are equally affected by motion artifacts.

4.1.4. Results

Bland–Altman scatter-plots were used to assess the agreement between the PhysioKit and the reference device (see Figure 7) [85,86]. These plots provide a combined comparison for all experimental conditions for HR (Figure 7A,C) and HRV metrics (Figure 7B,D). Since PPG signals were acquired from two different sites—finger and ear—we present the comparison for these two sensor sites separately.
Table 2 reports a detailed evaluation across each experimental conditions for both finger and ear sites. Both HR and PRV metrics from PhysioKit show a high correlation, as measured with Pearson correlation coefficient (r), and lower difference, as measured with root mean squared error (RMSE), mean absolute error (MAE) and standard deviation of error (SD). There is a higher correlation for HR (bpm) between PhysioKit and the reference device (ear: r = 0.97, finger: r = 0.97) than for PRV (pNN50) (ear: r = 0.87, finger: r = 0.89), which aligns with results from earlier studies listed in Table A1 in Appendix A.
For PRV, however, the agreement with the reference device during the difficult math task (ear: r = 0.64, finger: r = 0.75) and the face movement task (ear: r = 0.88, finger: r = 0.92) is lower than in the baseline. We can attribute this decrease in performance of PRV metrics for cognitively challenging and movement conditions to motion artifacts. Earlier studies involving commercial wearable PPG devices have also reported similar decreases in PRV accuracy [87].
In Figure 8A, we compare the SQI for PPG signals from both PhysioKit and the reference device under different experimental conditions. PhysioKit demonstrates consistent signal quality across different experimental conditions, while the reference device shows higher variance across each experimental condition. For PhysioKit, it can also be observed that the signal quality of the PPG sensor placed on the ear is higher than that of the finger for all conditions except face movements. Figure 8B compares the magnitude of facial movements across different experimental conditions, which is computed as a standard deviation of change in inter-frame rotation angle. A Wilcoxon Signed-Rank test found that motion increased significantly during math tasks and face-movement conditions from the baseline condition. From Figure 8A,B, it can be inferred that, under normal cognitive tasks not involving voluntary facial movement, the ear is less affected by motion artifacts.

4.2. Study 2: Usability Analysis

4.2.1. Use Cases

To demonstrate how PhysioKit can be applied in practice, PhysioKit was distributed to 10 different small-project teams (N = 44 members in total) from the department to use for their own research purposes for four to six weeks. In Table 3, we tabulate several examples of projects that made use of the toolkit for data acquisition as well as data analysis. Group members from each of these projects received a 1 h hands-on tutoring and spent four to six weeks utilizing the toolkit. Below, we summarize these projects by categorizing them into interventional and passive contexts (Section 2.1).

PhysioKit for Interventional Applications

Several groups explored using PhysioKit for developing affect recognition systems. For instance, some groups developed adaptive games that adjusted the level of gameplay difficulty by assessing acute stress using a combination of HR, HRV (e.g., pulse rate, pulse amplitude, interbeat interval), skin conductance and breathing rate. Another group examined how physiological responses from PPG and EDA could be mapped to a valence-arousal model to assess reactions to music. Other projects also examined the effects of biofeedback and social biofeedback visualizations of HR to overcome stressful scenarios (e.g., oral presentations) and promote mindfulness [4].

PhysioKit for Passive Applications

Project groups took advantage of PhysioKit’s diverse hardware and software functions to develop passive applications. For instance, a project that mapped stress during a virtual task assessed from an ear PPG showed that signals from this location were less subject to motion artifacts. Team members from a different group used PhysioKit’s event-marking function to generate a dataset for an affective music recommender system. Lastly, another research project team used the provision of synchronized acquisition for multi-user scenario to examine similarities in physiological responses during remote virtual reality experience.

4.2.2. Data Collection

To gain insights on the usability of PhysioKit, we designed a questionnaire in Qualtrics that included three parts. The first section focused on obtaining demographic data and information on participants’ prior experience with physiological sensing and toolkits. For the second part of the survey, we designed a modified version of the Usefulness, Satisfaction and Ease of Use questionnaire (USE; [88,89]) to gather insights on usability. For each question in this part of the survey, participants had five choices as follows: 1 (Strongly Disagree), 2 (Somewhat Disagree), 3 (Neither Agree Nor Disagree), 4 (Somewhat Agree) and 5 (Strongly Agree). The final section asked optional open-ended questions regarding participant’s favorite aspects, and suggestions for improvements and novel features.
One person representing each of the 10 project teams participated in completing the questionnaire (five female, five male; 18 to 34 years old), which took an average of 22.5 min to complete. Participation was voluntary, and no identifying information was collected. All participants had either completed a postgraduate degree in computer science or were currently in the process of completing one. Seven participants had used consumer-grade wearables (e.g., Apple Watch, FitBit Sense 2, Garmin, Withings Steel HR, Google Watch) either daily (N = 5) or occasionally (N = 2), while three had never used one before. In terms of previous experience with open source micro-controller kits, six participants had used them for a previous project or class (i.e., Arduino, Raspberry Pi, micro:bit Texas Instruments device), while four participants had no previous experience with micro-controllers.

4.2.3. Results

Usefulness and Ease of Use

Everyone who completed the survey acknowledged that PhysioKit was useful for their projects, with most strongly agreeing (N = 7). All participants also agreed that it gave them full control over their project activities (“Strongly agree”: N = 5) and facilitated easy completion of their project tasks (“Strongly agree”: N = 4). Several participants with limited computing experience (N = 3) considered using other physiological systems (e.g., Empatica, Apple Watch, fnv.reduce), but ultimately chose PhysioKit because it gave them full control over data and signals processing and setup.
Nearly all participants found that the provision of raw data were the most useful aspect of PhysioKit (N = 9), while almost all (N = 8) found that the ability to extract features, control data acquisition, and access supported data analysis was important. Also, having the flexibility to adapt PhysioKit to different experimental protocols was highly valued (N = 7). Participants found Physiokit easy to use (M = 4.30, SD = 0.675) and quick to set up (M = 4.00, SD = 0.943). They also appreciated that it enabled flexible configurations (M = 4.40, SD = 0.699).

Learning Process

Most participants learned to use PhysioKit quickly (M = 3.90, SD = 0.568) and with different sensor configurations for a diverse range of study designs (M = 3.90, SD = 0.568). Once they learned how to use the toolkit, everyone found it easy to remember how to use (M = 4.20, SD = 0.422), regardless of computing their prior experience.

Satisfaction

All participants were satisfied with the way PhysioKit worked (M = 4.20, SD = 0.422) and would recommend it to colleagues (M = 4.10, SD = 0.316). Many also found it essential for the completion of their projects (M = 4.60, SD = 0.699) and most would prefer to use it over other physiological systems for future projects (M = 3.50, SD = 0.527).

Open-Ended Questions

When participants were asked what they appreciated about PhysioKit, one person with limited programming experience responded: “It’s easy to understand and user-friendly for people without a coding foundation” (P3). People with high computing proficiency also found PhysioKit quick to setup, well-organized, simple and flexible to use. Lastly, participants left comments encouraging the further distribution of PhysioKit: “Promote it, make it accessible to more people, [help them] understand the difference between using this product and using physiological sensors directly.” (P3).

5. Discussion

In this section, we discuss our main findings and show how PhysioKit relates to and builds upon existing research.

5.1. Unique Propositions of PhysioKit

PhysioKit is a fully open source toolkit for physiological computing that implements an Arduino-based sensing and signal acquisition layer, offering researchers flexibility to configure one or more Arduino-compatible physiological sensors (PPG, EDA and RSP). The simple and user-friendly interface of the software application layer is effective in streamlining the physiological data acquisition for a variety of applications, including those involving biofeedback. Its provision to synchronize data acquisition for remotely located users is enabler for conducting remote studies that require acquiring physiological signals. Furthermore, it is cumbersome to manually inspect segments of acquired signals, specifically in case of long acquisition duration and high number of participants. SQA-Phys introduced in this work can be applied both in real-time and as a post-processing step to automate the quality assessment of PPG signals, significantly reducing efforts of researchers. Furthermore, access to raw data not only gives HCI researchers ownership and control over the data, but also enables computational research to develop robust algorithms for handling real-world environments. Taken together, these aspects all contribute to enhancing the usability of the toolkit for researchers.
The overall hardware cost of the PhysioKit is well below the least expensive of the commercial sensors mentioned in Table A1. For instance, with a setup of one Arduino Uno and one PPG sensor, the cost is less than USD 50. For the collection of sensors used in this work (i.e., two PPG sensors, one EDA sensor and a one RSP sensor), it amounts to less than USD 200, which is still less than the average cost of commercially available sensors in Table A1. The number of sensors that can be connected simultaneously is only limited by the number of analog input channels of the chosen Arduino board. The repository provides Arduino programs of up to four simultaneous sensors, though researchers can easily extend it to support higher numbers of sensors. The corresponding changes to configure the software application layer are limited to specifying the number and types of sensors in the experiment configuration file, making it a highly efficient and cost-effective solution to address different research needs.

5.2. Evaluating the Validation of PhysioKit

The validation study in Section 4.1 highlights very good agreement between PhysioKit and the gold standard for HR. The PRV metrics also show good agreement during the baseline condition, while there is acceptable agreement during experimental conditions involving significant movement. This difference between performance related to HR and PRV can be explained by the findings of a recent study that assessed the validity and reliability of PPG derived HRV metrics, and found that PPG sensors are less reliable for HRV measurements [90]. However, it is worth mentioning that the same study found PPG sensors to be accurate for measuring HR. It is also noteworthy that the overall performance of PhysioKit shows better agreement with the gold standard compared to the performance of existing PPG-based commercial devices mentioned in Table A1. The performance of PhysioKit can be largely attributed to its sensing and signal acquisition layer, as well as the processing pipeline that includes signal quality assessment, filtering, and extraction of physiological metrics.
Regarding flexible configuration, the validation study results show similar performance of the finger PPG sensor and ear PPG sensor, suggesting that alternative sensor sites for PPG can be explored to achieve different research objectives and support accessibility. The slightly low signal quality (pSQI) observed for finger PPG sensor can be attributed to the possible voluntarily movement of finger leading to more frequent motion artifacts, whereas inability to ambulate ear lobe makes it more promising site for acquiring PPG signals. This interpretation is further supported by the lower pSQI of ear-PPG signals for an experimental condition involving guided face movement (Figure 8A).

5.3. Assessing the Usability of PhysioKit

Projects with both interventional and passive application types utilized PhysioKit, demonstrating its versatility in supporting data acquisition in different settings. The usability survey results (see Section 4.2) highlight the usefulness, learning experience and favorable aspects of PhysioKit. Participants agreed that PhysioKit was essential for their projects because it provided them with control and flexibility over tasks and allowed them to accomplish what they wanted to do. While access to raw data were valued as the most important aspect of the toolkit, they also appreciated having control over data acquisition, feature extraction and data analysis. Participants felt PhysioKit was easy to use and seemed enthusiastic about using PhysioKit for future projects, as well as sharing it with colleagues and friends because of its open-source features.

5.4. Limitations and Future Work

In this work, we implemented PPG, EDA and RSP sensors. However, future work could explore integrating other contact-based physiological sensors commonly used in HCI research (e.g., EMG, ECG and EEG) [1,91]. The validation study [92,93,94] and signal quality assessment module (SQA-Phys) mentioned in this work currently only apply to data from PPG sensors. However, these can also be extended to other contact-based physiological sensors, such as the ones previously mentioned. While PhysioKit offers flexibility in choosing any physiological sensor compatible with Arduino, it does not introduce new sensor hardware. Therefore, in-the-wild scenarios such as physical activity were not evaluated in this work, since the challenges offered by such scenarios are associated with the sensor hardware and fitment.
The objectives of making PhysioKit an open-source toolkit are two-fold: (i) offering a flexible and cost-effective solution for research community, and (ii) leveraging the contributions from research community in addressing the existing limitations of PhysioKit towards introducing support for additional physiological sensors and biofeedback modalities, as well as extending the validation study for range of applications, sensor modalities and higher numbers of participants. The latter objective is expected to be achieved as a future work. Currently, the signal quality assessment module (SQA-Phys) mentioned in this work is limited to assess the quality of PPG signals, which could be extended to signals acquired using other sensors. Additionally, although the existing implementation of SQA-Phys provides high temporal resolution, its outcome is restricted binary classification at a given time-point. By using an appropriate activation function (e.g., ReLU) at the final layer of the 1D-CNN architecture, as well as replacing the classification loss with the regression loss function, it is possible to obtain a continuous score from 0 to 1 to gain better insights into the signal quality, which will be addressed as a future work.
While one project used event-marking function of PhysioKit to build a dataset to train affective music recommender system, no projects required performing data analysis based on the event-marking labels. In future work, this aspect of the toolkit can be further explored and validated. The hands-on training for groups participating in the usability study was provided owing to unavailability of elaborate installation and usage instructions for the toolkit at early development stage. The published repository of PhysioKit offers easy installation steps, along with detailed usage instructions. Furthermore, as a future work, tutorials for different application types will be made available at the repository homepage.
Our future implementation plan further considers implementing a processing pipeline for contact-less sensing methods, such as an RGB camera-based remote PPG [95,96] and a thermal infrared sensing pipeline, including optimal quantization [9] and semantic segmentation [97] for extraction of breathing [9] and blood volume pulse signals [56,98,99]. We also aim to further enhance the accessibility of the sensor interface (hardware) of PhysioKit for its use in real-world scenarios.

6. Conclusions

This paper introduced PhysioKit, an open-source and cost effective physiological computing toolkit that streamlines physiological signal acquisition and analysis for various HCI studies and applications. Uniquely, PhysioKit can synchronize signal collection in multi-user studies with both co-located as well as remote users. The evaluation study on heart rate and pulse rate variability measurements demonstrated the performance of PhysioKit compared to the reference system. The comparable signal reliability from the finger and ear PPG channels further indicates the possibility of enhancing accessibility to support participants with certain physical impairments (e.g., wheelchair users). Further, the usability study highlighted the usefulness, satisfaction and ease of use associated with PHysioKit, emphasizing its positive impact in various application scenarios. PhysioKit provides further useful features for researchers and practitioners: visual biofeedback, which can be extended to other forms of biofeedback such as audio and haptics; and machine-learning-driven signal quality assessment that can significantly reduce efforts and time on manual signal inspection (for discarding noisy signal segments). Also, raw physiological data are stored in an accessible format, which can be organized as per the study protocol and participant ID and can foster opportunities for researchers to easily apply state-of-the-art analysis. PhysioKit thus provides a one-stop approach that supports physiological sensing, data acquisition and computing supporting for a broad spectrum of studies and HCI applications.

Author Contributions

Conceptualization, Y.C. and J.J.; methodology, J.J., K.W. and Y.C.; programming, J.J.; study validation, J.J. and K.W.; artefacts validation, J.J. and Y.C.; investigation, Y.C.; data collection and analysis, J.J., K.W. and Y.C; manuscript preparation, J.J., K.W. and Y.C; visualization, J.J.; overall supervision, Y.C.; project administration, Y.C.; funding acquisition, Y.C. All authors have read and agreed to the published version of the manuscript.

Funding

UCL CS PhD Studentship; GDI - Physiological Computing and Artificial Intelligence.

Institutional Review Board Statement

The study protocol is approved by the University College London Interaction Centre ethics committee (ID Number: UCLIC/1920/006/Staff/Cho).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data are unavailable due to privacy or ethical restrictions.

Acknowledgments

Authors thank Luke Beddow for contributing to our 3D-printable model design for a wristband case. The authors also thank our participants who participated in the study.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
1D-CNN1-dimensional convolutional neural network
ADCAnalog-to-digital converter
APIApplication programming interface
BPMBeats per minute
BVPBlood volume pulse
CADComputer-aided design
ECGElectrocardiogram
EDAElectrodermal activity
EMGElectromyography
GSRGalvanic skin response
HCIHuman–computer interface
HRHeart rate
HRVHeart rate variability
LSTMLong short-term memory
PPGPhotoplethysmography
PRVPulse rate variability
RGBColor images with red, green an blue frames
RSPRespiration or breathing
SOTAState-of-the-art
SQISignal quality index
SVMSupport vector machine
TCP/IPTransmission control protocol/Internet protocol
UIUser interface

Appendix A. Validation Studies of Physiological Computing Systems

In Table A1, we highlight validation studies of commercially available physiological computing systems, and include studies that support analyzing heart-rate signals from PPG as well as ECG and provide validation results for the same. It is to be noted that this table is not exhaustive, and is presented for scoping purposes.
Most studies reported results as mean absolute error (MAE), interclass correlation (ICC) κ , or Pearson’s correlation coefficient (r) for HR (bpm). Where applicable, we also listed results of HRV validation, which were primarily reported as ICC κ for RMSSD. To illustrate how affordable, readily available, and useful the devices were, the table also includes the price (USD) and availability of the device, as well as whether raw signal data (e.g., BVP, ECG, EEG signals) is accessible. On average, consumer-grade commercial HR/HRV devices cost USD 291.77 (SD: USD 158.77), and had limited access to the mentioned raw signals. Also, by the time validation studies were published, most manufacturers had already discontinued the tested versions of the device. Low-cost research-grade devices range from USD 459.78 to USD 1690, and often provide access to raw physiological signals. However, existing low-cost research-grade devices and toolkits are not designed to readily support interventional studies as well as co-located or remote multi-user studies.
Of the physiological sensing devices included in validation studies (e.g., Xiaomi Miband 3, Apple Watch 4, Fitbit Charge 2, Garmin Vivosmart 3), consumer-grade wearables were significantly more accurate HR measurements at rest compared to research-grade wearables (i.e., Empatica E4, Biofotion Everion) when compared with a reference device (MAE: 7.2 ± 5.4 bpm compared to 13.9 ± 7.8 bpm [67]). The Apple Watch showed highest accuracy in several studies (Apple Watch 4 MAE: 4.4 ± 2.7 bpm [67]; Apple Watch 6 ICC: 0.96 [14,53]) compared to other commercial devices. Despite this, we could not find any studies that confirmed it was in good agreement with a gold standard reference device. Research-grade consumer devices showed the highest MAEs (Empatica: 11.3 ± 8.0 bpm, Bioevotion: 16.5 ± 6.4 bpm [67]) in a list of commercially available wearables.
In terms of results from validation studies, the RR interval signal quality of the Polar H10 HR chest belt was considered in good agreement with the gold standard (error rate 0.16 during rest), i.e., medilog AR12plus ECG Holter monitor [100]. Similarly, Garmin 920XT was in good agreement with the gold standard (i.e., ADInstruments Bio Amps) during rest conditions [101]. While validation studies involving physical activity conditions have shown moderate-to-strong agreement for devices, they have consistently revealed a decrease in accuracy as activity levels increase [102,103]. One common explanation for this discrepancy are motion artifacts from movement. Some devices (e.g., Polar H10), have designed elastic chest straps with multiple embedded electrodes to protect against measurement noise. The problem with elastic electrode straps is, however, the reduction in contact between skin and elastic electrode straps [100].
Table A1. Validation studies of commercially available consumer-grade devices as well as low-cost research-grade devices and toolkits with participants at rest.
Table A1. Validation studies of commercially available consumer-grade devices as well as low-cost research-grade devices and toolkits with participants at rest.
DeviceDevice
Category
Cost (USD)Access to Raw
Physiological Signals
Support for
Interventional Studies
Support for
Multi-User Studies
Validation Results
for HR (bpm)
Validation Results
for HRV
AliveCor KardiaMobileC79.00NoNoNo κ = 0.96 [53]N.A.
Apple Watch 4C399.00 *NoNoNoMAE = 4.4 ± 2.7 [67]; r = 0.99 [66]N.A.
Apple Watch 6C399.99 *NoNoNo κ = 0.96 [14,53]RMSSD: κ = 0.67 [14]
Fitbit Charge 2 FitbitC149.95 *NoNoNoMAE = 7.3 ± 4.2 [67]N.A.
Fitbit SenseC159.99 *NoNoNo κ = 0.88 [53]N.A.
Garmin Vivosmart 3C139.99 *NoNoNoMAE = 7.0 ± 5.0 [67]N.A.
Garmin Fenix 5C599.00 *NoNoNor= 0.89 [66]N.A.
Garmin Forerunner 245 MusicC349.00 *NoNoNo κ = 0.41RMSSD: κ = 0.24 [14]
Garmin Vivosmart HR+C219.99 *NoNoNoMAE = 2.98, κ = 0.90 [102]N.A.
Oura Gen 2C299.00NoNoNo κ = 0.85RMSSD: κ = 0.63 [14]
Polar H10 C89.95NoNoNoN.A.HRV (RR, ms) Spearman r = 1.00 [100]
Polar Vantage VC499.95NoNoNo κ = 0.93; r = 0.99 [66]RMSSD: κ = 0.65 [14]
Samsung Galaxy Watch3C399.99 *NoNoNo κ = 0.96 [53]N.A.
Withings ScanwatchC299.95NoNoNo κ = 0.95 [53]N.A.
Empatica E4R1690.00 *Yes (filtered signals
obtained from company servers)
NoNoMAE = 11.3 ± 8.0 [67]N.A.
BITalino (PsychoBIT)R459.78YesUsing third party
software
Yes,
only co-located
Morphology-based validation
with ECG signals: R 2 = 0.83 [104]
N.A.
OpenBCI (EmotiBit)R499.97YesUsing third party
software
NoN.A.N.A.
* Discontinued device: devices which are no longer being manufactured and sold; ECG device; C: consumer device; R: research-grade device.

References

  1. Wang, T.; Zhang, H. Using Wearable Devices for Emotion Recognition in Mobile Human–Computer Interaction: A Review. In HCII 2022: HCI International 2022—Late Breaking Papers. Multimodality in Advanced Interaction Environments; Kurosu, M., Yamamoto, S., Mori, H., Schmorrow, D.D., Fidopiastis, C.M., Streitz, N.A., Konomi, S., Eds.; Lecture Notes in Computer Science; Springer Nature Switzerland: Cham, Switzerland, 2022; pp. 205–227. [Google Scholar] [CrossRef]
  2. Malasinghe, L.P.; Ramzan, N.; Dahal, K. Remote Patient Monitoring: A Comprehensive Study. J. Ambient Intell. Humaniz. Comput. 2019, 10, 57–76. [Google Scholar] [CrossRef]
  3. Chanel, G.; Mühl, C. Connecting Brains and Bodies: Applying Physiological Computing to Support Social Interaction. Interact. Comput. 2015, 27, 534–550. [Google Scholar] [CrossRef]
  4. Moge, C.; Wang, K.; Cho, Y. Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, New York, NY, USA, 29 April 2022; pp. 1–16. [Google Scholar] [CrossRef]
  5. Cho, Y. Rethinking Eye-blink: Assessing Task Difficulty through Physiological Representation of Spontaneous Blinking. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, New York, NY, USA, 7 May 2021; pp. 1–12. [Google Scholar] [CrossRef]
  6. Ayobi, A.; Sonne, T.; Marshall, P.; Cox, A.L. Flexible and Mindful Self-Tracking: Design Implications from Paper Bullet Journals. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems; CHI ’18, New York, NY, USA, 19 April 2018; pp. 1–14. [Google Scholar] [CrossRef]
  7. Koydemir, H.C.; Ozcan, A. Wearable and Implantable Sensors for Biomedical Applications. Annu. Rev. Anal. Chem. 2018, 11, 127–146. [Google Scholar] [CrossRef] [PubMed]
  8. Peake, J.M.; Kerr, G.; Sullivan, J.P. A Critical Review of Consumer Wearables, Mobile Applications, and Equipment for Providing Biofeedback, Monitoring Stress, and Sleep in Physically Active Populations. Front. Physiol. 2018, 9, 743. [Google Scholar] [CrossRef]
  9. Cho, Y.; Julier, S.J.; Marquardt, N.; Bianchi-Berthouze, N. Robust Tracking of Respiratory Rate in High-Dynamic Range Scenes Using Mobile Thermal Imaging. Biomed. Opt. Express 2017, 8, 4480–4503. [Google Scholar] [CrossRef]
  10. Cho, Y.; Julier, S.J.; Bianchi-Berthouze, N. Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging. JMIR Ment. Health 2019, 6, e10140. [Google Scholar] [CrossRef]
  11. Ometov, A.; Shubina, V.; Klus, L.; Skibińska, J.; Saafi, S.; Pascacio, P.; Flueratoru, L.; Gaibor, D.Q.; Chukhno, N.; Chukhno, O.; et al. A Survey on Wearable Technology: History, State-of-the-Art and Current Challenges. Comput. Netw. 2021, 193, 108074. [Google Scholar] [CrossRef]
  12. Baron, K.G.; Duffecy, J.; Berendsen, M.A.; Cheung Mason, I.; Lattie, E.G.; Manalo, N.C. Feeling Validated yet? A Scoping Review of the Use of Consumer-Targeted Wearable and Mobile Technology to Measure and Improve Sleep. Sleep Med. Rev. 2018, 40, 151–159. [Google Scholar] [CrossRef]
  13. Arias, O.; Wurm, J.; Hoang, K.; Jin, Y. Privacy and Security in Internet of Things and Wearable Devices. IEEE Trans. Multi-Scale Comput. Syst. 2015, 1, 99–109. [Google Scholar] [CrossRef]
  14. Miller, D.J.; Sargent, C.; Roach, G.D. A Validation of Six Wearable Devices for Estimating Sleep, Heart Rate and Heart Rate Variability in Healthy Adults. Sensors 2022, 22, 6317. [Google Scholar] [CrossRef]
  15. Sato, M.; Puri, R.S.; Olwal, A.; Chandra, D.; Poupyrev, I.; Raskar, R. Zensei: Augmenting Objects with Effortless User Recognition Capabilities through Bioimpedance Sensing. In Proceedings of the Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology; Daegu, Republic of Korea, 8–11 November 2015, pp. 41–42. [CrossRef]
  16. Shibata, T.; Peck, E.M.; Afergan, D.; Hincks, S.W.; Yuksel, B.F.; Jacob, R.J. Building Implicit Interfaces for Wearable Computers with Physiological Inputs: Zero Shutter Camera and Phylter. In Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology—UIST’14 Adjunct, Honolulu, HI, USA, 5 October 2014; pp. 89–90. [Google Scholar] [CrossRef]
  17. Yamamura, H.; Baldauf, H.; Kunze, K. Pleasant Locomotion—Towards Reducing Cybersickness Using fNIRS during Walking Events in VR. In Proceedings of the Adjunct Publication of the 33rd Annual ACM Symposium on User Interface Software and Technology, Virtual Event, USA, 20 October 2020; pp. 56–58. [Google Scholar] [CrossRef]
  18. Pai, Y.S. Physiological Signal-Driven Virtual Reality in Social Spaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 16 October 2016; pp. 25–28. [Google Scholar] [CrossRef]
  19. Baldini, A.; Garofalo, R.; Scilingo, E.P.; Greco, A. A Real-Time, Open-Source, IoT-like, Wearable Monitoring Platform. Electronics 2023, 12, 1498. [Google Scholar] [CrossRef]
  20. Pradhan, N.; Rajan, S.; Adler, A. Evaluation of the Signal Quality of Wrist-Based Photoplethysmography. Physiol. Meas. 2019, 40, 065008. [Google Scholar] [CrossRef]
  21. Park, J.; Seok, H.S.; Kim, S.S.; Shin, H. Photoplethysmogram Analysis and Applications: An Integrative Review. Front. Physiol. 2022, 12, 808451. [Google Scholar] [CrossRef] [PubMed]
  22. Moscato, S.; Lo Giudice, S.; Massaro, G.; Chiari, L. Wrist Photoplethysmography Signal Quality Assessment for Reliable Heart Rate Estimate and Morphological Analysis. Sensors 2022, 22, 5831. [Google Scholar] [CrossRef] [PubMed]
  23. Lee, J.; Natarrajan, R.; Rodriguez, S.S.; Panda, P.; Ofek, E. RemoteLab: A VR Remote Study Toolkit. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22, New York, NY, USA, 28 October 2022; pp. 1–9. [Google Scholar] [CrossRef]
  24. Lee, J.H.; Gamper, H.; Tashev, I.; Dong, S.; Ma, S.; Remaley, J.; Holbery, J.D.; Yoon, S.H. Stress Monitoring Using Multimodal Bio-sensing Headset. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25 April 2020; pp. 1–7. [Google Scholar] [CrossRef]
  25. Maritsch, M.; Föll, S.; Lehmann, V.; Bérubé, C.; Kraus, M.; Feuerriegel, S.; Kowatsch, T.; Züger, T.; Stettler, C.; Fleisch, E.; et al. Towards Wearable-based Hypoglycemia Detection and Warning in Diabetes. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25 April 2020; pp. 1–8. [Google Scholar] [CrossRef]
  26. Liang, R.H.; Yu, B.; Xue, M.; Hu, J.; Feijs, L.M.G. BioFidget: Biofeedback for Respiration Training Using an Augmented Fidget Spinner. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 21 April 2018; pp. 1–12. [Google Scholar] [CrossRef]
  27. Clegg, T.; Norooz, L.; Kang, S.; Byrne, V.; Katzen, M.; Velez, R.; Plane, A.; Oguamanam, V.; Outing, T.; Yip, J.; et al. Live Physiological Sensing and Visualization Ecosystems: An Activity Theory Analysis. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 2 May 2017; pp. 2029–2041. [Google Scholar] [CrossRef]
  28. Wang, H.; Prendinger, H.; Igarashi, T. Communicating Emotions in Online Chat Using Physiological Sensors and Animated Text. In Proceedings of the CHI ’04 Extended Abstracts on Human Factors in Computing Systems, Vienna, Austria, 24 April 2004; pp. 1171–1174. [Google Scholar] [CrossRef]
  29. Dey, A.; Piumsomboon, T.; Lee, Y.; Billinghurst, M. Effects of Sharing Physiological States of Players in a Collaborative Virtual Reality Gameplay. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, New York, NY, USA, 2 May 2017; pp. 4045–4056. [Google Scholar] [CrossRef]
  30. Olugbade, T.; Cho, Y.; Morgan, Z.; El Ghani, M.A.; Bianchi-Berthouze, N. Toward Intelligent Car Comfort Sensing: New Dataset and Analysis of Annotated Physiological Metrics. In Proceedings of the 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), Nara, Japan, 28 September 2021; pp. 1–8. [Google Scholar] [CrossRef]
  31. Dillen, N.; Ilievski, M.; Law, E.; Nacke, L.E.; Czarnecki, K.; Schneider, O. Keep Calm and Ride Along: Passenger Comfort and Anxiety as Physiological Responses to Autonomous Driving Styles. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 21 April 2020; pp. 1–13. [Google Scholar] [CrossRef]
  32. Frey, J.; Ostrin, G.; Grabli, M.; Cauchard, J.R. Physiologically Driven Storytelling: Concept and Software Tool. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 21 April 2020; pp. 1–13. [Google Scholar] [CrossRef]
  33. Karran, A.J.; Fairclough, S.H.; Gilleade, K. Towards an Adaptive Cultural Heritage Experience Using Physiological Computing. In Proceedings of the CHI ’13 Extended Abstracts on Human Factors in Computing Systems, Paris, France, 27 April 2013; pp. 1683–1688. [Google Scholar] [CrossRef]
  34. Robinson, R.B.; Reid, E.; Depping, A.E.; Mandryk, R.; Fey, J.C.; Isbister, K. ’In the Same Boat’,: A Game of Mirroring Emotions for Enhancing Social Play. In Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland, UK, 2 May 2019; pp. 1–4. [Google Scholar] [CrossRef]
  35. Schneegass, S.; Pfleging, B.; Broy, N.; Heinrich, F.; Schmidt, A. A Data Set of Real World Driving to Assess Driver Workload. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’13, New York, NY, USA, 28 October 2013; pp. 150–157. [Google Scholar] [CrossRef]
  36. Wang, K.; Julier, S.J.; Cho, Y. Attention-Based Applications in Extended Reality to Support Autistic Users: A Systematic Review. IEEE Access 2022, 10, 15574–15593. [Google Scholar] [CrossRef]
  37. Fairclough, S.H.; Karran, A.J.; Gilleade, K. Classification Accuracy from the Perspective of the User: Real-Time Interaction with Physiological Computing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 18 April 2015; pp. 3029–3038. [Google Scholar] [CrossRef]
  38. Oh, S.Y.; Kim, S. Does Social Endorsement Influence Physiological Arousal? In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, San Jose, CA, USA, 7 May 2016; pp. 2900–2905. [Google Scholar] [CrossRef]
  39. Sato, Y.; Ueoka, R. Investigating Haptic Perception of and Physiological Responses to Air Vortex Rings on a User’s Cheek. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 2 May 2017; pp. 3083–3094. [Google Scholar] [CrossRef]
  40. Frey, J.; Daniel, M.; Castet, J.; Hachet, M.; Lotte, F. Framework for Electroencephalography-based Evaluation of User Experience. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, New York, NY, USA, 7 May 2016; pp. 2283–2294. [Google Scholar] [CrossRef]
  41. Nukarinen, T.; Istance, H.O.; Rantala, J.; Mäkelä, J.; Korpela, K.; Ronkainen, K.; Surakka, V.; Raisamo, R. Physiological and Psychological Restoration in Matched Real and Virtual Natural Environments. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25 April 2020; pp. 1–8. [Google Scholar] [CrossRef]
  42. Grassmann, M.; Vlemincx, E.; von Leupoldt, A.; Mittelstädt, J.M.; Van den Bergh, O. Respiratory Changes in Response to Cognitive Load: A Systematic Review. Neural Plast. 2016, 2016, e8146809. [Google Scholar] [CrossRef] [PubMed]
  43. Vlemincx, E.; Taelman, J.; De Peuter, S.; Van Diest, I.; Van Den Bergh, O. Sigh Rate and Respiratory Variability during Mental Load and Sustained Attention. Psychophysiology 2011, 48, 117–120. [Google Scholar] [CrossRef]
  44. Xiao, X.; Pham, P.; Wang, J. AttentiveLearner: Adaptive Mobile MOOC Learning via Implicit Cognitive States Inference. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle, WA, USA, 9 November 2015; pp. 373–374. [Google Scholar] [CrossRef]
  45. DiSalvo, B.; Bandaru, D.; Wang, Q.; Li, H.; Plötz, T. Reading the Room: Automated, Momentary Assessment of Student Engagement in the Classroom: Are We There Yet? Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2022, 6, 1–26. [Google Scholar] [CrossRef]
  46. Huynh, S.; Kim, S.; Ko, J.; Balan, R.K.; Lee, Y. EngageMon: Multi-Modal Engagement Sensing for Mobile Games. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2018, 2, 1–27. [Google Scholar] [CrossRef]
  47. Qin, C.Y.; Choi, J.H.; Constantinides, M.; Aiello, L.M.; Quercia, D. Having a Heart Time? A Wearable-based Biofeedback System. In Proceedings of the 22nd International Conference on Human–Computer Interaction with Mobile Devices and Services, MobileHCI ’20, New York, NY, USA, 5 October 2020; pp. 1–4. [Google Scholar] [CrossRef]
  48. Howell, N.; Niemeyer, G.; Ryokai, K. Life-Affirming Biosensing in Public: Sounding Heartbeats on a Red Bench. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, Glasgow, Scotland, UK, 2 May 2019; pp. 1–16. [Google Scholar] [CrossRef]
  49. Curran, M.T.; Gordon, J.R.; Lin, L.; Sridhar, P.K.; Chuang, J. Understanding Digitally Mediated Empathy: An Exploration of Visual, Narrative, and Biosensory Informational Cues. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland, UK, 2 May 2019; pp. 1–13. [Google Scholar] [CrossRef]
  50. Moraveji, N.; Olson, B.; Nguyen, T.; Saadat, M.; Khalighi, Y.; Pea, R.; Heer, J. Peripheral Paced Respiration: Influencing User Physiology during Information Work. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, Santa Barbara, CA, USA, 16 October 2011; pp. 423–428. [Google Scholar] [CrossRef]
  51. Wongsuphasawat, K.; Gamburg, A.; Moraveji, N. You Can’t Force Calm: Designing and Evaluating Respiratory Regulating Interfaces for Calming Technology. In Proceedings of the Adjunct Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, Cambridge, MA, USA, 7 October 2012; pp. 69–70. [Google Scholar] [CrossRef]
  52. Frey, J.; Grabli, M.; Slyper, R.; Cauchard, J.R. Breeze: Sharing Biofeedback through Wearable Technologies. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, New York, NY, USA, 21 April 2018; pp. 1–12. [Google Scholar] [CrossRef]
  53. Mannhart, D.; Lischer, M.; Knecht, S.; du Fay de Lavallaz, J.; Strebel, I.; Serban, T.; Vögeli, D.; Schaer, B.; Osswald, S.; Mueller, C.; et al. Clinical Validation of 5 Direct-to-Consumer Wearable Smart Devices to Detect Atrial Fibrillation: BASEL Wearable Study. JACC Clin. Electrophysiol. 2023, 9, 232–242. [Google Scholar] [CrossRef]
  54. Schneeberger, T.; Sauerwein, N.; Anglet, M.S.; Gebhard, P. Stress Management Training Using Biofeedback Guided by Social Agents. In Proceedings of the 26th International Conference on Intelligent User Interfaces, IUI ’21, New York, NY, USA, 14 April 2021; pp. 564–574. [Google Scholar] [CrossRef]
  55. Sweeney, K.T.; Ward, T.E.; McLoone, S.F. Artifact Removal in Physiological Signals—Practices and Possibilities. IEEE Trans. Inf. Technol. Biomed. 2012, 16, 488–500. [Google Scholar] [CrossRef]
  56. Akbar, F.; Mark, G.; Pavlidis, I.; Gutierrez-Osuna, R. An Empirical Study Comparing Unobtrusive Physiological Sensors for Stress Detection in Computer Work. Sensors 2019, 19, 3766. [Google Scholar] [CrossRef]
  57. Shin, H. Deep Convolutional Neural Network-Based Signal Quality Assessment for Photoplethysmogram. Comput. Biol. Med. 2022, 145, 105430. [Google Scholar] [CrossRef] [PubMed]
  58. Desquins, T.; Bousefsaf, F.; Pruski, A.; Maaoui, C. A Survey of Photoplethysmography and Imaging Photoplethysmography Quality Assessment Methods. Appl. Sci. 2022, 12, 9582. [Google Scholar] [CrossRef]
  59. Gao, H.; Wu, X.; Shi, C.; Gao, Q.; Geng, J. A LSTM-Based Realtime Signal Quality Assessment for Photoplethysmogram and Remote Photoplethysmogram. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 19 June 2021; pp. 3831–3840. [Google Scholar]
  60. Pereira, T.; Gadhoumi, K.; Ma, M.; Liu, X.; Xiao, R.; Colorado, R.A.; Keenan, K.J.; Meisel, K.; Hu, X. A Supervised Approach to Robust Photoplethysmography Quality Assessment. IEEE J. Biomed. Health Inform. 2020, 24, 649–657. [Google Scholar] [CrossRef]
  61. Elgendi, M. Optimal Signal Quality Index for Photoplethysmogram Signals. Bioengineering 2016, 3, 21. [Google Scholar] [CrossRef] [PubMed]
  62. Ahmed, T.; Rahman, M.M.; Nemati, E.; Ahmed, M.Y.; Kuang, J.; Gao, A.J. Remote Breathing Rate Tracking in Stationary Position Using the Motion and Acoustic Sensors of Earables. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, Hamburg, Germany, 19 April 2023; pp. 1–22. [Google Scholar] [CrossRef]
  63. Gashi, S.; Di Lascio, E.; Santini, S. Using Unobtrusive Wearable Sensors to Measure the Physiological Synchrony Between Presenters and Audience Members. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2019, 3, 1–19. [Google Scholar] [CrossRef]
  64. Milstein, N.; Gordon, I. Validating Measures of Electrodermal Activity and Heart Rate Variability Derived From the Empatica E4 Utilized in Research Settings That Involve Interactive Dyadic States. Front. Behav. Neurosci. 2020, 14, 148. [Google Scholar] [CrossRef]
  65. Stroop, J.R. Studies of Interference in Serial Verbal Reactions. J. Exp. Psychol. 1935, 18, 643–662. [Google Scholar] [CrossRef]
  66. Düking, P.; Giessing, L.; Frenkel, M.O.; Koehler, K.; Holmberg, H.C.; Sperlich, B. Wrist-Worn Wearables for Monitoring Heart Rate and Energy Expenditure While Sitting or Performing Light-to-Vigorous Physical Activity: Validation Study. JMIR MHealth UHealth 2020, 8, e16716. [Google Scholar] [CrossRef]
  67. Bent, B.; Goldstein, B.A.; Kibbe, W.A.; Dunn, J.P. Investigating Sources of Inaccuracy in Wearable Optical Heart Rate Sensors. Npj Digit. Med. 2020, 3, 1–9. [Google Scholar] [CrossRef] [PubMed]
  68. BIOPAC | Data Acquisition, Loggers, Amplifiers, Transducers, Electrodes. Available online: https://www.biopac.com/ (accessed on 23 June 2023).
  69. Shimmer Wearable Sensor Technology | Wireless IMU | ECG | EMG | GSR. Available online: https://shimmersensing.com/ (accessed on 9 August 2023).
  70. BITalino. Available online: https://www.pluxbiosignals.com/collections/bitalino (accessed on 28 May 2023).
  71. ThoughtTech—Unlock Human Potential. Available online: https://thoughttechnology.com/# (accessed on 9 August 2023).
  72. Makowski, D.; Pham, T.; Lau, Z.J.; Brammer, J.C.; Lespinasse, F.; Pham, H.; Schölzel, C.; Chen, S.H.A. NeuroKit2: A Python Toolbox for Neurophysiological Signal Processing. Behav. Res. Methods 2021, 53, 1689–1696. [Google Scholar] [CrossRef] [PubMed]
  73. van Gent, P.; Farah, H.; Nes, N.; Arem, B. Heart Rate Analysis for Human Factors: Development and Validation of an Open Source Toolkit for Noisy Naturalistic Heart Rate Data. In Proceedings of the 6th HUMANIST Conference, The Hague, The Netherlands, 13–14 June 2018. [Google Scholar]
  74. Föll, S.; Maritsch, M.; Spinola, F.; Mishra, V.; Barata, F.; Kowatsch, T.; Fleisch, E.; Wortmann, F. FLIRT: A Feature Generation Toolkit for Wearable Data. Comput. Methods Programs Biomed. 2021, 212, 106461. [Google Scholar] [CrossRef] [PubMed]
  75. Bizzego, A.; Battisti, A.; Gabrieli, G.; Esposito, G.; Furlanello, C. Pyphysio: A Physiological Signal Processing Library for Data Science Approaches in Physiology. SoftwareX 2019, 10, 100287. [Google Scholar] [CrossRef]
  76. Guo, Z.; Ding, C.; Hu, X.; Rudin, C. A Supervised Machine Learning Semantic Segmentation Approach for Detecting Artifacts in Plethysmography Signals from Wearables. Physiol. Meas. 2021, 42, 125003. [Google Scholar] [CrossRef]
  77. Lehrer, P.M.; Gevirtz, R. Heart Rate Variability Biofeedback: How and Why Does It Work? Front. Psychol. 2014, 5, 756. [Google Scholar] [CrossRef]
  78. Giggins, O.M.; Persson, U.M.; Caulfield, B. Biofeedback in Rehabilitation. J. Neuroeng. Rehabil. 2013, 10, 60. [Google Scholar] [CrossRef]
  79. Harris, C.R.; Millman, K.J.; van der Walt, S.J.; Gommers, R.; Virtanen, P.; Cournapeau, D.; Wieser, E.; Taylor, J.; Berg, S.; Smith, N.J.; et al. Array Programming with NumPy. Nature 2020, 585, 357–362. [Google Scholar] [CrossRef]
  80. Tonacci, A.; Billeci, L.; Burrai, E.; Sansone, F.; Conte, R. Comparative Evaluation of the Autonomic Response to Cognitive and Sensory Stimulations through Wearable Sensors. Sensors 2019, 19, 4661. [Google Scholar] [CrossRef]
  81. Birkett, M.A. The Trier Social Stress Test Protocol for Inducing Psychological Stress. J. Vis. Exp. JoVE 2011, 56, e3238. [Google Scholar] [CrossRef]
  82. Johnson, K.T.; Narain, J.; Ferguson, C.; Picard, R.; Maes, P. The ECHOS Platform to Enhance Communication for Nonverbal Children with Autism: A Case Study. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI EA ’20, Honolulu, HI, USA, 25 April 2020; pp. 1–8. [Google Scholar] [CrossRef]
  83. Alfonso, C.; Garcia-Gonzalez, M.A.; Parrado, E.; Gil-Rojas, J.; Ramos-Castro, J.; Capdevila, L. Agreement between Two Photoplethysmography-Based Wearable Devices for Monitoring Heart Rate during Different Physical Activity Situations: A New Analysis Methodology. Sci. Rep. 2022, 12, 15448. [Google Scholar] [CrossRef] [PubMed]
  84. Zanon, M.; Kriara, L.; Lipsmeier, F.; Nobbs, D.; Chatham, C.; Hipp, J.; Lindemann, M. A Quality Metric for Heart Rate Variability from Photoplethysmogram Sensor Data. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; pp. 706–709. [Google Scholar] [CrossRef]
  85. Altman, D.G.; Bland, J.M. Measurement in Medicine: The Analysis of Method Comparison Studies. J. R. Stat. Society. Ser. D Stat. 1983, 32, 307–317. [Google Scholar] [CrossRef]
  86. Giavarina, D. Understanding Bland–Altman Analysis. Biochem. Medica 2015, 25, 141–151. [Google Scholar] [CrossRef] [PubMed]
  87. Georgiou, K.; Larentzakis, A.V.; Khamis, N.N.; Alsuhaibani, G.I.; Alaska, Y.A.; Giallafos, E.J. Can Wearable Devices Accurately Measure Heart Rate Variability? A Systematic Review. Folia Medica 2018, 60, 7–20. [Google Scholar] [CrossRef]
  88. Lund, A. Measuring Usability with the USE Questionnaire. Usability User Exp. Newsl. STC Usability SIG 2001, 8, 3–6. [Google Scholar]
  89. Gao, M.; Kortum, P.; Oswald, F. Psychometric Evaluation of the USE (Usefulness, Satisfaction, and Ease of Use) Questionnaire for Reliability and Validity. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2018, 62, 1414–1418. [Google Scholar] [CrossRef]
  90. Speer, K.E.; Semple, S.; Naumovski, N.; McKune, A.J. Measuring Heart Rate Variability Using Commercially Available Devices in Healthy Children: A Validity and Reliability Study. Eur. J. Investig. Heal. Psychol. Educ. 2020, 10, 390–404. [Google Scholar] [CrossRef]
  91. Cowley, B.; Filetti, M.; Lukander, K.; Torniainen, J.; Henelius, A.; Ahonen, L.; Barral, O.; Kosunen, I.; Valtonen, T.; Huotilainen, M.; et al. The Psychophysiology Primer: A Guide to Methods and a Broad Review with a Focus on Human–Computer Interaction. Found. Trends® Hum.–Comput. Interact. 2016, 9, 151–308. [Google Scholar] [CrossRef]
  92. van Lier, H.G.; Pieterse, M.E.; Garde, A.; Postel, M.G.; de Haan, H.A.; Vollenbroek-Hutten, M.M.R.; Schraagen, J.M.; Noordzij, M.L. A Standardized Validity Assessment Protocol for Physiological Signals from Wearable Technology: Methodological Underpinnings and an Application to the E4 Biosensor. Behav. Res. Methods 2020, 52, 607–629. [Google Scholar] [CrossRef]
  93. Shiri, S.; Feintuch, U.; Weiss, N.; Pustilnik, A.; Geffen, T.; Kay, B.; Meiner, Z.; Berger, I. A Virtual Reality System Combined with Biofeedback for Treating Pediatric Chronic Headache—A Pilot Study. Pain Med. 2013, 14, 621–627. [Google Scholar] [CrossRef]
  94. Reali, P.; Tacchino, G.; Rocco, G.; Cerutti, S.; Bianchi, A.M. Heart Rate Variability from Wearables: A Comparative Analysis Among Standard ECG, a Smart Shirt and a Wristband. Stud. Health Technol. Informatics 2019, 261, 128–133. [Google Scholar]
  95. Wang, C.; Pun, T.; Chanel, G. A Comparative Survey of Methods for Remote Heart Rate Detection From Frontal Face Videos. Front. Bioeng. Biotechnol. 2018, 6, 33. [Google Scholar] [CrossRef] [PubMed]
  96. Yu, Z.; Li, X.; Zhao, G. Facial-Video-Based Physiological Signal Measurement: Recent Advances and Affective Applications. IEEE Signal Process. Mag. 2021, 38, 50–58. [Google Scholar] [CrossRef]
  97. Joshi, J.; Bianchi-Berthouze, N.; Cho, Y. Self-Adversarial Multi-scale Contrastive Learning for Semantic Segmentation of Thermal Facial Images. arXiv 2022, arXiv:2209.10700. [Google Scholar] [CrossRef]
  98. Manullang, M.C.T.; Lin, Y.H.; Lai, S.J.; Chou, N.K. Implementation of Thermal Camera for Non-Contact Physiological Measurement: A Systematic Review. Sensors 2021, 21, 7777. [Google Scholar] [CrossRef]
  99. Gault, T.; Farag, A. A Fully Automatic Method to Extract the Heart Rate from Thermal Video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA, 25–27 June 2013; pp. 336–341. [Google Scholar]
  100. Gilgen-Ammann, R.; Schweizer, T.; Wyss, T. RR Interval Signal Quality of a Heart Rate Monitor and an ECG Holter at Rest and during Exercise. Eur. J. Appl. Physiol. 2019, 119, 1525–1532. [Google Scholar] [CrossRef]
  101. Cassirame, J.; Vanhaesebrouck, R.; Chevrolat, S.; Mourot, L. Accuracy of the Garmin 920 XT HRM to Perform HRV Analysis. Australas. Phys. Eng. Sci. Med. 2017, 40, 831–839. [Google Scholar] [CrossRef]
  102. Chow, H.W.; Yang, C.C. Accuracy of Optical Heart Rate Sensing Technology in Wearable Fitness Trackers for Young and Older Adults: Validation and Comparison Study. JMIR MHealth UHealth 2020, 8, e14707. [Google Scholar] [CrossRef]
  103. Müller, A.M.; Wang, N.X.; Yao, J.; Tan, C.S.; Low, I.C.C.; Lim, N.; Tan, J.; Tan, A.; Müller-Riemenschneider, F. Heart Rate Measures From Wrist-Worn Activity Trackers in a Laboratory and Free-Living Setting: Validation Study. JMIR MHealth UHealth 2019, 7, e14120. [Google Scholar] [CrossRef]
  104. Batista, D.; Plácido da Silva, H.; Fred, A.; Moreira, C.; Reis, M.; Ferreira, H.A. Benchmarking of the BITalino Biomedical Toolkit against an Established Gold Standard. Healthc. Technol. Lett. 2019, 6, 32–36. [Google Scholar] [CrossRef]
Figure 1. PhysioKit is a novel physiological computing toolkit which is open-source and cost-effective. Its sensing and signal acquisition layer provides flexibility in using low-cost physiological sensors, and the software application layer enables data acquisition, visualization, and real-time ML-based signal quality assessment, while supporting passive as well as interventional studies for single and multi-user settings.
Figure 1. PhysioKit is a novel physiological computing toolkit which is open-source and cost-effective. Its sensing and signal acquisition layer provides flexibility in using low-cost physiological sensors, and the software application layer enables data acquisition, visualization, and real-time ML-based signal quality assessment, while supporting passive as well as interventional studies for single and multi-user settings.
Sensors 23 08244 g001
Figure 2. Flexible PPG setups available in PhysioKit to enhance accessibility.
Figure 2. Flexible PPG setups available in PhysioKit to enhance accessibility.
Sensors 23 08244 g002
Figure 3. (A) User-interface for Physiological Computing Toolkit; (B) configuration file for specifying experiment protocol; and (C) configuration file for controlling data acquisition.
Figure 3. (A) User-interface for Physiological Computing Toolkit; (B) configuration file for specifying experiment protocol; and (C) configuration file for controlling data acquisition.
Sensors 23 08244 g003
Figure 4. SQA-Phys: 1D-CNN-based encoder–decoder architecture for high temporal precision signal quality assessment.
Figure 4. SQA-Phys: 1D-CNN-based encoder–decoder architecture for high temporal precision signal quality assessment.
Sensors 23 08244 g004
Figure 5. Multi-user setup of PhysioKit for synchronized data acquisition and stimulus presentation.
Figure 5. Multi-user setup of PhysioKit for synchronized data acquisition and stimulus presentation.
Sensors 23 08244 g005
Figure 6. Data collection protocol for technical evaluation of PhysioKit. Four conditions that comprise the protocol are (A) baseline, (B) easy math task, (C) difficult math task, and (D) guided head movement task.
Figure 6. Data collection protocol for technical evaluation of PhysioKit. Four conditions that comprise the protocol are (A) baseline, (B) easy math task, (C) difficult math task, and (D) guided head movement task.
Sensors 23 08244 g006
Figure 7. Bland−Altman scatter plots to compare heart−rate (beats per minute) (A,C), and pulse−rate variability (pNN50) (B,D) with the reference device.
Figure 7. Bland−Altman scatter plots to compare heart−rate (beats per minute) (A,C), and pulse−rate variability (pNN50) (B,D) with the reference device.
Sensors 23 08244 g007
Figure 8. (A): Comparison of pSQI for PPG signals of PhysioKit and the reference device, under different experimental conditions; (B): experimental condition-wise comparison of facial movement.
Figure 8. (A): Comparison of pSQI for PPG signals of PhysioKit and the reference device, under different experimental conditions; (B): experimental condition-wise comparison of facial movement.
Sensors 23 08244 g008
Table 1. PhysioKit: specifications for sensing and signal acquisition layer along with package description.
Table 1. PhysioKit: specifications for sensing and signal acquisition layer along with package description.
Sensing and Signal Acquisition Layer Specifications
ParameterSpecifications
Sampling rate10–10,000 samples per second
ADC10/12 bit
Vref3.3V/ 5V
Baudrate9600–2,000,000
Data transmission modeUSB/ Bluetooth
No. of channels1—Max supported by specific Arduino board
Installation and package contents
Installation of the interfacepip install PhysioKit2
GitHub repositoryRepo link https://github.com/PhysiologicAILab/PhysioKit (accessed on 28 September 2023)
Configuration filesDownload path https://github.com/PhysiologicAILab/PhysioKit/tree/main/configs
 (accessed on 28 September 2023)
Codes to program ArduinoDownload path https://github.com/PhysiologicAILab/PhysioKit/tree/main/arduino
 (accessed on 28 September 2023)
3D-printable model for a wristband caseDownload path https://github.com/PhysiologicAILab/PhysioKit/tree/main/CAD_Models
 (accessed on 28 September 2023)
Table 2. Evaluation of PhysioKit: comparison of heart-rate (beats per minute) and pulse-rate variability (pNN50) with the reference device. Evaluation metrics include root mean-squared error (RMSE), mean absolute error (MAE), standard deviation of error (SD) and Pearson correlation coefficient (r), which are mentioned separately for each experimental condition as well as for all conditions combined.
Table 2. Evaluation of PhysioKit: comparison of heart-rate (beats per minute) and pulse-rate variability (pNN50) with the reference device. Evaluation metrics include root mean-squared error (RMSE), mean absolute error (MAE), standard deviation of error (SD) and Pearson correlation coefficient (r), which are mentioned separately for each experimental condition as well as for all conditions combined.
MetricsPPG SiteExperimental
Condition
RMSEMAESDPearson (r)
Heart
Rate
(beats
per minute)
FingerBaseline3.632.433.380.96
Math—Easy2.081.442.020.98
Math—Difficult2.411.752.410.98
Face Movement2.171.492.090.98
All sessions
combined
2.651.782.650.97
EarBaseline3.712.463.410.95
Math—Easy1.00.730.921.0
Math—Difficult2.081.462.070.98
Face Movement2.111.402.050.98
All sessions
combined
2.451.532.430.97
Pulse
Rate Variability
(pNN50)
FingerBaseline4.313.464.280.98
Math—Easy11.336.5410.60.91
Math—Difficult12.119.5312.090.75
Face Movement8.265.668.250.92
All sessions
combined
9.46.269.340.89
EarBaseline4.363.784.210.99
Math—Easy9.855.759.270.91
Math—Difficult15.2611.1315.120.64
Face Movement9.387.279.320.88
All sessions
combined
10.427.0210.260.87
Table 3. Project teams specific application cases of PhysioKit.
Table 3. Project teams specific application cases of PhysioKit.
ApplicationApplication
Type
No. of
Members
Project
Duration
Using physiological reactions
as emotional responses to music
Interventional64 weeks
Emotion recognition during
watching of videos
Interventional64 weeks
Using artistic biofeedback
to encourage mindfulness
Interventional16 weeks
Using acute stress response to
determine game difficulty
Interventional74 weeks
Generating a dataset for an affective
music recommendation system
Passive74 weeks
Adapting an endless runner
game to player stress levels
Interventional8
7
4 weeks
Influencing presentation experience
with social biofeedback
Interventional16 weeks
Mapping stress in virtual realityPassive16 weeks
Assessing synchronous heartbeats
during a virtual reality game
Passive26 weeks
See Section 2.1.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Joshi, J.; Wang, K.; Cho, Y. PhysioKit: An Open-Source, Low-Cost Physiological Computing Toolkit for Single- and Multi-User Studies. Sensors 2023, 23, 8244. https://doi.org/10.3390/s23198244

AMA Style

Joshi J, Wang K, Cho Y. PhysioKit: An Open-Source, Low-Cost Physiological Computing Toolkit for Single- and Multi-User Studies. Sensors. 2023; 23(19):8244. https://doi.org/10.3390/s23198244

Chicago/Turabian Style

Joshi, Jitesh, Katherine Wang, and Youngjun Cho. 2023. "PhysioKit: An Open-Source, Low-Cost Physiological Computing Toolkit for Single- and Multi-User Studies" Sensors 23, no. 19: 8244. https://doi.org/10.3390/s23198244

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop