Next Article in Journal
Achievement of Accurate Robotic Arm-based Bike Frame Quality Check Using 3D Geometry Mathematical Model
Previous Article in Journal
Total Least-Squares Iterative Closest Point Algorithm Based on Lie Algebra
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 9
Issue 24
10.3390/app9245353
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessing Drivers’ Physiological Responses Using Consumer Grade Devices

by
Timotej Gruden
*,
Kristina Stojmenova
,
Jaka Sodnik
and
Grega Jakus
University of Ljubljana, Faculty of Electrical Engineering, Tržaška Cesta 25, 1000 Ljubljana, Slovenia
*
Author to whom correspondence should be addressed.
Appl. Sci. 2019, 9(24), 5353; https://doi.org/10.3390/app9245353
Submission received: 30 October 2019 / Revised: 22 November 2019 / Accepted: 4 December 2019 / Published: 7 December 2019
(This article belongs to the Section Electrical, Electronics and Communications Engineering)
Browse Figures
Review Reports Versions Notes

Abstract

:
The ability to measure drivers’ physiological responses is important for understanding their state and behavior under different driving conditions. Such measurements can be used in the development of novel user interfaces, driver profiling, advanced driver assistance systems, etc. In this paper, we present a user study in which we performed an evaluation of two commercially available wearable devices for assessment of drivers’ physiological signals. Empatica’s E4 wristband measures blood volume pulse (BVP), inter-beat interval (IBI), galvanic skin response (GSR), temperature, and acceleration. Bittium’s Faros 360 is an electrocardiographic (ECG) device that can record up to 3-channel ECG signals. The aim of this study was to explore the use of such devices in a dynamic driving environment and their ability to differentiate between different levels of driving demand. Twenty-two participants (eight female, 14 male) aged between 18 and 45 years old participated in the study. The experiment compared three phases: Baseline (no driving), easy driving scenario, and demanding driving scenario. Mean and median heart rate variability (HRV), standard deviation of R–R intervals (SDNN), HRV variables for shorter time frames (standard deviation of the average R–R intervals over a shorter period—SDANN and mean value of the standard deviations calculated over a shorter period—SDNN index), HRV variables based on successive differences (root mean square of successive differences—RMSSD and percentage of successive differences, greater than 50 ms—pNN50), skin temperature, and GSR were observed in each phase. The results showed that motion artefacts due to driving affect the GSR recordings, which may limit the use of wrist-based wearable devices in a driving environment. In this case, due to the limitations of the photoplethysmography (PPG) sensor, E4 only showed differences between non-driving and driving phases but could not differentiate between different levels of driving demand. On the other hand, the results obtained from the ECG signals from Faros 360 showed statistically significant differences also between the two levels of driving demand.

1. Introduction

The human factor is still one of the leading causes for road traffic accidents. It has been reported that more years of life were lost due to traffic accidents than due to most human diseases [1]. Driving is a demanding process, mostly relying on the driver’s visual and manual senses, and largely to their auditory and cognitive capabilities. Vehicle manufactures have therefore been constantly working on reducing the driver’s role and improving the driving experience by adding a number of advanced driver-assistance systems (ADAS) and including user-friendly in-vehicle information systems (IVIS). Although the driver’s role is changing, people spend more time in a vehicle compared to ever before, thus monitoring the driver’s state is somewhat necessary for further research in current and higher levels of vehicle autonomy.
The measurements of drivers’ physiological signals can be very useful in the development of automated vehicles that tend to imitate human driving, since driving algorithms can be improved with the results of such measurements. Not only vehicles, driving simulators can also use such algorithms to make the scenarios (other vehicles in the scenarios) more realistic. Determining different drivers’ state and behavior from the measured signals is also beneficial to insurance companies for driver profiling and cost calculation. Among other factors, stress levels, cognitive demand, attention, arousal, and fatigue indicators have been used for evaluation of usability of new ADAS and IVIS systems. These indicators are mainly derived from drivers’ physiological signals that represent cardiovascular, electrodermal, pupil, or brain activity.

1.1. Physiological Responses for Assessment of Driver State

Among all studies that measure physiological signals of drivers, driver drowsiness and fatigue are the most often researched topics. Chronologically, the first studies on detection of physiological signals for drowsiness detection were using video analysis with image processing techniques [2,3]. Now, modern research often uses electroencephalography (EEG) [4,5,6,7] or a combination of EEG with other physiological signals [8,9]. Another common measure for detecting drowsiness besides EEG, is cardiovascular activity. Drowsiness has been assessed with different cardiovascular signals including photoplethysmography (PPG) [9,10,11,12], blood volume pulse (BVP) [11], and electrocardiogram (ECG) [8,12,13,14]. Some studies report on using heart rate variability (HRV) [9,12,13], electrooculography (EOG) [8,15,16], and galvanic skin response (GSR) [9,11].
Another often studied topic regarding the driver’s physiological signals is monitoring of individual driver’s state such as situational awareness [17], emotional states [18,19,20], or driver physical fitness [21,22,23]. Similar methods have also been used for observing effects of use of different modalities when interacting with in-vehicle information systems [24] and to assess drivers’ cognitive workload due to use of such systems [25,26] or experienced stress due to demanding driving conditions [27]. By observing EEG signals, physiological data can also be used for prediction of drivers’ intentions [28,29].
Difficult driving conditions, e.g., interaction with reckless drivers and bikers, impatient pedestrians crossing the road, and operating the vehicle, e.g., gear shifting, can significantly increase the load on drivers [30]. Different levels of driving demand reflect in drivers’ mental and also physical load. What makes the situation even worse is that the driving demand is increasing every day [31]. The driver’s mental workload is usually assessed by subjective reports, measures of task performance, or physiological measures [32]. As Brookhius and de Waard suggest, physical and mental workload have clear impact on physiological signals [31], especially on heart rate, heart rate variability, galvanic skin response, and blood pressure [33].
In summary, the list of common physiological measures for assessing drivers consists of:
  • Electroencephalography (EEG),
  • Electrocardiography (ECG),
  • Photoplethysmography (PPG),
  • Heart rate (HR),
  • Galvanic skin response (GSR),
  • Electromyography (EMG) and
  • Eye tracking (pupil diameter—PD).
Combinations often include EEG and ECG or PPG and GSR, sometimes in combination with eye tracking.

1.2. Devices Used to Measure Driver’s Physiological Responses

There is a great variety of available devices for capturing physiological signals. Most often higher reliability increases the price of the device, and thus reduces the availability. Devices used for signal capturing vary from study to study. Some even make their own custom capturing devices. Among the presented studies, Biopac MP-150 was often used as an overall acquisition system [7,20,27], also FlexComp Infinity system [8,34], Nexus-10 from Mind Media Schepersweg [11], Medac Systems/3 [25], and Geodesic EEG System 300 [35] were mentioned. EEG was measured with NicoletOne Ambulatory EEG [6], EMG was measured with Bagnoli-8™ [23], eye movement with FaceLAB® [25]. Bittium’s Faros was also used in many studies [36,37,38]. Based on the available research it can be concluded that physiological signals can give important information about the human’s state and prediction of human behavior or medical state. It is also noticeable that with the introduction of wearable devices, the research studies using such signals are no longer limited to medical and laboratory environments, but can be used also in dynamic environments such as vehicles. This can however bring “noise” (e.g., motion artefacts, quantization noise, electro-magnetic interferences from other devices, etc.) to the measured signals and therefore, it is important to be aware of this limitation when choosing a measuring device and later when performing the analysis. Therefore, we assessed suitability, accuracy, and robustness of two commonly used commercially available devices in a simulated driving environment.
The E4 wristband can measure BVP, inter-beat interval (IBI), heart rate (HR), electrodermal activity (EDA) or GSR, skin temperature (ST), and motion with an accelerometer. A detailed presentation of the device can be found in the next section.
In many studies, the raw data from the E4 device was used for assessing stress levels [39,40,41,42,43,44]. Sevil et al. used all of the available E4 output signals (GSR, ST, HR, BVP, and motion) to detect various types of acute stress such as social, completion, emotional and mental stress, and showed that using their algorithm, they can detect stress with 87% accuracy [39]. Park et al. on the other hand tried to estimate the relationship between EDA signals and increased cognitive workload using the E4 [45]. Their results showed that there is a somewhat linear relationship between these data. Furthermore, it has also been used in studies of affective computing in general [40,46], in medical researches for heart arrhythmia and atrial fibrillation detection [47,48,49]. Studies report also on using E4 data for assessing overall functional health of people with dementia [50] and creating of personal assistants for forgetful people [51]. Vandecasteele et al. even tried to use the HR data for seizure detection; however, their results showed that detection performance was considerably lower to the usually used hospital and wearable Faros device [52]. Additionally, there seems to be a growing trend of using E4 data in biofeedback applications [53,54,55].
When compared to other laboratory (stationary and wearable) devices, E4 showed comparable results. Ollander et al. compared the E4 device with stationary sensors for ECG and GSR [41]. Their results showed incomplete IBI data, however mean values of HR and standard deviation of HR showed good stress discrimination power. For the GSR, their results showed that due to the placement, data from the E4 yielded higher stress discrimination power than the signal measured with the stationary sensors placed at the fingers. When compared for emotion recognition to higher grade laboratory equipment—Biopac MP150, it was found that E4 can provide comparable results [40,56]. However, based on currently available data, no research was found on E4 being used in a driving environment.
Faros is a family of ECG devices by Bittium. Faros 360 is able to measure 3-channel ECG, body temperature, and accelerations. It is presented in detail along with different models in the next section.
Different versions of Faros are widely used in research studies for detection of various medical conditions, which affect humans’ cardiovascular activity [57,58,59,60].
Since Faros 360 is certified as a medical device, it is also often used as a reference for calibration or validation of new wearable devices [61,62,63]. However, Faros has been used also for monitoring different profiles of users’ states. For example, it has been used for the assessment of mental stress when interacting with robots, where, based on HRV data, it was found that human operators show higher engagement levels when being successful at completing a task [64]. It has also been used for monitoring effects of air pollution and noise due to traffic, and the results showed that HRV and blood pressure (BP) drop when noise increases [65].
Other studies have shown that the ECG signal from Faros can also be used to calculate other variables such as BP [66], or to extract core body temperature with respect to clothing and persons’ activity [67].
Since with wearable devices such as Faros, the experimental conditions differ and are not always performed in a controlled laboratory environment, the gathered data can be affected by motion artefacts. Alikhani et al. found that motion can describe the high frequencies in HRV up to 40% [68]. However, the noise of motion can be eliminated by adding an additional accelerometer on each electrode [69]. With this, it is possible to eliminate the baseline wandering without affecting the ECG signal.
Contrary to E4, Faros has already been used in a driving environment. Biondi et al. used HRV data gathered with Faros to observe the effects of semi-automated driving in Tesla model S [36]. The effects of different levels of automation on drivers’ state were studied also in a driving simulator environment by Radlmayr, et al. [38] and Murase et al. [37]. Their findings were however different, as Radlmayr et al. report on a decreased level of arousal and sympathetic nervous system activity when switching from automation level 2 to level 3, while Murase et al. did not find any effects on the ECG channel for different levels of autonomous driving [37].

1.3. Contribution

This research study explores two new and commercially available devices, which have shown promising results for assessing psychophysiological data in various medical research. The goal was to evaluate the two devices and their usefulness also in a driving environment, which is rather specific, and due to its dynamic component, differs from other experimental environments used in reported studies. Our motivation on evaluating these particular devices was the fact that they are wearable and nonintrusive, and therefore much easier to use in a simulated driving study and also in a real environment. We try to answer to the following research questions:
  • Can E4 and Faros be used for assessment of physiological signals in a driving environment?
  • If so, can E4 and Faros differentiate between different levels of driving demand?
The used methodology and experimental environment are presented in the Materials and Methods section and our findings are reported in Results. The possible applications and found limitations of both devices in a driving environment are discussed in the last section: Discussion and Conclusions.

2. Materials and Methods

The presented study had a within-subject (repeated measures) design and was conducted in a simulated driving environment. The drivers’ responses were assessed using two commercially available devices—E4 by Empatica [70] and Faros 360 by Bittium [71], whereas the driving measurements were recorded using a Nervtech driving simulator [72].

2.1. Empatica E4

E4 is a wearable device in the form of a wristband, equipped with a number of sensors for assessment of electrodermal and cardiovascular activity (Figure 1). The E4 wrist band can measure BVP, inter-beat interval (IBI), heart rate (HR), electrodermal activity (EDA) or GSR, skin temperature (ST), and motion with an accelerometer.
One of the main features of the E4 device is a photoplethysmography (PPG) sensor. Photoplethysmography sensors use an optical technique to detect changes in blood volume in the microvascular bed of tissue [73]. The PPG sensor in E4 consists of two red and two green LEDs, which provide two different wavelength light sources, and two sensors for detection of the reflected light, with a sampling frequency fs = 64 Hz. The output is a blood volume pulse (BVP) signal with an 8-bit resolution. Based on these measurements, a proprietary algorithm from Empatica detects the heart beats (peaks) in the signal and calculates the intervals between separate beats (inter-beat intervals (IBI)) in milliseconds. Unlike the BVP signal, which has a fixed sampling frequency, the IBI signal is provided when a beat is detected. However, average heart rate (HR) signal value is provided every second, as it is calculated from the last known IBI sample.
E4 is equipped also with an electrodermal activity (EDA) or galvanic skin response (GSR) sensor. With two 8-mm-wide silver-plated copper electrodes, located at the end of the wristband belt (see Figure 1), E4 captures skin conductance with a sampling frequency fs = 4 Hz.
Additionally, E4 also features an optical infrared temperature sensor that measures skin temperature assessed at the wrist, with a sampling frequency fs = 4 Hz.
E4 is equipped also with a 3-axis accelerometer, which allows monitoring of hand activity (movement). On every E4 device, the X-axis is defined as a vector from the center of device towards the USB connector, Y-axis is defined as a vector from the center of the device towards the shorter end of the wrist belt, and Z-axis is defined as a vector from the center of the device towards the bottom of the device. The dynamic range of the accelerometer is ±2 g, the output is an 8-bit signed integer with the resolution of 1/64 g. The accelerometer’s sampling frequency (fs) is 32 Hz.
With the E4 device, data can be captured in two ways: In a recording mode (retrieve after measurement) and in a streaming mode (retrieved in real-time). E4 has an internal storage of up to 60 h of measurements. It comes prepacked with a special USB dock station and proprietary software for downloading data from the device and uploading it to a cloud platform called Empatica connect [74]. When using the recording mode, raw data can only be downloaded from this cloud platform. In the streaming mode, E4 uses Bluetooth Low Energy (BLE) to transfer data in real-time.
Besides the official software solutions that come with the device, Empatica offers also iOS, Android and Windows development kits for researchers that want to customize and upgrade this solution to their needs. The development kits allow real-time data acquisition with custom software that gives the ability of real-time processing, feature extraction and on-demand visualization.

2.2. Bittium Faros 360

Faros by Bittium is a small electrocardiogram (ECG) measurement device. It comes in three versions:
  • Faros 90, which offers simple 1-channel ECG measurements,
  • Faros 180, which offers 1-channel ECG measurements and is able to stream data via Bluetooth, and
  • Faros 360, which enables 3-channels ECG measurements and is able to stream data via Bluetooth.
Each version can be mounted to the participant’s body in three ways: Fast-Fix, using a cable set, or using a stingray adapter. Fast-Fix, which is a Bittium’s proprietary electrode, is designed for quick mounting and is considered as the simplest and most convenient way of using the device. Faros can be mounted also by using cable sets (see Figure 2 left) with up to five electrodes for 3-channel measurements (or three electrodes for 1-channel). This version is considered as the most accurate and reliable version of Faros for measuring ECG because it enables best skin contact during the measurement process. The third mounting option does not require placing any electrodes on the participant’s body. It instead uses an elastic textile belt with two electrodes and a mounting pad for Faros (see Figure 2 right).
The most important feature of Faros is its ECG sensor. The sampling frequency of the provided raw ECG data can be fs1 = 125 Hz, fs2 = 250 Hz, fs3 = 500 Hz, or fs4 = 1000 Hz depending on the user’s needs. From the ECG signal, Faros automatically detects independent heart beats as points R in a PQRST model of the ECG signal, where P, Q, R, S, and T represent specific waves in the signal as presented in Figure 3. It also derives heart rate variability (HRV), sometimes also referred to as R–R or N–N interval, from the time intervals between the consecutive R points, and presents it as a single output in addition to raw ECG.
Faros is equipped also with a temperature sensor. According to the manufacturer’s specifications [75], its characteristic is linear only between 35 and 45 degrees Celsius when the sampling frequency is fs = 5 Hz.
Faros features also an accelerometer with two possible sampling frequencies: fs1 = 25 Hz and fs2 = 100 Hz. Its dynamic range can be set to ±2 g, ±4 g or ±16 g.
Faros 360 can operate in one of two modes: Datalogger mode and online mode. To use the datalogger mode, Faros has to be connected to a computer, and the appropriate measurement settings have to be set using the Faros Manager software [71]. Raw data is stored on Faros’ internal storage in a European data format (EDF) file. It can be viewed using the provided EDF viewer or can easily be converted to acceleration (ASC) or heart rate variability (SDF) file format. In online mode, Faros periodically (five times in a second) sends data via Bluetooth. After Faros is paired with a computer (or another Bluetooth device), the computer sees Faros as an external serial device accessible over the serial communication (COM) port. The protocol itself and data packet specifications are provided in Bittium’s document 810378 [75].

2.3. Nervtech Driving Simulator Overview

To ensure a controllable and repetitive driving environment, the study was performed in a driving simulator. We used Nervtech’s high fidelity driving simulator [72], using a triple screen set-up (Figure 4) with adjustable car seat, Fanatec ClubSport Wheel Base V2 with dynamic feedback, Fanatech ClubSport Pedals V3 with dynamic feedback, and Fanatec gear box. Total resolution of the three Samsung Curved 48′ TVs is 5760 × 1080 px, using Nvidia GeForce GTX 1070 Ti graphic card. The on-screen cockpit included speedometer and fuel gauge. Motion cues are provided by slightly moving the cockpit up and down. The simulator uses OKTAL’s Scanner Studio, which is managed by a supervising platform OktalControl by Nervtech. Based on the definition of high physical and functional fidelity as defined by Kinkade and Wheaton [76] and Hays [77], the simulator used in this study qualifies as a high-fidelity driving simulator.

2.4. Technical Setup

E4 was placed on the participant’s left hand (Figure 5). For this study, we used streaming mode to store data captured with E4 using Windows Streaming Server, which requires a specific BLE USB dongle to connect to the E4 wristband [78]. Once the Windows Streaming Server and E4 are connected, users or software can connect to the streaming server via the TCP socket. For this study, we created a custom software that connects to the streaming server and stores real-time data for further analysis.
From the three available versions for Faros, we used Faros 360, which enables 3-channel ECG measurement and is able to stream data via Bluetooth. For convenience of use, the stingray adapter was used for mounting the device on the participants (Figure 5). Following the recommendations by Laborde et al. we used the sampling frequency fs = 500 Hz [79].

2.5. User Study

2.5.1. Participants

Since the distribution of different factors that could affect the measurement in different ways is usually unknown, the American Association of Psychologists recommends using a slightly corrected Cohen’s effect sizes. This study was aimed to detect large effect sizes with 80% statistical power. Study [81] made some calculations regarding statistical considerations and it turned out that for statistical power of 80% at α = 0.05 in order to detect large effect sizes, a sample size of at least 21 participants is required.
Therefore, 22 drivers, aged from 18 to 45 participated in this study, eight of them (36%) were female. There were no drivers with any kind of heart diseases or other relevant health problems. A requirement for every driver to participate was a valid driving license.
All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the head of the Laboratory of Information Technologies, Faculty of Electrical Engineering, University of Ljubljana.

2.5.2. Tasks

This study had a within-subject design. It consisted of three phases: Baseline, easy driving, and demanding driving. The duration of each phase was 6–8 min to ensure at least a 5-min-long window of data, as recommended for short-term recordings of physiological signals by the European Society of Cardiology [79,82].

Phase 1: Baseline

Physiological signals such as HR and HRV differ among people therefore, we first had to collect baseline data for each participant. Since there is not a standardized procedure for baseline measurements, we followed the recommendations by Laborde et al. [79]. To create similar environment with the following driving tasks, the baseline measurements were also collected in a sitting position. The test drivers were told to sit in a chair with their legs straight (not crossed), keeping their knee angles at 90 degrees. They were asked to try to relax and stay calm, to lean their head back, keep their eyes closed, and to ensure normal blood flow to the wrists, place their arms on the supporters with hands facing up (Figure 6a).
The 5-min measurement began with a short beep from Faros that indicated a successful initialization of measurement system. During this period, all light and noise sources were turned off and remained off until the end of the study process in order to reduce potential distractions that could affect the measurement process.

Phase 2: Easy Driving

Phase 2 and 3 were performed in a driving simulator. The participants’ task was safe driving in a very simple driving scenario (Figure 6b). The participants were asked to drive on a two-way one-lane six-kilometer country road with no other vehicles or pedestrians. The speed limit was 90 km/h, and participants were not asked to follow any pre-defined path.

Phase 3: Demanding Driving

In the third phase, the participants’ primary task remained safe driving. However, in this task the participants had to follow a specific route communicated with a navigation system displayed on a built-in head-up display. In this phase, the driving scenario was much more demanding as it featured six kilometers of two-lane highway and two kilometers of city road with a lot of vehicles, intersections and pedestrians.
While on the highway (Figure 6c), the participants faced many critical situations, including a truck driving in the opposite direction, work on the road, unsafe overtaking by another vehicle, and traffic accident while driving in fog. The driver experienced critical situations also on the city road such as pedestrians crossing the road outside crosswalks, a bike forcing the driver’s right of way, a child on the road running after a ball, etc.

2.5.3. Variables

All output signals from both Faros 360 and E4 were recorded and processed. Comparison between the two devices could be made with HRV and temperature signals. Since only Faros provides raw ECG signal, the evaluation of other ECG features than the ones already extracted by the devices (HRV) is not feasible. The difference with HRV is only in signal capturing methods which could provide significantly different results. Faros measures ECG signal to obtain R–R intervals (HRV) while E4 uses photoplethysmography (PPG) to obtain BVP and IBI. Two temperature readings could also provide significantly different results since measures take part at different body parts with different thermal conductance. In addition to the comparison, GSR signal from E4 was also analyzed to provide the full picture of E4’s usability.
Independent variable in our tests was each phase’s driving difficulty that could take one of three values: 1 = baseline establishment, 2 = easy driving, 3 = demanding driving.
Dependent variables for both Faros and E4 were:
  • Mean and median HRV,
  • HRV SDNN—standard deviation of the R–R intervals (also known as normal-to-normal or N–N intervals, Figure 3), which reflects cyclic components during the measurement,
  • HRV SDANN—standard deviation of the average R–R intervals over a shorter period (10 s), it reflects changes due to longer cycles,
  • HRV SDNN index—mean of the standard deviations, calculated over a shorter period (10 s), it reflects changes due to shorter cycles,
  • HRV RMSSD—root mean square of successive differences, which reflects parasympathetic nerve system activity and is not affected by respiration process,
  • HRV pNN50—the number of successive differences, greater than 50 ms, derived by the number of total N–N differences, should be highly correlated to RMSSD,
  • Temperature mean and standard deviation.
Mean and standard deviation of recorded GSR signals were also analyzed as dependent variables from E4.

2.5.4. Procedure

The study began with a demographic questionnaire and a consent form for participants’ personal data processing. After that, Empatica E4 and Bittium Faros 360 were disinfected, turned on, and placed on participants’ wrist and chest respectively. Exact position of both devices is shown in Figure 5.
In [82], Task Force of the European Society of Cardiology set standards for ECG and HRV measurements. According to them, measurements of short events should take at least 5 min. Therefore, the baseline and both tasks lasted at least 5 min and to make the sample size equal, we only observed the last 5 min. Phase three was usually longer than 7 min (it depended on participants’ driving), so we only cut samples from the first half-minute to avoid having artefacts from initial driver movements.
Before the driving, each participant was introduced to and shown how to use the simulator. Participants were asked to follow the traffic rules and try to drive as close as possible as they would in real life.
After completing all three phases, the drivers were asked to subjectively assess their feelings regarding simulation sickness. The values are equally distributed on a 10-point scale, where 1 indicated “I am perfectly fine”, and 10 meant “I have to throw up”.

2.5.5. Statistical Analysis

The statistical analysis included Shapiro-Wilk normality test and Maunchly’s test of sphericity for every dependent variable. Where the distribution of the data was found normal, repeated measures ANOVA (RMANOVA) and (when the null hypothesis was rejected) Bonferroni post-hoc test. For normally distributed data with violated assumption of sphericity, Greenhouse-Geisser correction was applied.
For variables with non-normally distributed data we used Friedman’s non-parametric test and (when the null hypothesis was rejected) Dunn’s post-hoc test with Bonferroni correction.

3. Results

The results of HRV measurements are presented in four groups: Mean, standard deviation, shorter time-frame variables and successive differences, followed by skin temperature and GSR measurements. A summary of the results is provided in Table 1.
It is important to mention that E4’s built-in algorithm for calculating the HRV did often not provide any output. This resulted in missing HRV samples from E4 at easy driving (58%) and demanding driving (59%). It is important to notice that the missing values may have influenced the results of HRV successive differences, therefore they should be interpreted with caution. On the other hand, Faros provided all HRV samples during every measurement phase.

3.1. Mean Heart Rate Variability (HRV)

RMANOVA test for mean values of HRV (Figure 7) did not reveal any statistically significant differences among the data collected during the three phases for E4 (F(1.166, 22.148) = 0.308, p = 0.619) or Faros (F(1.331, 27.941) = 2.330, p = 0.131). The same test did not reveal any statistically significant differences also in median HRV among the observed phases for E4 (F(1.219, 23.154) = 0.366, p = 0.593) or Faros (F(1.323, 27.774) = 2.332, p = 0.131).

3.2. HRV SDNN

RMANOVA test for SDNN (Figure 8) showed statistically significant differences among the data from different phases for E4 (F(2, 38) = 10.096, p < 0.001). Bonferroni post-hoc test revealed that the HRV SDNN increased statistically significantly when comparing the baseline and easy driving (p = 0.002) and between baseline and demanding driving (p = 0.010). No statistically significant differences were found between the easy driving and demanding driving phase (p = 1.000).
RMANOVA test results for data captured with Faros 360 also showed that there are some statistically significant differences among the data from different phases (F(2, 42) = 6.967, p = 0.002). Bonferroni post-hoc test showed that the HRV SDNN increased statistically significantly when comparing the baseline and demanding driving (p = 0.010) and between easy driving and demanding driving (p = 0.027). The test did not show any statistically significant differences between the baseline and easy driving phase (p = 0.794).

3.3. HRV Variables for Shorter Timeframes

For further analysis, the data was divided into 10-s timeframes. The analysis showed statistically significant differences among the data from the observed phases for SDANN (Figure 9) but did not reveal any differences in the SDNN Index, for data from the both devices (Table 2).
When observing the SDANN data measured with E4, Bonferroni post-hoc test showed that it increased statistically significantly from the baseline compared to the easy driving phase (p < 0.001) and between the baseline and demanding driving phase (p < 0.001); no statistically significant differences were found between the easy driving and demanding driving phase (p = 1.000).
When observing the SDANN data measured with Faros 360, Bonferroni post-hoc test showed that it increased statistically significantly from the baseline compared to the demanding driving phase (p < 0.001) and between the easy driving and the demanding driving phase (p = 0.016). The test showed no statistically significant differences between baseline and easy driving (p = 0.086).

3.4. HRV Successive Differences

Statistically significant differences in the data from E4 data were found among the phases in both RMSSD (F(2, 38) = 44.822, p < 0.001) and pNN50 (F(1.411, 26.816) = 57.508, p < 0.001) (Figure 10 and Figure 11). Bonferroni post-hoc test showed that the values increased statistically significantly between the baseline and the easy driving phase (p < 0.001) and between the baseline and the demanding driving phase (p < 0.001), for both variables, RMSSD and pNN50. The test showed no statistically significant differences between easy driving and demanding driving (p = 0.964 (RMSSD), p = 0.530 (pNN50)).
In the data, captured with Faros 360, Friedman’s non-parametric tests did not reveal any statistically significant differences in RMSSD (χ2(2) = 3.545, p = 0.170) or pNN50 (χ2(2) = 0.364, p = 0.834).

3.5. Skin Temperature

During the experiment, the temperature was rising at what appears to be an exponential curve (see Figure 12).
Statistically significant differences among phases (Table 3) were found in mean and standard deviation of the skin temperature readings (Figure 13, Figure 14) from the both devices.
Bonferroni post-hoc test for mean skin temperature measured with E4 showed statistically significant increase between the baseline and the easy driving phase (p < 0.001) and between the baseline and the demanding driving phase (p < 0.001). No statistically significant differences were found between the easy and the demanding driving phase (p = 0.770).
Bonferroni post-hoc test for mean skin temperature measure with Faros 360 showed that it increased statistically significantly for each consecutive phase (p < 0.001).
Dunn’s post-hoc test with Bonferroni correction for standard deviation of the skin temperature measured with E4 showed it increased statistically significantly between the baseline and the easy driving phase (p = 0.020) and between the baseline and the demanding driving phase (p = 0.020). The test did not show any statistically significant differences between the easy driving and the demanding driving phase (p = 1.000).
Similarly, when analyzing the standard deviation of skin temperature measured with Faros 360, Bonferroni post-hoc test showed it increased statistically significantly from the baseline compared to the easy driving phase (p < 0.001), and from the baseline and compared to the demanding driving phase (p < 0.001). The test did not show any statistically significant differences between the easy driving and the demanding driving phase (p = 0.473).

3.6. E4’s GSR

Non-parametric Friedman test revealed that statistically significant differences among the data from the observed phases for mean GSR (χ2(2) = 8.273, p = 0.016) and standard deviation of GSR (χ2(2) = 6.909, p = 0.032) (Figure 15). In both cases, Dunn’s post-hoc test with Bonferroni correction revealed significant increase in the values from the baseline compared to the demanding driving phase (mean GSR: p = 0.013, standard deviation of GSR: p = 0.048), however there were no significant differences in these values when comparing the baseline and the easy driving, and the easy driving and the demanding driving phase (p > 0.1).

3.7. Sickness

The majority of drivers (86%) rated their sickness level with a score of equal to or less than 4 (out of 10). Only three participants reported higher sickness levels, with scores of 6, 8 and 9 (Figure 16).

4. Discussion and Conclusions

In this paper, we compared two commonly used wearable devices for measuring driver’s physiological signals—Empatica E4 and Bittium Faros 360. We performed a user study with which we wanted to evaluate the use of these devices in a dynamic driving environment. Additionally, we explored if they can be used for differentiating between different levels of driving difficulty.

4.1. Photoplethysmography (PPG) Limitations

E4 uses PPG, which is according to the literature less accurate than ECG, used by Faros [79]. PPG can provide somewhat different (delayed, malformed, etc.) results than ECG since the blood vessels are elastic and the pulse transit times may vary. Furthermore, it is considered that PPG can provide a valid representation of IBI during rest, but not during activity [83]. This suggests additional caution when assessing ECG data with wearable devices that use PPG and not ECG sensors.
When examining the raw data from E4, multiple inter-beat interval (IBI) samples were incomplete for most drivers in both of the driving phases; furthermore, the collected data were completely missing for two drivers. However, the IBI samples recordings were complete for the baseline phase, suggesting the set-up was performed correctly.
There are two potential reasons for incomplete IBI samples when recording with E4. As the missing samples occurred mainly in the dynamic driving environment, we can assume motion artefacts caused by steering negatively influence the E4. Furthermore, using the accelerometer and consequentially its internal algorithm for obtaining IBI from blood volume pulse (BVP), the device might detect intensive motion activity, which may have invalidated some of the IBI samples. Since there is an entire sequence of missing samples recorded, it is impossible to determine the differences between successive samples. Therefore, analysis of variables based on successive differences of such samples (i.e., RMSSD and pNN50) cannot be used for making reliable conclusions.

4.2. HRV Analysis

The results of HRV analysis based on mean and median tests did not show any statistically significant differences among the tested phases. However, the SDNN and SDANN signals showed that it is possible to differentiate between driving and non-driving situations, and to differentiate between different levels of driving demand.
When comparing the standard deviation (SDNN) of HRV, the analysis showed that E4 can differentiate only between non-driving and driving, but it cannot differentiate between different levels of driving demand, where drivers had to move their hands in order to steer the wheel. On the other hand, the analysis showed that with the signals recorded with Faros 360 it was possible to differentiate between the two different levels of driving difficulty. We speculate that the different results could be a result of the PPG limitations discussed earlier. Another possibility is that the driving demand did not differ significantly, and the sensors could not detect any differences because of the small differences in the driving demand.
The analysis of the HRV variables for shorter time frames (SDANN, SDNN index) showed statistically significant differences only in the values of SDANN, which suggests that changes in activities with periods longer than 10 s can be detected. However, there were no statistically significant differences in the SDNN index results, suggesting that neither E4 nor Faros can detect changes in different driving difficulties, based on recordings shorter than 10 s.
The analysis revealed statistically significant differences in RMSSD and pNN50 between the baseline and driving phases only in the data acquired with E4 but not in the data acquired with Faros although the HRV signal was measured with both devices simultaneously. We believe that the reason lies in E4’s limitations. As described in the third paragraph of the “PPG limitations” subsection, when there are missing samples in a sequence, successive differences cannot be determined. Therefore, we could perform analysis only on the signals measured with Faros, which however did not provide statistically significant results.
Based on these results, we can conclude that from the observed HRV signals, only SDNN or SDANN can be used for differentiating between the two observed levels of driving demand. However, we cannot rule out the possibility that the differences could be captured also in the other HRV variables, which could be investigated, for example, by using longer trials for each condition. Furthermore, it is better to use Faros than E4 as motion artefacts influence the measurement process and collected data.

4.3. Skin Temperature Analysis

Direct comparison of skin temperature data between the two devices is not possible, since they measure temperatures of different body parts. As shown in Figure 12 the temperature measured with Faros 360 was constantly rising. This could be due to the fact that the experiment lasted less than 20 min and it is possible that Faros was still warming up. Consequently, with slower warming up, the standard deviation significantly decreased in later phases. The skin temperature sensor in Faros 360 would provide more reliable data in studies that involve longer measurement periods (several hours) or, alternatively, participants would have to wear the measuring devices at least a few hours before the measurement process. This phenomenon was not evident with the E4’s skin temperature sensor.

4.4. Galvanic Skin Response (GSR) Analysis

The galvanic skin response (GSR) measurements with E4 revealed statistically significant increased conductance between demanding driving and baseline phases. However, there was a high standard deviation and standard error in the collected GSR signals during the driving phases, which can imply that the E4 may not have had skin contact during the whole recordings, probably due to the hand movements while steering.

4.5. Driver Sickness

Eighty-six percent of participants rated their feeling of sickness with a score of lower than 4 out of 10, and 91% of participants evaluated it with equal or lower than 6, suggesting that the results were not influenced by simulation/motion sickness. The measurements of the rest of the 9% of participants did not involve any outliers in the observed dependent variables.

4.6. Conclusions

The results from this study suggest that devices with PPG sensors such as E4 may be less reliable for assessing drivers’ signals compared to an ECG sensor-based device such as Faros 360. The main disadvantage is the sensitivity to motion artefacts, which can influence the signal recording quality. This is also in line with the placement of both devices, where Faros 360 is placed on the torso, which is relatively still during driving, whereas the E4 device is placed on the wrist, which is involved in movement due to steering the wheel. Therefore, in this particular case, we think Faros 360 could provide more reliable results for assessing drivers’ physiological signals.

Author Contributions

Conceptualization, T.G. and J.S.; methodology, K.S., G.J. and T.G.; software, T.G.; validation, K.S., J.S. and G.J.; formal analysis, T.G. and G.J.; investigation, T.G. and K.S.; resources, J.S.; data curation, T.G.; writing—original draft preparation, T.G.; writing—review and editing, K.S. and G.J.; visualization, T.G.; supervision, G.J. and J.S.; project administration, T.G. and K.S.; funding acquisition, J.S.

Funding

This work was partly supported by the Slovenian Research Agency within the research program ICT4QoL—Information and Communications Technologies for Quality of Life, grant number P2-0246, and the research project Neurophysiological and Cognitive Profiling of Driving Skills, grant number L2-8178.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Petridou, E.; Moustaki, M. Human factors in the causation of road traffic crashes. Eur. J. Epidemiol. 2000, 16, 819–826. [Google Scholar] [CrossRef] [PubMed]
  2. Vitabile, S.; Paola, A.D.; Sorbello, F. Bright Pupil Detection in an Embedded, Real-Time Drowsiness Monitoring System. In Proceedings of the 2010 24th IEEE International Conference on Advanced Information Networking and Applications, Perth, Australia, 20–23 April 2010; pp. 661–668. [Google Scholar]
  3. Bergasa, L.M.; Nuevo, J.; Sotelo, M.A.; Barea, R.; Lopez, M.E. Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Trans. Syst. 2006, 7, 63–77. [Google Scholar] [CrossRef] [Green Version]
  4. Lin, C.; Chen, Y.; Huang, T.; Chiu, T.; Ko, L.; Liang, S.; Hsieh, H.; Hsu, S.; Duann, J. Development of Wireless Brain Computer Interface With Embedded Multitask Scheduling and its Application on Real-Time Driver’s Drowsiness Detection and Warning. IEEE Trans. Biomed. Eng. 2008, 55, 1582–1591. [Google Scholar] [CrossRef] [PubMed]
  5. King, L.M.; Nguyen, H.T.; Lal, S.K.L. Early Driver Fatigue Detection from Electroencephalography Signals using Artificial Neural Networks. In Proceedings of the 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, USA, 30 August–3 September 2006; pp. 2187–2190. [Google Scholar]
  6. Li, W.; He, Q.; Fan, X.; Fei, Z. Evaluation of driver fatigue on two channels of EEG data. Neurosci. Lett. 2012, 506, 235–239. [Google Scholar] [CrossRef]
  7. Eoh, H.J.; Chung, M.K.; Kim, S.-H. Electroencephalographic study of drowsiness in simulated driving with sleep deprivation. Int. J. Ind. Ergon. 2005, 35, 307–320. [Google Scholar] [CrossRef]
  8. Khushaba, R.N.; Kodagoda, S.; Lal, S.; Dissanayake, G. Driver Drowsiness Classification Using Fuzzy Wavelet-Packet-Based Feature-Extraction Algorithm. IEEE Trans. Biomed. Eng. 2011, 58, 121–131. [Google Scholar] [CrossRef] [Green Version]
  9. Giusti, A.; Zocchi, C.; Rovetta, A. A Noninvasive System for Evaluating Driver Vigilance Level Examining Both Physiological and Mechanical Data. IEEE Trans. Intell. Trans. Syst. 2009, 10, 127–134. [Google Scholar] [CrossRef]
  10. Lee, B.; Chung, W. Driver Alertness Monitoring Using Fusion of Facial Features and Bio-Signals. IEEE Sens. J. 2012, 12, 2416–2422. [Google Scholar] [CrossRef]
  11. Bundele, M.M.; Banerjee, R. Detection of Fatigue of Vehicular Driver Using Skin Conductance and Oximetry Pulse: A Neural Network Approach. In Proceedings of the 11th International Conference on Information Integration and Web-based Applications & Services, Kuala Lumpur, Malaysia, 14–16 December 2009; ACM: New York, NY, USA, 2009; pp. 739–744. [Google Scholar]
  12. Shin, H.; Jung, S.; Kim, J.; Chung, W. Real time car driver’s condition monitoring system. In Proceedings of the 2010 IEEE SENSORS, Kona, HI, USA, 1–4 November 2010; pp. 951–954. [Google Scholar]
  13. Jung, S.-J.; Shin, H.-S.; Chung, W.-Y. Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel. IET Intell. Trans. Syst. 2014, 8, 43–50. [Google Scholar] [CrossRef]
  14. Matsuda, T.; Makikawa, M. ECG monitoring of a car driver using capacitively-coupled electrodes. In Proceedings of the 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Vancouver, BC, Canada, 20–25 August 2008; pp. 1315–1318. [Google Scholar]
  15. Thum Chia Chieh; Mustafa, M.M.; Hussain, A.; Hendi, S.F.; Majlis, B.Y. Development of vehicle driver drowsiness detection system using electrooculogram (EOG). In Proceedings of the 2005 1st International Conference on Computers, Communications, Signal Processing with Special Track on Biomedical Engineering, Kuala Lumpur, Malaysia, 14–16 November 2005; pp. 165–168. [Google Scholar]
  16. Ma, Z.; Li, B.C.; Yan, Z. Wearable driver drowsiness detection using electrooculography signal. In Proceedings of the 2016 IEEE Topical Conference on Wireless Sensors and Sensor Networks (WiSNet), Austin, TX, USA, 24–27 January 2016; pp. 41–43. [Google Scholar]
  17. Shih, C.; Cheng, S. A driver’s physiological monitoring system based on photodiode sensor with reflective optics element. In Proceedings of the 2016 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Beijing, China, 10–12 July 2016; pp. 1–4. [Google Scholar]
  18. Healey, J.; Seger, J.; Picard, R. Quantifying driver stress: Developing a system for collecting and processing bio-metric signals in natural situations. Biomed. Sci. Instrum. 1999, 35, 193–198. [Google Scholar]
  19. Healey, J.; Picard, R. SmartCar: Detecting driver stress. In Proceedings of the 15th International Conference on Pattern Recognition. ICPR-2000, Barcelona, Spain, 3–7 September 2000; Volume 4, pp. 218–221. [Google Scholar]
  20. Fairclough, S.H.; Spiridon, E. Cardiovascular and electrocortical markers of anger and motivation during a simulated driving task. Int. J. Psychophysiol. 2012, 84, 188–193. [Google Scholar] [CrossRef] [PubMed]
  21. Walter, M.; Eilebrecht, B.; Wartzek, T.; Leonhardt, S. The Smart Car Seat: Personalized Monitoring of Vital Signs in Automotive Applications. Personal Ubiquitous Comput. 2011, 15, 707–715. [Google Scholar] [CrossRef]
  22. Yang, C.; Wu, C.; Chou, C.; Yang, T. Vehicle driver’s ECG and sitting posture monitoring system. In Proceedings of the 2009 9th International Conference on Information Technology and Applications in Biomedicine, Larnaca, Cyprus, 4–7 November 2009; pp. 1–4. [Google Scholar]
  23. Balasubramanian, V.; Adalarasu, K. EMG-based analysis of change in muscle activity during simulated driving. J. Bodyw. Mov. Ther. 2007, 11, 151–158. [Google Scholar] [CrossRef]
  24. Reimer, B.; Mehler, B. The Effects of a Production Level “Voice-Command” Interface on Driver Behavior: Summary Findings on Reported Workload, Physiology, Visual Attention, and Driving Performance; MIT AgeLab White Paper No. 2013-18A: Cambridge, MA, USA, 2013. [Google Scholar]
  25. Son, J.; Oh, H.; Park, M. Identification of driver cognitive workload using support vector machines with driving performance, physiology and eye movement in a driving simulator. Int. J. Precis. Eng. Manuf. 2013, 14, 1321–1327. [Google Scholar] [CrossRef]
  26. Solovey, E.T.; Zec, M.; Garcia Perez, E.A.; Reimer, B.; Mehler, B. Classifying Driver Workload Using Physiological and Driving Performance Data: Two Field Studies. In Proceedings of the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada, 26 April–1 May 2014; ACM: New York, NY, USA, 2014; pp. 4057–4066. [Google Scholar]
  27. Lee, H.B.; Kim, J.S.; Kim, Y.S.; Baek, H.J.; Ryu, M.S.; Park, K.S. The relationship between HRV parameters and stressful driving situation in the real road. In Proceedings of the 2007 6th International Special Topic Conference on Information Technology Applications in Biomedicine, Tokyo, Japan, 8–11 November 2007; pp. 198–200. [Google Scholar]
  28. Liu, Y.; Ji, X.; Hayama, R.; Mizuno, T. A novel estimating method for steering efficiency of the driver with electromyography signals. Chin. J. Mech. Eng. 2014, 27, 460–467. [Google Scholar] [CrossRef]
  29. Kim, I.-H.; Kim, J.-W.; Haufe, S.; Lee, S.-W. Detection of braking intention in diverse situations during simulated driving based on EEG feature combination. J. Neural Eng. 2014, 12, 016001. [Google Scholar] [CrossRef]
  30. ZEIER, H. Concurrent physiological activity of driver and passenger when driving with and without automatic transmission in heavy city traffic. Ergonomics 1979, 22, 799–810. [Google Scholar] [CrossRef]
  31. Brookhuis, K.A.; de Waard, D. Monitoring drivers’ mental workload in driving simulators using physiological measures. Accid. Anal. Prev. 2010, 42, 898–903. [Google Scholar] [CrossRef]
  32. Brookhuis, K.A.; Waard, D. de On the assessment of (mental) workload and other subjective qualifications. Ergonomics 2002, 45, 1026–1030. [Google Scholar] [CrossRef]
  33. Mulder, L.B.J.; de Waard, D.; Brookhuis, A.K. Estimating Mental Effort Using Heart Rate and Heart Rate Variability. Available online: https://www.taylorfrancis.com/ (accessed on 14 November 2019).
  34. Wang, J.; Gong, Y. Recognition of multiple drivers’ emotional state. In Proceedings of the 2008 19th International Conference on Pattern Recognition, Tampa, FL, USA, 8–11 December 2008; pp. 1–4. [Google Scholar]
  35. Calderó-Bardají, P.; Longfei, X.; Jaschke, S.; Reermann, J.; Mideska, K.G.; Schmidt, G.; Deuschl, G.; Muthuraman, M. Detection of steering direction using EEG recordings based on sample entropy and time-frequency analysis. In Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016; pp. 833–836. [Google Scholar]
  36. Biondi, F.N.; Lohani, M.; Hopman, R.; Mills, S.; Cooper, J.M.; Strayer, D.L. 80 MPH and out-of-the-loop: Effects of real-world semi-automated driving on driver workload and arousal. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2018, 62, 1878–1882. [Google Scholar] [CrossRef] [Green Version]
  37. Murase, T.; Oiwa, K.; Nozawa, A. Quantitative Evaluation of Attention to the Driving Environment During Automated Driving at Levels 3 using ERP: Evaluation of Psychophysiological State of Driver. IEEJ Trans. EIS 2018, 138, 1148–1153. [Google Scholar] [CrossRef]
  38. Radlmayr, J.; Weinbeer, V.; Löber, C.; Farid, M.; Bengler, K. How Automation Level and System Reliability Influence Driver Performance in a Cut-In Situation. In Proceedings of the Advances in Human Aspects of Transportation, Orlando, FL, USA, 21–25 July 2018; Stanton, N.A., Ed.; Springer International Publishing: Cham, Switzerland, 2018; pp. 684–694. [Google Scholar]
  39. Sevil, M.; Hajizadeh, I.; Samadi, S.; Feng, J.; Lazaro, C.; Frantz, N.; Yu, X.; Brandt, R.; Maloney, Z.; Cinar, A. Social and competition stress detection with wristband physiological signals. In Proceedings of the 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Eindhoven, The Netherlands, 9–12 May 2017; pp. 39–42. [Google Scholar]
  40. Greene, S.; Thapliyal, H.; Caban-Holt, A. A Survey of Affective Computing for Stress Detection: Evaluating technologies in stress detection for better health. IEEE Consum. Electron. Mag. 2016, 5, 44–56. [Google Scholar] [CrossRef]
  41. Ollander, S.; Godin, C.; Campagne, A.; Charbonnier, S. A comparison of wearable and stationary sensors for stress detection. In Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, 9–12 October 2016; pp. 004362–004366. [Google Scholar]
  42. Kye, S.; Moon, J.; Lee, J.; Choi, I.; Cheon, D.; Lee, K. Multimodal Data Collection Framework for Mental Stress Monitoring. In Proceedings of the Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, Maui, HI, USA, 11–15 September 2017; ACM: New York, NY, USA, 2017; pp. 822–829. [Google Scholar]
  43. Kalimeri, K.; Saitis, C. Exploring Multimodal Biosignal Features for Stress Detection During Indoor Mobility. In Proceedings of the Proceedings of the 18th ACM International Conference on Multimodal Interaction, Tokyo, Japan, 12–16 November 2016; ACM: New York, NY, USA, 2016; pp. 53–60. [Google Scholar]
  44. Gouverneur, P.; Jaworek-Korjakowska, J.; Köping, L.; Shirahama, K.; Kleczek, P.; Grzegorzek, M. Classification of Physiological Data for Emotion Recognition. In Artificial Intelligence and Soft Computing; Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 619–627. [Google Scholar]
  45. Park, J.; Jeong, H.; Park, J.; Lee, B.C. Relationships Between Cognitive Workload and Physiological Response Under Reliable and Unreliable Automation. In Proceedings of the Advances in Human Factors and Systems Interaction, Washington, DC, USA, 24–28 July 2019; Nunes, I.L., Ed.; Springer International Publishing: Cham, Switzerland, 2019; pp. 3–8. [Google Scholar]
  46. Kutt, K.; Nalepa, G.J.; Giżycka, B.; Jemiolo, P.; Adamczyk, M. BandReader—A Mobile Application for Data Acquisition from Wearable Devices in Affective Computing Experiments. In Proceedings of the 2018 11th International Conference on Human System Interaction (HSI), Gdansk, Poland, 4–6 July 2018; pp. 42–48. [Google Scholar]
  47. McCarthy, C.; Pradhan, N.; Redpath, C.; Adler, A. Validation of the Empatica E4 wristband. In Proceedings of the 2016 IEEE EMBS International Student Conference (ISC), Ottawa, ON, Canad, 29–31 May 2016; pp. 1–4. [Google Scholar]
  48. Corino, V.D.A.; Laureanti, R.; Ferranti, L.; Scarpini, G.; Lombardi, F.; Mainardi, L.T. Detection of atrial fibrillation episodes using a wristband device. Physiol. Meas. 2017, 38, 787–799. [Google Scholar] [CrossRef] [PubMed]
  49. Mainardi, L.; Ferranti, L.; Laureanti, R. Atrial Fibrillation detection in PPG signal recorded through a wristband device. Thesis, Politecnico di Milano, Milano, It, 2015. [Google Scholar]
  50. Alam, M.A.U.; Roy, N.; Holmes, S.; Gangopadhyay, A.; Galik, E. Automated Functional and Behavioral Health Assessment of Older Adults with Dementia. In Proceedings of the 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), Washington, DC, USA, 27–29 June 2016; pp. 140–149. [Google Scholar]
  51. Bahrainian, S.A.; Crestani, F. Towards the Next Generation of Personal Assistants: Systems That Know When You Forget. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval, Amsterdam, The Netherlands, 1–4 October 2017; ACM: New York, NY, USA, 2017; pp. 169–176. [Google Scholar]
  52. Vandecasteele, K.; De Cooman, T.; Gu, Y.; Cleeren, E.; Claes, K.; Paesschen, W.V.; Huffel, S.V.; Hunyadi, B. Automated Epileptic Seizure Detection Based on Wearable ECG and PPG in a Hospital Environment. Sensors 2017, 17, 2338. [Google Scholar] [CrossRef] [PubMed]
  53. Marchiori, E.; Niforatos, E.; Preto, L. Measuring the Media Effects of a Tourism-Related Virtual Reality Experience Using Biophysical Data. In Proceedings of the Information and Communication Technologies in Tourism 2017, Rome, Italy, 24–26 January 2017; Schegg, R., Stangl, B., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 203–215. [Google Scholar]
  54. Cvetković, B.; Szeklicki, R.; Janko, V.; Lutomski, P.; Luštrek, M. Real-time activity monitoring with a wristband and a smartphone. Inf. Fusion 2018, 43, 77–93. [Google Scholar] [CrossRef]
  55. Hoecherl, J.; Schlegl, T.; Berlehner, T.; Kuhn, H.; Wrede, B. SmartWorkbench: Toward Adaptive and Transparent User Assistance in Industrial Human-Robot Applications. In Proceedings of the Proceedings of ISR 2016: 47st International Symposium on Robotics, Munich, Germany, 21–22 June 2016; pp. 1–8. [Google Scholar]
  56. Ragot, M.; Martin, N.; Em, S.; Pallamin, N.; Diverrez, J.-M. Emotion Recognition Using Physiological Signals: Laboratory vs. Wearable Sensors. In Proceedings of the Advances in Human Factors in Wearable Technologies and Game Design, Los Angeles, CA, USA, 17–21 July 2017; Ahram, T., Falcão, C., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 15–22. [Google Scholar]
  57. Salsone, M.; Vescio, B.; Fratto, A.; Sturniolo, M.; Arabia, G.; Gambardella, A.; Quattrone, A. Cardiac sympathetic index identifies patients with Parkinson’s disease and REM behavior disorder. Parkinsonism Relat. Disord. 2016, 26, 62–66. [Google Scholar] [CrossRef]
  58. Salsone, M.; Nistico’, R.; Vescio, B.; Novellino, F.; Morelli, M.; Lupo, A.; Arabia, G.; Quattrone, A. Heart rate variability in patients with essential tremor: A cross sectional study. Parkinsonism Relat. Disord. 2016, 33, 134–137. [Google Scholar] [CrossRef]
  59. Rajcani, J.; Solarikova, P.; Brezina, I. Effects of allergy and trait anxiety on heart rate variability: A naturalistic design study. Psychoneuroendocrinology 2017, 83, 40. [Google Scholar] [CrossRef]
  60. Kamp, M.R.; Thio, B.; De Jongh, F.; Driessen, J. Wearable Home-Monitoring in Asthmatic Children. In A60. Pediatric Allergy and Asthma, Proceedings of the American Thoracic Society 2018 International Conference Abstracts, San Diego, CA, 20 May 2018; American Thoracic Society: New York, NY, USA, 2018; p. A2028. [Google Scholar]
  61. Rapalis, A.; Petrėnas, A.; Šimaitytė, M.; Bailón, R.; Marozas, V. Towards pulse rate parametrization during free-living activities using smart wristband. Physiol. Meas. 2018, 39, 055007. [Google Scholar] [CrossRef]
  62. Mohapatra, P.; Premkumar, P.S.; Sivaprakasam, M. A Yellow-Orange Wavelength-Based Short-Term Heart Rate Variability Measurement Scheme for Wrist-Based Wearables. IEEE Trans. Instrum. Meas. 2018, 67, 1091–1101. [Google Scholar] [CrossRef]
  63. Vescio, B.; Salsone, M.; Gambardella, A.; Quattrone, A. Comparison between Electrocardiographic and Earlobe Pulse Photoplethysmographic Detection for Evaluating Heart Rate Variability in Healthy Subjects in Short- and Long-Term Recordings. Sensors 2018, 18, 844. [Google Scholar] [CrossRef] [Green Version]
  64. Drougard, N.; Roy, R.N.; Scannella, S.; Dehais, F.; Ponzoni Carvalho Chanel, C. Physiological Assessment of Engagement during HRI: Impact of Manual vs Automatic Mode. In Proceedings of the 2nd International Neuroergonomics Conference, Philadelphia, PA, USA, 27–29 June 2018; p. 0. [Google Scholar]
  65. Moshammer, H.; Panholzer, J.; Ulbing, L.; Udvarhelyi, E.; Ebenbauer, B.; Peter, S. Acute Effects of Air Pollution and Noise from Road Traffic in a Panel of Young Healthy Subjects. Proceedings 2018, 6, 10. [Google Scholar] [CrossRef] [Green Version]
  66. Simjanoska, M.; Gjoreski, M.; Gams, M.; Madevska Bogdanova, A. Non-Invasive Blood Pressure Estimation from ECG Using Machine Learning Techniques. Sensors 2018, 18, 1160. [Google Scholar] [CrossRef] [Green Version]
  67. Eggenberger, P.; MacRae, B.A.; Kemp, S.; Bürgisser, M.; Rossi, R.M.; Annaheim, S. Prediction of Core Body Temperature Based on Skin Temperature, Heat Flux, and Heart Rate Under Different Exercise and Clothing Conditions in the Heat in Young Adult Males. Front. Physiol. 2018, 9, 1790. [Google Scholar] [CrossRef] [PubMed]
  68. Alikhani, I.; Noponen, K.; Hautala, A.; Seppänen, T. Characterization and reduction of exercise-based motion influence on heart rate variability using accelerator signals and channel decoding in the time–frequency domain. Physiol. Meas. 2018, 39, 115002. [Google Scholar] [CrossRef] [Green Version]
  69. Hostettler, R.; Lumikari, T.; Palva, L.; Nieminen, T.; Särkkä, S. Motion Artifact Reduction in Ambulatory Electrocardiography Using Inertial Measurement Units and Kalman Filtering. In Proceedings of the 2018 21st International Conference on Information Fusion (FUSION), Cambridge, UK, 10–13 July 2018; pp. 1–8. [Google Scholar]
  70. Real-time Physiological Signals | E4 EDA/GSR Sensor. Available online: https://www.empatica.com/research/e4 (accessed on 23 May 2019).
  71. Bittium Faros. Available online: https:/www.bittium.com/medical/bittium-faros (accessed on 23 May 2019).
  72. Nervtech Simuation Technologies. Available online: https://www.nervtech.com (accessed on 29 April 2019).
  73. Allen, J. Photoplethysmography and its application in clinical physiological measurement. Physiol. Meas. 2007, 28, R1–R39. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  74. E4 connect. Available online: https://www.empatica.com/connect/login.php (accessed on 23 May 2019).
  75. Bittium Faros 3.x Bluetooth Protocol. Version 2.0.0. 2018.
  76. Kinkade, R.G.; Wheaton, G.R. Training device design. In Hum. Eng. Guide Equip. Des.; John Wiley & Sons: Hoboken, NJ, USA, 1972; pp. 668–699. [Google Scholar]
  77. Hays, R.T. Simulator Fidelity: A Concept Paper; Army Research Inst for the Behavioral and Social Sciences: , Alexandria, Virginia, USA, 1980. [Google Scholar]
  78. Bluegiga BLED112 Bluetooth Smart Dongle—Silicon Labs. Available online: https://www.silabs.com/products/wireless/bluetooth/bluetooth-low-energy-modules/bled112-bluetooth-smart-dongle (accessed on 23 May 2019).
  79. Laborde, S.; Mosley, E.; Thayer, J.F. Heart Rate Variability and Cardiac Vagal Tone in Psychophysiological Research—Recommendations for Experiment Planning, Data Analysis, and Data Reporting. Front. Psychol. 2017, 8, 213. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  80. Gruden, T. Uporaba komercialno dostopnih naprav za merjenje fizioloških signalov voznikov. Ph.D. Thesis, Univerza v Ljubljani, Fakulteta za elektrotehniko, Ljubljana, Slovenia, 2018. [Google Scholar]
  81. Quintana, D.S. Statistical considerations for reporting and planning heart rate variability case-control studies. Psychophysiology 2017, 54, 344–349. [Google Scholar] [CrossRef] [PubMed]
  82. Malik, M.; Bigger, J.T.; Camm, A.J.; Kleiger, R.E.; Malliani, A.; Moss, A.J.; Schwartz, P.J. Heart rate variabilityStandards of measurement, physiological interpretation, and clinical use. Eur. Heart J. 1996, 17, 354–381. [Google Scholar] [CrossRef] [Green Version]
  83. Schäfer, A.; Vagedes, J. How accurate is pulse rate variability as an estimate of heart rate variability? A review on studies comparing photoplethysmographic technology with an electrocardiogram. Int. J. Cardiol. 2013, 166, 15–29. [Google Scholar] [CrossRef] [PubMed]
Applsci 09 05353 g001 550
Figure 1. Empatica E4 with positions of its sensors.
Figure 1. Empatica E4 with positions of its sensors.
Applsci 09 05353 g001
Applsci 09 05353 g002 550
Figure 2. Faros 360 with cable set for measuring with electrodes (left) and Faros 360 with textile belt for electrode-free measurements (right).
Figure 2. Faros 360 with cable set for measuring with electrodes (left) and Faros 360 with textile belt for electrode-free measurements (right).
Applsci 09 05353 g002
Applsci 09 05353 g003 550
Figure 3. PQRST model and R–R interval.
Figure 3. PQRST model and R–R interval.
Applsci 09 05353 g003
Applsci 09 05353 g004 550
Figure 4. Nervtech driving simulator with three Samsung Curved TV 48’.
Figure 4. Nervtech driving simulator with three Samsung Curved TV 48’.
Applsci 09 05353 g004
Applsci 09 05353 g005 550
Figure 5. Positioning of Empatica E4 and Bittium Faros 360. E4 should be placed on the wrist of a non-dominant hand in a way that electrodes are on the line that goes between middle and ring finger. Faros with the textile belt should be placed right below the chest muscle [80].
Figure 5. Positioning of Empatica E4 and Bittium Faros 360. E4 should be placed on the wrist of a non-dominant hand in a way that electrodes are on the line that goes between middle and ring finger. Faros with the textile belt should be placed right below the chest muscle [80].
Applsci 09 05353 g005
Applsci 09 05353 g006 550
Figure 6. Driver’s position during three phases of measurement: (a) Baseline, (b) easy driving, (c) demanding driving [80].
Figure 6. Driver’s position during three phases of measurement: (a) Baseline, (b) easy driving, (c) demanding driving [80].
Applsci 09 05353 g006
Applsci 09 05353 g007 550
Figure 7. Mean heart rate variability (HRV) for each participant in each phase for E4 (left) and Faros 360 (right).
Figure 7. Mean heart rate variability (HRV) for each participant in each phase for E4 (left) and Faros 360 (right).
Applsci 09 05353 g007
Applsci 09 05353 g008 550
Figure 8. HRV SDNN for each participant in each phase for E4 (left) and Faros 360 (right).
Figure 8. HRV SDNN for each participant in each phase for E4 (left) and Faros 360 (right).
Applsci 09 05353 g008
Applsci 09 05353 g009 550
Figure 9. HRV SDANN for 10-s timeframes for each participant in each phase for E4 (left) and Faros 360 (right).
Figure 9. HRV SDANN for 10-s timeframes for each participant in each phase for E4 (left) and Faros 360 (right).
Applsci 09 05353 g009
Applsci 09 05353 g010 550
Figure 10. HRV RMSSD for each participant in each phase for E4 (left) and Faros 360 (right).
Figure 10. HRV RMSSD for each participant in each phase for E4 (left) and Faros 360 (right).
Applsci 09 05353 g010
Applsci 09 05353 g011 550
Figure 11. HRV pNN50 for each participant in each phase for E4 (left) and Faros 360 (right).
Figure 11. HRV pNN50 for each participant in each phase for E4 (left) and Faros 360 (right).
Applsci 09 05353 g011
Applsci 09 05353 g012 550
Figure 12. Faros’ temperature readings from the beginning of the baseline phase. Data for each participant is drawn with different color. Straight lines interpolate the temperature between the end of one phase and the beginning of the next one.
Figure 12. Faros’ temperature readings from the beginning of the baseline phase. Data for each participant is drawn with different color. Straight lines interpolate the temperature between the end of one phase and the beginning of the next one.
Applsci 09 05353 g012
Applsci 09 05353 g013 550
Figure 13. Mean skin temperature for each participant in each phase for E4 (left) and Faros 360 (right).
Figure 13. Mean skin temperature for each participant in each phase for E4 (left) and Faros 360 (right).
Applsci 09 05353 g013
Applsci 09 05353 g014 550
Figure 14. Standard deviation of skin temperature for each participant in each phase for E4 (left) and Faros 360 (right).
Figure 14. Standard deviation of skin temperature for each participant in each phase for E4 (left) and Faros 360 (right).
Applsci 09 05353 g014
Applsci 09 05353 g015 550
Figure 15. Mean and standard deviation of GSR for each participant in each phase for E4 measurements.
Figure 15. Mean and standard deviation of GSR for each participant in each phase for E4 measurements.
Applsci 09 05353 g015
Applsci 09 05353 g016 550
Figure 16. Drivers’ self-reports of sickness on a 10-point scale, where 1 indicates “I am perfectly fine”, and 10 means “I have to throw up”.
Figure 16. Drivers’ self-reports of sickness on a 10-point scale, where 1 indicates “I am perfectly fine”, and 10 means “I have to throw up”.
Applsci 09 05353 g016
Table
Table 1. Results of each device (columns) and each dependent variable (rows). ‘<’ indicates a statistically significant increase in the corresponding dependent variable between the two phases. X indicates that no significant differences among three phases of driving were found. N/A indicates the measurements were not applicable for the corresponding device.
Table 1. Results of each device (columns) and each dependent variable (rows). ‘<’ indicates a statistically significant increase in the corresponding dependent variable between the two phases. X indicates that no significant differences among three phases of driving were found. N/A indicates the measurements were not applicable for the corresponding device.
E4Faros 360
Mean and median HRVXX
HRV SDNNbaseline < easy driving
baseline < demanding driving		easy driving < demanding driving
baseline < demanding driving
HRV SDANNbaseline < easy driving
baseline < demanding driving		easy driving < demanding driving
baseline < demanding driving
HRV SDNN IndexXX
HRV RMSSDbaseline < easy driving
baseline < demanding driving		X
HRV pNN50baseline < easy driving
baseline < demanding driving		X
Mean skin temperaturebaseline < easy driving
baseline < demanding driving		baseline < easy driving
baseline < demanding driving
easy driving < demanding driving
Standard deviation of skin temperaturebaseline < easy driving
baseline < demanding driving		baseline < easy driving
baseline < demanding driving
Mean and standard deviation of GSRbaseline < demanding drivingN/A
Table
Table 2. Statistical tests’ results for HRV variables for shorter timeframes.
Table 2. Statistical tests’ results for HRV variables for shorter timeframes.
SDANNSDNN Index
E4RMANOVA:
F(2, 38) = 27.721, p < 0.001		RMANOVA with Greenhouse-Geisser correction:
F(1.586, 33.303) = 3.923, p = 0.0381 1
Faros 360RMANOVA:
F(2, 42) = 13.312, p < 0.001		RMANOVA with Greenhouse-Geisser correction:
F(1.398, 29.364) = 3.405, p = 0.062
1 Bonferroni post-hoc test showed no statistically significant differences between phases, p(1,2) = 0.166, p(2,3) = 0.641, p(1,3) = 0.091.
Table
Table 3. Statistical tests’ results for skin temperature.
Table 3. Statistical tests’ results for skin temperature.
Skin Temperature MeanSkin Temperature Standard Deviation
E4RMANOVA:
F(2, 42) = 29.313, p < 0.001		Friedman’s nonparametric test:
χ2(2) = 9.818, p = 0.007
Faros 360RMANOVA with Greenhouse-Geisser correction:
F(1.180, 24.786) = 279.574, p < 0.001		RMANOVA with Greenhouse-Geisser correction:
F(1.207, 25.350) = 78.100, p < 0.001

Share and Cite

MDPI and ACS Style

Gruden, T.; Stojmenova, K.; Sodnik, J.; Jakus, G. Assessing Drivers’ Physiological Responses Using Consumer Grade Devices. Appl. Sci. 2019, 9, 5353. https://doi.org/10.3390/app9245353

AMA Style

Gruden T, Stojmenova K, Sodnik J, Jakus G. Assessing Drivers’ Physiological Responses Using Consumer Grade Devices. Applied Sciences. 2019; 9(24):5353. https://doi.org/10.3390/app9245353

Chicago/Turabian Style

Gruden, Timotej, Kristina Stojmenova, Jaka Sodnik, and Grega Jakus. 2019. "Assessing Drivers’ Physiological Responses Using Consumer Grade Devices" Applied Sciences 9, no. 24: 5353. https://doi.org/10.3390/app9245353

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop