Physiological Driver Monitoring Using Capacitively Coupled and Radar Sensors

Featured Application

Physiological parameters of drivers, including the electrocardiogram (ECG), heart rate and heart rate variability, respiration activity and rate, can be measured in a contactless way using capacitively coupled and radar sensors. This potentially enables applications to monitor the driver’s physical and emotional status and well-being and contribute to preventing accidents and provide health-related services.

Abstract

Unobtrusive monitoring of drivers’ physiological parameters is a topic gaining interest, potentially allowing to improve the performance of safety systems to prevent accidents, as well as to improve the driver’s experience or provide health-related services. In this article, two unobtrusive sensing techniques are evaluated: capacitively coupled sensing of the electrocardiogram and respiration, and radar-based sensing of heartbeat and respiration. A challenge for use of these techniques in vehicles are the vibrations and other disturbances that occur in vehicles to which they are inherently more sensitive than contact-based sensors. In this work, optimized sensor architectures and signal processing techniques are proposed that significantly improve the robustness to artefacts. Experimental results, conducted under real driving conditions on public roads, demonstrate the feasibility of the proposed approach. R peak sensitivities and positive predictivities higher than 98% both in highway and city traffic, heart rate mean absolute error of 1.02 bpm resp. 2.06 bpm in highway and city traffic and individual beat R-R interval 95% percentile error within ±27.3 ms are demonstrated. The radar experimental results show that respiration can be measured while driving and heartbeat can be recovered from vibration noise using an accelerometer-based motion reduction algorithm.

1. Introduction

The monitoring of physiological parameters of drivers is becoming an important topic that is attracting the attention of many researchers and companies worldwide. By monitoring cardiac and respiratory parameters (e.g., heart rate, heart rate variability, ECG waveform, respiration rate) it is possible to gain insight in the physical and emotional state of the drivers, such as for example drowsiness [1], stress [2], emotional state [3], epileptic seizures [4], cardiovascular conditions such as ischemic stroke [5], myocardial infarction [6], heart failure [7], etc.

When such physiological sensors are combined with complementary non-physiological sensing systems inside a vehicle (e.g., steering wheel and pedal sensors, speed, lane-keeping, collision avoidance, etc.) it potentially allows to improve the performance of safety systems to prevent accidents, as well as to improve the driver’s experience and well-being or provide new services such as for example daily health check-ups during commuting.

For such a system to be successfully adopted and used it is essential that the sensors are unobtrusive, i.e., the driver should not have to take any action to connect any sensors to their body. Two sensing techniques that meet this requirement are capacitive-based sensors, which can measure physiological signals through clothing, and radar-based sensors, which can measure remotely. As will be shown below, these sensors have complementary characteristics. They do not rely on a driver-facing camera, avoiding what may be considered a potential privacy concern for some users.

The main challenge for the use of such unobtrusive sensors in vehicles is that the weak physiological signals need to be accurately detected while being reliably distinguished from motion artefacts due to unavoidable vibrations of the car while driving as well as the driver’s body motions.

The capacitive-based sensing system considered in this work can measure capacitively coupled ECG (ccECG) and capacitively coupled bioimpedance (ccBioZ). The ccECG measures the electrical signals generated by the heart (ECG) coupled capacitively through clothing to a conducting sensor plate. The ccBioZ is measured by capacitively (through the clothing) injecting a small a.c. current into the body from two conducting plates and measuring the corresponding voltage signal (also capacitively) through the clothing using two conducting sensor plates. The bio-impedance obtained in this way measures the respiration using the principle of impedance pneumography [8]. The ccECG can be measured through multiple layers of clothing [9] but not through e.g., thick jackets. The ccBioZ can be measured through very thick clothing layers or even small air gaps due to its 4-point measurement principle and due to the high frequency of the injected signal.

Radar-based sensing allows to measure the vital signs heartbeat and respiration through remote measurement (up to several meters) of small mechanical motions of the body surface. Remote radar-based vital signs sensing has been intensively investigated in the last two decades. The attention has been focused mainly on contactless vital signs monitoring in indoor environments [10,11,12,13,14,15,16,17,18]. Considerable research on multi-people vital signs sensing and tracking has been performed by imec [19,20,21,22,23,24], which also presented recently a Frequency-Modulated Continuous Wave (FMCW) integrated chip (IC) radar capable of detecting the vital signs up to a distance of 15 m while achieving a low-power consumption record of only 680 µW [24]. Radar has also been used for driver monitoring. In [25], the radar was placed inside the seat and the back of subject was directly in contact with the antennas. The conclusions of this work were that static measurements showed outstanding accuracy in detecting the vital signs while during the dynamic tests (i.e., while driving) the results were not robust enough to be used as input for medical or emergency prevention systems. In this work, a radar sensor, which integrates an accelerometer to measure the vibrations, was placed behind the driver’s seat.

These two sensor techniques are complementary: capacitive-based sensors can provide a complete ECG waveform which permits the most accurate determination of heart rate variability (HRV), but they only work through a number of clothing layers. On the other hand, radar-based sensors provide true remote sensing, but can only provide heartbeat information, not a complete ECG. Both techniques can provide respiration information. Hence, there is potential for sensor fusion combining these two techniques to improve overall performance under varying conditions. Initial work on sensor fusion of capacitively-coupled ECG and radar in an indoor environment was published by the Authors in [26].

In this work, both techniques are evaluated in real-world driving tests. In addition, signal processing techniques are presented that improve the robustness of these unobtrusive sensors to motion artefacts. For the ccECG, signal quality indicator (SQI) based motion artefact removal algorithms are implemented for this purpose. For the radar signals, a signal processing technique which correlates the radar signal to the accelerometer signal in order to remove the contribution of the vibrations from the vital signs information is presented.

This paper is organized as follows. In Section 2, the capacitively-coupled and radar systems and their physical positioning in the car are detailed together with the signal processing techniques. The experimental results are shown in Section 3, while the discussion and the conclusions are drawn in Section 4.

2. Materials and Methods

2.1. Capacitively-Coupled ECG and Bio-Impedance System and Positioning in the Car Seat

The capacitive sensor system that was evaluated is shown in Figure 1 and is described in detail in [27]. For this evaluation, the system was implemented on a seat-covering pillow for easy transfer between vehicles, but it can also be integrated directly into a car seat in case of a permanent installation. It consists of an array of ccECG sensing plates, implemented in this case using 15 rigid sensing plates in the back side, and 2 conductive textile sensing plates at the legs. This array configuration permits to pick up ECG signals for different driver body sizes, shapes and seating positions. Also 4 ccBioZ plates (2 for current injection and 2 for voltage sensing) are included in the back side, together with one unused plate in the center. In the bottom side a larger conductive textile plate is used for capacitive Driven Right Leg (DRL) feedback to reduce common mode interference.

2.2. Signal Processing on Capacitively-Coupled Signals

Figure 2a shows the signal processing steps performed on the capacitively coupled ECG. Besides standard processing steps such as bandpass filtering, alignment, beat detection etc., the lower branch indicates the specific additional quality-based steps for motion artefact handling applied in this work:

• A method to discard from the set of 4 ECG channels the channels with no potential valuable information. This uses a classifier described in more detail in [28];
• The identification of artefacts using a template-based signal quality metric previously evaluated by the authors [29], in combination with an amplitude-based metric calculated as the percentage of 0.5-s sub-windows that have data within [0.05–3] mV range;
• The selection of the channel. This selection uses the same signal quality metric as in step 2;
• The extraction of features from the signal parts with useful information. The parts of the signal that have been identified as containing artefacts in step 2 are omitted from the feature calculation.

Figure 2b shows the signal processing path for the capacitively coupled bioimpedance signal towards respiration metrics. No motion artefact algorithms have been included for respiration at this point.

2.3. Radar Sensor System and Positioning in the Car Seat

The radar sensor, shown in Figure 3a, consists of a radar unit, two 6 dBi antennas, and the Avnet PicoZed 7015 module which integrates a dual-core processor and a Field Programmable Gate Array (FPGA). The radar unit is based on an FMCW architecture and transmits series of FMCW signals, called chirps, whose instantaneous frequency increases linearly over time. The chirps have a total bandwidth B of 750 MHz, in the 7.3–8.05 GHz frequency range, involving a radar range resolution of 20 cm. Moreover, they are T = 102.4 µs long and are transmitted with a pulse repetition interval (PRI) of 3.072 ms, ensuring a sampling rate of the Doppler (vital signs) signal of 1/PRI = 325.52 Hz. The transmitted power is −6 dBm. Details on the radar-based vital signs sensing and on the FMCW theory can be found in the authors’ prior art [23,24].

The block diagram of the radar sensor system is show in Figure 3b. The chirp generator was designed through an analog Voltage-Controlled Oscillator (VCO) and a Fractional-N Phase Locked Loop (PLL). Each chirp is divided in two branches by a power splitter. One output feeds the transmitter antenna while the other output is connected to the mixer. The received chirps, which are reflected from the driver, are picked up by the receiver antenna, amplified by a low noise amplifier (LNA), and then mixed with the copies of the transmitted chirps. The resulting baseband signals are filtered and amplified before being acquired and digitized by an analog-to-digital converter (ADC). The latter are preprocessed in the FPGA which performs digital filtering, windowing function, Fast Fourier Transform (FFT), and then sends the resulting complex samples to the laptop by Ethernet.

The radar was placed behind the driver’s seat (Figure 3c,d). In order to maximize the heartbeat detection and to be able to monitor the respiration, the radar was positioned such that its line of sight (LoS) was in the middle of the back at the height of the chest. One advantage of operating at this frequency range is that the seat material does not obstruct the propagation of the electromagnetic waves. An accelerometer is integrated in the radar board to measure vibration of the radar caused by the car motion. It is fairly assumed that this vibration is very similar to the vibration of the driver’s back due to the motion of the car when he/she leans against the seat. The z-axis of the accelerometer coincides with the radar LoS, which gives the highest contribution in the Doppler shift.

2.4. Signal Processing on Radar Signals

Due to its ultra-wideband (UWB) nature, the FMCW radar is able to resolve temporally the reflections and, therefore, it is able to isolate the Doppler signals (i.e., phase shift) caused by the cardiopulmonary activity of the driver from the unwanted phase shifts present in the car generated by the vibrations of objects (e.g., dashboard, driving mirror, wipers) and/or other passengers.

The first step is to individuate the range bin wherein the subject’s back lies. Considering that each range bin (i.e., range resolution) is 20 cm and that the seat is about 10 cm thick, the vital signs information can be retrieved extracting the phase shift from the FFT’s complex samples which belong to the first range bin.

The second step is to extract and optimize the Doppler (phase) signal. In fact, static reflectors (e.g., seat, static body parts of the driver) in the range bin yield phase distortion. In order to overcome this issue, the linear demodulation technique, described in [23], is applied to recover phase signal.

In the car and with the radar positioned to the back of the seat, the Doppler signal is mainly a contribution of two sources: (1) The displacement of the back surface due to the cardiopulmonary activity; (2) the vibration of the subject’s back due to the car movement. An estimate of the latter vibration is measured by the accelerometer. The result of the linear demodulation is a signal that can be fairly considered directly proportional to the Doppler signal. Since the two sources are non-Gaussian distributed, statistically independent, and linearly combined, the independent component analysis (ICA) algorithm can be used to separate the two sources. The ICA takes as input the linear demodulation output and the acceleration in the z direction and provides two outputs: (1) The phase shift caused by the vibration; (2) the phase shift due to the vital signs. The latter can be further processed by Wavelet Transform to separate the heartbeat from the respiration and to remove the noise.

3. Results

The experimental tests were conducted on healthy volunteer drivers on public roads (i.e., city center, highway) under real conditions. The drivers were instructed to only pay attention to the driving task. Two additional passengers were always present in the car to collect the capacitive sensor and radar signals. In addition, a reference (FDA approved and CE class IIa conform) device was used (NeXus-10 MkII, MindMedia, 6049 CD Roermond, The Netherlands), which recorded contact ECG signals (Lead I) from the driver’s shoulders, as well as respiration signals using a strapped belt around the abdomen.

The experimental protocol was approved by imec-nl’s internal investigation approval board with approval reference IA-18-WATS-TIP2-041.

3.1. Experimental Results from Capacitively-Coupled ECG and Bio-Impedance

5 subjects were recorded. Each subject drove approximately 30 min in city traffic and 30 min in highway traffic (one subject only did the highway traffic driving). 2 subjects wore two layers of clothing, and 3 subjects wore a single layer of clothing. See

Table 1. Overview of recorded subjects for the capacitive sensor system evaluation.

Subject	Recording Length		Number of Clothing Layers Worn
	City	Highway
1	44 min	35 min	1
2	16 min	15 min	2
3	n/a	32 min	1
4	38 min	46 min	1
5	39 min	36 min	2

for details.

Table 2. R peak detection and heart rate performance before and after motion artefact algorithms. The percentage of signal removed is a result of the quality-based motion artefact handling. * The mean is not calculated for numbers expressed as percentages since the sample sizes are not identical.

Subj.	Traffic	Before Motion Artefact Algorithms			After Motion Artefact Algorithms
		R Peak Detection Sensitivity	R Peak Detection Positive Predictivity	HR Mean Absolute Error (bpm)	R Peak Detection Sensitivity	R Peak Detection Positive Predictivity	Hr Mean Absolute Error (bpm)	% of Signal Removed
1	highway	94.67%	95.15%	2.28	97.58%	98.26%	0.86	10.3%
	city	89.46%	92.34%	2.18	97.11%	97.23%	1.28	29.7%
2	highway	76.71%	84.65%	3.06	98.28%	99.03%	1.57	47.3%
	city	59.27%	66.53%	12.68	99.36%	100.00%	0.93	82.5%
3	highway	98.05%	98.04%	1.10	99.58%	99.49%	0.36	9.8%
	city	n/a	n/a	n/a	n/a	n/a	n/a	n/a
4	highway	99.63%	99.89%	0.05	99.82%	99.92%	0.01	2.8%
	city	99.28%	98.97%	0.35	99.31%	99.05%	0.22	0.3%
5	highway	89.92%	86.57%	3.37	96.65%	91.51%	2.30	20.6%
	city	92.74%	87.45%	6.56	93.99%	88.64%	5.80	11.2%
Median	highway	94.67%	95.15%	2.28	98.28%	99.03%	0.86	10.29%
	city	91.10%	89.90%	4.37	98.21%	98.14%	1.11	20.41%
	overall	92.74%	92.34%	2.28	98.28%	99.03%	0.93	11.15%
Mean	highway	*	*	1.97	*	*	1.02	*
	city	*	*	5.44	*	*	2.06	*
	overall	*	*	3.51	*	*	1.48	*

shows the performance for R peak detection and heart rate on each of the subjects together with the percentage of removed signal (as a result of the quality-based processing described in Section 2.2). The motion artefact algorithms improve the median R peak detection sensitivity from 92.74% to 98.28% and the median positive predictive value (PPV) from 92.34% to 99.03%, city and highway combined. This compares favorably to [30] (average sensitivity 91.75%, PPV 84.47%) and [31] (average sensitivity 88.02%, PPV 95.17%). The overall mean absolute error on the heart rate when compared to the reference system is 1.48 bpm, calculated over 60 s windows. The motion artefact algorithms removed 10.29% resp. 20.41% (medians) of the signal in highway resp. city traffic.

For applications where information regarding heart rate variability is desired, it is necessary to evaluate the performance on the detection of individual beat-to-beat R-R intervals.

Table 3. Beat-to-beat R-R interval performance metrics (after motion artefact algorithms), and respiration performance (without motion artefact algorithms). * The mean is not calculated for numbers expressed as percentages since the sample sizes are not identical.

Subj.	Traffic	ECG after Motion Artefact Algorithms					Respiration
		R-R Interval Mean Absolute Error (ms)	R-R Interval Mean Absolute Percentual Error (%)	95th Percentile Absolute R-R Interval Error (ms)	SDNN Mean Absolute Error (ms)	Coverage R-R Interval within 5 ms	Coverage Respiration Rate within 3 bpm
1	highway	18.67	1.39%	25.39	52.43	91.16%	68.57%
	city	25.44	1.98%	52.73	64.77	90.15%	61.36%
2	highway	28.64	2.54%	23.24	119.12	93.85%	26.67%
	city	25.00	0.84%	50.78	58.11	94.61%	25.00%
3	highway	5.21	0.39%	1.95	10.09	95.93%	68.75%
	city	n/a	n/a	n/a	n/a	n/a	n/a
4	highway	1.48	0.11%	1.95	0.13	97.50%	26.09%
	city	5.43	0.41%	5.86	8.17	94.34%	47.37%
5	highway	67.72	4.05%	391.21	88.34	59.37%	66.67%
	city	130.40	6.38%	369.48	132.58	58.94%	64.10%
Median	highway	18.67	1.39%	23.24	52.43	93.85%	66.67%
	city	25.22	1.41%	51.76	61.44	92.25%	54.37%
	overall	25.00	1.39%	25.39	58.11	93.85%	61.36%
Mean	highway	24.34	*	88.75	54.02	*	*
	city	46.57	*	119.71	65.91	*	*
	overall	34.22	*	102.51	59.30	*	*

shows performance metrics for all subjects. The mean absolute error on the R-R intervals is 34.22 ms. This is skewed by subject 5 where the reference contact ECG was noisy due to a poorly connected ground lead. If subject 5 is excluded, the mean absolute error on the R-R intervals reduces to 15.7 ms. Figure 4 shows the distribution of the R-R interval absolute error across the entire dataset, which is highly non-Gaussian. The 95th percentile of the absolute error is 27.3 ms. If subject 5 is excluded this reduces to 5.9 ms.

Table 3 also shows the coverage defined as the percentage of time that the respiration rate as calculated using an FFT-based method on 60-s windows from the ccBioZ signal has an error of 3 breaths per minute or less. The median coverage is 61%. This is significantly lower than the coverage numbers obtained for the ECG (median 93.85% coverage with R-R intervals within 5 ms), which can be explained by the facts that no signal quality based motion artefact reduction algorithm was yet included for the ccBioZ signals, and that only one, fixed ccBioZ measurement location was used, compared to an array for the ccECG measurements.

Figure 5 shows example signals from the capacitive ECG and respiration and reference signals while driving. It can be observed that the signals are robust to vibration noise from driving, both at high speed (120 km/h) and low speed (30 km/h) as well as during acceleration. Only when the driver’s body moves during the acceleration phase (in this case being ‘pushed’ back against the seat and hence generating electrode interface changes), motion artefacts are visible in the ECG signal. Similar behavior was observed in sudden deceleration as a cause of the back of the driver moving forward relative to the seat.

3.2. Experimental Results from Radar

In order to verify that the radar can monitor the vital signs from the back and through the seat, an initial test was performed with the car put in neutral with the engine on. The driver was invited to breathe normally. Figure 6 shows the results of this experiment. The respiration and heartbeat signals extracted with the radar and the references (i.e., belt and ECG) are in agreement. Moreover, in this static condition, as it can be seen from the small acceleration values, the vibrations from the car are negligible so no vibration removal algorithm is necessary.

Figure 7, Figure 8, Figure 9 and Figure 10 show a proof-of-concept result of vital signs monitoring while driving at 120 km/h on the highway. Figure 7 shows the time domain ECG signal from the reference and its relative spectrum. The heartbeat peak is clearly visible at 1.43 Hz in the ECG spectrum. Figure 8 shows the radar Doppler Signal after the linear demodulation, the z-acceleration, and their relative spectra. In the Doppler signal, the vital signs contribution is corrupted by the vibration. In fact, although the respiration signal is visible, the heartbeat is buried into the vibration. This is confirmed by the fact that there is no peak at 1.43 Hz in the Doppler signal spectrum while there is an in-band peak at about 1.17 Hz. Figure 9 is the result of the ICA and it shows the separated phase contributions of the vital signs and of the vibration. In the spectrum of the vital signs output of the ICA the 1.43 Hz heartbeat peak is clearly visible. Finally, Figure 10 shows the time domain signals of the extracted vital signs, of the reference, and the x-y-z accelerations. The average heart rate measured with the radar and with the ECG agree.

As seen in Figure 10, this proof-of-concept experiment shows that the radar is able to correctly measure the respiration signal and, after applying the proposed motion artefact algorithms, also the average heartbeat during driving. Further validation will be performed, and further improvements to both the signal processing algorithms and radar integration inside the car are expected to improve the performance further, including the measurement of individual beat-to-beat intervals during driving conditions.

4. Discussion

It was shown, in a study on 5 subjects in highway and city driving conditions, that the proposed capacitive ECG system can provide accurate beat detection, heart rate and individual R-R intervals, as well as full ECG waveforms, through clothing. By applying the proposed motion artefact algorithms, a real-world performance was achieved better than the previous state of the art. If desired for a specific application, the accuracy can be further improved by using a stricter classifier for the motion artefact algorithms, at the expense of discarding a higher fraction of the data. Further work is needed to permit capacitive ECG measurement also when thicker clothing (e.g., winter jacket) is worn.

Capacitive respiration measurement was also demonstrated, but with a lower coverage than for ECG. In future work, its performance is expected to benefit from signal quality algorithms and a multi-electrode array approach similar to ECG.

Proof-of-concept experiments have shown that the proposed radar sensor can provide respiration and, through the application of the proposed motion reduction algorithms, average heart rate during driving. Radar-based measurement of individual beat to beat intervals for heart rate variability remains a challenge during driving conditions. Further work on motion reduction algorithms as well as radar integration into the vehicle is needed to address those challenges.

It is expected that in the end a sensor fusion of both methods will provide the most robust results.