Motion-Tolerant Non-Contact Heart-Rate Measurements from Radar Sensor Fusion

Abstract

Microwave radar technology is very attractive for ubiquitous short-range health monitoring due to its non-contact, see-through, privacy-preserving and safe features compared to the competing remote technologies such as optics. The possibility of radar-based approaches for breathing and cardiac sensing was demonstrated a few decades ago. However, investigation regarding the robustness of radar-based vital-sign monitoring (VSM) is not available in the current radar literature. In this paper, we aim to close this gap by presenting an extensive experimental study of vital-sign radar approach. We consider diversity in test subjects, fitness levels, poses/postures, and, more importantly, random body movement (RBM) in the study. We discuss some new insights that lead to robust radar heart-rate (HR) measurements. A novel active motion cancellation signal-processing technique is introduced, exploiting dual ultra-wideband (UWB) radar system for motion-tolerant HR measurements. Additionally, we propose a spectral pruning routine to enhance HR estimation performance. We validate the proposed method theoretically and experimentally. Totally, we record and analyze about 3500 s of radar measurements from multiple human subjects.

1. Introduction

Vital-sign monitoring (VSM) devices are extremely important for human healthcare and wellness, whether it be consumer-grade devices that promote self-health monitoring or medical-grade devices that aid in early diagnosis and facilitate treatment. Remote sensing using radar is one of such technologies that supports non-contact vital-sign measurements [1,2,3]. Microwave ultra-wideband (UWB) radar systems have good penetrative capability and range resolution, which enables them to non-invasively monitor internal physiological motion of the organs of a body, such as the heart or lungs, by transmitting low-energy electromagnetic waves. Thus, such radar systems can extract the heart-rate (HR) and breathing rate (BR) of a subject remotely from a distance.

Radar-based health monitoring devices have a myriad of potential applications, such as infant monitoring [4], sleep monitoring [5,6,7], elder care [8], and animal care [9]. The non-contact feature of radar makes it extremely useful for healthcare applications, such as remote patient monitoring, and enabling a more comfortable and efficient caregiving. A non-contact way of measuring vital signs decreases the risk of infection for healthcare professionals, and thus reduces the risk of any contagious virus transmission. This cannot only be beneficial in the current coronavirus disease 2019 pandemic scenario [10], but also in the long run for general patient monitoring and rehabilitation care.

Recently, there has been a strong increase in demand for consumer-grade and medical-grade VSM devices. A significant portion of current vital-sign sensors are either wearable-based or use optical sensors such as cameras. These approaches have drawbacks such as requiring constant contact with the subject or a lack of privacy [11]. The optical sensors based on remote imaging photoplethysmography (PPG) [12] are a competing non-contact HR monitoring technology for radar approach. HR can be extracted using a low-cost color camera, such as a webcam, by processing a sequence of recorded video images. However, the camera approach is subjective to skin-tones [13], motion artifacts [14,15] and lighting condition [16,17]. Accordingly, significant efforts are investigated on imaging-based robust methods for PPG, such as algorithmic development [18,19,20] and exploration of non-visible light waves [21]. Besides raising privacy issues, the most obvious operational disadvantage of remote imaging PPG measurement is that optical sensors in general have poor penetrative capability. They do not penetrate clothes and blanket, which is required for non-interruptive sensing for continuous, long-term monitoring. Thus, optical sensors are limited for measurements from light-of-sight body spots and directly exposed skin areas, such as hand and face. Alternatively, microwave radar technology presents a very attractive option for ubiquitous short-range VSM due to its non-contact, see-through, and privacy-preserving features. Vast frequency bands have been explored using radar electronics for VSM, ranging from a few gigahertz (GHz) in low frequency band [1], tens of GHz in millimeter wave band [22,23], and hundreds of GHz in Terahertz wave band [24].

To date, most work on non-contact radar-based VSM sensors has focused on monitoring a single human stationary subject. Multiple human subjects in ideal situation were considered in these works [25,26,27,28]. Often, the subjects would typically be instructed to remain motionless by seating still or lying down, and would be in a quiet and controlled environment. This is because a small random body motion (a few centimeters) is often stronger than breathing-induced (millimeter to centimeter) and heartbeat-induced (much less than millimeter) physiological motions, potentially overwhelming the desired signal. A few attempts [29,30,31,32,33] have been made to cope with random body movement (RBM) noise via differential measurement/processing, such as multiple radars at different body sides [30], radar and camera fusion [33], multi-frequency radar systems [34], sensing system exploiting auxiliary sensors attached to the human body [32]. In particular, in reference [29] two continuous wave radars were placed on opposite sides of a person to cancel body movement through differential detection.

In this paper, we focus on investigation of motion-tolerant radar method for HR estimation in realistic settings with a dual-UWB impulse radar system. The two UWB radar sensors are strategically placed at the front side of human chest to create differential measurement of a common chest motion and upper body motion, for example, back-and-forth motion (BFM). The UWB sensor towards the left chest captures major heartbeat signal plus other common signals present in the other sensor on the right side of the chest. The directly sampled radio frequency (RF) signals are fused and converted in the complex baseband for spectral analysis. The recovery of useful vital-sign spectral structure by RF differencing operation to suppress BFM and breathing motion is theoretically and experimentally justified. Subsequently, a spectral-based HR pruning technique is proposed to exploit spectral heartbeat harmonics [26,35] when the fundamental heartbeat is masked by motion residual. The proposed method is tested against comprehensive motion-tolerant experiments. Finally, the impact of breathing dynamics and heartbeat dynamics on HR estimation performance are thoroughly discussed.

2. Materials and Methods

2.1. Experimental Setup and Design

The study was approved by the ethics committee of the Arizona State University (approval number: HPR-5-3b). All research was performed in accordance with relevant guidelines and regulations. Three human subjects were included during the study. In particular, the studied subjects have different physical conditions. One evidence is that some subjects have relatively lower resting HR while other subjects relatively higher resting HR. One of the goals of this experimental study is to demonstrate the limitations of conventional radar approaches. The existing methods mostly assume that (1) the testing subjects are stationary and (2) spectral separation is enough for HR calculation. These assumptions are in general not true. These facts are demonstrated through multiple relatively longer-term measurements, which capture involuntary RBM in the prolongated recordings. Additionally, multiple challenging poses are considered. These include stable seating (with back support), free-style seating (without back support), and standing. The second goal of this study is to provide a motion-tolerant radar method with active motion cancellation technique to solve the aforementioned issues. For validation, the proposed algorithm is tested in several carefully designed experiments, in which the test subjects are instructed to move back and forth constantly.

A novel dual-radar system setup is presented as illustrated in Figure 1. Both radars are placed at chest height in the front of the human body but with a tilted angle, 45°, with respect to a center line which is perpendicular to the chest plane. Exploiting spatial diversity with this frontal side dual-radar setup, similar radar channels can be created for RBM cancellation purpose, achieving the goal of improved vital-sign detection. The sensor pointing to the left chest area where the heart chamber is located captures all kinds of body motion including chest motion and heartbeat. As for the other sensor pointing to the right side of the chest, it captures the common body motion, as seen in another radar channel except that it only sees a small portion of heartbeat signal. Due to channel noise, the weak heartbeat signal is negligible in this sensor. Thus, the following assumption is made here that heartbeat signal is only present in one radar channel while the rest of body motions are common to both radar channels.

2.2. Measurement Devices

A fingertip oximeter together with a digital dataloger (NeuLog heart-rate and pulse logger sensor, model NUL-208) was used to collect standard PPG signal as pulse reference from the test subjects. The PPG signal is acquired at a sampling rate of 50 Hz. A chest air pressure belt (NeuLog respiration monitor belt logger sensor, model NUL-236) was used for providing breathing reference. The breathing signal is also acquired at a sampling rate of 50 Hz. The pulse reference device and breathing reference device are synchronized in hardware (Figure 2).

The dual-UWB radar system (Figure 1) consists of two independent radar sensors (Xethru X4M03 development kit) equal distant from the test subject. These radar sensors operate at center frequency 7.3 GHz with effective signal bandwidth 1.4 GHz. On board patch antennas have directional beamwidth about 60° in azimuth and elevation. Each of these sensors employs an UWB impulse signaling scheme and direct RF receiver architecture [36], critical sampling at RF and preserving all the information, including vital-sign signals, carried in the received signal in the digital domain [37]. For spectral masking, the transmitter employs a biphase coding modulation scheme, where the encoding bit sequence is independently generated at each sensor node using a pseudo-random noise pattern [36]. Thus, each sensor only sees its own signal backscattered from the target of interest. The two sensors are physically wired to a control unit and synchronized in time.

2.3. Proposed Signal Processing

The major signal-processing steps are highlighted in Figure 3 to extract human subject’s HR. The proposed signal-processing algorithm can be categorized into five processing blocks, including direct RF signal fusion, RF to complex baseband conversion, range-Doppler analysis to locate the range of interest, spectral peak detection and HR calculation. Each processing block is discussed in the following sections.

2.3.1. Active Motion Cancellation

Direct RF Signal Processing

Active motion cancellation is a key processing setup before any conventional signal processing is applied. With explicit suppression of any large RBMs, long-term heart-rate monitoring is feasible because no human subject can maintain an absolute stationary position for an extended period of time in any relaxed state. Two radar channels provide two slightly different observations from the two strategically positions for sensor fusion operation. The ability to access directly sampled RF samples allows mixing the two copies of RF radar scans (left side $S_L^{RF}$ and right side $S_R^{RF}$) at the same time. The differential measurement by differencing the two RF radar scans (left side $S_L^{RF}$ and right side $S_R^{RF}$) provides a desirable residual signal with enhanced pulse signal, such that the motion artifact and stronger breathing is suppressed prior to apply other signal-processing techniques. The received RF signal is modeled as a nonlinear function ($\pounds \left\{\right\}$) of normalized RBM $R(\tau ,t)$, breathing signal $B(\tau ,t)$ and pulse signal $H(\tau ,t)$ and receiver noise $W(\tau ,t)$. These components are organized in two-dimensional (2-D) matrix format, as follows,

$$\begin{array}{cc}\hfill S_L^{RF}(\tau ,t)& =\pounds \left\{R(\tau ,t),B(\tau ,t),H(\tau ,t),W_L(\tau ,t)\right\}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill S_R^{RF}(\tau ,t)& =\pounds \left\{R(\tau ,t),B(\tau ,t),W_R(\tau ,t)\right\},\hfill \end{array}$$

(2)

where $\tau$ denotes fast-time samples corresponding to the range domain while t denotes slow-time samples corresponding to the temporal domain. The HR calculation is operated along the slow-time samples. By differencing Equations (1) and (2) at $t=t_i$, where i denotes ith slow-time index, we have,

$$\begin{array}{c}\hfill \begin{array}{c}\hfill s_{Diff}^{RF}(\tau ,t_i)=s_L^{RF}(\tau ,t_i)-s_R^{RF}(\tau ,t_i)\end{array}.\end{array}$$

(3)

Baseband Signal Processing

The residual RF signal Equation (3) is digitally downconverted to complex baseband by mixing with a nominal carrier.

$$\begin{array}{c}\hfill s_{Diff}^{BB}({\tau }_j,t)=s_{Diff}^{RF}({\tau }_j,t)e^{-j2\pi f_c\tau }.\end{array}$$

(4)

The common approach is to extract the Doppler phase component from Equation (4), using $p_{Diff}^{BB}({\tau }_j,t)=\mathrm{unwrap}\left[\mathrm{arctangent}\left[s^{BB}({\tau }_j,t)\right]\right]$. “arctangent” denotes arctangent demodulation operating on the ratio of real component and imaginary component of the baseband signal $s^{BB}({\tau }_j,t)$ at the range of interest $d_j=2c{\tau }_j$. Moreover, “unwrap” operator is applied to deal with the wrapping problem when the absolute jumps between consecutive phase samples are greater than or equal to 180°.

However, it is found that the phase-based method does generate stable spectrum over time in the presence of RBMs in this study. Instead of phase extraction, the spectral peak detection for HR calculation is performed on the complex signals (Equation (4)) because the higher-order spectral features of heartbeat in complex signal frequency domain aid in HR calculation when the residual motion interference masks the fundamental heartbeat spectral energy.

2.3.2. Analytical Spectral Analysis of Fused Sensor Signal

The spectral representation of the complex-based signal $s_{Diff}^{BB}({\tau }_j,t)$ is derived to demonstrate that (1) the major frequency components related BFM are suppressed via differencing RF signals and (2) the heartbeat harmonic spectral features are maintained in the complex-based signal model and thus is helpful for rate calculation. For convenience, a simplified motion model is used for characterizing the RBM profile,

$$R\left(t\right)=A_R\sin \left(2\pi f_{R}t\right),$$

(5)

which matches the constant BFM considered in this study. $A_R$ is the amplitude of BFM and $f_R$ denotes the cyclic frequency of the activity. Similarly, the respiratory activity and the cardiac activity are defined using the same model but with appropriate amplitude values and frequency values to match their physical characteristics.

$$\begin{array}{cc}\hfill B\left(t\right)& =A_B\sin \left(2\pi f_{B}t\right)\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill H\left(t\right)& =A_H\sin \left(2\pi f_{H}t\right),\hfill \end{array}$$

(7)

where the amplitude values of these three activities follow $A_R\gg A_B\gg A_H$, $f_B$ for normal human subject is around 10 to 20 beats per minute (BPM) and $f_H$ 60 to 100 BPM. For UWB impulse radar, the received pulse is modeled as an attenuated and shifted version of the transmitted pulse $p\left(\tau \right)$. The BFM, breathing motion and heartbeat motion modulate the pulse and create a time-varying delay profile ${\tau }_D\left(t\right)$ around a nominal distance $d_0$ with the associated time-delay ${\tau }_0=2d_0/c$. The backscattered signal from the left chest is given as,

$$\begin{array}{cc}\hfill s_L^{RF}(\tau ,t)& =A_{T}p(\tau -{\tau }_{D,L}\left(t\right))\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\tau }_{D,L}\left(t\right)& =\frac{2\left(d_0+R\left(t\right)+B\left(t\right)+H\left(t\right)\right)}{c},\hfill \end{array}$$

(9)

where $A_T$ denotes the amplitude of the target response and c speed of light. The backscattered signal from the right chest is given as,

$$\begin{array}{cc}\hfill s_R^{RF}(\tau ,t)& =A_{T}p(\tau -{\tau }_{D,R}\left(t\right))\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\tau }_{D,R}\left(t\right)& =\frac{2\left(d_0+R\left(t\right)+B\left(t\right)\right)}{c}.\hfill \end{array}$$

(11)

Accordingly, the differential complex baseband signal $s_{Diff}^{BB}({\tau }_j,t)$ is obtained,

$$\begin{array}{}\mathrm{(12)}& \hfill s_{Diff}^{BB}(\tau ,t)& =s_L^{BB}(\tau ,t)-s_R^{BB}(\tau ,t)\hfill \mathrm{(13)}& & =A_T\left(p(\tau -{\tau }_{D,L}\left(t\right))-p(\tau -{\tau }_{D,R}\left(t\right))\right)e^{-j2\pi f_c\tau }.\hfill \end{array}$$

For convenience, the $s_L^{BB}(\tau ,t)$ and $s_R^{BB}(\tau ,t)$ in Equation (12) are evaluated separately. Through forward and backward Fourier transforms (FT) with respect to t and $\tau$, the FT of $s_L^{BB}(\tau ,t)$ is obtained as (see Appendix A for detailed derivation of $s_L^{BB}(\tau ,f)$),

$$\begin{array}{}\mathrm{(14)}& \begin{array}{l}{s}_L^{BB}(\tau ,f)=A_T{\displaystyle \sum _{k=-\infty }^{\infty }}{\displaystyle \sum _{l=-\infty }^{\infty }}{\displaystyle \sum _{q=-\infty }^{\infty }}\delta (f-k{f}_R-l{f}_B-q{f}_H)\times {\int }_{-\infty }^{\infty }d\nu [P\left(\nu \right)e^{\mathrm{j}2\pi \nu (\tau -{\tau }_0)}\\ \hspace{1em}\hspace{1em}\hspace{1em}\times J_k\left(4\pi \nu \frac{A_R}{c}\right)J_l\left(4\pi \nu \frac{A_B}{c}\right)J_q\left(4\pi \nu \frac{A_H}{c}\right)]\end{array}\mathrm{(15)}& \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=A_T\sum _{k=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }\sum _{q=-\infty }^{\infty }{C}_{L,(k,l,q)}\left(\tau \right)\delta (f-k{f}_R-l{f}_B-q{f}_H),\end{array}$$

where $J_k(.)$ denotes the Bessel function of the first kind [38] and $C_{L,(k,l,q)}\left(\tau \right)$ is given as the following equation and its absolute value achieves the maximum at ${\tau }_0$,

$$C_{L,(k,l,q)}\left(\tau \right)={\int }_{-\infty }^{\infty }d\nu [P\left(\nu \right)e^{\mathrm{j}2\pi \nu (\tau -{\tau }_0)}\times J_k\left(4\pi \nu \frac{A_R}{c}\right)J_l\left(4\pi \nu \frac{A_B}{c})J_q\left(4\pi \nu \frac{A_H}{c}\right)\right],$$

(16)

where $P\left(\nu \right)$ denotes the FT of the transmitted pulse $p\left(\tau \right)$. Then,

$$\begin{array}{cc}\hfill \left|{\mathbf{s}}_L^{BB}(\tau ,f)\right|& \le |A_T|\sum _{k=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }\sum _{q=-\infty }^{\infty }|C_{L,(k,l,q)}\left({\tau }_0\right)|\delta (f-k{f}_R-l{f}_B-q{f}_H)=|{\mathbf{s}}_L^{BB}({\tau }_0,f)|.\hfill \end{array}$$

(17)

On the other hand, the FT of ${\mathbf{s}}_R^{BB}({\tau }_j,t)$ with respect to t is derived in the same fashion,

$$\begin{array}{cc}\hfill s_R^{BB}(\tau ,f)& =A_T{e}^{-\mathrm{j}2\pi f_c{\tau }_0}\sum _{k=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }{C}_{R,(k,l)}\left(\tau \right)\delta (f-k{f}_R-l{f}_B),\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill |s_R^{BB}(\tau ,f)|& \le |A_T|\sum _{k=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }|C_{R,(k,l)}\left({\tau }_0\right)|\delta (f-k{f}_R-l{f}_B)=\left|s_R^{BB}({\tau }_0,f)\right|,\hfill \end{array}$$

(19)

where $C_{R,(k,l)}\left(\tau \right)$ is given as,

$$C_{R,(k,l)}\left(\tau \right)={\int }_{-\infty }^{\infty }d\nu [P\left(\nu \right)e^{\mathrm{j}2\pi \nu (\tau -{\tau }_0)}\times J_k\left(4\pi \nu \frac{A_R}{c}\right)J_l\left(4\pi \nu \frac{A_B}{c})\right].$$

(20)

By inspecting Equations (12), (15) and (19), the FT of ${\mathbf{s}}_{Diff}^{BB}({\tau }_j,t)$ with respect to t is approximated as,

$$\begin{array}{cc}\hfill \left|s_{Diff}^{BB}({\tau }_0,f)\right|& \approx |A_T|\sum _{k=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }\sum _{q=-\infty }^{\infty }|C_{L,(k,l,q)}\left({\tau }_0\right)|\delta (f-k{f}_R-l{f}_B-q{f}_H)\hfill \\ & -|A_T|\sum _{k=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }\left|C_{R,(k,l)}\left({\tau }_0\right)\right|\delta (f-k{f}_R-l{f}_B).\hfill \end{array}$$

(21)

The theoretical spectral power representation of the complex baseband signal $s_{Diff}^{BB}({\tau }_0,t)$ consists of weighted impulse pulse train located at frequency locations from combinations of $f_R,f_B$ and $f_H$. To focus on the most relevant spectral harmonics, only consider the orders of harmonics up to the second order in the analytical analysis and consider the positive frequency locations given the spectral symmetry in Equation (21), meaning that $k,l,q$ is either 0 or ±1 or 2 and $\left|k\right|+\left|l\right|+\left|q\right|\le 2$. The corresponding weights $|C_{L,(k,l,q)}|$ and $|C_{R,(k,l)}|$ significantly decrease when the order of harmonics increases (>2). Without loss of generality, let $f_R<f_B<f_H$. Therefore, Equation (21) is expanded out as,

$$\begin{array}{}\mathrm{(22)}& \begin{array}{c}\hfill \begin{array}{cc}\hfill & |{\mathbf{s}}_{Diff}^{BB}({\tau }_0,f)| \approx |A_T\left|\right\{|C_{L,(1,0,0)}\left({\tau }_0\right)|\delta (f-f_R)+ |C_{L,(0,1,0)}\left({\tau }_0\right)|\delta (f-f_B)+\left|C_{L,(0,0,1)}\left({\tau }_0\right)\right|\delta (f-f_H)\hfill \\ \hfill +& |C_{L,(2,0,0)}\left({\tau }_0\right)|\delta (f-2f_R)+ |C_{L,(0,2,0)}\left({\tau }_0\right)|\delta (f-2f_B)+\left|C_{L,(0,0,2)}\left({\tau }_0\right)\right|\delta (f-2f_H)\hfill \\ \hfill +& |C_{L,(1,-1,0)}\left({\tau }_0\right)|\delta (f+f_R-f_B)+|C_{L,(0,-1,1)}\left({\tau }_0\right)|\delta (f+f_B-f_H)+\left|C_{L,(-1,0,1)}\left({\tau }_0\right)\right|\delta (f+f_R-f_H)\hfill \\ \hfill +& |C_{L,(1,1,0)}\left({\tau }_0\right)|\delta (f-f_R-f_B)+|C_{L,(0,1,1)}\left({\tau }_0\right)|\delta (f-f_B-f_H)+\left|C_{L,(1,0,1)}\left({\tau }_0\right)\right|\delta (f-f_R-f_H)\}\hfill \\ \hfill -& |A_T\left|\right\{|C_{R,(1,0)}\left({\tau }_0\right)|\delta (f-f_R)+|C_{R,(0,1)}\left({\tau }_0\right)|\delta (f-f_B)+\left|C_{R,(2,0)}\left({\tau }_0\right)\right|\delta (f-2f_R)\hfill \\ \hfill +& |C_{R,(0,2)}\left({\tau }_0\right)|\delta (f-2f_B)+|C_{R,(-1,1)}\left({\tau }_0\right)|\delta (f+f_R-f_B)+\left|C_{R,(1,1)}\left({\tau }_0\right)\right|\delta (f-f_R-f_B)\}\hfill \end{array}\end{array}\mathrm{(23)}& \begin{array}{cc}\hfill & \approx A_T\left\{\right|C_{L,(1,0,0)}\left({\tau }_0\right)|\delta (f-f_H)+\left|C_{L,(2,0,0)}\left({\tau }_0\right)\right|\delta (f-2f_H)\hfill \\ \hfill +& |C_{L,(0,-1,1)}\left({\tau }_0\right)|\delta (f+f_B-f_H)+\left|C_{L,(-1,0,1)}\left({\tau }_0\right)\right|\delta (f+f_R-f_H)\hfill \\ \hfill +& |C_{L,(0,1,1)}\left({\tau }_0\right)|\delta (f-f_B-f_H)+\left|C_{L,(1,0,1)}\left({\tau }_0\right)\right|\delta (f-f_R-f_H)\}.\hfill \end{array}\end{array}$$

The result in Equation (23) is important since it implies that BFM $R\left(t\right)$ and breathing motion $B\left(t\right)$ is suppressed in the spectral domain and the dominant spectral components are the heartbeat harmonics. The last four terms in Equation (23) representing frequency intermodulations around the fundamental heartbeat frequency, but will not cause ambiguity because the weight of fundamental heartbeat is stronger than the other weights ($|C_{L,(1,0,0)}|>|C_{L,(0,-1,1)}|,|C_{L,(-1,0,1)}|,|C_{L,(0,1,1)}|$ and $|C_{L,(1,0,1)}|$) and more importantly the second-order harmonic of heartbeat ($2f_H$) further way from these intermodulations interference provides additional trace of HR. From Equation (22) to Equation (23), the weighted impulse responses at the common frequencies are cancelled out because of the following assumption,

$$C_{L,(k,l,0)}\left({\tau }_0\right)\approx C_{R,(k,l)}\left({\tau }_0\right).$$

(24)

This approximation is obtained by invoking the mean value theorem from Equation (16),

$$\begin{array}{}\mathrm{(25)}& \hfill C_{L,(p,q,0)}\left({\tau }_0\right)& \approx \Delta f\times P\left(f_c\right)J_k\left(4\pi f_c\frac{A_R}{c}\right)J_l(4\pi f_c\frac{A_B}{c})J_0\left(4\pi f_c\frac{A_H}{c}\right)\hfill \mathrm{(26)}& & \approx \Delta f\times P\left(f_c\right)J_k\left(4\pi f_c\frac{A_R}{c}\right)J_l\left(4\pi f_c\frac{A_R}{c}\right)=C_{R,(p,q)}\left({\tau }_0\right),\hfill \end{array}$$

where $\Delta f$ represents the signal bandwidth. The last term in Equation (25), $J_0(4\pi f_c{A}_H/c)\approx 1$. Given the radar system parameter $f_c=7.3$ GHz and $A_H=$ 0.08 mm [39], $J_0(4\pi f_c{A}_H/c)$ gives a value of 0.99986.

2.4. Spectral-Based Heart-Rate Calculation Algorithm

It is known that HR calculation from spectral peak selection in radar-based approaches suffers from overwhelming interference of breathing harmonics [35]. One typical such radar measurement is shown in Figure 4. The fundamental HR location overlaps with the higher-order breathing harmonic (5th in this example), which makes it challenging to calculate HR over time without external reference (Figure 4b). Therefore, the heartbeat spectral harmonic features are exploited to improve the detection robustness and help to identify the fundamental HR.

When RBM is not present, the received signal is dominated by the breathing signal and the pulse signal is often found at least 10-dB to 20-dB weaker than the breathing signal. The pulse signal in the temporal waveform is overwhelmed as seen in Figure 4a, and thus the spectral-based peak detection algorithm is commonly used. However, herein the scenario of interest is HR recovery when constant RBM occurs (Figure 5a,b). Directly applying conventional signal processing renders failure without explicit motion cancellation (Figure 5d,e). On the other hand, the proposed HR calculation is operated on RBM suppressed input signals, it is possible to reuse these existing signal-processing steps for rate calculation, as demonstrated in Figure 5f.

Spectral Peak Pruning Routine

Meaningful vital-sign spectrum is revealed with the aforementioned active motion cancellation technique (Figure 5f) when BFM occurs. Afterwards, a spectral peak pruning routine is proposed to provide consistent HR estimate. Based on common knowledge, a heartbeat bandpass filter is first applied. Please note that this frequency range should be tuned matching the testing human subject’s resting HR. Then, this range is divided into N equally spaced intervals to recover the fundamental HR and up to the Nth heartbeat harmonic. Within each frequency interval, the C most significant peaks are recorded. The cth peak location in the nth frequency interval, $Lo{c}_c^N$, is normalized relative to the order of harmonic n, and n = 1, …, N. This is because the HR harmonics are multiple of the fundamental HR.

$$e_c^n=\frac{{Loc}_c^n}{n},$$

(27)

where c denotes the cth largest peak and c = 1, …, C. A set $\mathcal{E}$ of potential HR estimates is obtained,

$$\widehat{hr}\in \mathcal{E}=\left\{e_1^1,\dots ,e_c^1,\dots ,e_C^1,|\dots \dots |,e_1^n,\dots ,e_c^n,\dots ,e_C^n,|\dots \dots |,e_1^N,\dots ,e_c^N,\dots ,e_C^N\right\}.$$

(28)

The final HR estimate is chosen from the set $\mathcal{E}$ using a majority vote strategy. The HR estimate is calculated as,

$$\widehat{hr}=\{\mathrm{mean}[\sum _{n_{*}}{e}_{c_{n_{*}}}^{n_{*}}];\underset{n_{*}}{argmax}\lambda \},$$

(29)

where $n_{*}$ belongs to a subgroup from $n=1,\dots ,N$ such that they ($e_{c_{n_{*}}}^{n_{*}}$) yields the most common votes $\lambda$, where $0\le \lambda \le \left(\genfrac{}{}{0pt}{}{N}{2}\right)$. A common vote is achieved when,

$$|e_{c_1}^{n_1}-e_{c_2}^{n_2}| <=\eta ,\mathrm{when}{n}_1\ne n_2,$$

(30)

where $c_1$ or $c_2$ = 1, …, C and $n_1$ or $n_1$ = 1, …, N. $\eta$ is a consensus tolerance parameter and is set to 2 BPM empirically. If no consensus ($\lambda$ = 0) is found using Equation (30) in the set $\mathcal{E}$, then the testing estimate that has smallest difference to the previous HR estimate is selected.

$$\widehat{hr}=\{e_{\mathring{c}}^{\mathring{n}};\underset{\mathring{c},\mathring{n}}{argmin}|e_c^n-{\widehat{hr}}^{prv}|\}.$$

(31)

The proposed spectral peak pruning routing is summarized in Algorithm 1.

Algorithm 1: Spectral-Based Peak Selection for Heart-Rate Calculation

2.5. Evaluation Metrics

Visual inspection and quantitative measures of the experimental results are used to justify the proposed radar algorithm. The RF waveform front single sensor and dual-sensor are compared to show the effect of RF fusion for motion cancellation (Figure 5a–c). Inspection of the corresponding the vital-sign spectra indicates the HR recovery performance (Figure 5d–i). For quantitative measures, HR error statistics are computed and displayed in terms of the empirical cumulative distribution function (CDF) of HR estimation errors (

Table 1. Summary of HR Estimation Error Statistics (BPM) in Various Scenarios.

	Subject 1		Subject 2		Subject 3
Algorithms	Proposed (RMSE, STD)	CSD	Proposed	CSD	Proposed	CSD
Stable Seating	1.05, 0.59	1.27, 0.92	1.38, 0.90	1.25, 0.88	1.55, 1.03	2.34, 1.03
Free-Style Seating	2.49, 1.60	3.98, 3.13	2.66, 1.80	4.59, 3.71	2.72, 2.02	3.68, 3.55
Standing	2.90, 2.20	9.71, 4.53	2.27, 1.80	10.30, 4.44	3.23, 1.98	9.08, 4.62
Seating + BFMs	3.89, 2.26	12.70, 4.72	4.16, 2.51	13.44, 4.56	3.74, 3.45	8.21, 5.02
Standing + BFMs	6.88, 5.26	11.25, 5.39	8.22, 4.98	15.21, 6.11	7.55, 6.22	13.46, 5.87

). The calculated empirical CDFs is used to compare different algorithms (Figure 6 and Figure 7). For visual inspection, the HR calculated with the proposed radar signal processing is compared against the reference HR from the acquired PPG signals (Figure 8 and Figure 9).

3. Motion-Tolerance Demonstrations

Two sets of studies are conducted for HR estimation error analysis in the presence of motion artifacts. The involuntary body motions from various poses and controlled BFMs are examined separately. These motion-tolerant experiments are repeated on three human subjects. The corresponding HR estimation error statistics for each subject and each case are summarized in Table 1 in terms of root mean squared error (RMSE) and standard deviation (STD), $RMSE\pm STD$. The HR results are generated with a 20 s sliding window with one sample increment. The number of equally spaced frequency intervals is chosen to be $N=3$ and up to 3rd heartbeat harmonics are used. The graphical representation of HR estimation performance of the subject 1 is displayed in Figure 6 and Figure 7 as an example.

The empirical CDFs of HR estimation error from stable seating, free-style seating and standing are shown in Figure 6. The proposed method includes active motion cancellation to recover meaningful spectrum at the frequency of interest and consequently the spectral peak pruning routine for consistent rate calculation. The complex signal demodulation method (CSD) [40,41] is added to the HR estimation performance comparison, in which only one sensor data from the left side is used and the dominant spectral peak in the heartbeat frequency region is selected to calculate HR. Three datasets, five minutes each, are used to produce the results in Figure 6a–c. From case in Figure 1a to case in Figure 6c, the stability of human body decreases, and the prolonged recording time captures realistic motion interference. The CSD performance significantly degrades from stable seating to standing as the human body stability decreases. On the other hand, the proposed method only degrades mildly in these cases. The performance gap between the proposed method and the CSD is largest in the standing example while their performance is almost comparable in sable seating example.

Controlled BFMs were introduced to the second set of experiments to challenge the motion-tolerant performance of the proposed algorithms. While the human subjects were seating (without back support) and standing with normal breathing, they were instructed to move back and forth slightly with maximum physical displacement about four centimeters as seen in Figure 7a,b. The accumulated BFM period is at least 60 s of the two minutes experiment time. Two datasets are used to produce Figure 7c,d. In these two challenging cases, the CSD method completely fails as the HR estimation is quite low with $RMSE$ 12.70 BPM and 11.25 BPM. For example, 12% of the time the HR estimation error is less than 5 BPM in seating with BFMs and 22% of the time in standing with BFMs. The proposed method with active motion cancellation is effective with $RMSE$ 3.89 BPM and 6.88 BPM. In addition, about 75% of the errors are within 5 BPM in seating with BFMs and about 67% in standing with BFMs.

4. Novel Breathing and Heartbeat Dynamics Analysis

Single radar platform with insufficient degree of freedoms (limited RF bandwidth and lack of spatial resolution) really limits the signal-processing capability for practical HR monitoring applications. The dual-radar fusion technique is demonstrated to be effective to handle HR monitoring in the presence of BFM to certain extent. However, still a few aspects are not well recognized and investigated. The dynamics of breathing activity makes a significant impact on HR monitoring and result in unreliable HR measurements. Three carefully designed experiments are demonstrated and compared side by side to illustrate this new insight on radar HR sensing. Three different breathing profiles are considered: consistent shallow breathing, varying breathing patterns from normal to fast breathing, consistent deep breathing. The test subject was seated in a stable chair with back support and instructed to perform these informed breathing activities while preventing any other body movements. The extracted breathing patterns from radar measurements are displayed in each subfigure. Their patterns match the pre-designed configuration. The breathing pattern in Figure 8a is regular shallow breathing with smaller amplitude, the breathing pattern in Figure 8b experiences a transition from slow to fast breathing as seen in the increased repetition cycle while the breathing pattern in Figure 8c has larger amplitude and increased breath interval corresponding to deep breathing profile. In the shallow breathing example, the CSD and the fusion-based method generates similar results. Both HR evolutions are consistent with the pulse reference. When breathing fast, the BR estimation increases and the strong breathing harmonics get closer to the fundamental HR. Thus, the CSD method starts deviating from the truth while the proposed method still provides accurate estimates. When breathing motion is exaggerated in the case of deep breathing, the conventional method fails to provide consistent results throughout the experiment. Robust measurement is achieved in the proposed method enabled by active motion cancellation in the RF stage. Note the exaggerated breathing motion is demonstrated as a source of RBMs that challenges the HR monitoring performance and validate our approach.

Another novelty of this study is the investigation of human subject dependent characteristics, the different resting HR among human subjects. HR statistics for normal healthy people is about 60–100 BPM. For robust radar algorithm it should work for most individuals. For example, resting HR for well-trained athletes can be well under 60 BPM and even close to 40 BPM while for some it can be close to 100 BPM. This is an important consideration, as it breaks the original basis of spectral-based HR calculation that HR and BR are further apart and thus separable. In this regard, people with lower resting HR are difficult to detect because the fundamental HR is inside the breathing harmonic frequency region. The following examples validate this new insight. Two test subjects were selected based on their resting HRs: one about 50 BPM and the other one about 90 BPM, both subjects were instructed to breath at 17–20 BPM range to ensure the ambiguity for HR detection for the subject with lower resting HR. For the low resting HR subject, the CSD method and phase-based method constantly underestimate HR due to stronger breathing harmonics however the proposed method can trace the HR over time (Figure 9a). On the contrary, for the high resting HR subject, both methods can perform HR monitoring by comparing against the pulse reference (Figure 9b).

5. Conclusions

A novel motion-tolerant non-contact HR estimation algorithm is presented and demonstrated with a dual-UWB impulse radar system. The active motion cancellation is achieved via direct RF signal fusion from the two radar sensors. The radar sensors are placed at chest height about 60 cm away at equal distance. They are pointing at different sides of the chest to create spatial diversity and differential measurements for enhanced HR detection when RBMs occur. A spectral-based HR peak pruning routing is proposed to deal with the ambiguity issue for detecting HR by exploiting higher-order spectral features of heartbeat. The proposed method is theoretically proved by an analytical analysis of the spectral representation of the differential complex baseband signal model. The developed algorithm is validated through a comprehensive experimental study for three human subjects. HR measurement performance are compared for three different poses, including stable seating, free-style seating and standing. A robustness test is conducted in the presence of BFMs, showing effectiveness of the proposed algorithm.

Additionally, two insights toward robust HR sensing using radar sensors are introduced and discussed. Radar sensors are motion-based measurement and thus they are sensitive to heartbeat and breathing motion. The latter one is stronger and always coupled with heartbeat. Naive spectral separation is not sufficient for differentiating the two activities, which is the basis for most popular methods for radar. In this regard, three different breathing profiles are investigated from suppressed breathing, fast varying breathing and deep breathing, in which the conventional radar signal processing behaves differently and fails to detect the fundamental HR due to stronger breathing motion and nearby breathing harmonics. A robust radar approach for HR is necessitated to monitor people with different resting HR. The HR diversity comes from the fact that people have different physical conditions, emotional states, lifestyles and more. It is shown to be difficult for the conventional radar signal processing to track HR for low resting HR human subject due to the reduced spectral gap between breathing and heartbeat. The proposed theory provides a new perspective on addressing these issues.