Radar-Based, Simultaneous Human Presence Detection and Breathing Rate Estimation

Human presence detection is an application that has a growing need in many industries. Hotel room occupancy is critical for electricity and energy conservation. Industrial factories and plants have the same need to know the occupancy status to regulate electricity, lighting, and energy expenditures. In home security there is an obvious necessity to detect human presence inside the residence. For elderly care and healthcare, the system would like to know if the person is sleeping in the room, sitting on a sofa or conversely, is not present. This paper focuses on the problem of detecting presence using only the minute movements of breathing while at the same time estimating the breathing rate, which is the secondary aim of the paper. We extract the suspected breathing signal, and construct its Fourier series (FS) equivalent. Then we employ a generalized likelihood ratio test (GLRT) on the FS signal to determine if it is a breathing pattern or noise. We will show that calculating the GLRT also yields the maximum likelihood (ML) estimator for the breathing rate. We tested this algorithm on sleeping babies as well as conducted experiments on humans aged 12 to 44 sitting on a chair in front of the radar. The results are reported in the sequel.


Introduction
The need for remote human presence detection is growing. Home security systems use cameras and passive infra-red (PIR) sensors to determine if a person approaching or is inside the house. Passive infra-red sensors suffers from many false alarm, but more importantly, they cannot sense minute movements like a still, sleeping person and cameras are not suitable for applications in which privacy is needed like monitoring an elderly in the restroom. Radars have become a readily available solution for the consumer, so in this paper we will focus on the usage of a radar to detect the micro-Doppler effect of breathing, and thereby determine if a person is there or not. The problem of presence sensing is basically an decision problem: is there someone in the radar search volume or not? If the person walks or moves inside the room, a simple moving target indicator (MTI) followed by a constant false alarm rate (CFAR) detector can be employed to detect the walking or movement. However, this method will not work properly on a person who is immobile and only breathing, such as the situation when a person is asleep. Another critical example is baby monitoring applications: inside the car (in-cabin) elimination of the forgotten baby syndrome, in-crib detection of sudden infant death syndrome (SIDS), both relies on accurate, true presence detection.
The topic of human presence detection using radars had been studied before both in the context of moving and stationary subjects. In [1], the authors showed the possibility of presence detection with a FMCW 24 GHz radar and compared it to passive-infrared (PIR) measurements. They utilized an energy detector on the range-Doppler map to decide whether a target is present or not, and did not deal with complete stationary or sleeping targets. The algorithm proposed in [2] is based on calculation of the Doppler power to identify doorway crossing and thus infer on the occupancy of a given room or space. This work also dealt with the problem of walking humans only. WiFi signals were used in [3] for passive-occupancy detection of humans. Though they did not deal with stationary targets, they reported promising results on people counting and presence detection of walking humans. Fourier processing with constant energy threshold detector was utilized in [4], however, stationary human presence accuracy was not reported.
The feasibility of using breathing to detect presence was proven in [5,6], where they showed that a mechanical target which simulates a breathing human can be detected in a room with 93% accuracy by using a Doppler radar with a threshold on the root mean square (RMS) of the received signal, while in [7] a non-adaptive energy threshold detector calculated from pre-recorded noise data was employed to determine if there is one person or two in the radar search volume, both in movement and stationary settings. However, high energy does not mean breathing, or presence. Finally, the usage of convolutional neural networks (CNN) were investigated in [8], where a Doppler radar and and infrared imaging device were jointly employed for presence detection of one human in a specific room. They reported an accuracy of 98.9%. The usage of CNNs bares a complex implementation for real-time purposes, as well as it relies heavily on visual information from the infrared imaging device.
Another application in which sensing the vital signs of a human subject is critical is through-wall and through-debris life sensing. In [9], the detection of vital signs through walls was investigated. They showed the feasibility of detecting the breathing and heartrate of a human subject standing behind a wall, while in [10] they proposed a continuous wave radar architecture for the purpose of detecting vital signs through highly dense construction materials of about 1.5 m thick. In [11], they used empirical mode decomposition to prove feasible the detection of breathing, hand waving and body bending behind an obstacle emulating debris. Through-debris breathing detection was also shown feasible in [12], where they experiment with a debris setup and a live person lying and breathing under it. They showed visual results that the human can be detected, though they did not show how to detect the human in an automated manner. The underlying assumption of the above papers is that the subject is there. The accurate estimation of the breathing frequency is also of importance for many applications, including baby monitoring, elderly care, sleep monitoring and more.
Breathing rate extraction with a pulse-Doppler architecture was presented in [13], where they visually showed feasibility of extracting the breathing rate of a stationary and moving human using Fourier analysis. They employed range-Doppler processing, but there was no outline of how to detect the 'breathing targets' and verify that it is in fact breathing, also the accuracy was not evaluated. The authors in [14,15] used the wavelet transform to overcome the discrete Fourier transform (DFT) resolution insufficiency, and for the same reason the chirp Z transform was used in [16] to estimate breathing rate. The chirp Z transform was also used in [17] coupled with an analytical model for the remote estimation of both breathing and heart-rate. The accuracy of these methods and a comparison against a known bound was not analyzed. More recent work on remote breathing extraction can be found in [18,19], in which breathing was extracted with a radar and a verification that the peak is falling within the breathing band of frequencies was done. While [18,19] reported accuracy results and evaluation of proposed methods, as we show in this paper, we achieve better results by using a maximum likelihood estimator. Furthermore, the verification presented there is not optimal, and does not test adherence to a breathing model, as the detector we present in this paper does.
The need to rely on the breathing movement for human presence detection stems from the fact that the use of a moving target indicator (MTI) [20], often fails to detect stationary humans. MTI is essentially a high-pass filter (HPF) that filters out close to zero-Doppler targets. This HPF caveat is that it most often filters out the breathing, being that breathing frequency is very close to the zero frequency (DC), hence, will fail to detect a sleeping person.
We propose an algorithm that detects presence using the minute movements of the abdomen and torso due to breathing, while at the same time, estimates the breathing frequency. Since detecting moving targets, even targets that slightly shift in their chair, can be done using MTI, we intentionally deal only with strictly stationary subjects.
Thus, this paper's contribution is three fold. First, we present a framework of detecting presence using only the breathing movement. We develop a GLRT detector which, as an input, takes the suspected breathing pattern and as an output decides whether its breathing (presence) or not. Second, a maximum likelihood (ML) estimator of the breathing rate is developed, and shown to asymptotically achieve the Cramer-Rao lower bound (CRB), and lastly, we show that the GLRT detector and ML estimator are the same mathematical expression so we inherently get both with one evaluation. We tested this algorithmic framework performance on various scenarios such as sleeping babies and stationary adults, and results are reported.
In Section 2, we explain the measurement setup, and immediately move to derive both the GLRT detector as well as the ML estimator in Section 3. The estimator is also compared to the CRB. In Section 4, we explain the experiments we have done to verify our algorithms as well as report the results. We discuss future work in Section 5 and we conclude the paper in Section 6.

Measurement Setup
The radar we use is an ultra wide-band (UWB) radar module named X4M300 (Novelda AS, Oslo, Norway) which carries XeThru X4 UWB radar chip. We collect the raw data to the PC through USB. The data are then feeding the algorithm we propose. The radar parameters are depicted in Table 1. The radar manufacturer tested this hardware for the specific application of breathing movement sensing for different angles and ranges. They concluded that even when the torso is 90 degrees rotated towards the radar, the micro-Doppler of the breathing is still present, and can be reliably detected even from 1.8 m away [21].

Radar Operation
The radar is transmitting a pulse with a pulse repetition frequency of 40.5 MHz and receiving the returned pulse. Integration is done on many pulses in order to increase the signal to noise ratio (SNR) so that the output is an integrated pulse, called a frame, every 0.1 s. In general, the return signal represents a superposition of reflected pulses from the environment, including, if present, a breathing target. Each sample of this returned frame represents a range bin or a fast-time bin. Let N rg be the number of range bins. If we wait T seconds in slow time or K frames we will get a slow time vs. fast-time matrix of size K × N rg , in which each row is a radar frame and each column is the change of radar return amplitude over T seconds (and K radar frames) of slow time.
If we know the specific range bin in which the breathing phenomenon is present and extract this column, then we will get a slow time signal that is periodic with a fundamental breathing frequency f b . The method to extract the relevant range-bin is detailed in the next section.

Extracting the Suspected Breathing Signal
Let D l denote the matrix of size K × N rg as described above, l is the current frame index such that the last row in D l is the lth received radar frame and the first row is the (l − K + 1) frame.
The columns of this matrix are then filtered with a HPF that filters out all frequencies below a minimal breathing frequency of 0.15 Hz. Next, each column spectrum is calculated using the fast Fourier transform (FFT), generating a range-Doppler map.
A maximum peak is then searched for inside the range Doppler map, and its corresponding range-bin is declared as the range bin of the target. The respective column out of the matrix D l is extracted and is denoted in this paper as the signal x.
This signal x is then used in to estimate breathing frequency while at the same time to detect the breathing signal, as seen in the next subsections.

Fourier Series Expansion
In this section, we show how we represent the extracted signal using Fourier series (FS), which constructs the detection and estimation problem as a least squares problem. Since the extracted signal is suspected to be a periodic breathing signal with a deterministic period inside the observation window, we can represent it as a FS model where n(k) is a zero mean circularly complex Gaussian r.v. with variance σ 2 , f b is the breathing frequency, θ m is the m'th Fourier coefficient and M is the number of relevant breathing harmonies in the signal. Note that f b is unknown as well as the Fourier coefficients θ m . Arranging (1) in a vector-matrix form to account for k = 0 · · · K − 1 samples we can write , θ = (θ 0 , · · · , θ K−1 ) T and n = [n(0), · · · , n(K − 1)] T , thus, n ∼ CN 0, σ 2 I .

Derivation of the ML Estimator
Define the parameters in (2) as ϑ = f b , θ T T , then the ML estimator of ϑ is given bŷ Note that the matrix H( f b ) is a function of the parameter f b , thus, for the sake of brevity will be referred to as H in the sequel.
Since the ML estimator of the coefficient vector θ for a given f b is given byθ where the last line in (6) is due to the property that a projection matrix P will satisfy P = P 2 = P H . Since the term x H x is independent of the parameter we can writê which can be solved by a line search across valid values of the breathing rate f b ∈ [0.1, 1.0] Hz. We will see in the next subsection that the value of P H x 2 2 evaluated atf b ML is also the GLRT value, so we get both the breathing rate estimation as well as the evaluation of the GLRT test value in one calculation.

Cramér-Rao Lower Bound
The CRB for the estimation of the breathing frequency can be divided to different cases, dependent on which parameters are assumed to be known. In our case, we are only interested in the frequency ω b = 2π f b while the amplitudes and phases are unknown and nuisance. For this case, the CRB for estimating ω b is given by [19,22] where K is the number of samples, the SNR is defined in (9) and the effective bandwidth, BW eff is defined in (10).
BW eff The effective bandwidth is also knows in the literature as r.m.s. bandwidth [23]. As seen in (8)-(10) if more harmonies are presented in the signal, the SNR effective bandwidth increases and the CRB decreases in an inverse relation.
The performance of the estimator in (7) for various SNR values compared to the CRB is depicted in Figure 1, where it is shown that for SNR > −5 dB the estimator meets the bound.

Generalized Likelihood Ratio Test (GLRT)
As derived in Appendix A the test is given by where γ is the test threshold and is computed in our experiments to a constant false alarm rate of P FA < 10 −7 . The calculation of this threshold was done empirically and numerically using Neyman-Pearson's theorem [24] . Monte-Carlo simulation was performed in order to study the separation between hypotheses. Thus, histograms of the detector value, under both hypotheses for various SNRs is shown in Figure 2. As shown, a very good separation is achieved for SNR greater than −5 dB. The Receiver Operating Characteristics (ROC) curve of the test for various SNR values is given in Figure 3. As depicted for SNR = −5 dB we might be able to estimate the breathing frequency and meet the CRB but at the operating point we will have quite a few misses and false-alarms in detection. For lower SNR, the values of the area under the (ROC) curve or AUC is more informative. It can be seen that as expected for very low (∼−20 dB) SNR regime the AUC is about 0.5 meaning there is no separation capability between the hypotheses, but as we go up in SNR and close to −5 dB the near 1 AUC suggest a very good separation between the hypotheses as shown in Figure 4.

Experimental Results
The algorithm was tested in two distinct settings (Experiments approved by IRB number FES-HSES1901): 1.
Baby sleeping. The proposed method was tested on 2 babies across 4 nights, each of which representing an uninterrupted, full night sleep, so total of 8 nights were tested. We placed the radar at half a meter and one meter away from the crib directly facing its long-side as depicted in Figure 5. Both babies were wearing a swaddle, and the babies were moving in the crib so random poses with respect to the radar were presented during the night. Analysis of the SNRs and test distribution are described below. No false alarms or misses were reported.

2.
Adults sitting: The proposed method was tested on nine test subjects, ages 12 to 44, sitting still without moving in front of the sensor, in various distances, breathing. The duration was approx. 60 s and sometimes more per test subject. The purpose of this these experiments was to analyze the errors in breathing rate as reported below. The setup is as seen in Figure 6. The test subject was sitting on the chair, leaning back, sitting as still as they can, looking straight to the radar. The distance was measured both with the radar as well as a measuring tape as seen in the figure.

Efficacy of Breathing Signal Extraction
The efficacy of the breathing signal extraction was discussed in [18]. It was tested on various subjects wearing on their abdomen a Neulog's respiration monitoring Belt logger NUL-236 (Neulog, Rochester, NY, USA) [25], used as ground truth. In [18], the efficacy of the extracted breathing signal was not tested under various distances so we decided to further test it with different radar ranges, from 1 m to 4 m. Table 2 summarizes the  subjects parameters, while Table 3 shows the correlation coefficients of the radar extracted breathing signal to the ground truth. A few examples of the two signals superimposed on each other are shown in Figures 7-9 . We report a maximum correlation of 0.971 and minimum of 0.781.The mean correlation coefficient we get over all distances and subjects is 0.881 which suggests a high efficacy of breathing signal extraction.

Examples of the Test Distribution for Various SNR Regimes
Baby I and II (see Table 4) were recorded sleeping for 4 nights each. We placed the radar at distances of half a meter and a meter away from the crib directly facing its long-side as depicted in Figure 5. The two different scenarios make for a variation in estimated SNR which we can use to visually test the adherence of the test distribution under both hypotheses to the theoretical above. As can be seen in Figures 10 and 11 the graphs resembles the theoretical results depicted in Figure 2 for SNR = 9, 14 dB.  Figure 11. Real example of test hypothesis separation. SNR∼= 9 dB. Extracted from an overnight sleep of Baby II.

Accuracy Results
This section will outline the detector performance and ML estimation accuracy results. In our experiments we took M = 2 harmonies and a time window of 10 s or K = 100 samples. The time window affects the estimation accuracy as per Equation (8). Shortening it will provide less information in the observation window and the estimation accuracy will drop, increasing it will improve it but at the same time increase the risk of losing local stationarity, i.e., breathing rate change inside the time window. Ten seconds is what we chose as a trade-off.

Detector Accuracy
The detector accuracy was evaluated across 8 combined nights of sleeping, split between two baby subjects. The ground truth was a camera synced to the radar. Over these 8 nights no false alarm or misses were reported. The reason for that is we work on a good SNR regime in which the separation between hypotheses is good and the sensitive detector probably needs months of operations to have a false or a miss. Figures 10 and 11 depicts the real, empirical, distribution of the test under both hypotheses. The babies move and change pose throughout the night, thus, the test value has a range of between 14 dB to 33 dB.

ML Estimation Accuracy
The results of the experiments described above performed on the nine subjects in Table 2 are shown in Tables 5-7. The root mean squared error (RMSE) in BPM for different distances are summarized. At times, the subjects were instructed to either breathe normally, or fast, to allow for variations in breathing rate. The table layout the breathing rate, the RMSE in BPM and the percentage of the RMSE with respect to the true breathing rate. The true breathing rate is reported and any rate higher than 20 BPM is considered fast. We report maximal error of 0.170 BPM and maximum percentage of 0.968% of true breathing rate across all experiments and subjects, which is much better than the performance of the algorithm we proposed in [18].

Future Work
The below points were identified for future work.

Sleep Stages Classification
The proposed algorithm is dealing only with presence detection, hence, there is a spread in values of the detector across time as shown in Figures 10 and 11. It is interesting to investigate if the GLRT value along with the breathing rate can be used as a descriptor to classify sleep stages, like awake, deep, light sleep, and Rapid Eye Movement sleep (REM).

Breathing Extraction under Movements
The proposed algorithm is detecting the movement of the breathing, however, when shifting in the chair or in bed, the breathing is "masked" by the bulk movement of the body. While a simple MTI will detect the movement, the breathing estimation during this movement will be wrong, because the movement is not taken into account in the proposed breathing model. Investigation of how to deal with remote breathing extraction under movement is something we plan for the future.

Conclusions
A method for the simultaneous presence detection and breathing rate estimation was presented. The method relies on the detection of minute movements of the torso due to breathing, such that even when the person is sitting still and breathing only, the detector is still effectively able to tell that there is a presence. The algorithm expands the breathing signal using Fourier series, and uses a simple line search for the maximum likelihood estimation of the breathing rate and the detector value simultaneously. We analyzed both the detector's performance as well as the estimator's performance, both in a Monte-Carlo setting as well as real life, and concluded that the estimator meets the CRB and for SNR values greater than −5 dB. Moreover, we report a maximum error of 0.170 BPM for distances up to 4 m. In order to analyze the detector performance in real life, we tested the algorithm on eight full nights recordings of two babies sleeping, reporting zero false alarm and zero misses.
Setting R n = σ 2 I, re-instating omitted the max{·} operation and moving the constant noise variance σ 2 into the RHS to be absorbed in the new threshold γ brings us to the next expression which concludes our derivation.