Deep Learning-Based Indoor Distance Estimation Scheme Using FMCW Radar

Abstract

In the distance estimation scheme using Frequency-Modulated-Continuous-Wave (FMCW) radar, the frequency difference, which was caused by the time delay of the received signal reflected from the target, is calculated to estimate the distance information of the target. In this paper, we propose a distance estimation scheme exploiting the deep learning technology of artificial neural network to improve the accuracy of distance estimation over the conventional Fast Fourier Transform (FFT) Max value index-based distance estimation scheme. The performance of the proposed scheme is compared with that of the conventional scheme through the experiments evaluating the accuracy of distance estimation. The average estimated distance error of the proposed scheme was 0.069 m, while that of the conventional scheme was 1.9 m.

1. Introduction

Along with the recent fourth industrial revolution, there has been a surge of interest and demand for indoor positioning systems for the prevention of safety accidents for workers and evacuation in the event of building fires, collapses, and disasters. In recent years, the various positioning technologies have been developed for both indoor and outdoor environments [1]. It can be used for prevention of collision, unmanned security, non-contact human detection, healthcare, and safety accident prevention in general location-tracking applications [2]. When indoor workers, such as construction sites, attempt to enter dangerous areas, the indoor positioning system can prevent safety accidents by warning the workers, and if the location information of victims can be obtained in case of the rescue, the rapid rescue and the safe evacuation can be expected in the indoor area. Global Navigation Satellite System is currently used for the location-based services in everyday life, but it has the limitation that precise positioning is impossible in indoor environments. Therefore, the need for an indoor positioning system that can accurately identify pedestrian position in indoor environment is increasing, and the indoor positioning technology is expected to be widely used to create a safe and convenient society such as emergency rescue, prevention of indoor worker safety accidents, and indoor road guidance [1,2].

The indoor positioning systems based on radio signals have been studied with various technologies such as Wi-Fi, Zigbee, RFID, Bluetooth, Ultra-wide Bandwidth (UWB) Radar, Frequency-Modulated-Continuous-Wave (FMCW) Radar, and so on. The lateration method and the fingerprint method are well known as positioning techniques using radio signals. The lateration method is based on ranging estimation and it computes the position of the target with each estimated distance [3]. The received signal strength or the time of flight (TOF) of radio signals is used for ranging estimation [4]. Meanwhile, the fingerprint method uses the characteristics of radio signals such as Wi-Fi, Bluetooth, and Zigbee at a specific point [5]. Furthermore, indoor positioning schemes based on the TOF camera or RGB-Data-based simultaneous localization and mapping are being studied to identify moving object information in dynamic environments [6,7]. While diverse technologies are being studied for indoor positioning systems, radar-based systems are considered because of its high performance in distance estimation. The UWB radars have the disadvantage of limited coverage, even though they can provide high time resolution for estimating distances and locations, compared to the communication-based positioning systems. One of the reasons for limited coverage is that noise and clutter in the indoor environment result in distance estimation distortion, because the transmit power is limited. In indoor environments, random factors such as indoor obstacles or complex structures of buildings, can cause various noises and clutters and result in distance estimation errors [8]. Therefore, the indoor positioning system is actively being researched and developed to reduce the effect of the factors causing distance estimation errors so that accurate distances can be estimated in complex indoor environments.

We propose a distance estimation scheme for indoor positioning system utilizing FMCW Radar with 24 GHz. Various indoor positioning schemes with FMCW radar have been introduced with various frequencies, which require hardware to be calibrated for increasing distance estimation accuracy [9,10]. On the other hand, this paper proposes a scheme for improving distance estimation accuracy with software rather than hardware. In the distance estimation scheme using FMCW radar, the frequency difference that was caused by the time delay of the received signal reflected from the target is calculated to estimate the distance information of the target [11]. In this paper, we propose a distance estimation scheme exploiting the deep learning technology of artificial neural network to improve accuracy over the conventional Fast Fourier Transform (FFT) Max value index-based distance estimation scheme. The performance of the conventional scheme can be degraded severely with noise and clutter in the indoor environment. Since the conventional scheme estimates the distance via Max value index, because of random factors, the incorrect measurement of Max value index can result in a distance estimation error. Another scheme is introduced for estimating distances using inverse tangent in the time domain [12]. It is a scheme for estimating distances by obtaining phase differences using inverse tangent and then converting phase differences into frequency domains. Like the state-of-the-art in the case of FMCW radars, this scheme also has a limitation of being able to estimate distances somewhat inaccurately due to random elements occurring in indoor environments [8,11,12].

Meanwhile, the deep learning method is considered to overcome these limitations [13]. By applying the deep learning technology to the received data from FMCW Radar, it can be classified in terms of different distance to the target even with noise and clutter, because the received data have different patterns at different distances. In the proposed deep learning-based distance estimation scheme, therefore, different patterns can be found in the received data, and thus the accurate distance can be estimated. Experiment results confirmed that the proposed scheme can provide enhanced performance in distance estimation.

2. System Description

2.1. System Overview

Unlike pulse radars, the FMCW radar emits a continuous wave that changes frequency linearly over time. Due to the radio wave delay, the received signal compared to the currently generated signal has a different frequency, which is proportional to the travel time. The FMCW radar shows a serrated signal waveform as shown in Figure 1. The signal transmitted by the radar is reflected back from the target and the received signal is returned with a delay of ∆t for the transmitted signal [11,14].

The distance from the target can be calculated by the time ∆t and the propagation speed c, when the propagation is reflected back from the target. If the distance to the target is R, the signal that is reflected after hitting the target has travelled over the distance of 2R, which is twice the distance R. These are summarized in terms of distance and time, as shown in Equations (1) and (2), respectively. The calculation formula is expressed as follows [11,12,13,14,15]:

$$R=\frac{c\times \Delta t}{2}$$

(1)

$$\Delta t=\frac{2R}{c}$$

(2)

2.2. Conventional Scheme

The conventional distance estimation scheme is to find max value in beat spectrum obtained by FFT of radar receiving signal and to estimate the distance through index of the location where maximum value was detected. By calculating the beat frequency caused by the time delay, the distance can be calculated by means of a formula with the signal cycle and frequency range, as shown in Equations (3) and (4), where $f_b$ is the beat frequency, B is the bandwidth of modulating frequency, R is the distance, T is the period of the signal emitted, and c is the speed of light. The calculation formula is expressed as follows [11,15,16,17,18,19,20]:

$$f_b=\frac{2B}{T}\times \frac{2R}{c}$$

(3)

$$R=\frac{c}{2B}\times f_b\times T$$

(4)

Experiments were conducted to estimate the distance from the target with the FMCW radar after fixing the target at a certain distance to evaluate the accuracy of the distance estimation of the conventional scheme. Figure 2; Figure 3 show the radar and the target used to collect FFT data from the FMCW radar, and further details of the experimental environment are discussed in Section 4. The collected data was used for the conventional distance estimation scheme to calculate the index with Max value among the FFT data and estimate the distance at which the target is located.

Figure 4 shows the mean of the FFT data collected at 1 to 10 m each from the first to the tenth graph, and it can be estimated that the largest part of the graph is where the target is located. The vertical bar indicates the distance at which the target is actually located for each distance, and the circle marked with a red dotted line is the part at which Max value was detected for each distance. This paper considered situations in which there was learning and utilization in a relatively short period of time in the field, so we wanted to estimate distance using relatively little data. The experiments collected 500 data for one test at each distance, which were carried out 10 times from 1 to 10 m, and 5000 data for one test. The process was repeated equally over 4 days, collecting a total of 20,000 data (5000*4 days). The 500 data collected from each distance were collected for approximately 3 min and 30 s (sampling approximately 2.54 times per second) and 10 iterations collected for about 35 min during one experiment. In summary, the data was collected from a distance of about 3 min and 30 s, repeated 10 times for about 35 min, and repeatedly collected after 4 days. Based on the data collected, the average calculation at each distance from 1 to 10 m is shown in Figure 4. The estimated average distance error for each distance at 1 to 10 m is shown in Figure 5, with the estimated overall distance error of approximately 1.9 m.

Other than the previously mentioned conventional FFT Max Index-based distance estimation scheme, another scheme is introduced for estimating distances using inverse tangent in the time domain [12]. The scheme as shown in Formula (5), applies reverse tangent to the I/Q signal, which is the raw data of the radar, to obtain the phase difference ($\varphi$), and then converts the phase difference into a frequency domain. After that, the frequency is divided by 2$\pi$, the average value is obtained, and the Doppler frequency is subtracted to obtain the beat frequency. The distance can be obtained by substituting the finally obtained beat frequency for the Formula (4) [11,12].

$$\varphi ={\tan }^{-1}\frac{Q}{I}$$

(5)

We used the same data as the existing FFT Max Index-based distance estimation scheme and applied it to this scheme, which resulted in an average error of 2.5317 m. As shown in Figure 6 and

Table 1. Estimated distance mean error of reverse tangent-based scheme.

True Distance	1m	2m	3m	4m	5m	6m	7m	8m	9m	10m
Mean error(m)	0.4515	0.7701	0.0144	0.0281	1.2653	3.2012	3.1036	3.8320	4.9379	7.7134

, the estimated average error of distance up to 4 m is 0.316 m, but it can be confirmed that the error rate becomes very severe as the distance increases. Beat frequencies are low in frequency, so they are environmentally sensitive, and due to various factors that can occur indoors, such as the environment and the effects of obstacles, there are severe errors [8,9,10,11,12]. To address the limitations of existing techniques for estimating distances with FMCW Radar, we propose a distance estimation scheme with deep learning techniques in Section 3.

3. Proposed Scheme

A deep learning-based distance estimation scheme is proposed to reduce the error rate of the conventional FFT Max index-based distance estimation scheme. The deep learning is a field of machine learning, and it can classify various data by extracting features from the data. In order to classify data, generally it is necessary to enter what characteristics it has in advance, but the deep learning algorithm does not require a prior process. Once the data is given, it can find the features from the data, and it can judge itself [21,22]. Since the received data have different patterns at different distances, it can be classified in terms of different distances to the target. In the proposed deep learning-based distance estimation scheme, therefore, different patterns can be found in the received data, and thus the accurate distance can be estimated.

The model of the proposed scheme consists of a total of five dense layers, one input layer, three hidden layers, and one output layer as shown in Figure 7. This paper considered the need to respond quickly to prevent safety accidents, so we wanted to achieve maximum performance with as little data while requiring as low complexity as possible. Thus, we considered a network with two, three and four hidden layers. Learning by holding the input and output layers equally and changing the number of hidden layers, the result was that the performance was the highest in learning and testing accuracy when there were three hidden layers [23,24,25].

Since FFT data collected from FMCW radar are collected in an array of 1 × 64, we enter 64 units as input, so layer 1 has 64 units. There are 32 units for layer 2 and layer 3, respectively, 16 units for layer 4, and 10 units for layer 5. Because the last layer (layer 5) estimates the distance from 1 to 10 m, the output consists of 10 units [23]. The activation function determines the output of the neural network, and the activation function of layer 1 to layer 4 is all the rectified linear unit (ReLU), which results in efficient results through simple and faster operation speed. The activation function of layer 5 used a linear function capable of multiple outputs by outputting the input value multiplied by a specific constant to obtain 10 output values [23,26]. We changed the hyper parameters of the model to proceed with the learning process, and set the highest performance value among them. The hyper parameter of the model has a learning rate of 0.001, training epochs 100, and batch size 500 as summarized in

Table 2. Hyperparameter in deep learning model.

	Layers	Learning Rate	Training Epochs	Batch Size
Value	5	0.001	100	500

.

The number of the FFT data of FMCW radars was 20,000 sets and it was used in the conventional scheme referred to in Section 2.2. The same data was used for the proposed deep learning-based distance estimation scheme. For a fair comparison, the same data collected by the FMCW Radar are applied for both the conventional and the proposed scheme to compare the accuracy of distance estimation. The data were divided into training data, validation data and test data, which are with 12,800, 4000, and 3200 sets, respectively. Since training, validation, and testing data need different data, respectively, to obtain accurate results, we divided all the data into training data, validation data, and test data, respectively, as shown in

Table 3. Number of data sets used.

	Total Data	Train Data	Validation Data	Test Data
Number	20000	12800	4000	3200
Ratio	1	0.64	0.2	0.16

. Note that the test data set is used only to evaluate performance without any involvement in learning [27]. The validation set is used to measure the performance of the model by engaging in training, and the test set can evaluate how accurately the distance is estimated by measuring the expected performance of the model without being involved in training. The train and the validation data were applied to the proposed model to train and validate, and the learned model estimated the distance for the test data. The learning accuracy and the loss of the model are shown in Figure 8; Figure 9, respectively, and the test accuracy was approximately 99%.

4. Experiment Results

Experiments were conducted to evaluate the proposed distance estimation scheme using FMCW Radar. The experiments were conducted in the corridor on the sixth floor of general office building of Kwangwoon University. The FMCW Radar was placed at one point and then the distance to the target was varied from 1 m to 10 m. The FMCW Radar was EVALKIT SMR-334 with 24 GHz and the target was a 140 cm × 35 cm metal plate, and the received FMCW Radar signals were 2000 × 64 samples for each distance, and a total of 20,000 × 64 FFT data were collected from 1 to 10 m. As mentioned in Section 2.2, 500 FFT data were collected every 1m between 1 to 10m in one test, and 5000 data (500 × 10 distances) were collected during one test. This process was repeated four times to test and collected a total of 20,000 data (5000 × 4 days). Since FFT data is collected in 1*64 arrays, a total of 20,000 × 64 data is collected.

Figure 10 shows the results of the distance estimation for the predicted data with the proposed scheme in Section 3. As shown in the figure, the distance predicted by the model is indicated with a circle and the estimated mean value of the distance at each distance is indicated with a bar with a circle. The predicted distribution and the mean value for each distance shows that the distance to the target was predicted very accurately, compared with the real distance. The average error of the proposed scheme in estimating distances from 1 to 10 m was approximately 0.069 m, while that of the conventional scheme was about 1.9 m. Therefore, it confirms the superiority of the proposed scheme to the conventional scheme in distance estimation.

Meanwhile, the regression analysis has been performed with the same data, and it results in an average distance estimation error of about 0.1647 m. We also tried to estimate the distance using the Multiclass Common Spatial Patterns (CSP), which is one of the classic machine learning classes [28,29]. The CSP is an algorithm that reveals the differences between signals collected under different conditions, and classification is done by extracting feature vectors based on the largest and smallest eigenvalues in the step of yielding covariance matrix consensus eigen decomposition. As the result of estimating distance with the CSP, the average error in estimating total distance was 0.2923 m. At this time, train data and test data are identical to the dataset used in the model proposed in this paper.

Table 4. Comparison of the average error between the conventional schemes and the proposed scheme.

	Conventional Scheme	Reverse Tangent Scheme	CSP Scheme	Regression Model	Proposed Scheme
Mean error	1.9 m	2.5317 m	0.2923 m	0.1647 m	0.069 m

shows the comparison of the average error for the conventional scheme, reverse tangent scheme, CSP scheme, regression model, and the proposed scheme. As summarized in the table, the proposed scheme based on deep learning outperforms the other schemes.

5. Conclusions

In this paper, we proposed a distance estimation scheme exploiting the deep learning technology of artificial neural network to improve the accuracy of distance estimation over the conventional FFT Max value index-based distance estimation scheme. The performance of the proposed scheme is compared with that of the conventional scheme through the experiments evaluating the accuracy of distance estimation.

The conventional distance estimation scheme calculates the index of Max value among FFT data in FMCW Radar to estimate the distance of the target, and the actual test results showed that the mean error in estimating distances from 1 to 10 m was about 1.9 m. To reduce the error of distance estimation, we proposed a deep learning-based distance estimation scheme. The results of the actual distance estimation experiments showed that the average distance estimation error of the proposed scheme was 0.069 m. By applying the deep learning technology to the received data from FMCW Radar in the proposed scheme, therefore, different patterns can be found in the received data, and thus the accurate distance can be estimated. Experiment results confirmed that the proposed scheme can provide the enhanced performance in distance estimation.

In this paper, we confirm that the distance information of the target can be obtained using FMCW radar, so we can later conduct a study for 2D positioning using two radars. We will later conduct a distance estimation study considering real-world situations in more diverse indoor environments, angles, and locations using two radars.