1. Introduction
Location-based services (LBSs) have gradually penetrated all aspects of people’s lives and have become an indispensable application technology. Examples of their application include pedestrian navigation or object searching in airports, shopping malls, hospitals, underground parking lots, tourist route planning services, unmanned medical care, intelligent manufacturing, and intelligent meal delivery [
1]. Therefore, LBSs have substantial commercial and economic value.
Reliable LBS cannot be achieved without accurate location information. Currently, the Global Navigation Satellite System (GNSS) in outdoor environments can satisfy all weather navigation requirements. Since satellite signals cannot penetrate buildings, research on indoor navigation methods has become crucial for ensuring seamless indoor and outdoor localization. However, though many indoor localization technologies seem rather similar, they are actually completely different. Indoor localization is greatly affected by the surrounding environment, and more obstacles and intricate environments, including building structures, materials, and various furniture in indoor environments, lead to complex multipath and signal attenuation effects and then cause a decrease in accuracy localization. Meanwhile, the accuracy requirement for indoor localization is much higher than that of outdoor localization.
Researchers have explored various indoor localization techniques, such as wireless-fidelity (Wi-Fi) [
2,
3,
4,
5], Bluetooth (BT) [
6,
7], radio frequency identification (RFID) [
8,
9], ultra-wideband (UWB) [
10,
11,
12,
13], ultrasound [
14], and visible light [
15,
16] technologies. These technologies need some infrastructure facilities, and are susceptible to environmental impacts, such as multipath propagation, signal absorption and building blocking. In contrast to the abovementioned techniques, geomagnetic localization technology [
17,
18] and pedestrian dead reckoning (PDR) [
19,
20] localization technology do not need extra hardware equipment and are less susceptible to the influence of the surrounding environment. Localization methods without infrastructure facilities have been favored by a large number of researchers and consumers and have become an interesting research topic.
Ubiquitous geomagnetic signals are generated by the earth’s geomagnetic field. Ferrogeomagnetic materials such as steer bars, irons, and iron-containing alloys in indoor environments result in geomagnetic abnormalities [
21]. Geomagnetic abnormalities provide obvious clues and are beneficial for indoor localization [
22]. Moreover, geomagnetic sequences can be collected using smartphones and they have high stability when the indoor environment does not change. Among the geomagnetic-based approaches, the dynamic time warping (DTW) algorithm [
23] is a typical method used for geomagnetic localization. Gong et al. [
24] used the DTW algorithm to warp the processed geomagnetic sequence and the known base, combining it with PDR to achieve geomagnetic matching localization. Subbu et al. [
25,
26] used the DTW algorithm to realize localization by matching the similarity of different geomagnetic sequences to solve the influence of different walking speeds on localization. The results showed that the localization accuracy is 2–6 m. Another widespread localization approach based on geomagnetic sequences is the particle filter [
27], which is a non-parametric implementation of Bayesian filtering [
28,
29]. Canciani et al. [
30] proposed an airborne navigation system using magnetic field. A particle filter is utilized to correct the errors in an inertial navigation system. Akai et al. [
31] proposed using the Gaussian model to describe the fingerprint database of the geomagnetic field, and the particle weight, which is determined by the residual between the established model parameters and the fingerprint database, to estimate the optimal location. Xie et al. [
32] proposed using MaLoc to achieve localization, which is based on the proposed augmented particle filter. Experimental results showed that the MaLoc algorithm can achieve 1~2.8 m localization accuracy in a large building with a 72 m × 64 m floor plan.
Pedestrian dead reckoning (PDR) [
19,
20] is a representative indoor localization method. It does not require additional infrastructure and can achieve relative localization in a short time. Kang et al. [
33] solved the hardware problem of using inertial measurement techniques for localization by replacing inertial sensors, which are measurement devices used in traditional PDR methods with smartphones. Poulose et al. [
34] performed a comparison analysis of various sensor fusion techniques for heading estimation to identify the optimal heading direction for PDR. Yao et al. [
35] proposed a robust step length and stride length detection method for smartphones to enhance localization accuracy by modifying the step length estimation model. A modified zero-velocity detection algorithm is designed for step and heading estimations in ref. [
36], which can obtain a more stable localization result.
However, existing geomagnetic indoor localization methods suffer from many problems. In the DTW algorithm, location determination is achieved when the maximum similarity of the geomagnetic magnitude is matched between online geomagnetic data and the fingerprint database in the overall region. However, the geomagnetic magnitudes do not fluctuate greatly without external metal changes. Large positioning errors will occur especially in symmetric and large area environments. The DTW algorithm has significant computational complexity. On the other hand, geomagnetic magnitudes are related to geological characteristics and the external human environment, and are nonlinear and complex problems. The particle filter algorithm can be used to solve various complex problems, such as moving object tracking. However, the particle filter is also severely impacted by motion estimation errors and suffers from the particle degradation problem. Increasing particle numbers can solve these problems, but a significant computational resource is required. Thus, the practical application of particle filters with smartphones is limited. In addition, different locations in symmetric environments may have identical geomagnetic readings, which cause localization ambiguity. Therefore, it is very challenging to construct a localization system using only geomagnetic sequences [
37]. PDR can achieve relative localization, which relies heavily on pre-set initial location. When long distance localization is involved, the cumulative errors and drift errors will gradually increase over time, resulting in poor localization accuracy in practical scenes. Additionally, different pedestrians have different heights, stride lengths, and walking habits, which may lead to errors in step detection and heading direction. Therefore, PDR is used more as an auxiliary optimization algorithm than other as a localization algorithm.
To address the above problems, we proposed a coarse-to-fine indoor localization algorithm based on geomagnetic and inertial data, which mitigates the effects of geomagnetic ambiguity and PDR cumulative errors. We exploited the application to collect geomagnetic sequences and inertial measurement unit (IMU) data using smartphones. A novel deep learning neural network is proposed to achieve indoor localization. The key contributions of this research are summarized as follows:
Multidimensional geomagnetic feature extraction: To facilitate distinctive geomagnetic feature extraction, we propose a multidimensional algorithm developed from spatial and temporal information, which improves the uniqueness of the geomagnetic features.
A novel neural network framework: To enhance the extraction of geomagnetic features and solve the problem of location ambiguity, we propose a novel neural network to realize localization with geomagnetic sequences. The hierarchical structure of this model can extract more unique features from multidimensional geomagnetic sequences and better study the correlation between geomagnetic sequences and locations. The experiments demonstrate that this network can improve localization performance.
A coarse-to-fine localization system: To enhance localization performance, a coarse-to-fine localization system based on geomagnetic signals and IMUs is proposed. The coarse localization is realized with the geomagnetic sequence and inertial data, and the fine localization is realized through the particle filter. We conducted several experiments with heterogeneous devices at three trial paths. The experimental results indicate that the coarse-to-fine localization system can achieve superior accuracy compared to the state-of-the-art algorithms. Through this system, the localization performance can be significantly improved and it can also be utilized in various scenarios.
The rest of the paper is structured as follows:
Section 2 reviews the related works. In
Section 3, the methodology is introduced. Experimental results are demonstrated in
Section 4. In
Section 5, discussion of the experimental results is presented. Finally, conclusion of this paper is stated in
Section 6.
2. Related Works
In recent years, deep learning methods have been used successfully in many academic studies. Wang et al. [
38] proposed DeepML, an indoor localization system based on deep long short-term memory (LSTM), which uses geomagnetic and light sensors to achieve localization. Ashraf Imran et al. [
39] used a deep neural network-based ensemble classifier to realize localization by extracting features of geomagnetic sequences. Liu et al. [
40,
41] proposed ST-Loc, which uses a hierarchical recurrent network to extract geomagnetic features from spatial and temporal representations for localization. Zhang et al. [
42] built an LSTM network to carry out real-time indoor localization and designed a method for preprocessing geomagnetic data based on double sliding window expansion, which had satisfactory localization performance. Jin et al. [
43] proposed a deep learning-based method that fuses the temporal and spatial features of geomagnetic fingerprints to obtain stable and reliable localization results. Lee et al. [
44] proposed a geomagnetic indoor localization method named AMID, which used a deep neural network as a classifier to recognize geomagnetic sequence patterns for localization. Bhattarai et al. [
45] utilized deep recurrent neural networks (DRNNs) to learn the spatial/temporal geomagnetic patterns and capture long-range dependencies in variable geomagnetic input lengths. Li et al. [
46] aimed to achieve localization by implementing a deep residual network (ResNet) to learn the key features from a massive fingerprint image database. Ding et al. [
47] designed a sliding window mechanism to expand the dimension of geomagnetic data and utilized a one-dimensional convolutional neural network and LSTM network to realize localization.
PDR is more commonly employed in combination with other localization algorithms. Zhang et al. [
48] proposed a novel fusing method integrating PDR characteristics and Wi-Fi characteristics and then realized data fusion and localization through LSTM networks. Ciabattoni et al. [
49] proposed a fused localization application that combines real-time model-based PDR techniques with received signal strength indicator analysis of Bluetooth Low Energy (BLE) beacons. Sun et al. [
50] presented fused indoor localization, which integrated pedestrian dead reckoning and geomagnetic localization using the genetic-particle filter algorithm. Tong et al. [
51] focused on integrating a PDR and a UWB-based localization algorithm, which can achieve high accuracy in harsh environments. Yan et al. [
52] proposed a cost-effective and high-efficiency localization algorithm that combines inertial sensors and acoustic signals, which has higher localization accuracy than the state-of-the-art methods. In ref. [
53], a novel fusion indoor localization system, CHAN-IPDR-ILS, based on acoustic signals and PDR was proposed. In this system, an improved step estimation method and a heading direction correction method were introduced to solve the problem of accumulated errors in the PDR algorithm. Qian et al. [
54] proposed a new hybrid method combining PDR and Wi-Fi information, and a modified particle filter algorithm was introduced to improve the precision of the fingerprint matching phase. Tao et al. [
55] introduced a smartphone PDR indoor localization method based on ambient light assistance, which implements PDR with self-correction capability using a light detection algorithm to detect the location of the light source. Sun Meng et al. [
56] constructed a smartphone fusion system with geomagnetic localization and PDR based on the extended Kalman filter, which can achieve localization performance with a mean localization error and a root mean square error of 1.25 m and 1.53 m, respectively. Yu et al. [
57] presented a new method for Wi-Fi/PDR localization fusion based on neural networks, which applied LSTM networks to train the historical motion states of pedestrians to reduce the cumulative error of PDR.
Inspired by the significant success of deep learning in geomagnetic indoor localization, we propose a coarse-to-fine indoor localization method using geomagnetic sequences. Our proposed method has tremendous advantages in the following aspects. First, we propose a multi-feature geomagnetic extraction method, which can extract more features from temporal and spatial information and solve the errors caused by different devices. Then, our novel deep learning neural network can achieve more accurate localization in the forward and backward directions. Finally, fine localization can be achieved via a particle filter with fusion PDR localization. In addition, the geomagnetic and IMU data can be acquired through a client application installed on the smartphone, which does not require excessive costs.
3. Methodology
The proposed coarse-to-fine localization model is elaborated in
Section 3.1. In
Section 3.2, we illustrate the multi-feature geomagnetic extraction method. Then, the hierarchical deep learning neural network is described in
Section 3.3. Step length estimation and heading angle estimation are presented in
Section 3.4 and
Section 3.5, respectively.
3.1. Coarse-to-Fine Localization Model
In this section, we introduce the proposed coarse-to-fine localization method, which contains data acquisition and preprocessing, coarse localization using deep learning-based geomagnetic localization and PDR localization, and fine location determination. The overall structure of the coarse-to-fine localization method is presented in
Figure 1.
In data acquisition and preprocessing, it is necessary to plan test scenarios and data acquisition paths. The pedestrian holds a smartphone to collect the geomagnetic sequences, acceleration, gyroscope, and orientation data through a preinstalled application software. The crowdsourcing method can realize fast and low-cost data in the planned region. Afterward, these data will be sent to the server in a *.txt format and preprocessed. The cubic spline interpolation method is used to construct a geomagnetic fingerprint database.
In coarse localization, we can gain the coarse location using deep learning and PDR methods. Deep learning-based geomagnetic localization consists of an offline phase and an online phase. In the offline phase, a multidimensional geomagnetic feature extraction method is used to extract the spatial and temporal information of geomagnetic sequences. Then, a multidimensional geomagnetic fingerprint database is constructed. In the online phase, we devise a hierarchical deep neural network model to predict the target location. In PDR localization, we can obtain the target location using inertial sensor data.
In fine localization, location estimation is achieved via a particle filter. The initial location is shown using deep learning-based geomagnetic localization. Then, according to PDR localization, we construct a particle set to test the hypothesis of the target’s state as the posterior distribution. Each particle set includes 100 random particles. The state of the target contains the location and the heading angle. Finally, all particle locations are compared with the geomagnetic localization locations to generate the weights of all particles and thus achieve localization.
In this coarse-to-fine localization model, the particles state
at time
is described as shown in Equation (1):
where
represent the location and heading angle of the particles at time
.
is the weight of particles.
Assuming the deep learning-based geomagnetic estimation as the system observation, the location estimation at time is the system state. During fine localization, the state of each particle is updated in the following steps:
Step 1:
particles are randomly sampled at time
, centered on the location system state
. The particles are characterized as
and the weight of each particle is equivalent. The particles at the next step may be updated using Equation (2):
where
is the step length of the pedestrian at time
t,
is the heading angle at time
t, and
is the Gaussian noise with 0 mean and variance 1.
Step 2: After updating the location of all the particles, the weight of each particle needs to be re-evaluated using Equation (3):
where
is the observation noise covariance matrix.
Step 3: Normalize the weight of each particle using Equation (4):
Step 4: Localization at time
can be achieved using Equation (5):
The procedure of the proposed localization method is presented in Algorithm 1.
Algorithm 1: The coarse-to-fine localization procedure |
Input: The geomagnetic sequence and inertial data from smartphone. Output: The localization of the target. 1: Collect data from smartphone. 2: Geomagnetic preprocessing and segmentation. 3: Multifeature geomagnetic extraction. 4: Geomagnetic localization using deep Learning. 5: Step length estimation of the pedestrian. 6: Coordinate transformation to navigation system. 7: Heading angle estimation. 8: Generate random particles based on location estimation. 9: for each step do 10: for each particle do 11: Resample particle location by Equation (2). 12: Re-evaluate each particle’s weight at time by Equation (3). 13: end for 14: Normalize the particle weights by Equation (4). 15: Resample the particles. 16: Achieve the fine target localization by Equation (5). 17: end for |
3.2. Multi-feature Geomagnetic Extraction
The geomagnetic field is highly affected by indoor man-made constructions, especially those involving ferrogeomagnetic materials such as steel and iron railings. Such anomalies caused by indoor infrastructure are observed to supply more clues at different locations.
To validate the stability of the indoor geomagnetic field over time, we conducted several experiments at a corridor of 32.4 m in length in two different time periods.
Figure 2 shows the three-direction geomagnetic data captured using smartphones on 23 February 2023 and 16 March 2023.
Figure 2a demonstrates the geomagnetic magnitude in the x-direction; the waveform of two dates has the same tendency.
Figure 2b,c show the geomagnetic magnitude in the y-direction and z-direction, respectively. It can be seen that the geomagnetic magnitude varies slightly in different time periods, but the overall trend has significant similarity.
In addition, we have compared the magnitudes and gradients of the geomagnetic data over time.
Figure 3a describes the geomagnetic magnitude of the same region on 23 February 2023 and 16 March 2023, respectively. The results show that the waveform of geomagnetic magnitude maintains great similarity after a period of time.
Figure 3b describes the geomagnetic gradient of the same region on 23 February 2023 and 16 March 2023, respectively. The gradient basically fluctuates around zero. Therefore, the gradient fluctuations are very small. From these two graphs, it can be seen that the geomagnetic field has good stability without external infrastructure changes.
Although the geomagnetic field strength and gradient have changed after a period of time, the fluctuations are less varied and basically negligible. Meanwhile, we calculate the mean and the RMS values of these geomagnetic sequence samples, as shown in
Figure 2 and
Figure 3. It can be seen that these values on 23 February 2023 and 16 March 2023 are very close. Therefore, the geomagnetic sequences exhibit great stability.
In summary, this stable variation feature of the indoor geomagnetic field facilitates indoor localization and endows the smartphone-based geomagnetic indoor localization system with very high potential application. Researchers have studied various works associated with geomagnetic sequence-based localization methods. However, most of them merely use a single raw sequence of geomagnetic values [
58], magnitudes [
26], or gradients [
18] for localization estimation, leading to rough localization accuracy in a large area scene. Among the geomagnetic sequence localizations, the three-direction vector
of geomagnetic sequences retains spatial information, the magnitudes of geomagnetic sequences
retain the geomagnetic characteristics, and the gradient of geomagnetic sequences
can effectively eliminate equipment heterogeneities. To address these issues, we propose a multidimensional geomagnetic feature extraction algorithm. Feature extraction includes data resampling, noise reduction, geomagnetic feature extraction, and normalization.
Geomagnetic data collected by different persons will lead to varying online and offline geomagnetic densities. To effectively match data, we employ the resampling method via upsampling and downsampling to satisfy different velocity targets. Based on the real-time and accuracy requirements of localization, we resample geomagnetic sequences in every step. The step length is estimated based on the work in [
52], which is related to ours and can predict pedestrian states more accurately.
Then, empirical mode decomposition (EMD) [
59] is adopted to reduce the noise generated during geomagnetic data acquisition and conversion. It is an adaptive signal separation method that decomposes nonstationary geomagnetic sequences
into a nonzero mean residue
and several intrinsic mode functions (Imfs), as shown in Equation (6).
where
is the number of Imfs and
is the
th Imf.
After preprocessing the geomagnetic data, the magnitude
and gradient
of the geomagnetic sequences can be obtained using Equation (7). Then, we can construct the geomagnetic vector
, in which
represents the spatial information features and
and
represent the geomagnetic field temporal information.
where
,
,
are the gradients of the geomagnetic field along the
x-,
y-,
z-axis, respectively.
,
,
are the respective unit vectors of the three axes.
The geomagnetic sequence samples are collected in a 32.4
1.9
area. The pedestrian walks along a 32.4 m corridor to collect data with a sampling frequency of 20 Hz. The geomagnetic magnitude of temporal representations is shown in
Figure 4. The arrows represent the pedestrian walking direction, the black dots are geomagnetic samples. Each sample in the figure corresponds to a location in the graph and is also associated with the pedestrian’s walking time. Each location corresponds to a geomagnetic magnitude value. It can be seen that the geomagnetic magnitude in the corridor varies strongly in temporal terms, and the geomagnetic magnitude is clearly differentiated and can be used for indoor localization.
To eliminate the undesirable effects caused by odd sample data, the geomagnetic data need to be normalized into the range [0, 1] with the min-max scaling approach, which is defined in Equation (8). To extract more geomagnetic features for localization, we utilize multiple geomagnetic data as input in the proposed deep learning network.
where
and
are the minimum and maximum values of geomagnetic sequence
s, respectively.
Long geomagnetic sequence data require more computational complexity during localization regardless of whether temporal correlation or fingerprint matching methods are used. Since it is very tedious and time-consuming to extract features from long input sequences, a sliding window with uniform size is adopted to fragment the geomagnetic sequences into several consecutive subsequences. The proposed localization method predicts the target location by matching the corresponding location of each frame in the subsequence.
3.3. Hierarchical Deep Learning Neural Network
In this section, we propose a hierarchical deep learning network model for target localization that can better extract features and learn the correlation between the geomagnetic sequences and location labels. The framework of the deep learning indoor geomagnetic localization is shown in
Figure 5. The proposed deep learning neural network sequentially assembles the LSTM, BiLSTM, dropout layer, and the fully connected (FC) layer.
In the proposed model, the LSTM layer is used to extract anomalous features from multidimensional geomagnetic sequences. To extract more context information of subsequences, the BiLSTM layer is introduced to enhance the extracted feature. Afterward, the last LSTM layer is applied to extract the enhanced features for more accurate prediction. Each layer is backed by a dropout layer with a dropout rate of 0.2, which resets certain values to zero during training and therefore avoids overfitting. Finally, the fully connected layer is applied to output the results. More specifically, the FC network contains three layers: a flattened layer and two dense layers. The first dense layer with 32 neurons is used to handle sparse output by the flattened layer. Then, the predicted value is output through the last dense layer, which has two neuron nodes activated by the sigmoid activation function. In the model training phase, the preprocessed feature data and location data are fed into the hierarchical deep learning network model for training. After completing the training stage, the target’s location can be determined.
Deep learning approaches solve the problem of localization delay and enable more accurate localization by learning more extensive abstractions and high-level features from geomagnetic sequences. Long short-term memory LSTM [
60], an improved recurrent neural network (RNN) with better long-term memory, can efficiently convey and represent information in long sequences without causing useful information from long periods to be forgotten. The forget gate, input gate, output gate and cellular state are used to control the message that will be stored in the LSTM structure. The expressions of LSTM are presented below:
where
,
, and
are the output, input and forget gates, respectively.
is the candidate memory cell, which is determined by combining the old cell state
and the cell input activation vector
.
and
are the outputs of the current neuron and the previous layer, respectively.
represents the input data at time
t.
The loss function of the LSTM network at time
t is defined as
where
are the output and input values of LSTM at time
t, respectively.
is the scale factor.
The gradient
of the output layer is expressed as follows:
The update of weight value
,
, and
at time
t is characterized as follows:
For the weight value
,
can be obtained from the gradient of the loss function at the current and previous times. The gradient
at the current time is obtained as follows:
We can determine the gradient
at time
t − 1 using Equation (15).
The gradient
at time
t − 2 is derived from Equation (16).
For the update of weight value
, all the gradients at the previous time include the gradient
of the cell state. In the LSTM model,
can be inferred in Equation (17). It is determined by the weight values
,
, and
. We can adjust the parameters so that the gradient is in a controllable range.
Similarly, for the parameters and update of the model, we can infer the weight value . Therefore, the model can effectively mitigate gradient explosion and gradient vanishing.
The BiLSTM contains two independent hidden LSTM layers arranged in a chronological and a reverse order. The state equations for the BiLSTM are the same as those mentioned above, except that both forward
and backward hidden states
are used in each time period. The BiLSTM is defined as follows:
where
is the output state of BiLSTM.
BiLSTM can capture the past and future contextual temporal and spatial correlations of the input geomagnetic sequence, keeping more detailed features and reducing the loss of information. Therefore, the geomagnetic sequence features can be effectively captured and extracted.
3.4. Step Length Estimation
The peak detection algorithm estimates the pedestrian step length based on the periodic variation in acceleration during walking. We set an acceleration threshold and time difference threshold to avoid invalid peaks. It is specified as follows:
First, assume , and are the accelerometer values at times t, t − 1, and t + 1, respectively. When is the maximum value among the values , then is labeled as the peak value. Otherwise, we will forward the time t + 1 and return this step.
Then, if the peak value is greater than the threshold , the peak value will be retained and transferred to the next step. Otherwise, we will return to the first step.
Finally, if the time difference between the current and the previous peak value is larger than the time threshold , then the peak is a valid peak.
The methods of step length estimation include mainly the constant model [
61], linear model [
62], and nonlinear model [
63]. The most representative nonlinear model is the Weinberg [
64] algorithm. These typical models are as follows:
Weinberg algorithm:
where
d and
are the walking distance and step number, respectively.
,
and
are model constants.
is the step frequency, and
and
are the maximal and minimal values of the acceleration in the vertical direction at each step, respectively.
The step length cannot always be constant due to the fluctuation of the pedestrians. In addition, different pedestrians have different step frequencies at different times. Therefore, the linear model and the constant model cannot accurately describe the variation in step length and meet the application requirements. The nonlinear model uses the correlation between the step length and acceleration data to estimate the step length, which is more in line with the actual pedestrian state. An improved nonlinear step length estimation method, which is our previous work [
52], can provide a more accurate pedestrian step length, as shown below:
where
and
are the former two-step lengths.
K is the model constant,
is the weight,
and
are the maximal and minimal values of the acceleration at the
i-th step, respectively, and
is the accelerometer compensation measured at stationary time.
We conducted experiments with the constant, linear, Weinberg, and Yan methods at three trial paths. Trial path 1 is 92 m long with fire hydrants, iron railings, and other iron substances. Trial path 2 is 158 m long with iron, steel, and ferrous substances. Trial path 3 is 72 m long with iron frames and iron bars.
Figure 6 shows the step length estimation of these methods when the pedestrian walks at a speed of 0.6 m per step. The experimental results demonstrate that our previous method can achieve more accurate step length estimation.
3.5. Heading Angle Estimation
During the data collection process, all sensor data are located in the built-in coordinate system of the smartphone. However, in indoor localization, a navigation coordinate system is needed. Therefore, it is necessary to transfer the coordinate system for heading direction estimation. We adopt the quaternion method, which converts the target from the phone coordinate system to the navigation coordinate system, as shown in Equation (23).
where
denotes the normalized posture quaternion and
is the scalar part of the quaternion.
are the vector parts.
The heading angle is expressed in Equation (24).
where
are the yaw, pitch and roll angle, respectively.
is the arctangent function, and
is the arcsine function.
4. Results
In
Section 4.1, we first introduce the experimental setting. The verification and analysis are presented in
Section 4.2. Finally,
Section 4.3 demonstrates the localization performance.
To evaluate the performance of our proposed localization method, we conducted numerous experiments and compared the proposed method with the state-of-the-art localization approaches in two experimental scenarios.
Figure 7 illustrates the floor plans of these two sites, a narrow corridor of office area in the library and a large-scale scenario of the gym, with dimensions of 34 × 17.2 × 4
and 56 × 35 × 10
, respectively. In
Figure 7, the red dashed line represents the object movement trajectory. In
Figure 7a,b, the planned paths are closed rectangular; the movement trajectory follows the red solid arrow. The start and end points are depicted in
Figure 7a,b. In
Figure 7c, the movement trajectory of the object is along the red dashed line with different start point and end point. The object moves along the solid red arrow. Trial path 1, which is 92 m long, is in a narrow corridor in the library with several fire hydrants and iron-containing substances at fixed locations. Trial path 2, which is 158 m long, contains more iron, steel, and ferrous material in the gymnasium. There are a few iron frames, ferrous substances, and iron bars in trial path 3 which is 72 m long.
4.1. Experimental Setting
The proposed coarse-to-fine localization system based on geomagnetic sequences and inertial data developed in this paper contains an application terminal and a server terminal.
The accelerometer, gyroscope, and magnetometer data are collected through smartphones wherein the application is preinstalled while the pedestrian walks along the planned path. In the three paths, the pedestrian is required to collect the data along the designed paths several times. In the experiments, this sampling frequency is 20 Hz. The crowdsourcing method is adopted to collect the geomagnetic signal strength and inertial sensor data.
A workstation acting as a server with the Windows 10 64-bit operating system is required to carry out the experiments in this paper. The CPU is an Intel Core i7-9750H with a base frequency of 3.6 GHz. The machine learning framework TensorFlow 2.9.0 with the deep learning library Keras 2.9.0 and Python 3.9.0 are utilized in the experiments.
We conducted a survey on smartphone usage in our university, which found that Vivo and Huawei are the common domestic phone models. Therefore, the Vivo X30 and Huawei Mate30 are selected in the experiments. The technical information of the mobile phones is presented in
Table 1.
4.2. Verification and Analysis
4.2.1. Localization Performance with Different Geomagnetic Sequence Lengths
To validate the localization performance with different magnetic lengths, we invite a volunteer from the local university to collect the data along the planned path at a normal speed with both Vivo X30 and Huawei Mate30 several times. We set geomagnetic sequence samples of 100 to 500 with a step size of 100 in trial paths 1 and 2 and geomagnetic sequence samples of 50 to 250 with a step size of 50 in trial path 3 to conduct the experiments.
Figure 8 shows the mean localization error with different geomagnetic sequence lengths in the three experimental paths. In
Figure 8a,b, the mean localization error is minimal when the geomagnetic samples are 200. When the geomagnetic samples exceed 200, the error becomes steady. In
Figure 8c, when the geomagnetic samples exceed 100, the decrease in error slows down. As can be seen in
Figure 8, the localization error decreases as the geomagnetic sequence samples increases. This is because more geomagnetic samples cover longer path with more unique geomagnetic features. Therefore, the proposed hierarchical deep neural network can learn more location clues from these unique features. However, longer geomagnetic sequences will take more time to achieve localization.
In addition, the time consumption of various geomagnetic sequence lengths with different devices is presented in
Table 2,
Table 3 and
Table 4. From the tables, we can see that in the three experimental paths, the time consumption increases as the sequence length increases. Longer input geomagnetic sequences require more time to achieve localization. Shorter sequences take less time; however, the deficiency of geomagnetic abnormalities will cause high localization error.
Considering the localization error and time overhead, the geomagnetic sequence lengths for the three trial paths are 200.
4.2.2. Localization Performance with Different Hidden Units
We have also evaluated the localization performance on different numbers of hidden units. Hidden units are the dimensional size of the output of a neural network layer; it is also an important factor that affects localization accuracy. In the experiment, the number of hidden units is set from 100 to 500 at a step size of 100.
Figure 9 shows the mean localization error with different numbers of hidden units at three trial paths using Vivo X30 and Huawei Mate30. The experimental results show that the mean localization error fluctuates with the number of hidden units. The mean localization error is minimized when the number of hidden units in all the three experimental paths is 200. In this paper, 200 hidden units for three trial paths are selected.
4.3. Localization Performance
In this section, the localization performances of the proposed method are evaluated with extensive experiments. We have compared the proposed method with the MaLoc method, DTW method, LSTM method, and PDR method in the three trial paths.
Figure 10 demonstrates the mean localization errors of LSTM, PDR, MaLoc, DTW, and the proposed algorithm for different step numbers on the three trial paths using Vivo X30. The experiments show that the proposed algorithm has a lower mean error at different step numbers compared with the state-of-the-art algorithms, and the localization has a more significant comparison, especially when the step numbers increase. This is because our method can gain more features from the temporal and spatial dimensions. In addition, the BiLSTM used in the novel neutral network matches the fingerprint from the forward and reverse order extracts more details, especially for long step numbers, and can effectively eliminate the location ambiguity of magnetic sequences and the accumulation errors of dead reckoning.
Figure 11 shows the mean localization errors with different step numbers using Huawei Mate30. Experimental results demonstrate that our proposed algorithm can also achieve comparable localization accuracy with different devices compared with the state-of-the-art algorithms. This is because the multidimensional feature extraction method can effectively mitigate the errors caused by heterogeneous devices and solve the cumulative error problem in PDR localization.
We present the localization error with Vivo X30 in the three trial paths in
Figure 12. Red symbols are the location anomalies. It shows that the median and maximum values of the proposed algorithm are the smallest compared to the LSTM, PDR, MaLoc, DTW algorithms at all three different trial paths. This is mainly because our proposed algorithm can efficiently extract the geomagnetic features in both spacious and narrow spaces, and thus more accurate localization can be achieved. It can be seen that our proposed algorithm can achieve good localization performance in different scenarios, overcoming the problem of ambiguous geomagnetic localization.
We also compare the localization error at the three trial paths using Huawei Mate30, as shown in
Figure 13. The experiments illustrate that our proposed method achieves comparable localization accuracy with different devices. These results occur because our multidimensional extraction algorithm can obtain enough information to realize localization, eliminating equipment heterogeneity.
Figure 14 presents the cumulative distribution function (CDF) of the localization error with Vivo X30 at the three trial paths. These experiments demonstrate that the proposed algorithm can achieve the best localization performance compared with the typical localization methods. This is mainly due to the following reasons. First, the distinctiveness and differences of features are increased with multidimension geomagnetic sequences. Then, the hierarchical neural networks can effectively estimate the location through the fully connected layer after fine-graining all the features. Finally, the coarse-to-fine localization algorithm can solve the ambiguous location on the basis of geomagnetic estimation and dead reckoning.
We also illustrate the CDF of localization error at the three trial paths using Huawei Mate30, as shown in
Figure 15. It shows that the proposed method achieves considerable accuracy compared to the state-of-the-art methods. This is attributed to the fact that the multidimensional geomagnetic sequence contains spatial and temporal representations, avoiding the extra effort needed to calibrate different devices to a uniform standard.
Table 5 shows the localization error of the 75th and 90th percentiles at the three experimental paths with both Vivo X30 and Huawei Mate30. The results demonstrate that our proposed method can achieve better accuracy than the other state-of-the-art methods. This is because our localization system can effectively extract the location information, thus eliminating the outliers caused by magnetic localization and PDR localization. Our method has great universality and feasibility in different trajectories and equipment.
Table 6 displays the mean errors and root mean square error (RMSE) of different localization algorithms with Vivo X30 at the three experimental paths. The results demonstrate that the localization performance of the proposed algorithm has significantly improved in comparison with the other schemes in the trial sites. The proposed localization is an efficient way to obtain sufficient information, using multidimensional geomagnetic sequences as location clues; thus, our framework is able to extract more distinctive features for localization.
Table 7 shows the RMSE of different localization algorithms with Huawei Mate30 at the three trial sites. The experiments show that our proposed method markedly improves the localization performance with different devices and trial paths. This is mainly because the proposed algorithm that uses a particle filter effectively eliminates the errors caused by localization. In addition, short subsequences processed using sliding windows reduce the loss of feature information that tends to occur in longer sequences. Consequently, the localization estimation route of the three trial paths fluctuates near the real reference value, and the localization performances are noticeably enhanced.
In addition, we compare the calculation complexity between our proposed algorithm and the DTW algorithm at the three trial paths. In trial path 1, our proposed method takes 312.87 s, and the DTW algorithm takes 1452.21 s. In trial path 2, our proposed method takes 521.64 s, and the DTW algorithm takes 2695.41 s. In trial path 3, our proposed method takes 343.58 s, and the DTW algorithm takes 400.89 s. The experiments illustrate that our coarse-to-fine localization method has a relatively low computational complexity and can satisfy the real-time localization requirements. This is because our proposed method can achieve more accurate localization in a smaller area by fusing the dead reckoning instead of matching the whole area as DTW does.
5. Discussion
In this section, we analyze the experimental results presented in
Section 4.2 and
Section 4.3, respectively. Then, we state the future research direction.
As already demonstrated in
Section 4.2, geomagnetic sequence length and hidden units of neural network are the two factors that affect localization performance. The results in
Section 4.2.1 show that short geomagnetic sequences take less time to achieve localization, but localization errors may be high due to the lack of geomagnetic features. Long geomagnetic sequences may have sufficient features, but may also lead to large localization errors due to the possibility of the presence of same geomagnetic features at different locations. Additionally, long geomagnetic sequences require more time for localization. Considering localization error and time consumption, the suitable geomagnetic sequence length should be within a certain range regardless of scenarios and mobile devices. In
Section 4.2.2,
Figure 9 demonstrates that the localization accuracy is affected by hidden units. Large number of hidden units will lead to high computational complexity and overfitting. Therefore, it is important to choose both an adequate geomagnetic sequence length and number of hidden units.
In
Section 4.3, we demonstrate the localization performance of our proposed algorithm in several aspects, including the localization error of step numbers, the boxplot and CDF of localization error, localization error of the 75th and 90th percentiles, and the mean and RMSE errors.
Figure 10 and
Figure 11 show the mean localization errors of different step numbers with Vivo X30 and Huawei Mate30 on the three trial paths, respectively. It can be seen that our proposed algorithm has the smallest localization error at different step numbers, even when different mobile devices on the same reference path are used.
Figure 12 and
Figure 13 illustrate the boxplot of localization error using Vivo X30 and Huawei Mate30, respectively. From the figures it is evident that our proposed algorithm can achieve considerable localization accuracy with both mobile devices on the same path. It implies that our multidimensional feature extraction method can alleviate the anomalies in the localization process, independent of the paths and mobile devices.
Figure 14 and
Figure 15 demonstrate the CDF of the localization error with Vivo X30 and Huawei Mate30, respectively. From the figures it is seen that when the accumulative probability accuracy is 80%, the localization error of our proposed algorithm with different devices in reference path 1 are 0.69 m and 0.80 m, in reference path 2 are 0.89 m and 0.80 m, in reference path 3 are 0.57 m and 0.76 m, respectively. On the other hand, the localization errors of the other state-of-the-art methods are all above 1 m. This implies that the localization accuracy of our proposed algorithm has significantly improved.
Table 5 presents the localization error of the 75th and 90th percentiles at the three experimental paths with Vivo X30 and Huawei Mate30. It can be noticed that the localization errors of our proposed algorithm at the 75th and 90th percentiles are all less than 1 m, exhibiting great improvement over other algorithms.
Table 6 and
Table 7 show that the mean error and RMS error of our proposed algorithm are less than 0.7 m with different smartphones at all the reference paths. Moreover, we calculate the computation complexity of our proposed algorithm, which takes less time than the DTW algorithm. As can be seen from the above experimental results, our proposed algorithm can achieve better localization performance and robustness than the other state-of-the-art algorithms. The device heterogeneity problem and location ambiguity problem can be effectively solved. Moreover, our proposed algorithm can sufficiently extract geomagnetic features whether it is in a spacious area or a narrow area. Therefore, our proposed algorithm has high generality and practicality.
In future research, we will explore more complicated environments to verify the applicability of our proposed algorithm. More different mobile devices will be adopted to validate robustness. Deep learning models with higher accuracy will also be considered for exploration. Furthermore, since the PDR method still suffers from cumulative and drift errors of long-term localization, methods to improve it for precise indoor localization may be another direction for future research.