Hybrid Indoor Positioning System Based on Acoustic Ranging and Wi-Fi Fingerprinting under NLOS Environments

Abstract

An accurate indoor positioning system (IPS) for the public has become an essential function with the fast development of smart city-related applications. The performance of the current IPS is limited by the complex indoor environments, the poor performance of smartphone built-in sensors, and time-varying measurement errors of different location sources. This paper introduces a hybrid indoor positioning system (H-IPS) that combines acoustic ranging, Wi-Fi fingerprinting, and low-cost sensors. This system is designed specifically for large-scale indoor environments with non-line-of-sight (NLOS) conditions. To improve the accuracy in estimating pedestrian motion trajectory, a data and model dual-driven (DMDD) model is proposed to integrate the inertial navigation system (INS) mechanization and the deep learning-based speed estimator. Additionally, a double-weighted K-nearest neighbor matching algorithm enhanced the accuracy of Wi-Fi fingerprinting and scene recognition. The detected scene results were then utilized for NLOS detection and estimation of acoustic ranging results. Finally, an adaptive unscented Kalman filter (AUKF) was developed to provide universal positioning performance, which further improved by the Wi-Fi accuracy indicator and acoustic drift estimator. The experimental results demonstrate that the presented H-IPS achieves precise positioning under NLOS scenes, with meter-level accuracy attainable within the coverage range of acoustic signals.

1. Introduction

The rise of smart city-based applications has brought about a multitude of opportunities for urban environments to become more efficient, sustainable, and people-centric. However, these applications can only be fully realized with the support of precise indoor localization-based services (iLBS). The value of iLBS is becoming increasingly clear in several fields such as intelligent transportation, human activity analysis, COVID-19 contact tracking, and precision marketing.

In the context of intelligent transportation, iLBS can facilitate optimized traffic flow within parking garages, airports, train stations, and other venues [1]. By using indoor localization technologies, it is possible to provide guidance to drivers, find available parking spots, minimize congestion, and reduce emissions caused by unnecessary driving. Moreover, iLBS can enhance the user experience when traveling inside buildings, such as guiding passengers to the right gate or providing real-time information on delays. Human activity analysis is another field in which iLBS can bring significant benefits. With its ability to track and monitor individuals’ movements within buildings, iLBS can be used to improve workplace safety, optimize retail layouts, and enhance security measures [2]. In manufacturing and logistics settings, iLBS can help to ensure that workers are following safe operating procedures, potentially preventing accidents and injuries. In addition, in the retail industry, iLBS can analyze shopping behaviors to optimize store layout, product placement, and ultimately increase sales.

The COVID-19 pandemic highlighted the critical need for iLBS for contact tracking purposes [3]. With the virus being highly contagious, fast and accurate tracing of the contacts of infected individuals is vital to contain outbreaks. Indoor localization technologies have been utilized to track the movements and contacts of people within buildings accurately. In the event of an outbreak, iLBS can provide valuable information to health authorities to notify people who may have been exposed to the virus to take necessary precautions. iLBS has tremendous potential in precision marketing. By using indoor localization technologies, retailers can deliver targeted marketing messages to customers based on their location and preferences [4]. For example, a retailer could send a message to a customer’s smartphone offering a discount on a product they are currently standing near in the store. This personalized approach has been shown to increase conversion rates and drive sales.

At this stage, various indoor positioning systems (IPSs) have been developed to provide indoor location information for users with different precision levels, including Wireless Fidelity (Wi-Fi) [5], Bluetooth Low Energy (BLE) [6], Ultra-wideband (UWB) [7], 5th Generation Mobile Communication Technology (5G) [8], acoustic source [9], and micro-electro-mechanical systems (MEMS) sensors [10]. However, due to the hardware and signal limitations and differences between consumer-level terminals, the single location source cannot meet the positioning requirements of universality, low-cost, and wide-coverage in large-scale and complex urban indoor environments. Thus, a multi-source fusion structure becomes the main solution for realizing an accurate indoor localization system [11,12], which also can be applied in other purposes, for instance, occupants’ movement tracking [13] and environmental monitoring [14].

The Wi-Fi positioning system (WPS) has become a popular choice for indoor localization owing to its wide coverage and low cost. WPS is supported by existing mobile devices and local wireless facilities, making it a universal location source for indoor positioning. Currently, there are two main methods of implementing WPS: distance measurement and fingerprinting.

Distance measurement can be achieved using two primary approaches: received signal strength indicator (RSSI) [15] and round-trip-time (RTT) [16]. The RSSI-based approach, however, suffers from the impact of multipath reflections and non-line-of-sight (NLOS) effects, which limit its accuracy in complex indoor environments. While RTT ranging provides meter-level precision between mobile terminals and Wi-Fi access points (APs), it requires support for the Wi-Fi Fine Time Measurement (FTM) protocol, which is currently only available on a small subset of mobile devices. Additionally, the performance of RTT can degrade in high-traffic scenarios or areas with dense personnel [17].

Fingerprinting algorithms have emerged as an alternative method that does not require knowledge of the position of local wireless stations. Instead, these algorithms acquire RSSI vectors from prior indoor environments to provide location information. Fingerprinting offers universal positioning performance and is not limited by the density of users or hardware support. However, it requires a large database of prior RSSI fingerprints and is affected by the quality of the matching algorithm used. Furthermore, Wi-Fi fingerprinting may struggle to maintain accuracy in open spaces with degenerate features [18].

Overall, while WPS offers several advantages for indoor localization, each approach has its limitations. The RSSI-based approach suffers from multipath and NLOS effects, while the RTT approach requires specialized hardware support and may degrade in high-traffic scenarios. Similarly, while fingerprinting algorithms offer universal positioning performance, they require a large database and may suffer from degraded performance in open spaces with degenerate features.

The acoustic positioning system (APS) is a promising technology for indoor localization that calculates the distance between a signal transmitter and receiver by measuring the propagation time of sound waves. This mode of operation is similar to that of the Global Navigation Satellite System (GNSS) and offers several advantages, including high-precision, good compatibility, and a high-capacity. As such, APS provides new possibilities for indoor positioning using consumer-grade smartphones [19].

One of the main advantages of APS is that it can utilize the built-in microphone of smartphones to achieve high-precision positioning without requiring any additional hardware modifications. This feature makes it a cost-effective solution for indoor localization. The technology has the ability to achieve decimeter-level ranging, sub-meter level positioning, and wide signal coverage, offering superior precision compared to other indoor positioning systems. However, the performance of acoustic ranging is also impacted by factors such as multipath propagation, NLOS, and complex pedestrian motion modes [20]. These factors can lead to errors in the distance calculation and ultimately affect the accuracy of the system. Therefore, further research is needed to develop algorithms that can effectively mitigate the impact of these factors on the performance of the APS.

In summary, the APS is a promising technology for indoor localization that leverages the built-in microphone of smartphones to achieve high-precision ranging and positioning. However, the impact of multipath propagation, NLOS, and complex pedestrian motion modes must be considered when deploying APS-based systems, and future research should focus on developing algorithms to address these challenges. Towards the large-scale indoor spaces, the cost of APS will increase with the area of real-world scenes. In some cases, the combination of different levels of positioning systems becomes an essential requirement. The mobile terminals integrated Micro Electromechanical System (MEMS) sensors are usually adopted to integrate or combine the existing positioning sources to achieve multi-source fusion-based localization purposes [21]. The MEMS sensors-based positioning algorithm usually contains the INS and pedestrian dead reckoning (PDR); both of these methods are easily affected by the cumulative error, the complex handheld modes of smartphones, and the changing motion features of different users [22].

To solve the above challenges, previous researchers have made a lot of effort. Yu et al. [5] developed a hybrid WPS that considers the merits of crowdsourced Wi-Fi fingerprinting, RTT ranging, and MEMS sensors to achieve a 3D indoor positioning function among complex environments and realizes accurate results among different scenes and meter-level accuracy under Wi-Fi RTT-supported indoor scenarios. Xu et al. [11] developed a wide-area navigation frame that uses the combination of BLE, floor plan, sound source, and low-cost sensors, in which a hybrid acoustic signal transmission solution is applied, and sub-meter level positioning accuracy can be applied in real-world scenes. Liu et al. [19] developed a novel time difference of arrival (TDOA) calibration algorithm for ranging drift error estimation by integrating acoustic ranging and inertial sensors data, and the final realized localization precision reached 0.2 m (RMS). Kuang et al. [22] proposed a novel MEMS sensors-based positioning framework by combining INS mechanization with multi-level observations, including the PDR-originated step-length and location results, a quasi-static magnetic vector, and extra acceleration, which effectively improved the performance of the attitude estimation and final location update.

According to the solutions provided by previous researchers, none of them applied both Wi-Fi and acoustic ranging together. Both WPS and APS have their own advantages and disadvantages, and the innovation and necessity of using the combination of Wi-Fi and acoustic for indoor positioning in large and diversified scenes are reflected in the following aspects: (1) the need for ubiquitous navigation, supporting all intelligent terminals, and achieving full-scene coverage. (2) The complementary advantages of Wi-Fi and acoustic signals in terms of spatial distribution and precision indicators, allowing for large-scale and controllable precision positioning. (3) The complementary advantages of Wi-Fi and acoustic signals in terms of signal transmission characteristics. Acoustic positioning is greatly affected by non-line-of-sight and multipath interference; therefore, it is not suitable for densely obstructed small-scale scenes but is more appropriate for open environments. Wi-Fi fingerprint positioning has a relatively high accuracy in situations where non-line-of-sight and multipath interference are strong, but in open areas, signal feature degradation can lead to a lower accuracy. (4) The long-term accuracy of combined Wi-Fi and acoustic positioning can effectively suppress the divergence error of MEMS sensors in multi-source fusion architectures, and when combined with uncertainty analysis models, it can effectively improve the effectiveness of the intelligent fusion algorithm.

To realize a hybrid indoor localization system and comprehensively consider the advantages of each location source, this work presents a hybrid indoor positioning system that uses a combination of acoustic ranging and Wi-Fi fingerprinting (H-IPS), which efficiently integrate the advantages of Wi-Fi fingerprinting, acoustic ranging, and MEMS sensors. The contribution of this paper is described as:

(1) This paper presents a novel data and model dual-driven framework (DMDD) for pedestrian motion information estimation, which combines the INS mechanization with a deep learning-based velocity estimator and can effectively increase the localization precision of MEMS sensors under complex handheld modes and indoor magnetic interference.

(2) This work proposed a double-weighted K-nearest neighbor (DW-KNN) based Wi-Fi fingerprinting algorithm and added the specific scene indexes. The matched location results are further evaluated by a Wi-Fi accuracy indicator (WAI) to provide the adaptive error reference in the final fusion phase.

(3) This paper develops an acoustic drift estimator (ADE) to feedback the drift error caused by the user motion, and the NLOS error ellipse is further built according to the calibrated acoustic ranging distance and scene detection results. The predicted NLOS error is finally applied to provide the acoustic error reference among the integration procedure.

(4) Based on the results of DMDD, DW-KNN, and ADE, H-IPS is proposed. An adaptive unscented filter (AUKF) is developed to combine different location sources together and feedback on the motion parameters in real-time. The WAI and NLOS detection results are further applied to adaptively evaluate the quality of each location source in the case of NLOS contained environments.

The rest of this paper is organized as follows. Section II describes the MEMS sensor-based data and model dual-driven structure. Section III details the improved methods of Wi-Fi and acoustic-based location sources and the overall structure of the H-IPS framework. Section IV describes the experimental results of the proposed H-IPS. Section V will conclude this paper and point out our future work.

2. MEMS Sensor-Based Localization Using the Data and Model Dual-Driven Structure

MEMS sensor-based localization can provide relatively precise short-term positioning performance, while the performance of the existing PDR or the INS mechanizations is constrained by the cumulative error, low sampling rate of the step-length calculation, and the time delay of the location update. This section presents a data and model dual-driven structure for MEMS sensor-based indoor localization, which combines the deep learning-based speed estimator (DLSE) with INS mechanization, and the fused results are further integrated by the AUKF with the Wi-Fi and acoustic-based location sources for the accurate positioning. The overall structure of H-IPS is described in Figure 1.

2.1. Deep Learning-Based Speed Estimation Framework

The performance of traditional pedestrian walking speed estimation models is constrained by the complex handheld modes, motion characteristics of different users, and simple step-length model. To solve these problems, the DLSE is developed to acquire precise walking speed measurements under complex motion modes, which considers motion and handheld features extracted from a time window of the accelerometer and gyroscope data instead of the instantaneous model in formal research. The network structure of the DLSE includes the integration of a one-dimensional convolutional neural network model (1D-CNN), gated recurrent unit (GRU), and multilayer perceptron (MLP), which is shown as follows.

Figure 2 describes the basic model of the proposed DLSE framework, in which the 1D-CNN is developed for fully learning and extracting the motion characteristics of the users, the GRU network further takes the time-related characteristics into consideration, and MLP is finally applied to integrate all the learned features and to predict the walking speed for users.

In the 1D-CNN structure, the mapping model among input features and outputted learned features is presented [2]:

$$O_j=\varpi ({\displaystyle \sum _{i=1}^N{x}_i}\ast k_{ij}+b_j)$$

(1)

where $x_i$ is the modeled input features, $k_{ij}$ is the kernel weights, $b_j$ is the calculated biases, $\varpi (·)$ is the activation function, and $O_j$ is the learned output features of 1D-CNN.

In the GRU layer, the updated model of the GRU parameters is described as [23]:

$$\left\{\begin{array}{l}{z}_t=\sigma \cdot (W_z\cdot [h_{t-1},x_t])\\ {\tilde{h}}_t=\tanh \cdot (W\cdot [r_t\ast h_{t-1},x_t])\\ r_t=\sigma \cdot (W_r\cdot [h_{t-1},x_t])\\ h_t=(1-z_t)\cdot h_{t-1}+z_t\ast {\tilde{h}}_t\end{array}\right.$$

(2)

where $r_t$ and $z_t$ are the forget and output units, $x_t$ is the input vector of GRU model, $h_t$ indicates the hidden state vector, and $\sigma$ is the sigmoid function. The output vector of the proposed GRU network is modeled as the input vector of MLP, and the predicted uncertainty error is presented as:

$$v_t=MLP(h_t)$$

(3)

where ${\mathit{v}}_t$ indicates the final predicted walking speed provided by the proposed DLSE.

Before inputting the inertial sensor data into the neural network, it is necessary to preprocess the data to reduce the complexity of the data so that the network can better fit the pedestrian walking model. Through the 9-axis sensor data, the real-time attitude quaternion is updated by the proposed DMDD. Finally, the acceleration and angular velocity are converted from the sensor coordinate system to the navigation coordinate system by using the attitude quaternion. The vector AG_INS in the sensor coordinate system can be converted into the variable in the navigation coordinate system by Formula (4):

$$A{G}_{INS}=\mathit{q}\times A{G}_{IMU}\times {\mathit{q}}^{-}$$

(4)

where AG_INS is the converted IMU data, which is regarded as the input values of the proposed DLSE.

This paper adopts the sliding window method to improve the output rate of neural network prediction. By using a fixed-size sliding window, the inertial sensor data will be divided into independent sequences. The sequence window size n is 100 frames (1 s), and the sliding window step size is for 10 frames. The velocity vector is predicted by the deep neural network of each sequence, and the position is generated by merging the link module. The current position is updated to the previous 200 frames. The overlapping window is used for the prediction update, and the output frequency will be increased to 20 Hz, and then passed through the low-pass filtering, and the position derived from the velocity vector is further processed by the controller to smoothly reconstruct the predicted trajectory.

2.2. Data and Model Dual-Driven Mode

In this part, a unified data and model dual-driven (DMDD) structure is constructed that contains the integration of the INS mechanization, magnetic vector, and DLSE result, which can realize an accurate trajectory estimation result under complex motion modes and interfered environments. The state vector of presented DMSS is modeled as [22]:

$$\delta \mathit{x}={\left[{(\delta {\mathit{p}}^n)}_{1\times 3} {(\delta {\mathit{v}}^n)}_{1\times 3} {\mathit{\varphi }}_{1\times 3} {({\mathit{\epsilon }}_g^b)}_{1\times 3} {({\mathsf{\nabla }}_a^b)}_{1\times 3}\right]}^T$$

(5)

where $\delta {\mathit{p}}^n$, $\delta {\mathit{v}}^n$, and $\mathit{\varphi }$ indicate the state vector of the position, velocity, and attitude using the n-frame, respectively. ${\mathit{\epsilon }}_g$ and ${\mathsf{\nabla }}_a$ are the biases of the gyroscope and accelerometer, respectively. The relationship of the state update of DMDD is modeled as:

$$\left[\begin{array}{c}\delta {\dot{\mathit{p}}}^n\\ \delta {\dot{\mathit{v}}}^n\\ \dot{\mathit{\varphi }}\\ {\dot{\mathit{\epsilon }}}_g^b\\ {\dot{\mathsf{\nabla }}}_a^b\end{array}\right]=\left[\begin{array}{c}\delta {\mathit{v}}^n\\ {\mathit{f}}_{s\mathit{f}}^n\times \mathit{\varphi }+{\mathit{C}}_b^n({\mathsf{\nabla }}_a^b+{\mathit{w}}_{\nabla })\\ -{\mathit{C}}_b^n({\mathit{\epsilon }}_g^b+{\mathit{w}}_{\epsilon })\\ {\mathit{\epsilon }}_g^b/{\tau }_{bg}+{\mathit{w}}_{\mathit{\epsilon }g}\\ {\mathsf{\nabla }}_a^b/{\tau }_{ba}+{\mathit{w}}_{\nabla a}\end{array}\right]$$

(6)

where ${\mathit{f}}_{s\mathit{f}}^n$ indicates the acceleration vector under n-frame, ${\tau }_{bg}$ and ${\tau }_{ba}$ indicate the correlation time parameters, and ${\mathit{w}}_{\mathit{\epsilon }g}$ and ${\mathit{w}}_{\nabla a}$ represent the state noise of ${\mathit{w}}_{\mathit{\epsilon }}$ and ${\mathit{w}}_{\nabla }$, respectively.

The original INS mechanization is subjected to the cumulative and divergence errors affected by low-cost sensors, thus cannot be applied individually. In the proposed DMDD structure, the DLSE predicted velocity is applied as the observations:

$$\left\{\begin{array}{l}\delta {\mathit{Z}}_{\mathit{v}}^n={\mathit{v}}_{DLSE}^n-{\mathit{v}}_{{}_{INS}}^n\\ \delta {\mathit{Z}}_{\mathit{p}}^n={\mathit{p}}_{DLSE}^n-{\mathit{p}}_{{}_{INS}}^n\end{array}\right.$$

(7)

where ${\mathit{v}}_{DLSE}^n$ and ${\mathit{p}}_{DLSE}^n$ represent DLSE predicted walking speed and location observations, ${\mathit{v}}_{{}_{INS}}^n$ and ${\mathit{p}}_{{}_{INS}}^n$ indicate the walking speed and location calculated by the INS mechanization.

To decrease the effect of the distorted local magnetic field, in this work, the magnetic observation among the recognized quasi-static magnetic field (QSMF) is extracted for modeling the difference vector between the first epoch data [5]:

$$\delta {\mathit{Z}}_{\mathit{m}}^n={\mathit{C}}_{n,k}^b\cdot {\mathit{m}}_k^b-{\mathit{C}}_{n,1}^b\cdot {\mathit{m}}_{k,1}^b$$

(8)

where ${\mathit{C}}_{n,1}^b$ and ${\mathit{m}}_{k,1}^b$ indicate the attitude matrix and magnetic data collected from the first QSMF epoch, respectively, and ${\mathit{C}}_{n,k}^b$ and ${\mathit{m}}_k^b$ indicate the attitude matrix and magnetic data collected from the following QSMF epochs, respectively.

3. Multi-Source Fusion Structure of Wi-Fi/Acoustic/MEMS Sensors

This paper presents a hybrid fusion structure based on Wi-Fi/Acoustic/MEMS sensors for the indoor positioning system (H-IPS). The developed H-IPS considers the advantages of each location source in order to provide indoor location information with different precision coverage.

3.1. Enhanced Wi-Fi Matching and Accuracy Indicator

The proposed H-IPS improves the accuracy of the Wi-Fi fingerprinting approach from two aspects: double-weighted K-nearest neighbor (DW-KNN) is proposed for the database matching algorithm and added the specific scene indexes. The matched location results are further evaluated by a Wi-Fi accuracy indicator (WAI).

Firstly, the weighted Euclidean distance $E_{t,other}$ is calculated based on the intensity of the collected RSSI values and further uses the nearest weighted distances $Di{s}_{t,r_1}$ for the fingerprinting location update procedure:

$$E_{t,other}={\displaystyle \sum _{i=1}^{\alpha }\eta \cdot \left|RS{S}_{scanned}^i-RS{S}_{database}^i\right|}$$

(9)

where $\eta$ is determined according to the relationship between the specific RSS value $RS{S}_{t,other}$ and max value $RS{S}_{t,MAX}$ in the collected RSS vector:

$$\eta =1-RS{S}_{t,other}/RS{S}_{t,MAX}$$

(10)

After acquiring the weighted Euclidean distance of each reference location in the database, the final matched location is calculated by:

$${\mathrm{P}\mathrm{O}\mathrm{S}}^{\prime }(x_r,y_r)=\frac{{\displaystyle \sum _{i=1}^K{\omega }_i^{\gamma }\mathrm{P}\mathrm{O}\mathrm{S}(x_i,y_i)}}{{\displaystyle \sum _{i=1}^K{\omega }_i^{\gamma }}}$$

(11)

where ${\mathrm{P}\mathrm{O}\mathrm{S}}^{\prime }(x_r,y_r)$ indicates the weighted location result, $\mathrm{P}\mathrm{O}\mathrm{S}(x_i,y_i)$ indicates each matched location in database, and ${\omega }_i^{\gamma }$ indicates the related weight value.

Secondly, the performance of matched Wi-Fi locations is further evaluated among the GRU-based Wi-Fi accuracy indicator (WAI) to obtain the positioning error prediction results for the final fusion phase, and the input features are extracted as:

(1) Relative location change during two adjacent Wi-Fi sampling timestamps provided by the accumulation of the calculated step-length $L_{\iota }$ and heading information ${\theta }_{\iota }$:

$$\left\{\begin{array}{l}\mathsf{\Delta }{D}_{MEMS}^x(\iota )={\displaystyle \sum _{\iota =1}^{\gamma }{L}_{\iota }\cdot \sin ({\theta }_{\iota })}\\ \mathsf{\Delta }{D}_{MEMS}^y(\iota )={\displaystyle \sum _{\iota =1}^{\gamma }{L}_{\iota }\cdot \cos ({\theta }_{\iota })}\end{array}\right.$$

(12)

where $\mathsf{\Delta }{D}_{MEMS}^x(\iota )$ and $\mathsf{\Delta }{D}_{MEMS}^y(\iota )$ indicate the increment in the x-axis and y-axis location provided by MEMS sensors.

(2) Euclidean distances during two adjacent Wi-Fi sampling timestamps provided by Wi-Fi and MEMS sensors:

$$\left\{\begin{array}{l}Eu{c}_{\iota }^{MEMS}=\sqrt{{\left({\displaystyle \sum _{\iota =1}^{\gamma }{L}_{\iota }\cdot \cos ({\theta }_{\iota })}\right)}^2+{\left({\displaystyle \sum _{\iota =1}^{\gamma }{L}_{\iota }\cdot sin({\theta }_{\iota })}\right)}^2}\\ Eu{c}_{\iota }^{Wi-Fi}=\sqrt{{\left(x_G^W(\iota )-x_G^W(\iota -1)\right)}^2+{\left(y_G^W(\iota )-y_G^W(\iota -1)\right)}^2}\end{array}\right.$$

(13)

where $\left\{x_G^W(\iota ),y_G^W(\iota )\right\}$ represents the Wi-Fi matching result.

(3) Instant speeds estimated with Wi-Fi and built-in sensors among two Wi-Fi sampling timestamps:

$$\left\{\begin{array}{l}{\mathit{V}}_{\iota }^{MEMS}=Eu{c}_{\iota }^{\mathit{M}E\mathit{M}S}/{\mu }_k^{\mathit{W}i-\mathit{F}i}\\ V_{\iota }^{Wi-Fi}=Eu{c}_{\iota }^{Wi-Fi}/{\mu }_k^{Wi-Fi}\end{array}\right.$$

(14)

where $V_{\iota }^{MEMS}$ and $V_{\iota }^{Wi-Fi}$ are the instant speeds provided by the MEMS sensors and Wi-Fi, respectively. ${\mu }_k^{Wi-Fi}$ is sampling rate of Wi-Fi.

(4) Virtual headings among two Wi-Fi sampling timestamps provided by MEMS sensors and Wi-Fi [5]:

$$\left\{\begin{array}{l}{\vartheta }_{virtual}^{MEMS}(\iota )=\arctan (\frac{{\displaystyle \sum _{\iota =1}^{\gamma }{L}_{\iota }\cdot \mathrm{s}\mathrm{i}\mathrm{n}({\theta }_{\iota })}}{{\displaystyle \sum _{\iota =1}^{\gamma }{L}_{\iota }\cdot \cos ({\theta }_{\iota })}})\\ {\vartheta }_{virtual}^{Wi-Fi}(\iota )=\arctan (\frac{x_G^W(\iota )-x_G^W(\iota -1)}{y_G^W(\iota )-y_G^W(\iota -1)})\end{array}\right.$$

(15)

where ${\vartheta }_{virtual}^{MEMS}(\iota )$ and ${\vartheta }_{virtual}^{Wi-Fi}(\iota )$ represent the low-cost sensors and Wi-Fi originated virtual headings, respectively.

(5) RSSI differences of adjacent Wi-Fi RSSI scanned results:

$$\mathsf{\Delta }\mathit{R}\mathit{S}\mathit{S}\mathit{I}(\iota )=\left\{RSS{I}_{\iota }^1-RSS{I}_{\iota -1}^1,\cdots ,RSS{I}_{\iota }^{\alpha }-RSS{I}_{\iota -1}^{\alpha }\right\}$$

(16)

where $RSS{I}_{\iota }^{\alpha }-RSS{I}_{\iota -1}^{\alpha }$ represents the difference between two adjacent scanned RSSI results from the same Wi-Fi AP.

3.2. Enhanced Acoustic Ranging and Accuracy Indicator

This work proposes the acoustic drift estimator (ADE) to compensate for the acoustic drift error caused by user motion. The frequency shift $f_s$ of acoustic ranging is presented as [19]:

$${\mathit{f}}_s={\mathit{f}}_t-{\mathit{f}}_r=\frac{{\mathit{v}}_{\mathit{p}}}{\varpi }{f}_r$$

(17)

where $f_t$ is the measured signal frequency, $f_r$ is the original signal frequency, $v_p$ is the projected velocity facing the acoustic station, and $\varpi$ is the transmission speed of the sound.

For the time-of-arrival (TOA)-based distance measurement, the calculated drift error is modeled as:

$${\zeta }_s=\varpi \cdot \left(\frac{f_s^i}{F_a^i}\times T_a^i\right)=\frac{f_r^i}{F_a^i}\times (v_p\cdot T_a^i)$$

(18)

where $F_a^i$ is the frequency range of transmitted acoustic signal and $T_a^i$ is the duration of signal transmission. Thus, the final observed acoustic distance is presented as:

$$L_{\mathrm{o}\mathrm{b}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{e}\mathrm{d}}=L_{\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}+{\zeta }_s+d_N+d_{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{o}\mathrm{m}}$$

(19)

where L_observed represents the measured result, which includes the drift error ${\zeta }_s$, NLOS deviation d_N, and measurement error d_random with the Gaussian distribution; $L_{\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}$ is the ground-truth acoustic distance measurement.

To further decrease the NLOS effect of acoustic ranging in real-world scenes, in this work, an acoustic NLOS error evaluation algorithm is proposed based on the detection of loop closure and scene recognition. During the first step, the Wi-Fi fingerprinting result is applied to provide the scene detection result to preliminarily find the NLOS affected stations, and the loop relationship between the current location and two received acoustic stations are described in Figure 3:

$$\left|\overrightarrow{OB}-\overrightarrow{OC}-\overrightarrow{BC}\right|=\mathsf{\Delta }{\zeta }_s^{BC}+d_N+d_{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{o}\mathrm{m}}+d_p$$

(20)

where d_p indicates the approximate error of the current location, and can be estimated according to the loop closure relationship with no NLOS effect:

$$\left|\overrightarrow{OC}-\overrightarrow{OD}-\overrightarrow{CD}\right|=\mathsf{\Delta }{\zeta }_s^{CD}+d_{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{o}\mathrm{m}}+d_p$$

(21)

Under the specific 2D scenario, the misclosure vector can be divided into X and Y coordinates:

$$\left\{\begin{array}{l}{X}_{B\mathit{C}}={\tilde{X}}_{OB}\pm {\tilde{X}}_{O\mathit{C}}\\ Y_{B\mathit{C}}={\tilde{Y}}_{OB}\pm {\tilde{Y}}_{O\mathit{C}}\end{array}\right.$$

(22)

where ${\tilde{X}}_{OB}$ and ${\tilde{Y}}_{OB}$ are the divided coordinate values, and are further applied to calculate the modeled vector $\left({\overline{X}}_{OB},{\overline{Y}}_{OB}\right)$:

$$\left\{\begin{array}{l}{\overline{X}}_{OB}=\frac{{\tilde{X}}_{OB}{L}_{OB}}{{\tilde{E}}_{OB}}\\ {\overline{Y}}_{OB}=\frac{{\tilde{Y}}_{OB}{L}_{OB}}{{\tilde{E}}_{OB}}\end{array}\right.$$

(23)

where $L_{OB}$ is the acquired RTT distance between points O and B at the current position A. ${\tilde{E}}_{OB}$ is the Euclidean distance among location B and current location A. Then, reconstruction of the misclosure vector under the coordinate forms:

$$\left\{\begin{array}{l}{V}_x={\overline{X}}_{AB}\pm {\overline{X}}_{A\mathit{C}}-X_{B\mathit{C}}\\ V_y={\overline{Y}}_{AB}\pm {\overline{Y}}_{A\mathit{C}}-Y_{B\mathit{C}}\end{array}\right.$$

(24)

where $\overrightarrow{V}={[V_x,V_y]}^T$ is the reconstructed misclosure coordinate, and the variance of $\overrightarrow{V}$ is described as:

$$D(\overrightarrow{V})=\left(\begin{array}{cc}{\left(\frac{{\tilde{X}}_{AB}}{{\tilde{R}}_{AB}}+\frac{{\tilde{X}}_{A\mathit{C}}}{{\tilde{R}}_{A\mathit{C}}}\right)}^2& 0\\ 0& {\left(\frac{{\tilde{Y}}_{AB}}{{\tilde{R}}_{AB}}+\frac{{\tilde{Y}}_{A\mathit{C}}}{{\tilde{R}}_{A\mathit{C}}}\right)}^2\end{array}\right)\left(\begin{array}{cc}{D}_r& 0\\ 0& D_r\end{array}\right)$$

(25)

In the reconstructed misclosure coordinate form, the t-test [24] is conducted for NLOS evaluation:

$$\sqrt{D_N}=‖\overrightarrow{V}‖-\eta \cdot (\sqrt{D_r}+\sqrt{D_p})$$

(26)

where $D_N$ is final evaluated NLOS error variance, $D_r$ and $D_p$ are the random error and approximate error related variances.

3.3. Multi-Source Integration Based H-IPS

After the error evaluation of Wi-Fi and acoustic-based positioning sources using the WAI and NLOS detection algorithm, the evaluated Wi-Fi fingerprinting and acoustic ranging results are model as the observations in the final fusion phase.

$$\delta {\dot{\mathit{X}}}_s={\mathit{F}}_s\delta {\mathit{X}}_s+{\mathit{G}}_s{\mathit{\epsilon }}_s$$

(27)

where $\delta {\mathit{X}}_s$ consist of 15 dimensions of the state vector described in Equation (5), ${\epsilon }_s=[{\epsilon }_1\cdots {\epsilon }_{15}]$ indicates the state error with the Gaussian distribution, and ${\mathit{G}}_s$ is the noise driven matrix.

In the case where acoustic TOA is received, the tightly-coupled integration model is constructed according to the MEMS sensors and ADE compensated acoustic ranging results:

$$\delta {\mathit{z}}_L=\left[\begin{array}{c}\delta {\mathit{z}}_{1,\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}\\ \delta {\mathit{z}}_{2,\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}\\ ⋮\\ \delta {\mathit{z}}_{n,\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}\end{array}\right]=\left[\begin{array}{c}{L}_{{}_{\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}}^1-{\zeta }_s^1-d_{MEMS,1}\\ L_{{}_{\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}}^2-{\zeta }_s^2-d_{MEMS,2}\\ ⋮\\ L_{{}_{\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}}^n-{\zeta }_s^n-d_{MEMS,n}\end{array}\right]$$

(28)

where $\delta {\mathit{z}}_{n,\mathrm{A}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}}$ is the compensated acoustic ranging result, and the MEMS sensor-based ranging $d_{MEMS,n}$ is described as follows:

$$d_{MEMS,n}=\sqrt{{(E_{{}_{MEMS}}^n-P_n^E)}^2+{(N_{{}_{MEMS}}^n-P_n^N)}^2}$$

(29)

where (Eⁿ_MEMS, Nⁿ_MEMS) represents the built-in sensors originating in the 2D location and (P_n^E, P_n^N) indicates the position of the n^st acoustic anchor.

After the Wi-Fi RSSI fingerprinting matching results acquired, the Wi-Fi and built-in sensors loosely coupled integration structure is proposed and enhanced by the WAI. In this case, the observation model is described as:

$$\left\{\begin{array}{l}\delta {\mathit{z}}_r={\mathit{r}}_{rssi}^{}-{\mathit{r}}_{{}_{MEMS}}^{}\\ \delta {\mathit{z}}_v={\mathit{v}}_{rssi}^{}-{\mathit{v}}_{{}_{MEMS}}^{}\end{array}\right.$$

(30)

where ${\mathit{r}}_{rssi}^{}$ and ${\mathit{v}}_{rssi}^{}$ indicate the matched result of Wi-Fi fingerprinting, and $r_{{}_{MEMS}}^{}$ and ${\mathit{v}}_{{}_{MEMS}}^{}$ represent the INS updated results.

4. Experiment Results

In this part, a series of experiments are designed to estimate the accuracy of the developed DMDD, location sources accuracy indicators, and final proposed H-IPS. Two experimental sites contain the office and corridor scenes and NLOS effects are selected for evaluation purposes. In this case, the Google Pixel 3, 4, and 6 are applied as the test terminals, and the sampling rate of the Wi-Fi, acoustic signals, and sensors are 0.3 Hz, 1 Hz, and 50 Hz, respectively.

4.1. Accuracy Estimation of DMDD

In this work, a novel data and model dual-driven model is proposed in order to solve the problems of changeable handheld modes, low-accuracy of speed estimation, and heading drift. The dataset is the fundamental cornerstone of data-driven-based pedestrian dead reckoning. In this paper, the high-precision simultaneous localization and mapping (SLAM) system is used to collect the ground-truth speed and heading information, and the smartphone is used to collect MEMS sensors data at the same time [25]. After the time synchronization procedure, the fulfilled dataset is used to train the DMDD model. In order to reflect various situations in daily use, this paper recruited 10 volunteers to collect sensors data with different smartphone postures, including reading, phoning, pocket, and swaying, and the motion modes included stop, slow walking, normal walking, and fast walking, as well as other motion modes. The total movement distance of the training dataset is more than 120 km, and the total time is more than 38 h. In the training phase, some public data sets are mixed together for training purposes. The test dataset contains 20 trajectories with the ground-truth value for the positioning error evaluation, which is collected and modelled in the same way as in the training dataset.

To evaluate the accuracy of the DMDD, a 2D test route including the indoor and outdoor areas is conducted, which is shown in Figure 4. Testers started at the point A, passed the point B, C, and D, and returned to point A, and repeated the same route three times in order to evaluate the long-term precision. The comparison of the DMDD-based trajectory estimation among complex motion modes is described in Figure 5.

Figure 4 describes that the reading mode achieves the best localization performance, and the phoning mode achieves second-order accuracy, which is lower than the reading mode, the pocket mode achieves the third-order accuracy, and the swaying mode achieves the worst accuracy.

Then, two state-of-art data-driven structures are adopted for comparison: RONIN [26] and IONet [27]. The same test dataset is adopted for performance evaluation, which contains 20 trajectories with ground-truth value, and the comparison results are presented in

Table 1. Average errors of different handheld modes.

Algorithms	Reading	Phoning	Swaying	Pocket
DMDD	2.69 m	3.01 m	4.47 m	3.74 m
RONIN	3.87 m	4.12 m	5.69 m	4.28 m
IONet	4.57 m	4.84 m	6.22 m	4.99 m

.

Table 1 indicates that the developed DMDD shows the best location estimation precision when comparing with the existing algorithms RONIN and IONet under complex motion modes. The reading mode achieves the highest accuracy of within 2.69 m, the phoning mode has the second-order accuracy of within 3.01 m, the pocket mode shows the third-order accuracy of within 3.74 m, and the swaying mode proves to have the worst positioning performance of within 4.47 m.

4.2. Accuracy Estimation of Signal Accuracy Indicators

This work proposes DW-KNN for improving the accuracy of Wi-Fi fingerprinting matching and applies WAI for evaluating the performance of Wi-Fi matched locations. In this work, the performance of DW-KNN is compared with three existing matching algorithms, the classical WKNN, RWKNN [18], and SAWKNN [28], and a hybrid dataset provided by IPIN-2018 [29] and IPIN-2020 [30] competitions is applied for comparison purposes. The matching results are described in Figure 5:

Figure 5 shows that the proposed DW-KNN has a much better matching performance using the same database, and the achieved average matching accuracy reaches 2.96 m, compared with WKNN, RWKNN, and SAWKNN with 4.21 m, 3.87 m, and 3.96 m, respectively.

In addition, this work proposes the GRU model-based Wi-Fi fingerprinting error indicator to provide error reference in the final fusion phase. The state-of-art MLP network [5] is applied for comparison purposes using the public dataset described above. The evaluation performance of the two different error prediction approaches is compared in Figure 6.

Figure 6 indicates that the proposed GRU model proves better error prediction performance compared with the existing MLP model, and the estimated prediction errors of the two models are 1.17 m in 75% and 1.99 m in 75%, respectively.

In addition, this work proposes the ADE to compensate for the acoustic drift error for more accurate acoustic TOA ranging results. To estimate the accuracy of ADE, the SLAM system is applied to provide the ground-truth ranging distance. An additional hardware module is connected to the smartphone for synchronization between the smartphone and acoustic stations using a 433 MHz wireless signal, which is shown in Figure 7.

This work applies ADE under the light-of-sight (LOS) condition to compensate for the drift error of acoustic ranging, which effectively improves the accuracy of acoustic ranging by compensating for the motion-related drift. In this work, a LOS contained environment is selected to test the performance of the proposed ADE, which is described in Figure 8, and the walking route of the testers is A–K, H, I, F, G, D, E, B, A. The state-of-art acoustic/dropper positioning (ADP) method [19] is applied for comparison, and the trajectory comparison is described in Figure 9:

Figure 9 indicates that in the developed DMDD algorithm exists cumulative error, which increases with the time period. The AD- based acoustic and MEMS sensor integration algorithm significantly decrease the positioning error, and is proven to have a better performance that the ADP approach. The positioning errors of three structures are compared in

Table 2. Comparison of different algorithms under the LOS scene.

Algorithms	RMSE	Maximum
DMDD	0.73 m	1.32 m
ADP	0.49 m	1.06 m
ADE	0.38 m	0.91 m

.

Table 2 indicates that the RMSEs of the proposed DMDD, ADP, and ADE reach 0.73 m, 0.49 m, and 0.38 m in office environment, respectively. Compared with the other two algorithms, the proposed ADE effectively improves the positioning accuracy by 47.9% and 22.4%.

4.3. Experiment Results of the H-IPS Framework

The aim of the proposed H-IPS framework is to realize universal indoor positioning by combining the advantages of Wi-Fi, acoustic, and MEMS sensors, and also to maintain the localization precision under NLOS conditions. In this work, a comprehensive and large-scale indoor environment is applied for the estimation of H-IPS, which is described in Figure 8, in which the acoustic covered area is described in Figure 10.

In the above section, we tested the performance of ADE in the case of the LOS condition. In this section, a NLOS contained environment is applied for the evaluation of the proposed acoustic NLOS detection algorithm. In the acoustic covered area, adjacent rooms are separated by a wall; thus, the NLOS error is unavoidable in selected environments. The tester walks alongside the adjacent offices and the real-time received acoustic ranging results are affected by the additional NLOS bias, which needs to be detected and evaluated. The performance comparison between [19] and the proposed ADE and NLOS detector algorithm is described in Figure 11 and

Table 3. Comparison of different algorithms under the NLOS scene.

Algorithms	RMSE	Maximum
ADE/NLOS	0.26 m	0.82 m
ADE	0.67 m	1.41 m

:

It can be seen in Table 3 that the proposed ADE and NLOS detector realizes a much better error compensation performance, and the estimated RMSE reaches 0.26 m, the maximum positioning error is lower than 0.82 m.

Finally, a large-scaled environment containing the corridor scene and office scene is applied to realize Wi-Fi fingerprinting and acoustic positioning at the same time. We also make a comprehensive comparison of the proposed DMDD, Wi-Fi, and built-in sensors fusion structure, and the final H-IPS. The test route of pedestrians is described in Figure 8, the trajectories estimated by different approaches are shown in Figure 10, and the accuracy comparison is shown in Figure 12:

As shown in Figure 12 and Figure 13, the DMDD approach alone is still susceptible to cumulative errors when used over a prolonged time period. The accuracy of DMDD positioning reaches only 6.44 m in 75% of trials. However, our proposed fusion structure that combines Wi-Fi and MEMS sensors effectively decreases the cumulative error of DMDD and realizes a positioning accuracy within 2.37 m in 75% of trials.

By further integrating different location sources with the adaptive unscented Kalman filter, the H-IPS achieves even higher accuracy levels, with an average precision of 2.01 m in 75% of trials. This result confirms the effectiveness of the proposed system in combining multiple sources of data for accurate indoor localization.

Notably, our experimental results also demonstrate that meter-level positioning precision can be achieved in acoustic signal cover areas with NLOS conditions. This is a significant achievement as it allows for highly accurate indoor positioning, even in challenging environments where traditional approaches may fail.

Overall, our findings highlight the potential of the proposed H-IPS for achieving reliable and robust indoor localization. By leveraging multiple sources of data and innovative algorithms, the H-IPS offers a promising solution for smart city applications that require high-precision indoor positioning. The results of this study contribute to advancing the field of indoor localization and provide valuable insights for future research and development in this area.

5. Conclusions

This paper presents the H-IPS, a novel indoor positioning system that leverages the combination of acoustic ranging, Wi-Fi fingerprinting, and low-cost sensors to achieve accurate and robust localization. The proposed system offers several contributions, including the DMDD model for MEMS sensor-based trajectory estimation under complex pedestrian motion modes. Additionally, we propose the use of ADE and WAI-based quality evaluation algorithms for the error evaluation of acoustic signals and Wi-Fi fingerprinting in NLOS environments. Finally, an adaptive unscented Kalman filter is applied for intelligent fusion of different location sources and error evaluation results.

Our comprehensive experimental results demonstrate that the proposed H-IPS significantly improves the performance of multi-source fusion-based localization. With an accuracy of 2.01 m achieved in 75% of the trials, the final integrated accuracy reaches meter-level precision, even under NLOS scenes and complex motion modes. Compared to other existing indoor positioning systems, the proposed H-IPS offers superior performance in terms of accuracy, robustness, and adaptability.

The DMDD model used in our approach is a significant innovation as it considers both the data-driven and model-driven approaches to address the challenges of pedestrian motion modes in complex indoor environments. The proposed quality evaluation algorithms also play a crucial role in improving the accuracy of the H-IPS by accounting for the impact of NLOS environments on the acoustic signals and Wi-Fi fingerprinting. Moreover, the adaptive unscented Kalman filter enables intelligent fusion of different location sources and error evaluation results, further enhancing the overall accuracy of the system.

The proposed H-IPS also has disadvantages in real-world applications. First of all, the estimation of ranging and location errors of acoustic signal and Wi-Fi matching needs to be enhanced using a more robust network and dataset in order to adapt to the changeable environments. In addition, the deep learning-based walking speed prediction model can use more robust networks, such as a transformer, while the capacity of the training dataset also needs to be expanded to improve the accuracy.

In summary, the proposed H-IPS offers a promising solution for accurate and robust indoor localization. By leveraging the advantages of multiple sources, integrating the DMDD model, developing quality evaluation algorithms, and applying the adaptive unscented Kalman filter, the H-IPS achieves meter-level accuracy in challenging indoor environments. The results of this study provide valuable insights for the development of future indoor positioning systems, opening new opportunities for the use of such systems in smart city applications.