Scene Recognition for Indoor Localization Using a Multi-Sensor Fusion Approach
Abstract
:1. Introduction
- (1)
- To imitate the visual cognition ability of human, a scene recognition module implemented with deep learning is adopted to improve the accuracy and robustness of infrastructure-free indoor localization system.
- (2)
- A practical indoor localization scheme is designed which digging the full potential of built-in sensors on a smartphone. Particle filter algorithm is used in sensor data fusion.
- (3)
- The database construction in this paper is labor-saving and easily to be extended as a crowdsourcing solution for the reason that data is automatically collected while users’ walking.
- (4)
- The proposed system is implemented on Android platform and performance is evaluated in a challenging indoor environment which includes several glass walls and a patio.
2. Related Work
3. System Overview and Methods
3.1. System Overview
3.2. Data Acquisition and Fingerprints Processing
3.2.1. Data Acquisition
- Firstly, locations with the same scene label must have same semantic significance. For instance, as shown in Figure 4, point 3 belongs to room 313 while point 4 belongs to patio, it is obvious they are in different scenes.
- Secondly, locations with the same semantic significance but in sensitive areas are preferrentially separated into different scenes. The sensitive areas here are more likely to achieve high error rate in the positioning phase. For instance, magnetic fingerprinting methods are difficult to distinguish locations at each side of a glass wall or door. As illustrated in Figure 4, the wall between room 313 and patio is a glass one. So the corridor inside room 313 and the corridor on the left of patio are defined as two different scenes to distinguish locations such as point 3 and point 4 even better.
3.2.2. Fingerprints Processing
3.3. Indoor Scene Model Training
3.3.1. Deep Convolutional Neural Networks (CNNs)
3.3.2. Fine-Tuned Deep CNNs for Indoor Scenes Recognition
3.4. Location Estimation
3.4.1. Indoor Scene Recognition
Algorithm 1. Ttransition Point Detection. | |
Input: | Samples of WiFi RSSI |
Output: | TP’s postion |
Begin: | minimum distance equals infinite, nearest TP equals TP1 |
1: | for every TP fingerprint in database do |
2: | while n < number of APs scanned do |
3: | compare mac address of every AP with TP fingerprint |
4: | if mac address is matched |
5: | number of matched APs ++ |
6: | end if |
7: | end while |
8: | if number of matched APs > 3 |
9: | compute signal distance between samples and TP |
10: | if distance < minimum distance |
11: | update the minimum distance |
12: | update nearest TP |
13: | end if |
14: | end if |
15: | end for |
16: | compare scene label of nearest TP to current scene |
17: | if not matched |
18: | return ture |
19: | else return flase |
20: | end if |
3.4.2. Space Narrowing and Localization
4. Performance Evaluation
4.1. Experiment Setup
4.2. Performance of Indoor Scene Recognition
4.3. Localization Results and Analysis
5. Discussion
6. Conclusions
Acknowledgments
Author Contributions
Conflicts of Interest
References
- Li, D.; Shan, J.; Shao, Z.; Zhou, X.; Yao, Y. Geomatics for smart cities-concept, key techniques, and applications. Geo-Spat. Inf. Sci. 2013, 16, 13–24. [Google Scholar] [CrossRef]
- Bahl, P.; Padmanabhan, V.N. RADAR: An in-building RF-based User Location and Tracking System. In Proceedings of the Nineteenth Joint Conference of the IEEE Computer and Communications Societies, Tel Aviv, Israel, 26–30 March 2000; pp. 775–784. [Google Scholar]
- Youssef, M.; Agrawala, A. The Horus WLAN location determination system. In Proceedings of the International Conference on Mobile Systems, Applications, and Services, Seattle, WA, USA, 6–8 June 2005; pp. 205–218. [Google Scholar]
- Rai, A.; Chintalapudi, K.K.; Padmanabhan, V.N.; Sen, R. Zee: Zero-effort crowdsourcing for indoor localization. In Proceedings of the 18th Annual International Conference on Mobile Computing and Networking, Istanbul, Turkey, 22–26 August 2012; pp. 293–304. [Google Scholar]
- Yang, Z.; Wu, C.; Liu, Y. Locating in fingerprint space: Wireless indoor localization with little human intervention. In Proceedings of the 18th Annual International Conference on Mobile Computing and Networking, Istanbul, Turkey, 22–26 August 2012; pp. 269–280. [Google Scholar]
- Yang, S.; Dessai, P.; Verma, M.; Gerla, M. Freeloc: Calibration-free crowdsourced indoor localization. In Proceedings of the 32nd IEEE International Conference on Computer Communications, Turin, Italy, 14–19 April 2013; pp. 2481–2489. [Google Scholar]
- Li, F.; Zhao, C.; Ding, G.; Gong, J.; Liu, C.; Zhao, F. A reliable and accurate indoor localization method using phone inertial sensors. In Proceedings of the 14th ACM International Conference on Ubiquitous Computing, Pittsburgh, PA, USA, 5–8 September 2012; pp. 421–430. [Google Scholar]
- He, S.; Chan, S.H.G. Wi-Fi Fingerprint-Based Indoor Positioning: Recent Advances and Comparisons. IEEE Commun. Surv. Tutor. 2016, 18, 466–490. [Google Scholar] [CrossRef]
- Yang, Z.; Wu, C.; Zhou, Z.; Zhang, X.; Wang, X.; Liu, Y. Mobility increases localizability: A survey on wireless indoor localization using inertial sensors. ACM Comput. Surv. 2015, 47, 1–34. [Google Scholar] [CrossRef]
- Zheng, Y.; Shen, G.; Li, L.; Zhao, C.; Li, M.; Zhao, F. Travi-navi: Self-deployable Indoor Navigation System. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking, Maui, HI, USA, 7–11 September 2014; pp. 471–482. [Google Scholar]
- Chen, Z.; Zou, H.; Jiang, H.; Zhu, Q.; Soh, Y.; Xie, L. Fusion of WiFi, Smartphone Sensors and Landmarks Using the Kalman Filter for Indoor Localization. Sensors 2015, 15, 715–732. [Google Scholar] [CrossRef] [PubMed]
- Liu, J.; Chen, R.; Pei, L.; Guinness, R.; Kuusniemi, H. A hybrid smartphone indoor positioning solution for mobile LBS. Sensors 2012, 12, 17208–17233. [Google Scholar] [CrossRef] [PubMed]
- Chen, L.H.; Wu, E.H.K.; Jin, M.H.; Chen, G.H. Intelligent fusion of Wi-Fi and inertial sensor-based positioning systems for indoor pedestrian navigation. IEEE Sens. J. 2014, 14, 4034–4042. [Google Scholar] [CrossRef]
- Shu, Y.; Bo, C.; Shen, G.; Zhao, C.; Li, L.; Zhao, F. Magicol: Indoor localization using pervasive magnetic field and opportunistic WiFi sensing. IEEE J. Sel. Area. Comm. 2015, 33, 1443–1457. [Google Scholar] [CrossRef]
- Oquab, M.; Bottou, L.; Laptev, I.; Sivic, J. Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; pp. 1717–1724. [Google Scholar]
- Khan, S.H.; Hayat, M.; Bennamoun, M.; Togneri, R.; Sohel, F.A. A Discriminative Representation of Convolutional Features for Indoor Scene Recognition. IEEE Trans. Image Process. 2016, 25, 3372–3383. [Google Scholar] [CrossRef] [PubMed]
- Yu, J.; Hong, C.; Tao, D.; Wang, M. Semantic embedding for indoor scene recognition by weighted hypergraph learning. Signal Process. 2015, 112, 129–136. [Google Scholar] [CrossRef]
- Gupta, S.; Arbelaez, P.; Malik, J. Perceptual organization and recognition of indoor scenes from RGB-D images. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 25–27 June 2013; pp. 564–571. [Google Scholar]
- Kawewong, A.; Pimup, R.; Hasegawa, O. Incremental Learning Framework for Indoor Scene Recognition. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, Bellevue, WA, USA, 14–18 July 2013; pp. 496–502. [Google Scholar]
- Espinace, P.; Kollar, T.; Roy, N.; Soto, A. Indoor scene recognition by a mobile robot through adaptive object detection. Robot. Auton. Syst. 2013, 61, 932–947. [Google Scholar] [CrossRef]
- Espinace, P.; Kollar, T.; Soto, A.; Roy, N. Indoor scene recognition through object detection. In Proceedings of the IEEE International Conference on Robotics and Automation, St Paul, MN, USA, 14–18 May 2012; pp. 1406–1413. [Google Scholar]
- Quattoni, A.; Torralba, A. Recognizing indoor scenes. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA, 20–25 June 2009; pp. 413–420. [Google Scholar]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; pp. 580–587. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 1904–1916. [Google Scholar] [CrossRef] [PubMed]
- Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Object detectors emerge in deep scene cnns. arXiv, 2014; arXiv:1412.6856v2. [Google Scholar]
- Yosinski, J.; Clune, J.; Bengio, Y.; Lipson, H. How transferable are features in deep neural networks? In Proceedings of the Neural Information Processing—21st International Conference, ICONIP 2014, Kuching, Malaysia, 3–6 November 2014; pp. 3320–3328. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. In Proceedings of the Neural Information Processing—19th International Conference, ICONIP 2012, Doha, Qatar, 12–15 November 2012; pp. 1097–1105. [Google Scholar]
- Chen, W.; Wang, W.; Li, Q.; Chang, Q.; Hou, H. A Crowd-Sourcing Indoor Localization Algorithm via Optical Camera on a Smartphone Assisted by Wi-Fi Fingerprint RSSI. Sensors 2016, 16, 410. [Google Scholar] [CrossRef] [PubMed]
- IndoorAtlas. Available online: http://www.indooratlas.com/ (accessed on 15 October 2017).
- Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; Li, F.-F. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
- Chen, R.; Pei, L.; Chen, Y. A smart phone based PDR solution for indoor navigation. In Proceedings of the 24th International Technical Meeting of the Satellite Division of the Institute of Navigation, Portland, OR, USA, 20–23 September 2010; pp. 1404–1408. [Google Scholar]
- Lane, N.D.; Georgiev, P. Can deep learning revolutionize mobile sensing? In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, Santa Fe, NM, USA, 12–13 February 2015; pp. 117–122. [Google Scholar]
- Latifi Oskouei, S.S.; Golestani, H.; Hashemi, M.; Ghiasi, S. CNNdroid: GPU-Accelerated Execution of Trained Deep Convolutional Neural Networks on Android. In Proceedings of the 2016 ACM on Multimedia Conference, Amsterdam, The Netherlands, 15–19 October 2016; pp. 1201–1205. [Google Scholar]
- Caffe|Deep Learning Framework. Available online: http://caffe.berkeleyvision.org/ (accessed on 15 October 2017).
- Model Zoo BVLC/Caffe Wiki. Available online: https://github.com/BVLC/caffe/wiki/Model-Zoo (accessed on 15 October 2017).
- Caffe/Models/Bvlc_Reference_Caffenet at Master BVLC/Caffe. Available online: https://github.com/BVLC/caffe/tree/master/models/bvlc_reference_caffenet (accessed on 15 October 2017).
- Goodfellow, I.; Bengio, Y.; Aaron, C. Deep Learning; MIT Press: Cambridge, MA, USA, 2016; pp. 120–121. ISBN 0262035618. [Google Scholar]
Scene Labels | TPs 1 | Steps | Heading | WiFi RSSI | MFS 1 |
---|---|---|---|---|---|
S1 | (x0, y0) | k1 | (r1, r2, ..., rk1) | (vrssi 1, vrssi 2, ..., vrssi k1) | (vmfs 1, vmfs 2, ..., vmfs k1) |
S2 | (x1, y1) | k2 | (r1, r2, ..., rk2) | (vrssi 1, vrssi 2, ..., vrssi k2) | (vmfs 1, vmfs 2, ..., vmfs k2) |
… | … | … | … | … | … |
Sn−1 | (xn, yn) | kn−1 | (r1, r2, ..., rkn−1) | (vrssi 1, vrssi 2, ..., vrssi kn−1) | (vmfs 1, vmfs 2, ..., vmfs kn−1) |
Layers | Kernel Size | Stride | Pad | Output |
---|---|---|---|---|
conv1 | 11 | 4 | 0 | 96 |
pool1 | 3 | 3 | 0 | 96 |
con2 | 5 | 1 | 2 | 256 |
pool2 | 3 | 2 | 0 | 256 |
conv3 | 3 | 1 | 1 | 384 |
conv4 | 3 | 1 | 1 | 384 |
conv5 | 3 | 1 | 1 | 256 |
pool5 | 3 | 2 | 0 | 256 |
fc6 | / | / | / | 4096 |
fc7 | / | / | / | 4096 |
fc8 | / | / | / | 20 |
Model | Iterations | Time Cost | Top-1 (val) | Top-1 (Test) |
---|---|---|---|---|
CaffeNet | 6000 | 23′39″ | 97.5% | 89.8% |
Method | Mean Error | Variance | 95% Accuracy |
---|---|---|---|
PDR + WiFi + Magnetic 1 | 2.36 m | 2.84 | 5.93 m |
IndoorAtlas (Hybird platform) | 1.08 m | 2.69 | 3.01 m |
PDR + WiFi + Magnetic + Indoor Scene 2 | 0.53 m | 0.16 | 1.32 m |
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, M.; Chen, R.; Li, D.; Chen, Y.; Guo, G.; Cao, Z.; Pan, Y. Scene Recognition for Indoor Localization Using a Multi-Sensor Fusion Approach. Sensors 2017, 17, 2847. https://doi.org/10.3390/s17122847
Liu M, Chen R, Li D, Chen Y, Guo G, Cao Z, Pan Y. Scene Recognition for Indoor Localization Using a Multi-Sensor Fusion Approach. Sensors. 2017; 17(12):2847. https://doi.org/10.3390/s17122847
Chicago/Turabian StyleLiu, Mengyun, Ruizhi Chen, Deren Li, Yujin Chen, Guangyi Guo, Zhipeng Cao, and Yuanjin Pan. 2017. "Scene Recognition for Indoor Localization Using a Multi-Sensor Fusion Approach" Sensors 17, no. 12: 2847. https://doi.org/10.3390/s17122847
APA StyleLiu, M., Chen, R., Li, D., Chen, Y., Guo, G., Cao, Z., & Pan, Y. (2017). Scene Recognition for Indoor Localization Using a Multi-Sensor Fusion Approach. Sensors, 17(12), 2847. https://doi.org/10.3390/s17122847