Comparison of Hybrid Localization Methods Using Images and Wi-Fi Signals ^†

Abstract

Indoor localization is important for many applications, such as navigation, movement tracking, geotagging, and augmented reality. Most studies have used either Wi-Fi or image signals to determine the user’s location. However, each localization method has advantages and disadvantages. In this study, we propose a hybrid localization system combining the advantages of Wi-Fi- and image-based methods. The localization is calculated based on the best four outputs of either image or Wi-Fi localization system. The system was evaluated by comparing the accuracy and unit errors of image-based, Wi-Fi-based, hybrid (image + Wi-Fi), and hybrid (Wi-Fi + image) methods. The results showed accuracies of 77.2%, 49.5%, 73.1%, and 81.6% in the image-based, Wi-Fi-based, hybrid (image + Wi-Fi), and hybrid (Wi-Fi + image) methods, respectively. The hybrid (Wi-Fi and image) method has the lowest error and highest accuracy of the four methods compared. In addition, the image-based localization system shows the highest error, while the Wi-Fi-based localization system shows the lowest accuracy. The robot tests prove that the proposed hybrid system can achieve excellent performance in indoor localization. The proposed hybrid system uses both image processing and Wi-Fi fingerprinting methods to determine the location of the mobile device by creating the two-phase framework, which can help improve the accuracy of indoor localization.

1. Introduction

Indoor localization is important for many applications, such as navigation, geotagging, movement tracking, and augmented reality. Recently, many methods have been proposed for indoor localization, including infrared light, ultrasonic waves, radio-frequency identification (RFID), Wi-Fi signals, and image processing [1]. Of these, most studies have focused on localization based on Wi-Fi signals in different scenarios [2,3]. Wi-Fi-based indoor localization uses the signal strength of various wireless access points to determine the user’s position. There have also been previous studies that match an image captured using a cell phone camera to images from a database for indoor localization [1,4,5,6]. Image-based indoor localization uses image features, which are key points that are extracted by feature extraction algorithms, such as speeded-up robust features (SURF) or scale-invariant feature transform (SIFT).

Although each method (image or Wi-Fi-based) can be used individually for indoor positioning, both methods have advantages and disadvantages. Wi-Fi localization has relatively high accuracy but requires high infrastructure costs. Sometimes, Wi-Fi access points (APs) do not work temporarily or change, requiring re-calibration of the whole system. The most challenging problems in image-based approaches are an intensive computational cost for extraction of features from the images and image recognition in a large building due to locations with similar appearances. Thus, various methods have been developed to decrease the limitations related to indoor localization using smartphones with built-in Wi-Fi and a camera, since the development of phones has advanced spectacularly in recent years. Several studies have proposed a hybrid indoor location method using Wi-Fi signals and camera images [7,8,9]. Hattori et al. [7] demonstrated the high accuracy of a hybrid indoor localization system using image processing and Wi-Fi signals. Ching et al. [8] proposed a two-phase framework that uses both Wi-Fi signals and taken images for indoor localization, where the Wi-Fi-based localization method calculates a rough location of the device, then refine the search using the images taken by the user. Niu et al. [9] developed a system that combined data of Wi-Fi signals and the images of the surrounding area. In this study, we propose a hybrid indoor localization system using either a combination of images and Wi-Fi signals or Wi-Fi signals and images to estimate the location of users more accurately.

Preliminary experiments showed that Wi-Fi-based localization has a relatively small error, but is less accurate for positioning. In contrast, image-based localization can predict the position more accurately, but it may produce a larger error if it fails to recognize the images correctly. This proposed hybrid localization system works based on the advantages of both the image and Wi-Fi-based methods. The localization is calculated based on the best four outputs of either image or Wi-Fi localization system. In the hybrid image + Wi-Fi method, image localization works on the best four Wi-Fi signals from the Wi-Fi localization. Alternatively, Wi-Fi localization can use the best four images from image localization for the hybrid Wi-Fi + image localization.

2. Basic Process of the Proposed System

2.1. Hybrid Localization (Image + Wi-Fi)

The proposed hybrid localization system used both image processing and Wi-Fi fingerprinting methods to determine the location of the user as described above (Figure 1). Two moduli will reside on the user’s mobile device with Wi-Fi scanning and a camera. On the server, the Wi-Fi and image localization algorithms will reside. The Wi-Fi localization uses the fingerprinting method, while the image localization uses the feature extraction algorithms to find the position. All taken images and Wi-Fi RSS are sent to the server as input. The hybrid localization method has a two-phase framework. In the first phase, the four best-matched images were found using the image localization method based on the generated score between query and database images. The second phase included the Wi-Fi localization which refined the search using the RSS information of these four images in the fingerprint database, and the final position was defined by the smallest Euclidean distance.

2.2. Hybrid Localization (Wi-Fi + Image)

The collecting and processing data are the same as the image + Wi-Fi method (Figure 1). In the first phase, the best four RSS measurements were defined using the Wi-Fi fingerprinting method which calculates the Euclidean distance between the query RSS measured from different access points and the pre-stored RSS fingerprinting database using the KNN algorithms. Second, the four images with the best Wi-Fi RSS measurement were determined from the vocabulary tree database. Then, the image localization method was performed on these four images to find the best-matched image.

3. Experiments and Results

3.1. Image-Based Localization

We conducted experiments in an indoor environment (fifth floor of Kyung Hee University, Yongin, Korea) (Figure 2a). In the training phase, we collected 490 images from 49 rooms in two directions (two corridors), and at five different times (8:30 a.m, 11:30 a.m, 2:30 p.m, 5:30 p.m, 8:30 p.m) using a Samsung Galaxy Tab S2 which has an 8 Megapixel rear camera (Figure 2b). The resolution of the tablet was 612 × 816 pixels (with an initial resolution of 2448 × 3264 pixels). In the testing phase, we captured 294 images of 49 rooms in two directions (two corridors), and at three different times (10:00 a.m, 4:00 p.m, 7:00 p.m). All collected images were sent to the server, which has an Intel Pentium (R) 2.90 GHz CPU and 4 GB of RAM, and runs on a Windows 8.1 Pro 64-bit operating system. All server-side components were implemented in Python 2.7.

In total, we tested 294 query images for three different time groups. The results showed accuracies of 87.76%, 80.61%, and 63.27% for the 10:00 a.m, 4:00 p.m, and 7:00 p.m groups, respectively. The error ranged from 0 to 24 and the average error ranged from 1.02 to 2.21. The average accuracy and error were 77.21% and 1.44, respectively, for the three different groups (Figure 3).

3.2. Wi-Fi-Based Localization

To build the Wi-Fi database, the Wi-Fi RSS was recorded in front of 49 rooms for the training phase. On each measurement, we received Wi-Fi signal strength 100 times over 16 min. The experiment was performed in the same place as the image-based localization (Figure 2). For the testing phase, we collected Wi-Fi RSS 10 times for 49 rooms. The measured RSS of the testing phase was compared with the fingerprint database. The KNN was used to determine the mobile user position. We tested a total of 489 query Wi-Fi measurements.

The results showed an accuracy of 49.49%. The error ranged from 0 to 6 and the average error was 0.75 (Figure 3). The accuracy of the Wi-Fi method was not as good as those of other methods; this may be because the neighbor rooms could have similar RSS fingerprints due to the small distance between rooms. Although the location was estimated, the true location of the room was mistakenly matched. This can be shown by the lower error than for the image-based method.

3.3. Hybrid Localization (Image + Wi-Fi)

As described above the hybrid localization method has a two-phase framework. The previously collected training (490 images) and testing (294 images) images in the image-based localization were used as the first phase of the hybrid localization method. Here, the four best-matched images with the room numbers were extracted using image processing. Then, the Wi-Fi RSS of these rooms was determined from the fingerprint database. In the second phase, the Wi-Fi localization method was used to refine the location within the four Wi-Fi measurements.

The results showed an accuracy of 73.13%. The error ranged from 0 to 19 and the average error was 0.66 (Figure 3). Although the overall results were improved relative to the Wi-Fi-based method, the accuracy was 4% lower than that of the image-based localization method. When we chose the image-based method as a base method, there was a possibility of mismatched user location due to feature similarities between query and database images. If the best four images were incorrectly matched, the Wi-Fi method could not detect the true position of the user.

3.4. Hybrid Localization (Wi-Fi + Image)

The collected Wi-Fi RSS for the training and testing phases in the Wi-Fi-based localization was used as the first phase of the hybrid localization method. The 294 of 489 query Wi-Fi measurements were tested. Then, the four best-matched Wi-Fi measurements with room numbers were extracted using the fingerprinting method. Then, the images of these rooms were determined from the image training database. In the second phase, the image localization method was used to match images within the four images.

The results showed an accuracy of 81.63%. The error ranged from 0 to 5 and the average error was 0.28 (Figure 3). When we compare the results with the above methods, both accuracy and error were improved. The main advantage of the hybrid (Wi-Fi + image) method is that the best four images were selected based on the Wi-Fi fingerprinting method, which helps to narrow down the user’s location efficiently. Then, the image-based method finds the query images from the database within the four images, which improved the accuracy and helped the hybrid (Wi-Fi + image) system find the location faster than in other approaches.

4. Robot Test

To verify the effectiveness of the hybrid localization method, we collected the image and Wi-Fi signals using a robot, which can capture images and collect Wi-Fi signals automatically at specified intervals. The experiment was performed in the same indoor environment. For the database used in the image localization method, the robot captured 1440 images for 20 rooms at two times. For the testing phase, 720 images were collected for 20 rooms at three different times. For the database used in the Wi-Fi localization method, Wi-Fi signals were recorded in front of 20 rooms. On each measurement, we received Wi-Fi signal strength 100 times over for 16 min. For testing, we collected Wi-Fi signal strength 30 times for 20 rooms.

The results showed accuracies of 75.32%, 61.72%, 78.54%, and 86.25% in the image-based, Wi-Fi-based, hybrid (image + Wi-Fi), and hybrid (Wi-Fi + image) localization methods, respectively. The average errors were 1.26, 0.57, 0.63, and 0.21 in the image-based, Wi-Fi-based, hybrid (image + Wi-Fi), and hybrid (Wi-Fi + image) localization methods, respectively (Figure 4). The robot tests prove that the proposed hybrid system can achieve excellent performance in indoor localization.

5. Conclusions

We proposed an indoor hybrid localization method that uses both image processing and Wi-Fi fingerprinting methods to determine the location of the mobile device by creating the two-phase framework. The system effectivity was evaluated by comparing the accuracy and unit errors with different localization techniques. The hybrid (Wi-Fi + image) method had the lowest error and the highest accuracy of the four methods performed. In addition, image-based localization showed the highest error, while Wi-Fi-based localization showed the lowest accuracy. The robot tests prove that the proposed hybrid system can achieve excellent performance in indoor localization. The proposed hybrid system combined the advantages of image-based and Wi-Fi-based methods, which can help improve the accuracy of indoor localization.