Side ‐ Information ‐ Aided Preprocessing Scheme for Deep ‐ Learning Classifier in Fingerprint ‐ Based Indoor Positioning

: Deep ‐ learning classifiers can effectively improve the accuracy of fingerprint ‐ based indoor positioning. During fingerprint database construction, all received signal strength indicators from each access point are combined without any distinction. Therefore, the database is created and utilised for deep ‐ learning models. Meanwhile, side information regarding specific conditions may help characterise the data features for the deep ‐ learning classifier and improve the accuracy of indoor positioning. Herein, a side ‐ information ‐ aided preprocessing scheme for deep ‐ learning classifiers is proposed in a dynamic environment, where several groups of different databases are constructed for training multiple classifiers. Therefore, appropriate databases can be employed to effectively improve positioning accuracies. Specifically, two kinds of side information, namely time (morning/afternoon) and direction (forward/backward), are considered when collecting the received signal strength indicator. Simulations and experiments are performed with the deep ‐ learning classifier trained on four different databases. Moreover, these are compared with conventional results from the combined database. The results show that the side ‐ information ‐ aided preprocessing scheme allows better success probability than the conventional method. With two margins, the proposed scheme has 6.55% and 5.8% improved performances for simulations and experiments compared to the conventional scheme. Additionally, the proposed scheme, with time as the side information, obtains a higher success probability when the positioning accuracy requirement is loose with larger margin. With direction as the side information, the proposed scheme shows better performance for high positioning precision requirements. Thus, side information such as time or direction is advantageous for preprocessing data in deep ‐ learning classifiers for fingerprint ‐ based indoor positioning.


Introduction
With the dual development of wireless communications and smart mobile devices, locationbased services (LBSs) have developed rapidly and shown broad market prospects [1]. LBSs have penetrated most aspects of daily life and working environment, affording great convenience to people. In general, an LBS can be divided into outdoor and indoor positioning services according to the positioning location. With the maturity of the outdoor positioning technology, research in highprecision positioning has gradually shifted toward indoor locations [2]. At present, the leading indoor positioning technologies involve infrared, wireless local area network (WLAN), ultrawideband, radio frequency identification, ultrasonic, and Bluetooth technologies [3]. WLAN refers to Wi-Fi networks that communicate over a wireless medium in a local area and has the advantages of full area coverage and low installation cost [4]. Therefore, Wi-Fi-based indoor positioning has been widely studied and applied. Wi-Fi-based positioning techniques can be divided into two types: the spatio-temporal attribute method and the received signal strength indicator (RSSI)-based scheme. The latter is also known as fingerprint recognition and is one of the most popular indoor location techniques. In the fingerprint-based indoor positioning system, a k-nearest neighbor (KNN) approach is commonly used, which is simple for operation and does not require many data points for training. However, for the KNN, the output completely relies on the nearest neighbors. That is, it is sensitive to the noise and the multipath loss [5]. When an error occurs in measuring the nearest neighbor, it is necessary to manually impute the missing values, which complicates the positioning process and deteriorates its accuracy. To solve the problem, a deep-learning method was proposed in the fingerprint-aided positioning and its performance was evaluated in [6] and [7]. In [8], the convolutional neural network (CNN) was proposed to show the applicability of complex image classification to the emerging fields. Thus, many studies have adopted the CNN classifier to achieve improved positioning results. As described in [9], a CNN-based indoor fingerprint positioning framework was introduced to improve the accuracy of fingerprint positioning. In our past research [10], we compared the performance of our CNN framework with other CNN frameworks for fingerprint positioning; our results showed that our constructed framework provided better performance for fingerprint positioning than conventional methods. The fingerprint positioning system with a deep-learning classifier is shown in Figure 1. The Wi-Fi-based indoor fingerprinting positioning system has two phases: the online and offline phases. First, in the offline phase, a fingerprint database was built by collecting RSSI via smartphones, and the processed database was then inputted to the CNN model to train the classifier. The trained classifier is then passed to the online phase. Second, in the online phase, the Wi-Fi fingerprint at each unknown position is collected and stored in the database to be matched with the fingerprint for the test point by the trained classifier. The trained classifier subsequently returns the decision on the user's location, and the decision results are passed to the user's smartphone. Currently, the critical issues in Wi-Fi fingerprint-based indoor positioning systems are related to dynamic environments, such as fading of multipath propagation signals caused by obstacles, pedestrians near the user location, and addition/deletion of Wi-Fi access points (APs) [11]. Therefore, it is necessary to consider methods to deal with positioning errors arising from dynamic environments [12]. Note that several environmental factors, such as human activity, temperature, and humidity, are not easily controlled in actual experiments [13]. In [14], it was noted that a database built in a static environment might not be suitable for positioning tests in a dynamic environment. Therefore, it would be necessary to build the fingerprint database under various conditions. For the deep-learning classifier, a large-scale database is usually constructed by collecting RSSI values under various conditions, so that the impact of the dynamic factors is reduced. However, in [15] and [16], where RSSI values were collected at different times (morning/afternoon) and directions (forward and reverse), it was shown that building a large-scale database may not always be suitable for deeplearning classifiers in dynamic environments. The authors of [9] observed that it was not necessary to learn all the features of a reference point (RP) to improve positioning accuracy and, hence, more advantageous to selectively learn the key features of an RP in the CNN model. In [17], the authors proposed to divide the database structure for the location area according to information that is beneficial for positioning accuracy. Therefore, we believe that it would be helpful for the deeplearning classifier to decide the position to separately store the RSSI features. In [18] and [19], it was shown that design of a suitable classifier and correct fingerprint feature learning for fingerprint positioning was essential. In [20], it was shown that, in a dynamic environment, the fingerprint recognition points could be improved by analysis of the time and space for the location. Further, it was shown in [7] that the preprocessing technique could improve positioning accuracy performance. In order to reduce the impact of the dynamic environment on the data fingerprint database and to improve the positioning accuracy, we subdivided the database according to different side information and then selected the database suitable for the current environment for positioning according to the positioning conditions. Therefore, in the preprocessing stage of the database generation, the collected data were classified and processed according to different conditions to construct multiple types of databases for training the classifier. In this study, we propose a sideinformation-aided preprocessing scheme that utilises information such as time or direction to train the deep-learning classifier. In other words, the side information was used to segment and construct the multiple databases for training classifiers to evaluate the performance of the system. This not only reduces the impact of dynamic RSSI values from the same APs for feature learning but also improves the positioning accuracies. To verify its usefulness, we categorise four types of data sets, namely morning/afternoon and forward/backward. Database preprocessing is an effective method to improve fingerprint positioning. In a previous work, we built a large-scale database by collecting a large quantity of data and used data augmentation preprocessing to improve the positioning accuracies [6]. However, we observed that the fingerprints measured at different times and directions were different, so we proposed a preprocessing method to classify the database based on the side information and in combination with the data augmentation method to effectively improve the positioning success rates. The positioning results were analysed for both simulations and real-time experiments. The remainder of this paper is organised as follows. In Section 2, the system model is described. The simulation and real-time experiment results are discussed in Section 3. Finally, Section 4 summarises the conclusions of our work.

Environment Setup
The user divides the collection space into several smaller grids, where the size of each grid is determined as 1 m 2 and 4 m 2 centred at the collector. The size of the grid can affect the accuracy of positioning. That is, the denser the grid area, the higher the positioning accuracy is and the longer the time required to collect the signal samples is. The environment setup for a floor map with 74 reference points is shown in Figure 2. For the experiments, the RSSI data collection and experiments were carried out on the seventh floor of the Engineering Building of Dongguk University, Seoul, Korea. The 52 × 32 m area was divided into 74 reference points (RPs), and the interval between any two RPs was 2 m. For the hardware, we adopted the Dell Alienware model P31e as the positioning server. Meanwhile, a Samsung SM-E310K smartphone was used to measure the RSSI values of the available APs on each RP. In terms of the software, the RSSI database, data preprocessing process, deep-learning classifier and online experiment programme were constructed using Python and the TensorFlow platform.

CNN Model and Data Augmentation
In our work, the deep-learning code was designed for comma-separated value (CSV) input files; hence, we converted the collected data files into CSV files for training and testing. The training and testing CSV files contained 257 columns. In column 257, a total of 74 RPs were used as labels. As shown in Figure 3, the NN represents the RSSI value. For each RP, the AP has a unique media access control (MAC) address. The programme sets each MAC address that can be detected into a separate column. At the same time, the corresponding RSSI values are placed under the MAC addresses. In the process of filling all the RSSI values of the RPs into the corresponding MAC addresses, if the RSSI values of certain MAC addresses do not exist in a certain RP, the corresponding MAC columns are filled with zeros. When the programme arranges all the RSSI values of the 74 RPs into the corresponding MAC columns, if the total number of columns in the CSV file is less than 256, the remaining columns are filled with zeros. If the total number of columns is greater than 256, the MAC address column with the lowest RSSI value is deleted. If the total number of columns in the final CSV file is equal to 256, the reference data set is complete. The CNN model used in this work is shown in Figure 4, with five layers of network structure [6]. This five-layer network is used to predict the 74 classes. For each RP, the RSSI values of the 256 APs will be recorded. The RSSI values of these different APs are collated into 16 × 16 images. As shown in the input in Figure 4, different RSSI values are represented by different brightnesses. The greater the RSSI value, the higher the brightness is. That is, the RSSI value of 0 represents black. In our CNN model, the first layer has an input grayscale image of size 16 × 16 × 1, rectified linear unit (ReLU) and dropout. Because of the small size of the input data set, the first layer does not use maximum pooling, and the second layer consists of a 16 × 16 convolution with ReLU, which then consists of a max pooling layer of size 8 × 8, with a total of 18,496 parameters. This produces an output for the third 8 × 8 convolutional layer with ReLU and a 4 × 4 max pooling layer. This output feeds directly into a fully connected (FC) layer with 3072 nodes, which results in the next hidden FC layer with 1024 nodes. Finally, a softmax layer with 74 nodes is used to calculate the output, which is the number of RPs in the setup [6]. In turn, 74 RPs are used as input to the CNN to analyse its data characteristics. The input RSSI files for the 74 RPs will produce different images for input into the CNN model. Data augmentation is a useful technique which is employed for training general-purpose deep learning classifiers. It can speed up convergence or act as a regularizer, thereby avoiding overfitting and increasing generalization [21]. The data augmentation usually involves applying a set of transformations to the data space or feature space, or both. The most common augmentation generates new samples, which creates a larger number of data sets to prevent overfitting, to normalize the model, to balance classes in the database, and even to synthesize more representative new samples for use cases or tasks at hand. In this work, we plan to use data augmentation schemes to expand the existing data so that deep-learning algorithms could more effectively extract the necessary features. As more data input is beneficial for the CNN model to learn data features, the data is augmented. In [6], two data augmentation schemes were discussed for deep-learning architectures. In our work, an augmentation scheme that randomly changes the original data value within a difference range based on the difference between the calculated average value and original data is used in the data augmentation method. In our work, an augmentation scheme randomly changes the original data value within the determined range using mean and uniform random numbers to increase the number of the RSSI values in the reference data set. To identify the potential range for the uniform random number generator, the mean RSSI value for each RP is evaluated and then the current RSSI value is subtracted from it. The global repetition number is declared as N. The input data set is established in the form of CSV files. For example, let the global N be 60, the current RSSI be 33, and the mean value be 45. The potential range for the uniform random numbers becomes 12 (= 45−33), which generates any numbers within the range of 33 to 45 sixty times. If the range is equal to zero, the current RSSI value is repeated sixty times.

RSSI Database Setup
The Wi-Fi fingerprint-based indoor positioning system is divided into two phases: online and offline phases. The offline phase is used for database generation and training of the CNN classifiers. In the online phase, the trained CNN classifier is applied to the actual positioning experiment. In the offline phase, the RSSI fingerprint of each reference point is mapped using the smartphone and computer to build the database. The database in the offline phase consists of two types of databases, namely the trial database and training database. The trial database is used to perform the offline test simulation. The training database is used for training the deep-learning classifier. During the data collection stage, the RSSI values are collected for different times and directions. The specific data collection plan is as follows. In the morning, we conduct forward and backward data collections, respectively. We do the same thing in the afternoon. Forward data collection refers to collecting the RSSI value of each RP, from RP1 to RP74, in sequence. Backward data collection is the opposite of forward data collection, which collects the RSSI value of each RP from RP74 to RP1 sequentially. For data collection in each direction, the RSSI values of each RP are collected five times. The sampling time for each RP measurement is 5 s, which means 25 s for a total of five measurements. The data collection is performed for seven consecutive days. In total, about 28 RSSI data files are obtained. Since time and direction are employed as side information for the preprocessing stage, the collected data files are classified into four data sets: morning, afternoon, forward, and backward. Each data set includes 14 data files. The data set structure is shown in the following table. In Table 1, M and A indicate the time of data collection for morning and afternoon, respectively, and F and B indicate the direction of data collection for forward and backward, respectively. Finally, the numbers represent the days for which data were collected. Note that we combine all 28 data files to form a conventional data set to generate the conventional databases. Here, we divide the data files in each set into two groups, and one group forms the training database while the other group forms the trial database. For each data set, except for the conventional data set, we select 13 data files to perform the data augmentation in [6] for the training database and one rest data file for the trial database. For the conventional database, we select 24 data files, except for four data files used to generate the trial database, to perform data augmentation and then create a conventional training database. The specific division method and database types are shown in Table  2. During the training database construction phase, we performed four repeated measurements on the 74 RP points in one day (forward and backward measurements in the morning and afternoon, respectively) and measured the RSSI values of each RP five times for each measurement. Each data file thus contains 74 kinds of RPs, and each RP is measured five times. Therefore, each file contains 370 RPs. For each training database except the conventional database, there are 13 data files, and then, after the data augmentation scheme, each training database contains 288,600 RPs whose size is approximately 250 MB. The conventional training database contains 24 data files, and, after data augmentation, it contains a total of 532,800 RPs and the size is 350 MB.

Side-Information-Aided Preprocessing Scheme
The basic idea is that, in the offline phase, we input the training database constructed with side information and the trial database into the CNN model to train the classifier and select the classifier with the highest success rate among all simulation results. Then, we pass this classifier to the online phase. In the online phase, real-time measurement information on unknown user location is passed to the trained classifier. The trained classifier returns the user's location decision, which is then passed on to the user's smartphone. In [15] and [16], we found that when using conventional methods to test positioning performance, there were large differences between the positioning results obtained during the morning and afternoon. There was also a gap between the forward and backward test results that cannot be ignored. Therefore, we believe that the difference in test time and direction can affect the positioning results. We subdivide the fingerprint database according to the side information of time and direction and then use different databases to perform positioning tests under the corresponding test conditions. The specific test plan is shown as follows. The plan is divided into simulation, that is, an offline phase, and experiment, which is an online phase. In the simulation (offline phase), we use a mobile phone to collect the RSSI value of each RP and store it as a text file. We call the collected text files as data sets and then collate the collected data sets into CSV files. We call these CSV files as databases of two types: training databases and trial databases. Both training and trial databases are preprocessed using the proposed preprocessing method. Since we choose time and direction as side information to perform the preprocessing, here, we have created different databases based on morning and afternoon, as well as forward and backward. The two types of databases constructed with side information are simultaneously passed to the CNN model for feature learning. The specific process involves taking the training database and trial database as inputs and then performing feature learning in the CNN model. In our work, the Python-based CNN classifier produces a metafile after each epoch, which is a set of weights and biases. After multiple epochs, many metafiles are generated, and we select the metafile with the highest success rate for real-time positioning experiments (online phase). Based on the four kinds of side information, we perform four corresponding simulation tests. At the same time, we also use the conventional method for simulation. In the experiment plan (online phase), we use the classifier trained in the simulation phase in the software used for the experiment and later use the mobile phone and computer for location testing. In the actual measurements, RSSI values are collected at each unknown RP point, and the collected values are inputted to a trained classifier for comparative analysis to determine user position. For each experiment, we obtained five RSSI measurements at the same RP point and then the trained classifier returned five positioning decisions. Since we chose time and direction as side information to perform the preprocessing, we also strictly followed the side information requirements when conducting experiments. For example, in the morning, the classifier trained on the morning database was used to perform the positioning experiment in the front-rear direction. To verify the feasibility of this preprocessing, we used the conventional method to conduct experiments for the same time and direction.
We use a database constructed with side information to perform the positioning tests and analyse the positioning performances from simulated and real-time experimental results. We call this scheme the side-information-aided preprocessing scheme. The simulation and experimental environments of the side-information-aided preprocessing scheme are shown in Table 3. For the realtime experiment in each case, we conducted four tests over two days. We also used a conventional training database to simulate and perform a two-day real-time experiment, which we call the conventional scheme. We compare the results under the two schemes to verify the feasibility of the proposed design.

Simulation Results
For the side-information-aided preprocessing scheme, we performed simulation tests as follows. We inputted the training database into the CNN model with the help of the side information, trained the classifier through data feature learning, and inputted the corresponding trial database into the CNN model to verify the positioning performance of the trained classifier. Additionally, the simulation for the conventional scheme was performed. We inputted a conventional training database to the CNN model to train the classifier and then used the trial database. The positioning performances were compared under two schemes. In the simulation, we employed the success probability to describe the positioning accuracy. When using the trial database as the input, the success probability is the probability of correct position prediction with an error margin of two RPs. As mentioned earlier, the interval between two consecutive RPs is 2 m, so the precision of success was 4 m in this work. Here, we used three different margins (0, 1, and 2) to analyse the simulation results. The 0 margin indicates that the decision obtained at a certain RP position is entirely correct; the 1 margin means that the positioning decision result is within the range of (RP−1, RP+1), and the 2 margin indicates that the positioning decision value is within the range of (RP−2, RP+2). Since there are 74 RPs in the database, the total number of samples is 74. The success probability of the classifier is thus given as follows [7]: In the CNN model, the simulation results with the highest success probabilities were selected for comparisons and analyses. The simulation results refer to the success probabilities calculated by matching the 74 RPs in the trial database with the 74 RPs in the training database individually. The simulation results, in terms of success probabilities, are shown in Table 4. Here, we call the sideinformation-aided preprocessing scheme as the proposed scheme. It can be seen from all the results in Table 4 that, irrespective of 0, 1, or 2 margins, all types of databases preprocessed with side information can achieve better performance than conventional databases. With the margin of two RPs, the success probabilities of the proposed scheme for cases 1 and 2 (time as the side information for preprocessing) are higher than those of the conventional scheme by 6.55% and 4.97%, respectively. For example, according to Table 4, with margin 2, the positioning success probability of case 1 is 92.64% and that for the conventional case is 86.09%; thus, the difference between the two cases is 6.55%. Meanwhile, the success probabilities of the proposed scheme for cases 3 and 4 (direction as the side information for preprocessing) are higher than those for the conventional scheme by 3.07% and 4.42%, respectively. The principal component analysis (PCA) plot for training and test data is shown in Figure 5. In the Figure 5, graphs a-d correspond to four different positioning cases for the side-information-aided preprocessing scheme. The green points in the figure represent the reference points in the training database, and the red points represent the unknown location points (test data set). We analyze the numerical characteristics of the unknown points in the test database, and then compare them with the RP characteristics in the trained classifier. As shown in Figure 5, the green point closest to the red point is considered the final positioning decision.

Experimental Results
For real-time experiments, we use the trained classifier for location testing. The specific process involves inputting the measured RSSI value of the unknown point into the CNN model to analyse the numerical features. Then, the obtained features are compared with those in the trained classifier. The RP with the most similar features in the trained classifier is determined as the final position decision. Based on the side-information-aided preprocessing scheme, we used four different trained classifiers to perform the forward and backward positioning tests in the morning and afternoon as planned. For this scheme, we performed a total of 16 positioning experiments and conducted four experiments in each case. According to the conventional scheme, the eight-time positioning experiment was carried out using the conventionally trained classifier under the same conditions as the side-information-aided preprocessing scheme. When processing the experimental results, we stipulated that the test results with measurement errors less than or equal to 2 were successful, and the data represent the probability of successful results obtained in the measurement results. Table 5 shows the experimental results. As shown in Table 5, for each RP positioning test in an experiment, five positioning decisions were performed continuously for the same RP. In order to facilitate comparison between the results of the side-information-aided preprocessing scheme and the results of the conventional scheme, we merged all the experimental results into Table  6, which depicts the data results summarised according to different positioning times and directions. When analysing the data for all cases, we averaged the forward and backward results for cases 1 and 2 and averaged the morning and afternoon results for cases 3 and 4, respectively. The average results of the experiments are shown in Table 7. According to the results in Table 7, using the sideinformation-aided preprocessing for the classifier in the real-time experiments can achieve better positioning performance. Specifically, with the margin of two RPs, the side-information-aided preprocessing scheme achieved a gain of 4.93% to 5.8% as compared to the conventional positioning scheme. It is worth noting that, for cases 1 and 2, the performance differences between the proposed and conventional schemes increase as the margin increases. Meanwhile, for cases 3 and 4, the differences between the two schemes decrease as the margin increases. This means that, as the requirements for positioning accuracy become looser with larger margins, the difference in terms of success probability becomes larger when using time as the side information. In other words, the proposed scheme using time as the side information achieve higher gain with lower precision. Meanwhile, under the requirement of high-precision positioning, the proposed scheme, using direction as the side information, achieves higher gain. For example, for cases 3 and 4 with a margin of 0, the proposed scheme using the forward and backward directions as the side information shows gains of 13.72% and 15.06%, respectively. This means that the proposed scheme with time as the side information enables a higher success probability when the requirement of positioning accuracy is loose. Further, with direction as the side information, the proposed scheme shows better performance for the high positioning precision requirement. It is generally known that multiple sampling can help reduce the multipath effect on RSSI [22,23]. Therefore, a large-scale comprehensive database construction method is used to generate the fingerprint database. However, owing to the indiscriminate learning of data with multiple information, the unique features for the side information may be ignored. Therefore, it is necessary to build various databases based on the types of side information and to select a suitable database for each test condition. When constructing the database, it was found that some RSSI values of the adjacent RPs collected from different directions were the same. For example, a certain RSSI value of RP1 collected in the forward direction is identical to a certain RSSI value of RP2 collected in the backward direction. This is not conducive to the CNN model training classifier. For the RSSI values collected in different directions, the multipath effects may also be different [14]. Therefore, we consider that the database constructed with RSSI values collected at different times or in different directions can reduce the effects of dynamic environments. In the same environment, the data collected at different times may have different characteristics. Therefore, if indoor positioning is obtained at different times, it may be best to use a database positioning built at the corresponding time. Since the RSSI value is affected by various factors, we need to consider different conditions to generate multiple databases. Through location testing, we can select the database that is best suited to the current positioning environment. When the impact of external factors on positioning cannot be changed, multiple databases can be built based on different external factors, and the trained classifier can then be used for positioning experiments. Here, we choose the classifier that is suitable for the current positioning environment based on the experimental results, which can not only reduce the influence of external factors on positioning but also improve positioning accuracy.

Conclusions
In this work, multiple fingerprint databases based on different types of side information were constructed and employed to determine the classifiers that would be trained by the corresponding fingerprint databases. It was shown that selection of a suitable trained classifier for a particular situation could effectively improve indoor positioning accuracies. The simulated and experimental results showed that the side-information-aided preprocessing scheme used herein allowed better positioning success probability than conventional schemes. With two margins, the proposed scheme could improve the simulated and experimental performances by 6.55% and 5.8%, respectively, in comparison to the conventional scheme. The proposed scheme, with time as the side information, also obtained a higher success probability when the positioning accuracy requirement was loose with a larger margin. Furthermore, with direction as the side information, the proposed scheme showed better performance for the high positioning precision requirement. As a future work, we plan to consider other factors, such as weather, temperature, humidity, and traffic, to establish more positioning test databases and to investigate which of these databases could better handle positioning experiments under various conditions.