Indoor Localization Using Semi-Supervised Manifold Alignment with Dimension Expansion

: Location estimation plays a crucial role in Location-Based Services (LBSs) with satisfactory user experience. The Wireless Local Area Network (WLAN) localization approach is preferred as a cost-efﬁcient solution to indoor localization on account of the widely-deployed WLAN infrastructures. In this paper, we propose a new WLAN Received Signal Strength (RSS)-based indoor localization approach using the semi-supervised manifold alignment with dimension expansion. In concrete terms, we ﬁrst construct an innovative objective function based on the augmented physical coordinates and the corresponding WLAN RSS measurements. Second, the closed-form solution to the objective function is derived out according to the Lagrange multiplier equation, which results in the manifold in physical coordinate space. Third, the target location is estimated by matching the transformed newly-collected RSS against the manifold. The localization performance with noise perturbation is analyzed upon the constructed objective function, and meanwhile, the closed-form solution to the objective function with respect to multiple types of measurements is also derived out for the sake of leveraging all of the potential measurements for indoor localization. The extensive testing results show that the proposed approach performs well in localization accuracy even at low calibration load, and its performance can be further improved by using multiple types of measurements for localization.


Introduction
In modern society, people spend almost 90% of their time within indoor environments [1], which provides great potential and vast development prospects towards indoor Location-Based Services (LBSs) [2,3].Although the Global Navigation Satellite System (GNSS) [4] is regarded as the most prestigious and prevalent solution to outdoor localization, it has deteriorated localization performance or even has no assistance for localization by considering the serious signal attenuation in the structure-complex indoor environment, such as shopping malls, airports and underground parking lots [5].Therefore, various indoor localization alternatives are proposed to make up for the GNSS in indoor environments, like the Assistant Global Positioning System (A-GPS) [6], cellular networks [7], Wireless Local Area Networks (WLAN) [8], Bluetooth [9], Radio Frequency Identification (RFID) [10], Near Field Communication (NFC) [11], Visible Light Communications (VLC) [12], Infrared Radiation (IR) [13] and motion sensor-based [14] localization approaches.Among them, the WLAN-based localization approach is preferred due to the wide deployment of WLAN infrastructures [15].
The measurements, like Time Of Arrival (TOA) [16], Time Difference Of Arrival (TDOA) [17] and Angle Of Arrival (AOA) [18], as well as the propagation model [19] and Received Signal Strength (RSS) fingerprinting [20], have attracted significant attention for WLAN localization.As the most frequently-used measurement, RSS fingerprinting [21] is featured with no extra hardware requirement.However, the RSS fingerprinting involves time-consuming and labor-intensive location calibration, which significantly limits its application to the large-scale indoor environment.
To solve this problem, the semi-supervised learning approach [22] has been studied intensively for indoor localization based on a limited number of calibrated locations with known physical coordinates and the corresponding collected RSS measurements.On the one hand, the semi-supervised learning approach requires a small number of calibrated locations, which can effectively relieve the load for the off-line site survey of the target environment.On the other hand, the satisfactory localization performance is normally reached by the semi-supervised learning approach without any motion sensor assistance.As one of the most popular representatives in semi-supervised learning, the manifold alignment approach has been carefully investigated [23].The conventional manifold alignment approaches generally assume that the measurements collected at nearby locations, which are used for localization, are featured with good neighborhood relations in a low-dimensional manifold.Based on this, the solution to the objective function in manifold alignment is calculated as the closed-form in the low-dimensional manifold.Then, the target locations are estimated by the closed-form solution to the objective function in the low-dimensional manifold.This approach heavily depends on the similarity of the calibrated fingerprints in the low-dimensional space, which cannot well preserve the neighborhood relations of the calibrated fingerprints, and meanwhile, its objective function involves the online RSS measurements, which decrease the stability of transformation relations among the calibrated fingerprints.
Different from the conventional manifold alignment approaches, we propose a new one that can better preserve the neighborhood relations of the physical coordinates and the corresponding RSS measurements at nearby locations.The basic idea of the proposed approach is to perform the augmentation of physical coordinates to make the dimensions of the augmented physical coordinates and RSS measurements equal and then construct the two transformation matrices with respect to the augmented physical coordinates and the corresponding raw RSS measurements.After that, the target locations can be estimated by using the neighbors in the common manifold transformed from the augmented physical coordinates, as well as the newly-collected RSS measurements.In sum, the three main contributions of this paper are summarized as follows.First of all, an innovative objective function in manifold alignment is constructed based on the augmented physical coordinates and the corresponding RSS measurements, which can be easily generalized to the scenario with multiple types of measurements used for localization.Second, the localization performance under different noise perturbation is discussed for the sake of exploring the robustness of the function-based localization solution in various noise environments.Third, by the extensive testing results, a higher localization accuracy is verified to be achieved by the proposed approach compared with the existing Weighted K-nearest Neighbor (WKNN), Bayesian, kernel and manifold alignment approaches.
The rest of this paper is organized as follows.In Section 2, we survey some related work on the manifold alignment approaches used in RSS fingerprinting-based indoor localization.In Section 2, the proposed approach is described in detail.Then, the testing results are presented in Section 4. Finally, in Section 5, we conclude the paper and provide some future directions.

Related Work
As we know, the RSS fingerprinting-based indoor localization requires the site survey of the target environment, which is always time-consuming and labor-intensive.To solve this problem, the semi-supervised learning approach has attracted significant attention by using a small number of calibrated locations to achieve the satisfactory localization performance [24].As one of the most popular representatives in semi-supervised learning, the manifold alignment approach is much preferred for indoor localization.This approach is based on the assumption that the measurements collected at nearby locations, which are used for localization, like the RSS, TOA and AOA, are featured with good neighborhood relations in a low-dimensional manifold [25].By perturbing the local geometries of physical coordinates, the manifold alignment approach in [26] locates the target with a limited number of calibrated locations.The authors in [27] give a solution to the fast radio map construction in indoor WLAN environment by using the crowdsourced manifold alignment approach.Additionally, the locations of the target are estimated by matching the newly-collected RSS measurements against the radio map in a low-dimensional manifold.By employing the unsupervised manifold alignment approach, the authors in [28] utilize the WLAN RSS fingerprints and newly-collected RSS measurements to construct a source dataset and meanwhile rely on the coordinates of RSS fingerprints to construct a destination dataset.After performing the transformation from the source into destination datasets, the target is located based on the neighbor matching of the transformed data with respect to the newly-collected RSS measurements and coordinates of RSS fingerprints in the common manifold.The authors in [29] apply the local tangent space alignment and Multidimensional Scaling (MDS) approaches to develop a new manifold learning approach, by which the pairwise distance between every two RSS, TOA and TDOA measurements is considered to construct various local maps, and the overlap of different local maps is aligned to obtain the relative global coordinates in physical space.The manifold flattering approach for anchor-less localization is proposed in [30].This approach relies on the Euclidean distance matrix to gradually infer the unknown distances among the newly-collected RSS measurements and then applies the MDS approach to calculate the relative locations of the target.The authors in [31] propose to use the semi-supervised Laplacian regularized least squares approach for localization.Specifically, this approach depends on an alignment criterion to learn the appropriate kernel function with respect to the similarities between different anchors and then uses the manifold regularization-based semi-supervised learning approach to locate the target.A motion sensor-free localization approach is discussed in [32], in which the floor plan and temporal RSS sequences are used to perform the graph matching-based manifold alignment.The authors in [33] embed the RSS measurements collected on different floors into a common low-dimensional manifold and then align the labeled and unlabeled RSS measurements for localization.In all, the previous literature mainly focuses on the similarity of the calibrated fingerprints in the low-dimensional space and involves the online RSS measurement in the objective function, which sometimes cannot preserve the neighborhood relations of the calibrated fingerprints well and guarantee the stability of transformation relations with respect to the physical coordinates and the related RSS measurements.
Different from the approaches mentioned above, the proposed semi-supervised manifold alignment with dimension expansion is based on an innovative objective function that can better preserve the neighborhood relations of the augmented physical coordinates and the corresponding RSS measurements at nearby locations.Our goal is to obtain the transformation matrices with respect to the augmented physical coordinates, as well as the corresponding RSS measurements, which form the common manifold, and then achieve the localization of the target based on merely the known transformation relations.In addition, the localization performance under different noise perturbation conditions is analyzed, and meanwhile, the generalization of the objective function to the scenario with multiple types of measurements used for localization is also discussed.

Location Calibration
In the target environment, we denote the sets of l calibrated locations, as well as corresponding RSS measurements as RSS value from the k-th AP at the i-th calibrated location and m is the number of Access Points (AP).To guarantee that the physical coordinates and RSS measurements have the same dimension, we simply replicate the physical coordinate of each calibrated location m/2 times (when m is odd, the replication will do (m − 1) 2 times, which is ended by adding the former element of the two-dimensional physical coordinate, such that H =

Manifold Alignment with Dimension Expansion
Let X = {x i } and Y= {y i } (1 ≤ i, j ≤ l) be the manifolds corresponding to the augmented physical coordinates and the corresponding RSS measurements, respectively.Here, the augmented physical coordinates and the corresponding RSS measurements are two different types of measurements.Then, the objective function in the proposed manifold alignment for indoor localization is constructed as: where ; h i and g j stand for the i-th augmented physical coordinate and j-th RSS measurement, respectively; and the notation "|| • ||" represents the two-norm operation.In (1), the first term is to guarantee that the manifolds corresponding to the augmented physical coordinates and the corresponding RSS measurements can be well aligned, while the second and third terms are to preserve the neighborhood relations of the augmented physical coordinates and the corresponding RSS measurements at nearby locations, respectively.Since the manifolds X and Y are obtained from the transformation of the augmented physical coordinates and the corresponding RSS measurements, we can convert the objective function into the one by optimizing the transformation matrices with respect to the augmented physical coordinates and the corresponding RSS measurements, P x and P y , such that: Since: ∑ i,i ∈{1,2,...,l} ∑ j,j ∈{1,2,...,l} are the diagonal matrices satisfying By applying the scale and translation invariance constrains of the solution to (7), we set P T ZZ T P = I and P T Ze = 0, where I is a m × m unit matrix and e is a 2l × m all one matrix.Using the Lagrange multiplier approach, we can construct the Lagrange equation in (8).L(P) = P T ZΩZ T P − λ(P T ZZ T P − I ) − µP T Ze (8) where λ and µ are the Lagrange coefficients.Calculating the partial derivative of L(P) with respect to P, we obtain: By setting (9) equal to zero, one has: Meanwhile, by multiplying VP T to (10), we can have: where V is a 2m × m matrix satisfying VP T = I" and I" is a 2m × 2m unit matrix; we note that P T Ze = 0; thus, we can obtain: Let ZΩZ T = E and ZZ T = M.Then, we can rewrite (12) as: Finally, since P T ZΩZ T P=λP T ZZ T P=λI', the solution to ( 7) is equivalent to the solution of the m generalized eigenvectors, which are corresponding to the m non-zero smallest generalized eigenvalues in (13).

Target Localization
For each location query, we transform the newly-collected RSS measurement r = (rss new 1 , rss new 2 , ..., rss new m ) into P y T r T , where rss new is the newly-collected RSS value from the k-th AP.After that, we locate the target at the location, which is closest to rss new

Noise Perturbation
Considering the noise perturbation in the indoor environment, we focus on the impact of an increment of G, ∆, on the localization performance in this section.In this case, the objective function in (6) Similarly, we rewrite (15) as: To obtain the solution of P in ( 16), we construct the Lagrange equation as follows.
L(P) = P T ZΩZ T P + P T ZΩ 0 0 0 Then, the partial derivative of L(P) with respect to P equals: Since Ω T = Ω, (18) can be equivalent to: Due to: we can rewrite (19) as:

Impact of Noise Perturbation on P
We now explore the difference between the partial derivative in ( 9) and (21).From (21), if P stays unchanged as the value ∆ increases, then the value −H∆ T P y will decrease, and consequently, we have: Thus, P will change as the value ∆ increases.In addition, since: where L x =D x − S x and L y =D y − S y are the symmetric matrices, we can obtain that the expectation of each column in I + 2L x − λI, C t (1 ≤ t ≤ l), equals: where the notation "E(•)" represents the expectation operation.At the same time, it is obtained that λ l from (15), and therefore, the value E(C t ) should be zero from the statistical aspect.In this case, we have To make (21) equal to zero, −H∆ T P y − HG T P y is required to be a constant, and thereby, the value P y will decrease as the value ∆ increases.Similarly, −H∆ T P x + (G(I + 2L y )∆ T + ∆(I + 2L y )G T + ∆(I + 2L y )∆ T )P y − GH T P x is also required to be a constant by the zero expectation constraint.Therefore, we conclude that the value P decreases as the value ∆ increases.

Impact of Noise Perturbation on Localization Accuracy
As the value ∆ increases, we assume that the decreased values with respect to the solutions to (7), P x T and P y T , are P x T −Φ 1 and P y T −Φ 2 , respectively.We notate h p (1 ≤ p ≤ l) as the estimated coordinate in H T for the newly-collected RSS measurement r without the variation ∆.For simplicity and without losing generality, we select the one-norm operation for the localization.Then, we have: where the notation "|| • || 1 " represents the one-norm operation.Under the variation ∆, we assume that the transformed vector for the newly-collected RSS measurement, r, is (P x T −Φ 1 )r T , and the manifold corresponding to H T is {(P y T −Φ 2 )h 1 , (P y T −Φ 2 )h 2 , ..., (P y T −Φ 2 )h l }.Based on this, the distance between h p and r T is changed into: and meanwhile, the distance between h k (k ∈ {1, 2, ..., l}, k = p) and r T equals: Then, it is obtained that ||(P y

and due to the arbitrariness of
Therefore, h k becomes the estimated location with respect to r, which will be deteriorated as the value ∆ increases.

Impact of System Parameters on Localization Accuracy
First of all, with the increase of number of samples at each calibrated location, the RSS variation can be described as: where r ij is the RSS from the j-th AP in the i-th RSS sample; n is the number of samples; and m j1 and m j2 are the mean of n and n + u RSSs from the j-th AP, respectively.Based on (29), we can easily obtain that as the sample number increases, the value ∆ will decrease, and consequently, higher localization accuracy is achieved.Second, as the number of calibrated locations increases, the values L x and L y decrease, which will result in the decrease of the values H(I + 2L x − λI) and G(I + 2L y − λI) accordingly.Then, to make (23) equal to zero, the value P should be increased, which indicates that an increase of the number of calibrated locations results in a higher localization accuracy.
Third, with the increase of augmentation dimension from m to 2m, 3m, etc., the values L x and L y increase, which will result in the increase of the values H(I + 2L x − λI) and G(I + 2L y − λI) accordingly.In this case, to make (23) equal to zero, the value P should be decreased, which indicates that the increase of augmentation dimension results in the lower localization accuracy.
, and , where P u is the transformation matrix with respect to D u .Similarly, we can convert (30) After the solution of P in ( 31) is obtained, we locate the target by: arg min where P 1 and P p (2 ≤ p ≤ t) stand for the transformation matrices with respect to the augmented physical coordinates I aug and p-th type of measurement, respectively, and new p is the p-th type of newly-collected measurement in (33).
where new p k is the newly-collected value corresponding to p-th type of measurement from the k-th AP.

Testing Results
To demonstrate the analytical results derived in the previous section, we perform the simulations about the proposed approach under different noise perturbations, signal propagation models (see Table 1) and the number of calibrated locations and RSS samples in an indoor environment with the dimensions of 56 m × 19.3 m, as shown in Figure 2.This environment is covered by five WLAN APs, notated as AP1, • • • , AP5, and meanwhile, there are 93 labeled locations (with "•"), which are evenly distributed with 2-m intervals.Here, the simulation parameters are set as follows.The standard deviation of noise ranges from 2 dB to 5 dB; the ratio of calibrated fingerprints ranges from 30% to 100%; and the number of RSS samples at each RP ranges from 150 to 200.

Types of Models Models Formulation Parameters Setting
Log-distance [33]  Figure 3a shows the sum of elements in P x and P y under different values ∆.The purple and green lines stand for the fitted curves with respect to P x and P y , respectively.From this figure, we can find that as the value ∆ increases, the sum of elements in both P x and P y decreases, which demonstrates that the value P will decrease with the increase of ∆.
Figure 3b shows the Cumulative Distribution Functions (CDFs) of errors by using the proposed approach under different standard deviations of noise.From this figure, we can find that a slighter noise perturbation (or smaller standard deviation of noise) results in a higher localization accuracy as expected.In Figure 3c, we show the localization performance of the proposed approach when the signal attenuation property satisfies the log-distance [34], multi-walls [35], Supachai [36] and breakpoint [37] models, respectively.From this figure, we can find that the breakpoint model performs best in localization accuracy due to the fact that most of the labeled locations are distributed in the straight corridors, which is applicable to the breakpoint model [38].In the results that follow, we set the initial parameters as follows.The standard deviation of noise is 2 dB; the ratio of calibrated fingerprints is 100%; the number of RSS samples at each RP is 200; the signal attenuation property satisfies the breakpoint [37] model in the simulation; and the augmentation dimension is m, where m is the number of APs.Based on this initial parameter setting, we use the control variates method to perform each testing.Specifically, when the value of a parameter varies, the initial values are distributed to the other parameters.
Figure 3d shows the CDFs of errors by using the proposed approach under different ratios between the number of calibrated and labeled locations (or called different ratios of calibrated fingerprints).From this figure, we can find that as the ratio between the number of calibrated and labeled locations increases, the localization accuracy also increases.For instance, the mean error under the ratio of 30% (or by using 28 calibrated locations) is 3.3 m, while the one under the ratio of 100% (or by using 93 calibrated locations) decreases to 2.1 m. Figure 3e shows the variation the CDFs of errors as the number of RSS measurements collected at each calibrated location increases from 150 to 200.As expected, a larger number of collected RSS measurements results in a higher localization accuracy.
Finally, based on the same layout in Figure 2, we give a simple example about the localization performance by using the single RSS measurement, double RSS and AOA measurements and triple RSS, AOA and TOA measurements for localization, respectively, in Figure 3f.In the testing, the standard deviations of AOA and TOA measurements are assumed to be evenly distributed in the ranges of [0.5 • , 5 • ] and [0.3 ns, 6 ns], respectively, which are also used in [39].From this figure, we can find that more types of measurements used for localization generally result in a higher localization accuracy, which triggers the motivation of generalizing the proposed approach to the scenario with multiple types of measurements used for localization.

Experimental Results
In this section, we will continue to examine the performance of the proposed approach in the real indoor environment with the same layout in Figure 2. We set that the ratio of calibrated fingerprints ranges from 30% to 100%, the number of RSS samples at each RP ranges from 10 to 150 and the augmentation dimension ranges from m to 5m, where m is the number of APs.The CDFs of errors by using the proposed approach under different ratios between the number of calibrated and labeled locations are shown in Figure 4a, from which we can find that the localization accuracy also increases as the ratio increases.
Figure 4b shows the CDFs of errors by using the proposed approach under different numbers of RSS measurements collected at each calibrated location.As expected, a larger number of collected RSS measurements results in a higher localization accuracy, which is in accordance with the previous discussion.The CDFs of errors by the proposed approach under different augmentation dimensions are shown in Figure 4c.As can be seen from this figure, the localization performance deteriorates with the increase of augmentation dimension.Meanwhile, as for the augmentation corresponding to the physical coordinates, we also explore the localization performance with random replication with which the replicated value is selected randomly from the initial two-dimensional physical coordinate.Figure 4d shows the localization errors under different replications for the physical coordinates augmentation.Each running time relates to a specific replication, and the corresponding CDFs of the mean and standard deviation of errors are shown in Figure 4e.From these figures, we can find that more than 80% mean errors are smaller than 3 m, and meanwhile, the localization accuracy can be well guaranteed under most of the replications for the physical coordinates augmentation.
Figure 4f compares the CDFs of errors of the target by using the proposed and other four popular approaches, i.e., WKNN, Bayesian, kernel and the conventional manifold alignment without transformation matrices [28].From this figure, we can find that the proposed approach outperforms the other four in localization accuracy, which demonstrates the significant benefit provided by the dimension expansion to the manifold alignment for indoor localization.In addition, the mean and standard deviation of errors are also illustrated in Table 2, from which we can find that both the smallest mean and standard deviation of errors, i.e., 2.4 m and 1.1 m, are achieved by the proposed approach.Furthermore, Figure 5 shows the mean of errors under different numbers of RSS samples at each RP and the ratios of calibrated fingerprints, from which we can find that the proposed approach achieves good localization performance especially under the small number of RSS samples at each RP and low ratios of calibrated fingerprints.

Conclusions
We have proposed a new semi-supervised manifold alignment approach with dimension expansion for indoor WLAN localization.Based on the innovative objective function constructed from the augmented physical coordinates and the corresponding RSS measurements, the neighborhood relations of the nearby locations can be well preserved in the corresponding manifolds.In addition, the localization performance has been carefully analyzed by considering different noise perturbations and system parameters, and meanwhile, the objective function corresponding to the multiple types of measurements used for localization has also been deduced.Finally, the testing results demonstrate that the proposed approach is able to achieve the satisfactory localization accuracy especially with limited location calibration load.In future, the application of the proposed approach to a large-scale indoor multi-floor environment forms an interesting topic.that the manifolds corresponding to the t types of measurements can be well aligned, while the second to (t + 1)-th terms are to preserve the neighborhood relations of different types of measurements at nearby locations.Since the manifold X Nu is obtained from the transformation of D u , we can convert the objective function into the one by optimizing the transformation matrix with respect to D u , P u , such that: and , we can simplify (A5) into: arg min P {Tr(P T ZΩZ T P)} (A6) Similarly, by applying the scale and translation invariance constrains of the solution to (A6), we have P T ZZ T P = I t and P T Ze t = 0, where I t is an m × m unit matrix and e t is a (t × l) × m all one matrix.Using the Lagrange multiplier approach, we can construct the Lagrange equation in (A7).L(P) = P T ZΩZ T P − λ t (P T ZZ T P − I t ) − µ t P T Ze t (A7) where λ t and µ t are the Lagrange coefficients.Calculating the partial derivative of L(P) with respect to P, we obtain: ∂L(P) ∂P = 2ZΩZ T P − 2λ t ZZ T P − µ t Ze t (A8) By setting (9) equal to zero, one has: 2ZΩZ T P − 2λ t ZZ T P − µ t Ze t = 0 (A9)
where the notation "Tr(•)" represents the trace of the matrix; L x = D x − S x and L y = D y − S y ; S x = S x ii and S y = S y jj ; and D x = D x ii and D y = D y jj min P {Tr(P T ZΩZ T P + P T ZΩ 0 s.t.P T ZZ T P = I', P T Ze = 0 into (the details of the derivation are shown in the Appendix): arg min P {Tr(P T ZΩZ T P)} s.t.P T ZZ T P = I t , P T Ze t = 0.

Figure 2 .
Figure 2. Layout of an indoor environment.

Figure 3 .
Figure 3. Parameter discussion in simulation.(a) Sum of elements in P x and P y under different values ∆; (b) CDFs (Cumulative Distribution Functions) of errors under different noise perturbations; (c) CDFs of errors under different signal attenuation properties; (d) CDFs of errors under different numbers of calibrated locations; (e) CDFs of errors under different numbers of RSS measurements; (f) CDFs of errors by using different types of measurements for localization.

Figure 4 .Figure 5 .
Figure 4. Parameter discussion in the actual environment.(a) CDFs of errors under different numbers of calibrated locations; (b) CDFs of errors under different numbers of RSS measurements; (c) CDFs of errors under different augmentation dimensions; (d) localization errors under different replications for physical coordinates augmentation; (e) CDFs of mean and standard deviation of errors; (f) CDFs of errors by different approaches.
can be converted into: min P x ,P y {Tr(P x T HH T P x + P y T (G+∆)(G+∆) T P y − P x T H(G+∆) T P y − P y T (G+∆)H T P x )+ 2Tr(P x T HL x H T P x ) + 2Tr(P y T (G+∆)L y (G+∆) T P y )} = min P x ,P y {Tr(P x T HH T P x + P y T GG T P y − P x T HG T P y − P y T GH T P x ) + Tr(P y T G∆ T P y + P y T ∆ T GP y + P y T ∆∆ T P y − P x T H∆ T P y − P y T ∆H T P x ) + 2Tr(P x T HL x H T P x ) + 2Tr(P y T GL y G T P y )+ 2Tr(P y T GL y ∆ T P y ) + 2Tr(P y T ∆L y G T P y ) + 2Tr(P y T ∆L y ∆ T P y )}

Table 1 .
Parameters in different signal propagation models.

Table 2 .
Mean and standard deviation of errors by different approaches.
min {P 1 ,P 2 ,...,P t } { ∑ w,v∈{1,2,...,t} and w =v q∈{1,2,...,l} (||P w T d wq − P v T d vq || 2 ) + ∑ ,2,...,t} and w =v q∈{1,2,...,l} (||P w T d wq − P v T d vq || 2 ) = ∑ w,v∈{1,2,...,t} and w =v Tr(P w T D w D w T P w+ P v T D v D v T P v − P w T D w D v T P v − P v T D v D w T P w ) ∈{1,2,...,l} (||P u T d uq − P u T d uq || 2 S Nu qq ) = 2Tr(P u T D u L u D u T P u ) (A4)whereL u = E u − S u ; S u = S Nu qq ; and E u = E u qq is a diagonal matrix satisfying E u qq = ∑ l q =1 S Nuqq , we can convert (A2) into: arg min {P 1 ,P 2 ,...,P t } {∑ t u=1 2Tr(P u T D u L u D u T P u ) + ∑ w,v∈{1,2,...,t} and w =vTr(P w T D w D w T P w + P v T D v D v T P v − P w T D w D v T P v − P v T D v D w T P w )