GDCPlace: Geographic Distance Consistent Loss for Visual Place Recognition

Abstract

Visual place recognition (VPR) is essential for robots and autonomous vehicles to understand their environment and navigate effectively. Inspired by face recognition, a recent trend for training a VPR model is to leverage classification objective, where the embeddings of images are trained to be similar to corresponding class centers. Ideally, the predicted similarities should be negative correlated to the geographic distances. However, previous studies typically used loss functions from face recognition due to the similarity between the two tasks, which cannot guarantee the rank consistency above as face recognition is unrelated to geographic distance. Current methods for distance-similarity or ordinal constraint are either for sample-to-sample training, only partially meet the constraint, or are incapable for the VPR task. To this end, we provide a mathematical definition geographic distance consistent defining the above consistency that the loss function for VPR should adhere to. Based on it, we derive the upper bound of cross-entropy softmax loss under the desired constraint to minimize, and propose a novel loss function for VPR that is geographic distance consistent, called GDCPlace. To the best of our knowledge, GDCPlace is the first classification loss function designed for VPR. To evaluate our loss, we collected 11 benchmarks that have high domain variability to test on. As our contribution is on the loss function and previous classification-based VPR methods mostly adopt face recognition loss function, we collect several additional loss functions to compare, e.g., loss for face recognition, image retrieval, ordinal classification, and general purpose. The results show that GDCPlace performs the best among different losses and former state-of-the-art (SOTA) for VPR. It is also evaluated for ordinal classification tasks to show the generalizability of GDCPlace.

1. Introduction

Visual place recognition (VPR) helps robotics and autonomous systems recognize familiar locations, which is crucial for tasks like mapping, localization, and path planning. Stated simply, it aims to predict the geographical localization of where a photo was captured. It is frequently depicted as a retrieval task, meaning that the system stores a large number of images with localization to retrieve for an input image [1,2,3]. There are two classic types of training and inference strategies: (1) sample-to-sample training, inference in a retrieval form [4,5,6,7]; (2) training and inference both in a classification form [8,9]. Recently, motivated by other retrieval tasks like face recognition [10,11] and landmark retrieval [12,13], there are some methods that train the retrieval model using a classification proxy but carry out inference as a retrieval task, achieving a significant boost in convergence speed and decreasing implementation difficulties while maintaining comparable performance [14,15].

There is a reasonable constraint that the visual place recognition methods should address, namely, the distances from the location where the shot was taken to locations A, B, and C progressively increase, then the similarity of image embeddings to locations A, B, and C should correspondingly decrease. However, current methods are incapable of achieving this for classification training. Specifically, they are either (1) implemented in sample-to-sample training paradigms, which are not suitable for classification training [16,17]; (2) only aiming to place correct images ahead of wrong ones, but cannot guarantee the exact order [18,19]; or (3) ordinal classification methods deal with 1d ranking data (e.g., age) [20,21], whereas VPR has 2d geographic data. Furthermore, there is currently no classification loss designed specifically for VPR, with most approaches replicating CosFace [10] and ArcFace [11]. A list of related works can be viewed in

Table 1. List of related state-of-the-art papers.

References	Training Strategies	Inference Strategies	Methods	Limitations
Arandjelović et al. [4]	Sample-to-Sample	Retrieval	NetVLAD integrates a differentiable VLAD layer into a CNN, aggregating local features into a global descriptor.	High computational and memory requirements, along with sensitivity to variations in local features.
Ali-Bey et al. [5]	Sample-to-Sample	Retrieval	MixVPR fuses CNN feature maps using cascaded lightweight MLP blocks to generate compact global descriptors.	Its reliance on fixed pretrained backbones and MLP fusion can make it sensitive to noise.
Shen et al. [6]	Sample-to-Sample	Retrieval	StructVPR distills structural knowledge by dynamically weighting training samples to guide global descriptor learning.	Its effectiveness relies on accurate structural cues, making it potentially sensitive to noisy scene structures.
Ali-Bey et al. [7]	Sample-to-Sample	Retrieval	GSV-Cities introduces a large-scale, accurately labeled urban dataset to train VPR models	Its approach may be less effective in non-urban settings and relies on extensive, high-quality annotations.
Trivigno et al. [8]	Classification	Classification	Divide & Classify reformulates VPR task as a fine-grained classification task by partitioning dense urban maps into non-adjacent cells.	Its performance depends heavily on dense, well-distributed training data and may struggle with small-scale regions.
Muller-Budack et al. [9]	Classification	Classification	The method treats geolocalization as a classification task by hierarchically partitioning the Earth into cells and integrating scene context.	It suffers from ambiguity in scene content in defining effective partitions, reducing localization accuracy in transitional or unclear areas.
Berton et al. [14]	Classification	Retrieval	CosPlace reformulates visual geo-localization as a classification task by partitioning the city into discrete regions and training a CNN with an CosFace loss.	Its performance is highly sensitive to the partitioning strategy. Errors in defining region boundaries or ambiguous scene cues can lead to misclassifications.
Berton et al. [15]	Classification	Retrieval	EigenPlace trains a CNN to produce viewpoint-robust global descriptors by incorporating eigen-decomposition techniques that emphasize stable, discriminative features.	Its reliance on eigen-decomposition and the need for diverse, extensive training data can limit generalization and add computational complexity.

.

Motivated by the above reasons, we first give a mathematical definition called geographic distance consistent, which is defined for those loss functions that encourage embedding similarities negatively correlated to the geographic distances. We derive an upper bound of the cross-entropy softmax loss under the desired constraint. Supported by a few lemmas, we prove that the resulted formulation, GDCPlace, meets the definition we require. We also reveal several other useful patterns it exhibits. Further, inspired by hard negative mining (HNM) [22,23], we propose to implicitly mine hard negative classes by merely modification to the formulation of our loss function, thus free of time and memory overhead. We call this technique hard negative class mining (HNCM).

Extensive experiments were conducted to evaluate the performance of our method. To evaluate the performance and generalization capacity, we collect 11 benchmarks which cover different cities, weathers, and viewpoints [4,24,25,26,27,28,29]. We test our loss function on the proposed method, EigenPlace [15]. We first compare with other classification loss functions that have been/have the potential to be adopted to VPR training [10,11,30,31,32,33,34]. Then, we also follow recent works to adopt DINOv2 backbone [35] to further evaluate its performance compared with other SOTAs. Our method exhibits the overall best performances in the comparisons. Remarkably, under the identical training setup, substituting the CosFace loss [10] utilized by EigenPlace [15] with GDCPlace loss results in an increase of recall@1 by over 11% on SVOX Night [25]. The generalizability is also evaluated on ordinal classification benchmarks where GDCPlace is capable to make precise estimation. GDCPlace is both experimentally and theoretically evaluated to demonstrate its effectiveness in training a VPR model. It is also proven to be capable of benefiting other methods.

The main contributions can be summarized as follows:

(1)

To help models learn reasonable relationships between embedding of different localization in VPR, we propose GDCPlace, a classification loss function that holds similarity-distance constraint. To the best of our knowledge, this is also the first loss function designed specifically for training a classification proxy in a VPR retrieval model.

(2)

The mathematical analysis is presented to demonstrates the effectiveness of GDCPlace in a theoretical view.

(3)

We propose hard negative class mining through loss function design to help classification training with no overhead.

(4)

Experimental results show that GDCPlace outperforms previous classification loss functions and other SOTA methods in various VPR benchmarks. Experiments on ordinal classification further demonstrate its generalizability.

The rest of our article is organized as follows. We introduce and discuss the related works in Section 2. The details of our proposed method is in Section 3. The effectiveness of the proposed method is evaluated on extensive benchmarks, as introduced in Section 4. Finally, the conclusion and the future work directions are presented in Section 6.

2. Related Works

2.1. Classification Loss for Retrieval

To tackle open-set classification tasks like VPR [10,12,14,36], it has been firstly de facto to adopt sample-to-sample loss [4,19,37,38,39], as it well fits the training object. Although it works well in several cases, it has some unpleasant drawbacks recognized in general. For instance, it is difficult to converge [40], there is a requirement for hard negative mining [41], and there is training time overhead [42]. It has been proposed to set up a proximal classification task, giving samples of the same kind a center to represent each category of images [43]. From then on, several subsequent works were about the design of such proximal classification loss [30,44,45]. Subsequently, it has begun a long period of trending to work on the margin. Just as most sample-to-sample loss has additional margin penalty, it has been shown that it is also beneficial to classification loss functions [46]. The works can be, in general, divided into three lines: (1) Find where the best location is to place the margin term [10,11,45]; (2) Determine how to dynamically tuning the margin during training [31,32,47]; (3) Perform the theoretical analysis of margin loss [34,48]. For instance, ArcFace [11] places the margin terms to the angle between embedding and central weights, contributing to a reasonable manifold explanation. AdaFace [31] further achieves dynamic tuning of the margin term. LM-Softmax [34] pushes the inter-class distance to maximum backup by theoritical analysis. Recent VPR works utilize face recognition loss functions, especially CosFace and ArcFace, to train models [8,14,15]. They take different localizations as different faces in face recognition to perform the training. However, they are not designed for VPR, and are not capable of considering the relationship between embedding similarities and geographic distances, which motivates this work.

2.2. Visual Place Recognition

VPR aims to predict the localization given photos taken in a certain place [28,49,50,51,52]. In contrast to place recognition based on LiDAR [53], it is more cost-effective and has lower power consumption. A dataset with photos and the corresponding localization are constructed such that we can match the correct image given the query photo, and the localization is, thus, obtained. Like many other computer vision areas, it began with hand-craft local features processing [54,55], such as SIFT [56], SURF [57], BRIEF [58], and the fusion of different works [59]. As we came to the deep learning era, much attention has been dedicated to the design of the head layer to aggregate local features to a global one. NetVLAD is a popular CNN module for solving VPR tasks in an end-to-end style [4], with a novel weakly-supervised ranking loss proposed. It works as a global feature integrator that can be plugged into neural networks like NetBoW [60] and NetFV [61]. To alleviate the influence of the noisy label, SFRS [62] has been proposed to train with image-to-region supervision signal. GeM pooling [63], a method used to aggregate local features by their exponents, was also adopted in VPR. On top of that, MixVPR [5] has been proposed to perform accurate feature aggregation given various pretrained backbones. They are all based on sample-to-sample loss. CosPlace [14] firstly adopted the idea of CosFace [10], in a proximal classification form, to VPR and showed satisfying performance. For achieving view-point invariance, singular value decomposition (SVD) was adopted to classify photos in different viewpoints [15]. A line of works also converted VPR to a pure classification task covering inference, with significant speed boost and memory saving [8,64]. There are also works trying to address the similarity-distance constraint in a sample-to-sample training style [16,17]. However, a classification method that can handle this constraint remains uncovered.

2.3. Hard Negative Mining

Hard negative mining focuses on identifying and utilizing challenging negative examples to enhance model robustness and discrimination capabilities, particularly in contrastive learning and object detection. The evolution starts by collecting the hard-to-predict samples according to the specific metrics [65,66]. For instance, ref. [67] introduced an online sample mining strategy, collecting samples in each batch. Built upon this trend, there are works synthesizing hard negative samples and automatically selecting from them [68,69,70]. DCL [70] and HCL [71] sample hard negatives without knowing true labels by leveraging positive-unlabeled learning [72]. Ref. [13] utilized curricular learning [32] with hard negative mining to achieve SOTA for landmark retrieval. There are also works that set up a dynamic dictionary to train momentum networks to obtain more online samples in a contrastive style [13,73,74,75]. HCA [76] enables hard negative mining in the design of loss function but for sample-to-sample training. In contrast to the above works, we propose a loss function that implicitly incorporates the idea of hard negative mining. Specifically, compared to focal loss [77], which is also a loss function side work to balance negative samples, we directly drop those easy negative classes and maintain the geographic distance consistency, which is important for VPR tasks.

3. Methods

In this section, we first formulate the VPR task and give the desired definition in Section 3.1. The derivation of our GDCPlace, and the analysis for showing the GDCPlace geographic distance consistent is in Section 3.2. We introduce hard negative class mining in Section 3.3. Last, but not least, the implementation details of GDCPlace are in Section 3.4. All the methods introduced in this section are evaluated in Section 4.

3.1. Problem Formulation

For VPR with retrieval-style inference, users upload given photos to the system. The algorithm then predicts the location where the photos were taken by finding the most similar photo in the database. As in previous works [14,15], we interpret VPR as such a retrieval task, but also leveraging classification training proxy. Here, we formulate the classification training settings for it.

The geographic map $\mathcal{U}$ is divided into different blocks which form a set $\left\{u^i\right\}$ with a total number of N blocks. Each block has a geographic center $z^i$ and maintains learnable normalized unit-length weights $w^i$. There are multiple photos distributed in different blocks, with localization $z_j$, and the corresponded vision signal $x_j\in \mathcal{X}$. We aim to train our network $f_w:\mathcal{X}\to \mathcal{F}$ to obtain the latent feature $g_j\in \mathcal{F}$ (normalized, unit length) such that $g_j\ast w^{i_1}>g_j\ast w^{i_2}\iff d_{j{i}_1}<d_{j{i}_2}$, where $d_{xy}=\left|\right|z_x-z^y\left|\right|$ is the distance of the given x-th photo to the y-th block. Let $p_j=\underset{i}{arg\; min}{d}_{ji}$ be the index of block that the given photo belongs to. Note that we mainly follow the partition strategies of EigenPlace [15], where more details on field of view (FoVs) and group construction are given in Appendix B. We discuss an arbitrary single input image, and let $d_i$ be the distance from this photo to the i-th block. The dot-product between the latent feature of this photo and the learnable weight of the i-th block are denoted as $cos{\theta }_i$, as both vectors are in unit length, following previous works [10,11]. The loss function $\mathcal{L}{\displaystyle (}\left\{d\right\},\{cos\theta \},p{\displaystyle )}$ is minimized to optimize the parameter w. The goal of our paper is to find such a loss function that is geographic distance consistent. We give a formal definition of geographic distance consistent:

Definition 1

(Geographic Distance Consistent). A loss function $\mathcal{L}$ is called geographic distance consistent if, given $d_1<d_2<\dots <d_N$, and a set of values $\{cos{\theta }_1,\dots ,cos{\theta }_N\}$, the order that minimizes $\mathcal{L}$ is $cos{\theta }_1^{\prime }>cos{\theta }_2^{\prime },\dots >cos{\theta }_N^{\prime }$, where $cos{\theta }_i^{\prime }$ represents the cosine value from $\{cos{\theta }_1,\dots ,cos{\theta }_N\}$ that is placed in pair with $d_i$.

The meaning of GDCPlace is also depicted visually in Figure 1. By ensuring geographic distance consistent, we can obtain a loss function that encourages a correct relationship between embedding similarities and geographic distances. This is intuitive as we want the cosine similarity to be higher when the sample is closer to the center, and vice versa.

3.2. Geographic Distance Consistent Loss (GDCPlace)

Our goal is to find a proper formulation of the loss function that meets the consistency condition. We start from the cross-entropy softmax loss function as a reasonable statistical view. The aim to make it related to the geographic constraint suggests that integrating the constraint of Definition 1 into the initial form could be a possible attempt towards this goal. Now that p is the index for the positive block, let $n_{\ast }$ denote the index of other negative blocks. We can always relabel the corresponding geographic distances such that $d_p<d_{n_1}<d_{n_2}<\dots <d_{n_{N-1}}$. According to our constraint, we should have $cos{\theta }_p>cos{\theta }_{n_1}>\dots >cos{\theta }_{n_{N-1}}$. Then, we can define a monotonic decreasing function h of geographic distance d such that (the choice of h appears in Section 3.4):

$$\begin{array}{c}\hfill min{\displaystyle (}cos{\theta }_i,h{\displaystyle (}{d}_i{\displaystyle )}{\displaystyle )}\ge max{\displaystyle (}cos{\theta }_j,h{\displaystyle (}{d}_j{\displaystyle )}{\displaystyle )},\forall d_i<d_j\end{array}$$

(1)

We start from the cross-entropy softmax loss, and desire to have a derivation of cross-entropy loss under this constraint. However, it is not easy to derive an analytic form in this way. Instead, we focus on the upper bound of the cross-entropy loss under our constraint. Here, we present the derivation:

$$\begin{array}{}\hfill \mathrm{(2)}& \hfill {\mathcal{L}}_{ce}=-log{\displaystyle \frac{exp{\displaystyle (}scos{\theta }_p{\displaystyle )}}{exp{\displaystyle (}scos{\theta }_p{\displaystyle )}+{\displaystyle \sum _{i=1}^{N-1}}exp{\displaystyle (}scos{\theta }_{n_i}{\displaystyle )}}}\hfill \hfill & \hfill ={\displaystyle \frac{1}{2}}{\displaystyle (}log{\displaystyle (}1+\sum _{i=1}^{N-1}{\displaystyle \frac{exp{\displaystyle (}scos{\theta }_{n_i}{\displaystyle )}}{exp{\displaystyle (}scos{\theta }_p{\displaystyle )}}}{\displaystyle )}\hfill \hfill \mathrm{(3)}& \hfill +log{\displaystyle (}1+\sum _{i=1}^{N-1}{\displaystyle \frac{exp{\displaystyle (}scos{\theta }_{n_i}{\displaystyle )}}{exp{\displaystyle (}scos{\theta }_p{\displaystyle )}}}{\displaystyle )}{\displaystyle )}\hfill \hfill & \hfill \le {\displaystyle \frac{1}{2}}{\displaystyle (}log{\displaystyle (}1+\sum _{i=1}^{N-1}{\displaystyle \frac{exp{\displaystyle (}scos{\theta }_{n_i}{\displaystyle )}}{exp{\displaystyle (}scos{\theta }_p{\displaystyle )}}}{\displaystyle )}\hfill \hfill \mathrm{(4)}& \hfill +log{\displaystyle (}1+\sum _{i=1}^{N-1}{\displaystyle \frac{exp{\displaystyle (}scos{\theta }_{n_i}{\displaystyle )}}{exp{\displaystyle (}sh{\displaystyle (}{d}_{n_i}{\displaystyle )}{\displaystyle )}}}{\displaystyle )}{\displaystyle )}\hfill \hfill & \hfill \le {\displaystyle \frac{1}{2}}{\displaystyle (}log{\displaystyle (}1+exp{\displaystyle (}s{\displaystyle (}h{\displaystyle (}{d}_p{\displaystyle )}-cos{\theta }_p{\displaystyle )}{\displaystyle )}{\displaystyle )}\hfill \hfill \mathrm{(5)}& \hfill +log{\displaystyle (}1+\sum _{i=1}^{N-1}exp{\displaystyle (}s{\displaystyle (}cos{\theta }_{n_i}-h{\displaystyle (}{d}_{n_i}{\displaystyle )}{\displaystyle )}{\displaystyle )}{\displaystyle )}+log{\displaystyle (}{\displaystyle \frac{N}{2}}{\displaystyle )}{\displaystyle )}\hfill \end{array}$$

where s is the hyperparameter for scaling utilized by [10,11,15,30] to facilitate training. The detailed proof can be found in Appendix A. Minimizing Equation (5) is equally minimizing without constant term. We, therefore, obtain the first formulation of GDCPlace:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{GDC}& ={\displaystyle \frac{1}{s}}{\displaystyle (}log{\displaystyle (}1+exp{\displaystyle (}s{\displaystyle (}h{\displaystyle (}{d}_p{\displaystyle )}-cos{\theta }_p{\displaystyle )}{\displaystyle )}{\displaystyle )}\hfill \\ \hfill & +log{\displaystyle (}1+\sum _{i=1}^{N-1}exp{\displaystyle (}s{\displaystyle (}cos{\theta }_{n_i}-h{\displaystyle (}{d}_{n_i}{\displaystyle )}{\displaystyle )}{\displaystyle )}{\displaystyle )}{\displaystyle )}\hfill \end{array}$$

(6)

The following lemma gives a condition that can be pushed forward to satisfy Definition 1, and it is also a property worth showing for our GDCPlace.

Lemma 1.

Given a function f, if for a list of cosine similarities and distances to different classes: ${\displaystyle (}{\displaystyle (}cos{\theta }_1,d_1{\displaystyle )},\dots ,{\displaystyle (}cos{\theta }_N,d_N{\displaystyle )}{\displaystyle )}$, $d_1<\dots <d_N$, suppose there exists $cos{\theta }_i<cos{\theta }_j$ where $i<j$. We have

$$\begin{array}{c}\hfill f{\displaystyle (}{\displaystyle (}\dots ,{\displaystyle (}cos{\theta }_j,d_i{\displaystyle )},\dots ,{\displaystyle (}cos{\theta }_i,d_j{\displaystyle )},\dots {\displaystyle )}{\displaystyle )}\\ \hfill <f{\displaystyle (}{\displaystyle (}\dots ,{\displaystyle (}cos{\theta }_i,d_i{\displaystyle )},\dots ,{\displaystyle (}cos{\theta }_j,d_j{\displaystyle )},\dots {\displaystyle )}{\displaystyle )}\end{array}$$

(7)

then f is geographic distance consistent.

Proof. 

It is not hard to show that we can keep executing order-changing till the loss reaches the minimum under the correct order of cosine values and distances. See Appendix A. □

The above lemma indicates that we can first prove that switching the pairs of $cos\theta$ that are not aligned with distance d decreases the value of our loss function. It follows that we are ready to show the following.

Theorem 1.

${\mathcal{L}}_{GDC}$ is geographic distance consistent.

Proof. 

The proof starts from proving that ${\mathcal{L}}_{GDC}$ meets the condition described in Lemma 1. By the convex nature with a special case of Jensen’s inequality, Mercer type [78] (also shown in Lemma A1 in Appendix A), we prove that each of the order changing in Lemma 1 decreases the loss. It follows that it guarantees that ${\mathcal{L}}_{GDC}$ is geographic distance consistent. See Appendix A for details. □

Theorem 1 indicates that ${\mathcal{L}}_{GDC}$ is a loss function that meets our requirement. Figure 1 depicts the effect of GDCPlace. Note that it is discussed for a single input image. For the whole training set, we can further expand our theorem to prove that it still guarantees the consistency. Here, ${\mathcal{L}}_{GDC}^i$ represents the loss of the i-th input image out of a total number of M.

Corollary 1.

Given all M inputs from the training dataset, ${\displaystyle \sum _i^M}{\mathcal{L}}_{GDC}^i$ is still geographic distance consistent.

Proof. 

It can also be supported by Lemmas 1 and A1. Compared with Theorem 1, an extra discussion is about the condition mentioned by Lemma 1 that appears between ${\mathcal{L}}_{GDC}^i$ of different i. The proof is in Appendix A. □

Recent work, UniFace [33], suffers from an extreme gradient unbalance when the class number rises, namely, the negative class gradient will lead when we consider more classes. UniFace overcomes this issue by adding an extra term to balance the gradients. In contrast, GDCPlace is, by nature, gradient balanced, as we will show. Derive the gradient with respect to the correct and wrong classes, respectively:

$${\displaystyle \frac{\partial {\mathcal{L}}_{GDC}}{\partial cos{\theta }_p}}=-{\displaystyle \frac{exp{\displaystyle (}s{\displaystyle (}h{\displaystyle (}{d}_p{\displaystyle )}-cos{\theta }_p{\displaystyle )}{\displaystyle )}}{1+exp{\displaystyle (}s{\displaystyle (}h{\displaystyle (}{d}_p{\displaystyle )}-cos{\theta }_p{\displaystyle )}{\displaystyle )}}}$$

(8)

$${\displaystyle \frac{\partial {\mathcal{L}}_{GDC}}{\partial cos{\theta }_{n_i}}}={\displaystyle \frac{exp{\displaystyle (}s{\displaystyle (}cos{\theta }_{n_i}-h{\displaystyle (}{d}_{n_i}{\displaystyle )}{\displaystyle )}{\displaystyle )}}{1+{\displaystyle \sum _{i=1}^{N-1}}exp{\displaystyle (}s{\displaystyle (}cos{\theta }_{n_i}-h{\displaystyle (}{d}_{n_i}{\displaystyle )}{\displaystyle )}{\displaystyle )}}}$$

(9)

The absolute values of positive and negative gradients are all restricted between 0 and 1, regardless of the number of negative classes $N-1$:

$$-1<{\displaystyle \frac{\partial {\mathcal{L}}_{GDC}}{\partial cos{\theta }_p}}<0<\sum _{i=1}^{N-1}{\displaystyle \frac{\partial {\mathcal{L}}_{GDC}}{\partial cos{\theta }_{n_i}}}<1$$

(10)

Other useful properties that GDCPlace exhibits are presented in Appendix C.

3.3. Hard Negative Class Mining (HNCM)

In various sample-to-sample training strategies [13,37,42,79], hard negative mining or other engineering variants are seen as a must for better convergence. They mine hard negative samples given an anchor sample for it to learn the differences. Although the performance boost is well examined, it is at the cost of massive time overhead. In our training loss design, we incorporate this idea into the formulation of our GDCPlace, with no obvious overhead (see Section 4.9 for complexity analysis and benchmarking). Given a list of similarities and geographic distances to negative classes, $\{{\displaystyle (}cos{\theta }_{n_1},d_{n_1}{\displaystyle )},\dots \}$, we sort the pairs by $cos\theta$ in a descending order and we obtain the ordered list ${\displaystyle (}{\displaystyle (}cos{\theta }_{n_1}^{\prime },d_{n_1}^{\prime }{\displaystyle )},\dots ,{\displaystyle (}cos{\theta }_{n_{N-1}}^{\prime },d_{n_{N-1}}^{\prime }{\displaystyle )}{\displaystyle )}$. Then, we further derive GDCPlace with top-K HNCM for $K<N-1$:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{GDC-K}& ={\displaystyle \frac{1}{s}}{\displaystyle (}log{\displaystyle (}1+exp{\displaystyle (}s{\displaystyle (}h{\displaystyle (}{d}_p{\displaystyle )}-cos{\theta }_p{\displaystyle )}{\displaystyle )}{\displaystyle )}\hfill \\ \hfill & +log{\displaystyle (}1+\sum _{i=1}^{K}exp{\displaystyle (}s{\displaystyle (}cos{\theta }_{n_i}^{\prime }-h{\displaystyle (}{d}_{n_i}^{\prime }{\displaystyle )}{\displaystyle )}{\displaystyle )}{\displaystyle )}{\displaystyle )}\hfill \end{array}$$

(11)

which only considers the top-similar negative classes. In the perspective of geographic distance consistency, this form of GDCPlace adheres to a variant mathematical constraint as follows:

Corollary 2.

${\mathcal{L}}_{GDC-K}$ is top-${\displaystyle (}K+1{\displaystyle )}$ geographic distance consistent, i.e., let $d_1<\dots <d_{K+1}$ represent the top-($K+1$) shortest distance, then ${\mathcal{L}}_{GDC-K}$ is minimized when $cos{\theta }_1^{\prime }>cos{\theta }_2^{\prime }>\dots >cos{\theta }_{K+1}^{\prime }>max{\displaystyle (}cos{\theta }_{K+2}^{\prime },\dots ,cos{\theta }_N^{\prime }{\displaystyle )}$. This conclusion can be expanded to ${\sum }_i^M{\mathcal{L}}_{GDC-K}^i$ for arbitrary M samples, as per Corollary 1.

Proof. 

The idea is similar to the previous theorem. See Appendix A. □

This variant of geographic distance consistency only ensures the order of similarities with respect to the distance to the positive center and the top-K shortest distances to negative ones, which is actually desired. It is because there is less semantic information for the model to judge when the distance is too large. Specifically, when retrieving images in a database given a query image, it is observed that the relatively closer places share some common semantic clues for the model to predict the distances. For those farther ones, however, there is hardly no semantic information to be utilized. HNCM only guarantees geographic distance calibration among top-K images, which prevents the far blocks with less semantic cue being considered. The shape of function h is also capable of exhibiting a large value gap for those closer objects, whereas a smaller gap for the distant ones. The visualization is shown in Figure 2.

3.4. Implementation of GDCPlace

The key aspect of implementing GDCPlace revolves around selecting the function h. As per our analysis, the function h should exhibit a monotonic decreasing behavior with respect to geographic distance. Additionally, we aim to constrain its values to ensure stable training. Taking factors into consideration, we adopt sigmoid function as h, which is given by

$$\begin{array}{c}\hfill h{\displaystyle (}x{\displaystyle )}={\displaystyle \frac{1}{1+exp{\displaystyle (}\gamma {\displaystyle (}x-\zeta {\displaystyle )}{\displaystyle )}}}\end{array}$$

(12)

where we take $\gamma =0.2$ and $\zeta =6$ by comparing the results on the SF-XL [14] validation set. More selections on $\gamma$ and $\zeta$ are examined by ablation studies. K is the key variable to achieve HNCM, where we take $K=2$ and show the ablations on K in Section 4.8. We set the scale factor as $s=30$. The unit is meters. The shape of the function can also solve the problem that the distant objects share less semantic information, as in Figure 2. Concretely, when we measure the difference in the function value between a pair of distances, $x_1$ and $x_2$, we assume that the differences in x, i.e., $|x_2-x_1|$, remain constant. Then, when x is larger, the difference in output function values is smaller. This meets the phenomenon we show in Figure 2, whereby the visual cues vanish as the distance becomes larger. The values of h for the far class centers are very similar and the differences mainly appear for those closer ones, as also shown in Figure 2. Note that the formulation of h is similar to the positive and negative influence functions $g^{+}$ and $g^{-}$ in [16] (Janine’s Loss), but (1) when we expand the formulation, one will find they are actually different, for which the h is added and g’s are multipliers. They are like the margin terms placed different in various loss functions for face tasks [10,11,45]. (2) By the formulation differences, GDCPlace actually exhibits more useful properties that Janine’s Loss does not, e.g., gradient balance (Section 3.2). (3) Janine’s Loss is used for sample-to-sample style training, while GDCPlace is used for classification. (4) We use HNCM, while Janine’s Loss needs hard negative mining. The reasons above together make our approach significantly different from Janine’s Loss. Detailed analysis and performance comparisons are given in Section 4.5.

4. Experiments

We conduct extensive experiments to evaluate the effectiveness of our GDCPlace. In this section, we first introduce our common settings in Section 4.1, for experimental platform, datasets selection, and training settings. In Section 4.2, we compare our GDCPlace with other classification loss functions that have been adopted in VPR, or any potential sensed, with a wide range including face recognition, landmark retrieval, and theoretical analysis. In Section 4.3, we compare with other SOTA methods, showing that GDCPlace performs the best among other classification-training-based methods and contrastive-learning-based methods in the same ResNet-50 backbone [80]. The ranking results, discussion of previous loss functions, application to ordinal classification task, and application to other methods are given in Section 4.4, Section 4.5, Section 4.6 and Section 4.7, respectively. As very recently a few works have adopted the large-scale pretrained vision foundation model, DINOv2 [35], we also conduct experiments to evaluate under the DINOv2 backbone. The ablations on backbone scalability, components, and hyperparameters are reported in Section 4.8. Section 4.9 reports the efficiency analysis. Finally, Section 4.10 shows the visualizations of our methods, compared with EigenPlace [15].

4.1. Common Settings

4.1.1. Experimental Platform

We used Pytorch 1.10.0 with Python 3.8 for data processing, model building, training, and inference. For mathematical analysis, we use MATLAB© R2022b. For hardware, the platform is equipped with two NVIDIA A100 GPUs with a 20-core Intel© Xeon© Gold 6248 Processor. The experimental hardware setting is identical for all experiments.

4.1.2. Datasets Selection

For a fair comparison, we adhere to EigenPlace’s approach [15] by training on the SF-XL dataset [14] and strictly replicating EigenPlace’s training settings. For benchmark to test on, we collect 11 standard VPR datasets [14,27,81,82,83], of which the significance has been well acknowledged in previous studies. These datasets share strong domain shift, extreme weather, poor photo-taking condition, or day/night variances. For instance, in NordLand [26], most photos were taken in a train which has a significantly different domain to the SF-XL [14] training set. SVOX Night [25] is difficult for the night condition and long exposure time of photos. AmsterTime [24] has a notable distinction between scanned archival images and street view images. We follow [15], which divides the datasets into multi-view and frontal-view (facing the road ahead) ones. A more comprehensive introduction of each dataset is given in Appendix D.

4.1.3. Evaluation Metrics

Following the convention of previous works to evaluate the performance of algorithms for VPR, we adopt Recall@1 for most experiments, and @5 and @10 for ablation studies. Here, Recall@k is calculated by the ratio of queries of which the top-k confident predictions contain at least one of the photos whereby the location is within a specified proximity to the exact coordinates. For most datasets we considered, this proximity is 25 m. In the case of Nordland [26], it corresponds to a span of 10 frames. For Amstertime [24], a correct prediction is identified by matching it with its equivalent in the database. It is a standard metric for evaluating VPR methods.

To further evaluate the ranking performance, several experiments adopt mAP@k as the metric to assess the ranking results of GDCPlace. This metric is designed to evaluate the quality of rankings within the top-k predictions. It has been used for large-scale retrieval problems, as in [84]. The formula is provided as follows:

$$mAP@k={\displaystyle \frac{1}{Mmin{\displaystyle (}{n}_i,k{\displaystyle )}}}\sum _i^M\sum _j^{min{\displaystyle (}{n}_i,k{\displaystyle )}}P{\displaystyle (}i,j{\displaystyle )}rel{\displaystyle (}i,j{\displaystyle )}$$

(13)

where $P{\displaystyle (}i,j{\displaystyle )}$ is the precision of the top-j confident predictions of query i; $rel{\displaystyle (}i,j{\displaystyle )}$ is an indicator function outputting 1 if the j-th predictions of i-th queries is within the proximity threshold introduced above; M is the total number of the queries; $n_i$ is the number of correct photos in database given the i-th query. We select this metric but not complete mAP because there are some images that were taken very close to the query image, but they are not in the same orientation and, thus, contain no semantic cue. Thus, the complete mAP will be largely decreased for these implicit biases.

4.2. Comparison with Other Classification Loss Functions

4.2.1. Baseline Selection

As our work introduces a novel loss function and there is a lack of loss functions specifically for VPR classification training, we also gathered additional classification loss functions from similar areas for sufficient comparison. Most of them are from face recognition, as they are mostly adopted by VPR classification works [14,15], which include various famous and new SOTA loss functions: NormFace [30], SphereFace [45], CosFace [10], ArcFace [11], CurricularFace [32], AdaFace [31], and UniFace [33]. We also select MadaCos [85] and GeM-AP [19] from landmark retrieval, SORD [86] from ordinal classification, and LM-SoftMax [34] for general purpose to compare with. Many ordinal classification methods are difficult to transferr to VPR, as they are often designed for 1d data. For these types of method, we pick SORD as a good fit for VPR. Meanwhile, although GeM-AP is for sample-to-sample training, it appears as a competitor that partially meets the consistency (but only prioritizes the correct images instead of the wrong ones), so we include it into our comparison. All of them perform SOTA in their tracks for some time and are capable of being reproduced for VPR. Each method follows the same training settings as EigenPlace [15], except for ArcFace [11], for which we extended the training epochs to 60, as it took longer time to converge in our experiments. All the loss functions used in this study were retrained on the SF-XL training set [14], except for CosFace, whose results were adapted from the CosPlace paper [14]. The hyperparameters were chosen based on the results of the SF-XL validation set, which are shown in Appendix E.

4.2.2. Results

The results can be viewed in

Table 2. Comparison with other loss functions (recall@1). Bold denotes the best, and underline denotes the second best. Abbreviations: PI for Pitts30k. AM for AmsterTime. EY for Eynsham. TO for Tokyo-24/7. NL for NordLand. SN for SVOX Night. SO for SVOX Overcast. SW for SVOX Snow. SR for SVOX Rain. SU for SVOX Sun. MS for MSLS val.

Method	Track	Multi-View					Frontal-View							AVG
		PI	AM	EY	TO		NL	SN	SO	SW	SR	SU	MS
NormFace [30]	Face	90.6	41.3	90.0	88.3		58.4	40.3	88.2	87.2	85.2	41.3	86.8	72.5
SphereFace [45]	Face	86.4	43.1	79.5	63.2		59.3	52.0	90.7	87.7	79.8	63.1	83.4	71.7
CosFace [10]	Face	92.5	48.9	90.7	93.0		71.2	58.9	93.1	93.1	90.0	86.4	89.1	82.4
ArcFace [11]	Face	89.2	40.5	83.2	91.2		64.2	46.1	82.8	90.2	83.6	81.4	85.7	76.2
CurricularFace [32]	Face	92.3	47.1	90.8	91.4		68.8	61.6	93.7	93.6	90.6	84.4	89.1	82.1
AdaFace [31]	Face	86.7	44.8	75.4	74.6		61.1	44.7	89.1	83.9	74.8	51.9	79.6	69.7
SORD [86]	Ordinal	91.8	49.2	90.1	91.6		70.8	63.9	93.2	93.5	90.6	85.3	88.0	82.5
LM-Softmax [34]	General	92.2	49.6	90.4	90.5		73.2	65.2	94.0	93.4	90.1	85.4	89.6	83.1
MadaCos [85]	Landmark	91.7	46.8	90.6	90.8		68.8	61.4	95.0	93.6	90.3	85.1	88.5	82.1
Gem-AP [19]	Landmark	90.6	45.1	88.6	91.5		68.2	54.1	91.9	92.0	86.4	82.9	85.0	79.7
UniFace [33]	Face	92.2	46.5	90.6	91.4		65.5	61.2	94.2	94.0	91.5	84.7	88.1	81.8
GDCPlace	VPR	92.6	51.6	90.9	92.1		77.6	70.1	95.0	94.4	90.8	86.8	89.6	84.7

. GDCPlace performs the best in 9 out of 11 datasets compared with other loss functions. It is not surprising that some old methods, e.g., NormFace [30] and SphereFace [45], underperform other new SOTA methods. But it is interesting that ArcFace [11] cannot surpass CosFace [10] in VPR. It might need many more training epochs (D&C [8] trained ArcFace on SF-XL [14] for 300 epochs, for example). AdaFace [31] relies on the quality of the face images, which is not helpful for VPR so the result is intuitive. Although CosFace [10] is adopted for several VPR works [14,15], it turns out that LM-Softmax [34] performs the second best overall thanks to its robust theoretical backup. The ranking-related loss functions, SORD and GeM-AP, fall short in comparison with GDCPlace. This shows that GDCPlace is better than other competitive loss functions, and is, thus, the most robust classification loss function for VPR.

4.3. Comparison with State-of-the-Art Methods

4.3.1. Baseline Selection

We collected well-known and the most recent advanced works in VPR for comparison, including NetVLAD [4], SFRS [62], MixVPR [5], Conv-AP [7], D&C [8], CNN-Transformer [87], CosPlace [14], EigenPlace [15], HCA [76], and $R^2$Former [88]. The methods are categorized into two groups: training based on classification or sample-to-sample. As there are only limited works training a VPR model in the classification style but using retrieval for inference, we also included the landmark retrieval model CFCD [85] for comparison of the similarity between tasks. Additionally, as we only consider the single-stage method, we compare it with $R^2$Former single-stage results. We adopt the public accessible weights of these methods.

4.3.2. Vision Foundation Model

There are some very recently proposed methods that are based on the large-scale pretrained vision foundation model, DINOv2 [35]. It is built on the Vision Transformer [89] architecture, distinct from the common backbone in VPR, ResNet [80]. For a fair comparison, here, we also report the results adopting our methods with DINOv2. To evaluate the performance, we compare with SALAD [90], CricaVPR [91], and AnyLoc [1], which are all recently proposed DINOv2-based methods. We also reproduced EigenPlace [15] in DINOv2 backbone under the same settings to evaluate the performance of the previous SOTA.

4.3.3. Results on ResNet50

We report the results in

Table 3. Comparison with SoTAs equipped with ResNet50 and VGG16 backbones (recall@1). Blue denotes the best for both categories, and bold denotes the best in each category.

Method	Backbone	Dim.	Multi-View					Frontal-View							AVG
			PI	AM	EY	TO		NL	SN	SO	SW	SR	SU	MS
Classification Training
D&C [8]	ResNet50	1280	84.6	17.5	87.2	60.0		11.9	4.7	69.3	57.7	55.1	42.5	69.6	50.9
CFCD [85]	ResNet50	512	62.2	28.8	76.3	54.9		6.1	16.8	60.8	50.9	39.3	25.6	50.7	42.9
CosPlace [14]	ResNet50	2048	90.9	47.7	90.0	87.3		71.9	50.7	92.2	92.0	87.0	78.5	87.4	79.6
EigenPlace [15]	ResNet50	2048	92.5	48.9	90.7	93.0		71.2	58.9	93.1	93.1	90.0	86.4	89.1	82.4
GDCPlace	ResNet50	2048	92.6	51.6	90.9	92.1		77.6	70.1	95.0	94.4	90.8	86.8	89.6	84.7
Sample-to-Sample Training
SFRS [62]	VGG16	4096	89.1	29.7	72.3	80.3		16.0	28.6	81.1	76.0	69.7	54.8	70.0	60.7
NetVLAD [4]	VGG16	4096	85.0	16.3	77.7	69.8		13.1	8.0	66.4	54.4	51.5	35.4	58.9	48.8
$R^2$ Former-global  [88]	ResNet50	256	73.1	12.8	82.4	48.3		24.6	13.5	75.8	60.8	47.6	28.0	80.3	49.7
CNN-Transformer [87]	VGG16 + Swin-B	4096	86.3	-	-	78.9		-	-	-	-	-	-	-	-
HCA [76]	VGG16	4096	-	-	-	86.7		-	-	-	-	-	-	76.4	-
Conv-AP [7]	ResNet50	8192	90.5	35.0	87.6	72.1		62.9	43.4	91.9	91.0	82.8	80.4	82.4	74.5
MixVPR [5]	ResNet50	4096	91.5	40.2	89.4	85.1		76.2	64.4	96.2	96.8	91.5	84.8	87.2	82.1

. GDCPlace outperforms SOTA in most datasets considered. It is remarkable that GDCPlace significantly outperforms MixVPR [5], the SOTA of sample-to-sample training. MixVPR exhibits consistent advantages compared to other methods in the same training pattern, except for Pitts30k, where HCA performs the best. It is also a satisfying boost from EigenPlace [15], considering that only loss function change can bring such a score increase. However, GDCPlace underperforms MixVPR [5] in some frontal-view datasets. It is known that one of the main drawbacks of SF-XL-based methods, e.g., EigenPlace and CosPlace, is that it often displays inferior performance compared to MixVPR and SALAD. As GDCPlace is trained with EigenPlace, it also inherits this nature. But progress has also been made, namely, it maintains the best performance in rather hard SVOX datasets. It is always challenging to compare the performance of classification methods in front-view datasets with sample-to-sample ones [15].

4.3.4. Results on DINOv2

The results are reported in

Table 4. Comparison with recent DINOv2 methods (recall@1). Blue denotes the best for both categories, and bold denotes the best in each category.

Method	Backbone	Dim.	Multi-View					Frontal-View							AVG
			PI	AM	EY	TO		NL	SN	SO	SW	SR	SU	MS
Classification Training
EigenPlace [15]	DINOv2-B/14	2048	93.3	53.0	91.3	95.2		86.2	92.8	96.9	97.2	96.3	94.1	89.2	89.6
GDCPlace	DINOv2-B/14	2048	93.5	57.7	91.7	96.5		88.8	94.9	97.2	97.9	96.8	95.6	91.4	91.1
GDCPlace	DINOv2-B/14	8448	94.2	58.5	91.9	96.2		90.7	94.9	97.6	97.8	97.0	96.1	91.4	91.5
Sample-to-Sample Training
AnyLoc [1]	DINOv2-G/14	49152	87.5	43.9	87.5	91.4		35.7	76.5	91.9	88.0	85.6	86.5	70.3	76.8
SALAD [90]	DINOv2-B/14	8448	91.9	56.9	91.4	94.3		86.1	94.8	98.3	98.9	98.7	96.7	92.2	90.9
CricaVPR [91]	DINOv2-B/14	4096	94.9	64.7	91.6	93.0		90.7	85.1	96.7	96.0	95.0	93.8	90.0	90.1

. Although GDCPlace fails to surpass EigenPlace in Tokyo-24/7 [27] under ResNet50, it outperforms EigenPlace [15] under DINOv2 in all measures. This suggests a better scaling capacity compared with CosFace [10] for VPR. For the other recent DINOv2-based methods, although SALAD performs SOTA in a large portion of datasets, GDCPlace shows the best average performance compared with other sample-to-sample methods. Remarkably, this is achieved by a 4× smaller descriptor size compared with SALAD [90], contributing to a more memory-friendly VPR system.

4.4. mAP Results of GDCPlace

We provide quantization ranking results for GDCPlace in

Table 5. mAP@k results of SoTA methods. S2S represents sample-to-sample, and CLS stands for classification. Bold denotes the best in each measure, and underline denotes the second best.

Methods	Training Strategy	NordLand			SVOX Night			AmsterTime
		@3	@5	@7	@3	@5	@7	@3	@5	@7
CosPlace [14]	CLS	60.8	51.8	44.6	40.1	31.8	26.0	54.8	56.2	56.7
EigenPlace [15]	CLS	57.0	46.8	39.4	47.7	34.1	27.6	56.0	57.1	57.7
Conv-AP [7]	S2S	49.1	40.4	34.3	31.6	24.3	19.5	40.5	41.9	42.4
MixVPR [5]	S2S	60.7	50.2	42.7	47.2	35.9	28.4	46.6	47.6	47.9
GDCPlace	CLS	63.6	52.8	44.8	54.6	42.3	33.7	58.3	59.7	60.0

and

Table 6. Comparison on mAP@k with other classification loss functions. Bold denotes the best in each measure, and underline denotes the second best.

Methods	Tracks	NordLand			SVOX Night			AmsterTime
		@3	@5	@7	@3	@5	@7	@3	@5	@7
NormFace [30]	Face	39.5	31.2	25.7	29.0	22.5	18.0	47.9	49.0	49.6
SphereFace [45]	Face	47.2	38.8	32.9	39.1	30.4	24.6	50.6	51.9	52.3
CosFace [10]	Face	60.8	51.8	44.6	40.1	31.8	26.0	54.8	56.2	56.7
ArcFace [11]	Face	38.9	31.7	26.8	19.3	14.6	11.7	45.3	46.6	47.2
CurricularFace [32]	Face	55.1	45.4	38.3	46.7	36.0	28.8	55.3	56.5	57.0
AdaFace [31]	Face	48.4	40.1	34.2	33.7	26.1	21.0	51.7	52.9	53.5
SORD [86]	Ordinal	61.2	50.6	43.2	51.4	39.8	32.4	58.0	57.6	59.4
LM-Softmax [34]	General	59.8	49.5	42.0	49.2	37.7	30.2	57.1	58.2	58.7
MadaCos [85]	Landmark	55.9	46.5	39.5	46.8	36.2	29.1	54.7	55.8	56.1
GeM-AP [85]	Landmark	58.2	47.9	41.5	48.7	37.8	31.2	56.2	57.2	58.0
UniFace [33]	Face	52.5	43.3	36.7	44.3	33.8	27.0	55.0	56.1	56.6
GDCPlace	VPR	63.6	52.8	44.8	54.6	42.3	33.7	58.3	59.7	60.0

. It is shown that, although EigenPlace [15] outperforms CosPlace [14] for recall@1, the performance on mAP@k is different. CosPlace [14] performs even better in NordLand [26] across all $k=3,5,7$. Furthermore, it is shown that CosFace [10] outperforms LM-Softmax [34] in Nordland [26]. On the other hand, although MixVPR [5] leads on the recall metric, it falls behind compared with EigenPlace [15] when evaluating mAP@k. GDCPlace consistently performs the best for all mAP@k, which fits the motivation. The results are obtained with the public weights.

4.5. Comparison with Janine’s Loss

First, let us recap the formulation of Janine’s Loss [16]. It is designed for sample-to-sample training, but not for classification training, as GDCPlace does. Given a set of anchor images $\left\{a_i\right\}$, we are interested in the Euclidean distances of the features to anchors $l_{ji}=\left|\right|f_{\theta }{\displaystyle (}{x}_j{\displaystyle )}-f_{\theta }{\displaystyle (}{a}_i{\displaystyle )}\left|\right|$, where $f_{\theta }$ is neural network, and $x_j$ j-th images. The corresponded geographic distances in map are $d_{ji}$. Janine’s Loss is formulated as follows:

$$\begin{array}{c}\hfill {\mathcal{L}}_{Janine}{\displaystyle (}i{\displaystyle )}=log{\displaystyle (}1+\sum _{j}exp{\displaystyle (}\eta g^{+}{\displaystyle (}{d}_{ji}{\displaystyle )}{l}_{ji}-\mu {\displaystyle )}/\eta {\displaystyle )}\\ \hfill +log{\displaystyle (}1+\sum _{j}exp{\displaystyle (}\mu -\eta g^{-}{\displaystyle (}{d}_{ji}{\displaystyle )}{l}_{ji}{\displaystyle )}/\eta {\displaystyle )}\end{array}$$

(14)

where $g^{+}{\displaystyle (}y{\displaystyle )}={\displaystyle \frac{1}{1+exp{\displaystyle (}\lambda -\gamma y{\displaystyle )}}}$, and $g^{-}{\displaystyle (}y{\displaystyle )}={\displaystyle \frac{1}{1+exp{\displaystyle (}\gamma y-\lambda {\displaystyle )}}}$. On the other hand, for GDCPlace:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{GDC-K}& ={\displaystyle \frac{1}{s}}{\displaystyle (}log{\displaystyle (}1+exp{\displaystyle (}s{\displaystyle (}h{\displaystyle (}{d}_p{\displaystyle )}-cos{\theta }_p{\displaystyle )}{\displaystyle )}{\displaystyle )}\hfill \\ \hfill & +log{\displaystyle (}1+\sum _{i=1}^{K}exp{\displaystyle (}s{\displaystyle (}cos{\theta }_{n_i}^{\prime }-h{\displaystyle (}{d}_{n_i}^{\prime }{\displaystyle )}{\displaystyle )}{\displaystyle )}{\displaystyle )}{\displaystyle )}\hfill \end{array}$$

(15)

where $cos\theta$ of GDCPlace and their l are measures of similarities and distances between latent features, respectively. The h is added to $cos\theta$, while the g is multiplied to l. Because of these formulation differences, Janine’s Loss does not meet gradient balance (see Section 3.2) as GDCPlace does. Here, we give the proof:

Proof. 

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial {\mathcal{L}}_{Janine}{\displaystyle (}i{\displaystyle )}}{\partial l_{ji}}}={\displaystyle \frac{{\sum }_j{g}^{+}{\displaystyle (}{d}_{ji}{\displaystyle )}exp{\displaystyle (}\eta g^{+}{\displaystyle (}{d}_{ji}{\displaystyle )}{l}_{ji}-\mu {\displaystyle )}}{1+{\sum }_{j}exp{\displaystyle (}\eta g^{+}{\displaystyle (}{d}_{ji}{\displaystyle )}{l}_{ji}-\mu {\displaystyle )}/\eta }}\\ \hfill -{\displaystyle \frac{{\sum }_j{g}^{-}{\displaystyle (}{d}_{ji}{\displaystyle )}exp{\displaystyle (}\mu -\eta g^{-}{\displaystyle (}{d}_{ji}{\displaystyle )}{l}_{ji}{\displaystyle )}}{1+{\sum }_{j}exp{\displaystyle (}\mu -\eta g^{-}{\displaystyle (}{d}_{ji}{\displaystyle )}{l}_{ji}{\displaystyle )}/\eta }}\end{array}$$

(16)

It is not hard to see that Equation (16) is not bounded. It can be arbitrary high or low via adjusting $g^{+}$ and $g^{-}$. □

Furthermore, GDCPlace is derived by integrating Definition 1 to cross-entropy loss, which makes a reasonable foundation. HNCM is also proposed for efficient hard class mining instead of the common time-consuming hard negative mining. To evaluate the advantages of the formulation of GDCPlace for classification, we incorporate Janine’s Loss for classification training with the same training settings as GDCPlace obeys. As shown in

Table 7. Comparison with Janine’s Loss (Recall@1). Bold denotes the best in each dataset.

Formulation	Multi-View					Frontal-View							AVG
	NI	AM	EY	TO		NL	SN	SO	SW	SR	SU	MS
Janine’s Loss	78.3	47.5	76.1	69.2		48.4	36.8	88.3	89.8	80.2	66.2	80.1	69.2
Janine’s Loss + HNCM	89.9	51.6	89.2	81.6		51.3	44.7	92.4	92.2	87.4	78.3	86.6	76.8
GDCPlace	92.6	51.6	90.9	92.1		77.6	70.1	95.0	94.4	90.8	86.8	89.6	84.7

, the performance gap is also examined.

4.6. Application to Ordinal Classification Task

To demonstrate the generalizability of GDCPlace, we also compare our loss function with several other SOTA methods for ordinal classification, including regression [20], classification [92], DLDL-v2 [21], SORD [86], and OR-CNN [20]. We reimplement all the methods chosen and conduct our experiments on two popular benchmarks, UTKFace [93] and AgeDB [94]. These two benchmarks are tasked with estimating people’s ages given their face photos, which is one of the typical ordinal classification tasks. It is not the main focus of this work, but the results, shown in

Table 8. GDCPlace applied to ordinal classification tasks (MAE). Bold denotes the best in each dataset.

Method	Regression [20]	Classification [92]	DLDL-v2 [21]	SORD [86]	OR-CNN [20]	GDCPlace
UTKFace	4.78	4.95	4.67	4.77	4.64	4.59
AgeDB	6.72	7.29	6.89	6.86	6.68	6.55

, reveal that GDCPlace generalizes well to the ordinal classification tasks. The improvements, while appearing to be minor, are shown accordingly in both datasets. This suggests that further domain-specific designs could benefit the ordinal classification. Meanwhile, as it is not the main contribution, we did not conduct enough experiments to completely evaluate the method on the ordinal classification.

4.7. Application to CosPlace

To evaluate GDCPlace as an out-of-the-box loss function, we replace the loss function of CosPlace [14] (CosFace [10]) with our GDCPlace to see if it can also work with other methods. CosPlace has a different map partition strategy compared to EigenPlace, where views of the same direction are categorized as being in the same class. The results are reported in

Table 9. GDCPlace applied to CosPlace (recall@1). Bold denotes the best in each dataset.

Method	Backbone	Dim.	Multi-View					Single-View							AVG
			PI	AM	EY	TO		NL	SN	SO	SW	SR	SU	MS
CosPlace	ResNet50	2048	90.9	47.7	90.0	87.3		71.9	50.7	92.2	92.0	87.0	78.5	87.4	79.6
CosPlace + GDCPlace	ResNet50	2048	91.5	50.2	90.0	89.5		75.8	54.2	93.0	93.5	87.8	79.0	88.0	81.1

. It is shown that GDCPlace + CosPlace performs consistently better than vanilla CosPlace in every benchmark.

4.8. Ablation Studies

4.8.1. Ablation on HNCM

In this section, we showcase the experiments demonstrating the effect of HNCM and its impact based on K, which determines the number of hard classes.

Table 10. Ablations on different K of HNCM (recall). Bold denotes the best in each measure.

K in HNCM	Pitts30k				Tokyo-24/7				SVOX Night
	R@1	R@5	R@10		R@1	R@5	R@10		R@1	R@5	R@10
1	92.4	96.5	97.3		92.1	96.8	97.1		50.3	72.2	78.1
2	92.6	96.4	97.3		92.1	97.5	98.4		51.6	72.5	77.0
3	92.4	96.5	97.4		92.7	96.2	97.1		51.1	73.4	78.1
50	92.0	96.4	97.2		90.2	95.6	96.8		50.4	72.5	77.0
No HNCM	92.3	96.5	97.3		91.4	97.1	97.8		50.5	72.1	77.3

first illustrates the advantage of using HNCM compared to not using it. When using HNCM, it is shown that a small number of K will further boost the performance. This suggests that only a small portion of hard negative classes needs to be compared during each loss computation. Additional visualization on HNCM is introduced in Section 4.10.

4.8.2. Ablation on Function h

There are two important hyperparameters, $\gamma$ and $\zeta$, for the implementation of function h. $\gamma$ controls the shape of the function (to be more like sigmoid or not), whereas $\zeta$ decides the phase of the function (the graph moves left or right).

Table 11. Ablation on different $\gamma$ in function h (recall). Bold denotes the best in each measure.

$\mathit{\gamma }$ in Function h	Pitts30k				Tokyo-24/7				SVOX Night
	R@1	R@5	R@10		R@1	R@5	R@10		R@1	R@5	R@10
0.1	92.4	96.4	97.4		91.4	97.5	97.8		50.5	72.0	77.8
0.2	92.6	96.4	97.3		92.1	97.5	98.4		51.6	72.5	77.0
0.4	92.3	96.5	97.4		91.8	97.1	98.0		50.4	71.6	77.6
0.6	92.4	96.5	97.2		91.1	96.8	97.5		50.9	72.6	77.8
0.8	92.3	96.2	97.2		90.8	97.1	97.8		50.9	72.3	77.3

and

Table 12. Ablation on different $\zeta$ in function h (recall). Bold denotes the best in each measure.

$\mathit{\zeta }$ in Function h	Pitts30k				Tokyo-24/7				SVOX Night
	R@1	R@5	R@10		R@1	R@5	R@10		R@1	R@5	R@10
2	92.4	96.4	97.4		94.0	97.8	98.1		68.5	85.4	88.2
4	92.3	96.4	97.3		92.1	96.8	97.8		68.5	84.0	88.6
6	92.6	96.4	97.3		92.1	97.5	98.4		70.1	84.8	89.2
8	92.1	96.5	97.3		90.8	96.8	97.5		70.4	83.1	87.5
10	92.2	96.5	97.3		90.2	97.5	97.5		66.3	81.7	85.7

show that GDCPlace is not fragile with respect to the selection of $\gamma$ and $\zeta$. The hyperparameter tuning based on the SF-XL validation set is also reliable.

4.9. Time Complexity of GDCPlace

For N classes, a classification loss function typically has $\mathcal{O}{\displaystyle (}N{\displaystyle )}$ time complexity (omit the descriptor size). For GDCPlace, an additional operation to pick up top-K negative classes is required, which is $\mathcal{O}{\displaystyle (}KN{\displaystyle )}$. As K is constant, and is set to 2, GDCPlace still exhibits $\mathcal{O}{\displaystyle (}N{\displaystyle )}$ time complexity, as per other classification loss functions. The time cost was also benchmarked, as shown in

Table 13. The training efficiency.

Method	Time (min/5000 Steps)
EigenPlace	$30.74\pm 0.21$
GDCPlace	$30.69\pm 0.36$

. We trained EigenPlace and GDCPlace on SF-XL for 5 epochs and recorded the average time interval. This proves that the time gain is ignorable, but there is an increase in standard derivation.

4.10. Qualitative Results

To see whether GDCPlace can guarantee a correct similarity order during inference, we present some visualizations of EigenPlace [15] and our method in Figure A2. It can be observed that GDCPlace outputs the correct top six results compared to EigenPlace [15], where several mistakes were made for two queries from SVOX Night [25] and NordLand [26], respectively. The query photos in SVOX Night [25] were taken during night and unclear, and photos in Nordland [26] appear to be extremely hard to distinguish. But GDCPlace still performs well and give robust results. It is less informative to show the ranking capacity, as the photos in AmsterTime [24] are generally far from each other. But we still show the performance of GDCPlace whereby it can predict the correct image more accurately compared with EigenPlace [15], as shown in Figure A3. It is less easy to be fooled by the photos from Amsterdam City Archive, which exhibit a strong domain shift. The qualitative results of using HNCM are shown in Figure 3. It is demonstrated that training with HNCM is in favor of correct orders of images. We also provide more visualizations on SVOX Night [25] and NordLand [26] in Figure A4. It is consistently proven that GDCPlace is stable and reliable in outputting a correct order of photos. Except for SVOX Night [25], the query from Nordland [26] similarly demonstrates a significant change in weather conditions, under which GDCPlace is still robust.

5. Discussion

Visual place recognition (VPR) serves as a critical enabling technology for autonomous systems, allowing them to reliably identify previously visited locations through visual inputs. This capability forms the foundation for large-scale navigation in GPS-denied environments, enabling loop closure in SLAM (Simultaneous Localization and Mapping) systems and facilitating long-term autonomy through cross-session place matching. By combining GDCPlace with robust matching algorithms, modern VPR systems have the potential to demonstrate remarkable resilience to viewpoint variations, illumination changes, and seasonal appearance shifts, making them indispensable for autonomous vehicles, delivery robots, and aerial drones operating in dynamic real-world conditions. The GDC condition can also serve as a standard mathematical constraint for future loss functions design. For the limitations, although GDCPlace shows the best performance overall and has backup from theoretical analysis, the FOVs and group partition strategy are taken care by EigenPlace [15]. The hyperparameters are also decided via tuning on the validation set, where an auto mechanism for this goal is also appealing. Meanwhile, although GDCPlace shows strong potential in performing ordinal classification tasks, there is still room for improvement such that the 1d data nature of ordinal classification can be further utilized to refine our loss function.

On the other hand, as it is clear that most of the face recognition and landmark retrieval methods can be applied to VPR, conversely, it is also highly possible that GDCPlace can be adapted to other retrieval tasks. However, defining a distance to replace with the geographic distance in VPR for other retrieval tasks poses a significant challenge. These are all our future work directions.

6. Conclusions

In summary, this work resolves the challenge in VPR classification training that loss functions should encourage the embedding similarities to be negatively correlated to geographic distances. We gave a formal definition, geographic distance consistent, and proved that our proposed loss function, GDCPlace, meets this requirement. As a novel loss function, we compared GDCPlace with other classification loss functions, from areas including face recognition, ordinal classification landmark, and visual place recognition itself. We also conducted comparisons with other SOTA methods proposed for the VPR task, both with traditional ResNet-50 and DINOv2 backbones. As GDCPlace encourages a correct rank of geographic distances, we also evaluated the mAP results to test the rank prediction performance. Ablations were conducted on each of the hyperparameters, and qualitative results were provided to visually showcase the performance of GDCPlace. Overall, extensive experiments revealed that our GDCPlace performance is SOTAcompared with other cutting-edge methods. It is also the first VPR-specific loss function with robust theoretical support.