Learning Part-Based Features for Vehicle Re-Identification with Global Context

Abstract

Re-identification in automated surveillance systems is a challenging deep learning problem. Learning part-based features augmented with one or more global features is an efficient approach for enhancing the performance of re-identification networks. However, the latter may increase the number of trainable parameters, leading to unacceptable complexity. We propose a novel part-based model that unifies a global component by taking the distances of the parts from the global feature vector and using them as loss weights during the training of the individual parts, without increasing complexity. We conduct extensive experiments on two large-scale standard vehicle re-identification datasets to test, validate, and perform a comparative performance analysis of the proposed approach, which we named the global–local similarity-induced part-based network (GLSIPNet). The results show that our method outperforms the baseline by 2.5% (mAP) in the case of the VeRi dataset and by 2.4%, 3.3%, and 2.8% (mAP) for small, medium, and large variants of the VehicleId dataset, respectively. It also performs on par with state-of-the-art methods in the literature used for comparison.

1. Introduction

CCTV cameras are an effective and pervasive component of most modern surveillance systems that have numerous applications, including law enforcement, event monitoring, process monitoring, crime analysis, and other smart city processes [1]. They generate vast amounts of image and video data for these systems. The tasks performed by surveillance systems that deal with such data can be narrowed down to activities like recognition, identification, re-identification, tracking, or the retrieval of objects, persons, or vehicles [2]. Various tasks, like object detection [3] and classification [4], can be performed on the data to generate important insights. In this paper, we focus specifically on vehicle re-identification (VReID), which involves retrieving an image of a particular vehicle from a database of vehicle images, given a particular vehicle image as a query. In surveillance applications, VReID is used in traffic- and fleet-management-based industrial processes. The VReID problem is closely related to person re-identification, which has been more thoroughly studied and researched than VReID. Successful results in person re-identification, usually achieved through deep learning, have inspired parallel research on VReID, and in some aspects, the latter often poses more challenges to the problem of re-identification matching.

In any re-identification problem, discriminating features are extracted from the ‘query’ image to find matches from a collection of ‘gallery’ images. Methods used in person re-identification have been found to work well in the case of vehicle re-identification. Initially, some handcrafted image-processing-based features were used along with deep learning features [5,6]. However, due to the low performance of these features, subsequent works have utilized deep learning-based methods only for feature extraction. Most methods use backbone models pre-trained on ImageNet [7], like AlexNet [5,7], VGG [8,9], GoogLeNet [6,10], ResNet [11,12], and MobileNet [13], for feature extraction. The features can be attribute-based, local component-based, part-based, or global deep features. Attribute-based features include global features like vehicle color, make, model, and viewpoint. The attributes are first learned using a separate dedicated module/network, and then the learned attributes are mapped to labels. This causes an inherent disadvantage in such methods with respect to increased complexity, as a separate module is required for attribute identification. Local features can be one of two types: (i) salient features directly extracted from vehicle images like headlights, mirrors, number plates, back lights, etc., or (ii) features extracted from a deeply learned feature vector. The latter are termed part-based local features. In this approach, the convolutional tensor obtained after the backbone model is partitioned into different parts to obtain local representations [14]. Attention layers have also been employed in some works to generate localized discriminative features [15,16]. The global deep features are generally features generated by the backbone model. Conventionally, approaches used in VReID use a combination of part-based layers along with a global layer. The layers obtained from the deeply learned global features proceed in parallel in different branches, which are all trained by fully connected layers. These local features can be used independently or together with a global network to generate a feature map. We propose a different approach in which we unify global and part-based features without actually training any global branches.

In this paper, we set out to increase the accumulated body of knowledge on part-based re-identification methods by proposing a novel approach to unifying global and part-based features. In this method, we initially perform a generalized uniform partition on the convolutional tensor generated after passing the image through the backbone network. Then, at a later stage, we introduce a global component into the model during training by applying a weighted classification loss function to the parts. The weights are calculated according to the distance of the part features from the global feature, rewarding those parts that are closer to the global feature. Our focus and contribution in this paper are on learning better discriminative features from localized parts during training by incorporating a global component without training a separate global branch. The major significance of this method is that it introduces a global component during training without explicitly training a global tensor, as the scores generated by the similarity generator reflect agreement between global and local features. This results in finer feature learning. In this work, we compare the performance of the proposed model with a model comprising only part-based features without the global component, which is considered the baseline in this paper. Our approach is found to generate better results than training part-based features alone.

The major contribution of this work is the proposed GLSIPNet model, which effectively utilizes cues from the global feature to train the part-based features via the distance between the global and part-based features, without explicitly training the global features. This makes the process less complex than training global features directly. To the best of our knowledge, this method of incorporating global features without explicit training of the global features has not been reported in the literature, which is the novelty of the proposed method. Experiments are conducted on two large-scale datasets—VeRi and VehicleId—using the proposed method to show the extent of improvement in matching performance over the baseline and other related results in the recent literature.

The rest of this paper is organized as follows. Section 2 presents a literature review and the motivation for this work. Section 3 describes the methodology used. The results of the experiments and the comparative performance analysis with the state of the art are discussed in Section 4, and the overall conclusions are summarized in Section 5.

2. Motivation and Related Works

In this section, we discuss the standard datasets for vehicle re-identification and the state-of-the-art methods for solving the vehicle re-identification problem.

Vehicle Re-Identification Datasets

A few vehicle re-identification datasets, like VeRi [5], VehicleId [8], VRIC [17], VeRi-Wild [18], CityFlow [19], PKU-VD [20], and ToyCars [21], have been developed for use in vehicle re-identification tasks. Most existing works report results on the VeRi and VehicleId datasets. Hence, we use these datasets in our experiments, which are described as follows:

(a) VeRi Dataset: This dataset contains 49,357 images of 776 vehicles captured by 20 cameras over a 1 sq km area. The images were captured in a real-world, unconstrained surveillance scene. Each vehicle was captured by 20 cameras from different viewpoints, under varying illuminations, and with different resolutions and occlusions. The training set comprises 575 vehicle identities, and the remaining images are used for testing. The test set comprises a query set and a gallery set. The query set includes only one image per vehicle identity, which is used to find matches in the gallery set. The vehicle identities of the training and test sets are different. Hence, no cross-validation is used during testing.

(b) VehicleId Dataset: This dataset contains 221,763 images of 26,267 vehicles. Each vehicle image was captured from either the front or the back, i.e., from two different viewpoints. The training set contains 110,178 images of 13,134 vehicles. The test data are split into three subsets containing 800, 1600, and 2400 vehicles, with 7332, 12,995, and 20,038 images, respectively. The query set for each of the three test subsets contains one vehicle image for each vehicle identity, and the gallery set contains the rest of the vehicle images used to determine the match. In this dataset, the vehicle identities of the training and test sets are different; hence, cross-validation is not used during evaluation to ensure a proper comparison with the state of the art.

Re-identification methods based on global and local feature learning are broadly divided into two categories:

• Attribute-based models, which generally use a separate dedicated network for the identification of attributes or local parts.
• Part-based models, which use single or multiple branches to extract local features with different schemes of partitioning. These methods may or may not include a global branch that learns the global features. Single-branch models are inherently less complex than multi-branch models due to their smaller number of convolutional layers.

In this section, we first review the existing works on VReID in the literature that use either global, local, or a combination of the two feature learning methods. Methods for learning attributes utilize a wide variety of techniques for feature extraction. A joint approach for learning deep features along with attributes such as camera view, vehicle color, and vehicle type was proposed by Wang et al. [22]. In another work, a single-stage, single-shot detector is used to detect regions of interest in vehicle images, and the deep features of the regions are used as local features [23]. VDG, a method for viewpoint-aware learning in an unsupervised context, where novel views are generated and then a contrastive loss is used to learn viewpoint-invariant features, was proposed in [24]. He et al. introduced a two-pronged approach which involved first generating the local regions of interest and then projecting them onto the global feature map, thereby ensuring a mix of local attribute-based and global re-identification features [25]. This approach followed a scheme similar to that used in object detection and segmentation models [26]. Qian et al. used a two-branch model in which local features are extracted by applying horizontal stripes to the output of a backbone layer, and these are combined with a global layer. The global layer comprises two fully connected layers, one with ID predictions and the other with attribute information, that generate a combined feature [27]. Wang et al. extracted region feature vectors for multiple region segmentation results using a convolutional neural network and obtained appearance feature vectors of the target vehicle by fusing them with global feature vectors [28]. Other works used short and densely connected convolutional units without a backbone network to develop an architecture from scratch for feature learning. This is then utilized with a Siamese network-type architecture for improving feature detection and with a modified quadrature pooling layer for feature generation [29,30,31]. Peng proposed a feature learning approach that employs a spatial transformer network-based localization model for integrating global and local features into a representation for VReID [32]. The major disadvantage of models that learn attributes or local features is that a separate module is required to extract attributes, which increases complexity.

Methods for learning part-wise features follow a strategy that involves dividing either the raw image or the convolutional tensor into different parts or directing attention to specific regions within them. Convolutional features are then learned from these targeted areas. This approach addresses a major limitation of the methods described above, i.e., it does not require any additional networks or training to identify the local features and can be used in a straightforward manner [33]. The part-based convolutional baseline, or PCB [14], which uses horizontal part-based features as local features, has been widely used in person re-identification. Refined part pooling has also been used to address partial inconsistencies by training the learned parts. Another work [34] introduced a two-branch, part-based network with horizontal and vertical partitioning, as well as an external identity memory to store the features from the two branches separately. The network also includes a triplet center loss and softmax loss for metric learning. The DGPM (Distance-Based Global and Partial Multi-Regional Feature) [35] model consists of three branches—one global and two local branches, one with horizontal parts and the other with vertical parts. The local branches incorporate a self-attention module prior to partitioning in order to suppress background information inside the parts. The authors of [36] utilized three branches, namely a global, local, and complementary branch, for feature extraction. Their method uses partition-based as well as local attribute extraction for robust feature learning. A two-branch model, with one global branch and another part-based local branch using horizontal parts and an attention block prior to partitioning, was described in [37]. In [38], the authors proposed the Multi-Granularity Network (MGN), a three-branch network with one global branch and two part-based branches with a varying number of parts. Each part-based branch includes its own global branch. In unsupervised re-identification methods, complementary part-based and global features are used to generate labels. The complementary association of these features is achieved by exploiting the k-nearest neighbors in the part and global feature spaces [33]. A global–local supervision scheme with channel partitions was proposed in [15]; however, it involves a complex attention layer applied to the channel parts and training of the global layer, which increases complexity. URRNet addresses the problem of viewpoint variation while preserving vehicle details in the re-identification problem by integrating global and local features. The global features are enhanced using a graph convolutional network, and then the keypoints, viewpoint, and multiscale characteristics of the vehicle are extracted [39]. DRReid was trained on multiple source domains and evaluated on unseen target domains for person re-identification [40]. An identity-guided spatial attention mechanism was used to mask discriminative areas, without the need for manual annotation, to address vehicle re-identification in [41]. MDFE-Net addresses re-identification by incorporating a non-local attention block that enhances long-range dependence in discriminative features [42]. A method that integrates convolutional and vision transformer features was proposed in [43] by applying channel-by-channel convolution within the feed-forward layer. To address the VReID problem, a three-branch structure with one global and two local branches was proposed in [16]. The local branches include channel attention modules to generate subtle discriminative features. The authors of [44] utilized a transformer architecture to model the relationship between global and local features for VReID applications. A multilayer fusion approach was designed to achieve this. A two-stage feature generation method was proposed in [45], where the pixel-level feature map is first transformed into a patch-level map within a global context; then, the patches are further refined in a local context using self-attention.

Reported results show that incorporating both global and local features usually gives rise to higher accuracy due to completeness in learning [38]. However, a major drawback of training the global and local branches is the large number of trainable parameters, which leads to higher complexity. Also, the attention layers used to generate local features, along with the multi-branch architecture, result in increased complexity. This drawback can be addressed by our proposed single-branch method, which is discussed in the next section.

3. Proposed Unified Framework for Global and Part-Based Local Features

The structure of the part-based network is shown in Figure 1. Here, the images are pre-processed and passed through a backbone network to obtain a convolutional tensor T, which is then partitioned into a number of uniform parts by a pooling layer. The parts are then trained independently using an equal number of fully connected (FC) layers. During evaluation, the obtained parts before the FC layer are concatenated to form the feature vector, which is used for re-identification.

In the images from the VeRi dataset shown in Figure 2, we can see that there are certain parts of the image, such as background information and occlusion, which do not contain meaningful information and which can give rise to unnecessary, irrelevant information in the parts obtained from the image. To address this and improve image retrieval accuracy, we need contextual global information with respect to the entire image. However, this comes with added complexity. We illustrate this below.

A convolutional or a fully connected layer contains trainable parameters. Let the number of trainable parameters in the backbone network be denoted as x. For a part-based network with p parts, each part is accompanied by a pooling layer and a fully connected layer. Let the number of parameters for a fully connected layer be denoted as c.

So, for a part-based network, say W with p parts, the number of trainable parameters would be

$$No.ofParameters\left(W\right)=x+pc$$

(1)

For a global branch, an additional pooling layer and a fully connected layer are added. Let the branched network containing a part-based branch and a global branch be denoted as G.

$$No.ofParameters\left(G\right)=x+pc+c=x+(p+1)\left(c\right)$$

(2)

If the number of global branches is g, then the number of parameters of network F becomes

$$No.ofParameters\left(F\right)=x+(p+g)\left(c\right)$$

(3)

So, we can see that just by adding a single global branch, the number of trainable parameters increases. In order to overcome these shortcomings, we propose a framework in which the global features are accounted for without adding any trainable parameters to the network. In our present work, we propose a model where the learning of uniform horizontal parts is augmented by taking into account their relative importance with respect to the global feature without introducing any new trainable layer. The learning is induced and influenced by the similarity between the global feature and the partitioned features. In this light, we call our approach the global–local similarity-induced part-based network (GLSIPNet). This approach does not add any extra trainable parameters other than those from the part-based network, and the number of parameters is static.

$$No.ofParameters\left(GLSIPNeT\right)=x+pc$$

(4)

3.1. Data Pre-Processing

This is a normalization step for the images in the datasets. Raw images are subject to the following transformations in this step:

• The images are first resized to a fixed size (384,192). The resizing is performed to match the input size expected by the pretrained backbone model.
• Random Horizontal Flip—This is an image data augmentation technique in which an input image is flipped horizontally with a given probability.
• Random Erasing—Using this technique, a region in an image is randomly selected, and the pixels are erased. This is useful for the robust training of the given data.
• Finally, the images are tensorized and then normalized to the same mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225) as standard ImageNet [7] images. This is done to ensure the best results due to the use of a backbone architecture pretrained on ImageNet.

3.2. Backbone Network

Any image classification model can be selected as the backbone model. In this case, we use ResNet50 [11], pre-trained on the ImageNet dataset [7]. ResNets (residual networks) are used in many computer vision applications. They use interconnections from higher to lower layers to mitigate the problem of vanishing gradients, particularly in very deep networks. The building block of a ResNet is defined as

$$y=F(x,w_i)+x$$

(5)

Here, x and y are the input and output vectors of the layers. The function $F(x,w_i)$ represents the layers in between, where $w_i$ represents the weights. The output is obtained by adding the result of the intermediate layers to the original input, which helps preserve information from the input layer, which in effect is the essence of ResNet. ResNet has been used in similar part-based networks [12,14,26], which inspired us to use it as one of the base models.

3.3. Proposed GLSIPNet Model

The framework of the proposed GLSIPNet model is shown in Figure 3. The method includes two consecutive cycles of training with an equal number of epochs. The first cycle involves partitioning the convolutional tensor generated by the backbone network into uniform horizontal partitions, following a strategy similar to PCB [14]. The convolutional tensor $T^b$ is partitioned uniformly, and the parts are then pooled using average pooling to form the partitioned convolutional tensor set $T^p\{t^1,t^2,\dots t^n\}$. The parts or vectors are then trained separately, and the weights are updated using the cross-entropy loss, which is calculated separately for each part and then combined.

The second cycle is similar to the first one in terms of structure but involves a global component in addition to the local part-based features. A global–local similarity score is generated for each local part using the Euclidean distance of each part from the whole tensor. For every batch of input images, the respective scores are generated by the network. The score serves as the combination of weights for the individual losses. The cross-entropy losses for the individual trainable parts are combined in the ratio of their respective similarity scores.

3.4. Global–Local Similarity Score Generator

A copy of the convolutional tensor $T^b$ is fed into the score generator network. Similar to the first cycle, the partitioned convolutional tensor set $T^p$ is generated. Subsequently, the scores are generated as described below.

Let $T^b$ be the convolutional backbone tensor of size $BxCxHxV$, which is then passed through an average pooling layer to obtain the tensor $T^{bn}$ of size $Bxcx1x1$ (global feature vector). Let $T^p=\{t^1,t^2,\dots t^n\}$ denote the partitioned convolutional tensor set comprising n parts, each of size $BxCx1x1$ (part-based local feature vector).

Then, we have

$$D^p=\{d^1,d^2,\dots ,d^n\}=distanceset$$

(6)

where $d^n$ is the distance between each $T^n$ and $T^b$. The distance may be any type of vector distance.

The similarity score between global and local features is then defined as

$$S=f\left(D^p\right)$$

(7)

where f is any function such that the score is the inverse of the distance.

$$S=\{s^1,s^2,\dots ,s^n\}$$

(8)

The above score set is returned by the network. We then modify our training scheme during the calculation of the total loss in order to incorporate the effect of the weighted scores S.

During the training of a convolutional neural network, the weights are updated according to a loss function. For classification problems, the cross-entropy loss is the widely used loss function. For an N-category classification problem, with the probabilities of each class denoted by $p_i$ the cross-entropy loss is given by

$$L_{CE}=-\sum _{i=1}^N{q}_i\ast log{p}_i$$

(9)

where $p_i$ is the probability of each class and $q_i$ is the ground-truth label for that class. Let the cross-entropy loss for each part i be $L_i$. The total cross-entropy loss becomes

$$L_{tot}=\sum _{i=1}^n{s}^i\ast L_i$$

(10)

where $s^i$$ϵ$ S is the similarity score for a part. Here, the set S can be considered the unifying factor.

From Equation (9), we find that the loss function is the negative logarithm of the probabilities of the classes; the loss value will be lower for higher-ranked classes. Also, from (4), we find that the total loss is updated in such a way that parts with higher similarity with the global feature will have a lower absolute value of the loss. The framework penalizes parts that are farther from the global feature and reinforces parts that are more similar to it. This creates a scenario in which fine-grained local features are learned by training the network to focus more on parts that are relevant to the global context.

The rest of the training process, including weight updates after backpropagation, follows the standard process for training CNNs.

4. Experimental Results and Analysis

We conducted a series of experiments on the VeRi and VehicleId datasets using the proposed method. The input images were resized to 384 × 192 during the training and evaluation stages. The training batch size was set to 32. We performed uniform partitioning in the horizontal dimension with six parts. Training was conducted for 60 epochs in the first cycle and another 10 epochs in the second cycle using the VeRi dataset, and for 60 epochs using the VehicleId dataset. This was due to the larger size of the VehicleId dataset. The plots of the loss values on the training set and the mAP values on the testing set for the VeRi dataset are given in Figure 4 and Figure 5, respectively. It can be seen from the plots that the model was trained to convergence. For our experiments, we used the inverse function f in the second cycle such that $f\left(x\right)=1/x$.

We also conducted experiments using only the first cycle for 120 epochs on both datasets as a baseline and compared the results with those of the proposed method. We also used re-ranking with k-reciprocal nearest neighbors as a post-processing step during evaluation. This step generated a ranked list of gallery output images for each query image using a mixture of Euclidean and Jaccard distances, following the process described in [46]. This method has been shown to improve accuracy in the existing literature. In line with conventional re-identification works in the literature, we used mean average precision and Rank-1, Rank-5, and Rank-10 accuracies for both datasets.

Table 1. Comparison of GLSIPNet with the baseline on the VeRi and VehicleId datasets.

Dataset	Model	mAP	Rank-1	Rank-5	Rank-10
VeRi Dataset	GLSIPNet	76.76	94.63	97.55	98.92
	Baseline	74.17	94.04	97.37	98.56
	GLSIPNet-RR	80.99	95.70	97.55	98.68
	Baseline-RR	78.75	94.69	96.36	97.67
VehicleId Dataset (Small)	GLSIPNet	88.46	85.78	98.34	99.24
	Baseline	86.01	82.90	97.15	98.68
	GLSIPNet-RR	88.84	86.26	98.26	99.29
	Baseline-RR	86.54	83.48	97.36	98.71
VehicleId Dataset (Medium)	GLSIPNet	85.08	82.21	95.73	98.00
	Baseline	81.78	78.55	93.78	96.92
	GLSIPNet-RR	85.08	82.15	95.92	98.04
	Baseline-RR	82.04	78.78	94.04	97.07
VehicleId Dataset (Large)	GLSIPNet	82.66	79.63	93.88	96.98
	Baseline	79.87	76.81	91.16	95.52
	GLSIPNet-RR	83.01	80.12	93.80	97.02
	Baseline-RR	80.16	77.13	91.22	95.58

compares the results of our proposed method with those of the baseline on the VeRi and VehicleId datasets.

Table 2. Comparison of different methods on the VeRi dataset. NR: not reported.

Sl. No.	Method	mAP (%)	R-1 (%)	R-5 (%)
1	SLSR [47]	65.13	91.24	NR
2	VANeT [48]	66.34	89.78	95.99
3	Batch Sample [49]	67.55	90.23	96.42
4	Part-regularized near duplicate [25]	74.30	94.30	98.70
5	MRL + Softmax Loss [50]	78.50	94.30	98.70
6	MRM [23]	68.55	91.77	95.82
7	SAN [27]	72.5	93.3	97.1
8	TCPM [34]	74.59	93.98	97.13
9	DGPM [35]	79.39	96.19	98.09
10	AttributeNet [51]	80.1	97.1	98.6
11	PGAN [52]	79.30	96.5	98.30
12	Swin Transformer [53]	78.6	97.3	NR
13	TransReid [54]	78.2	96.5	NR
14	VehicleGAN [55]	74.2	93.6	97.3
15	URRNet [39]	72.2	93.1	97.1
16	MIMA Net [16]	79.89	94.99	98.81
17	CFSA [45]	79.89	94.99	98.81
18	GLSIPNet + RR (Ours)	80.99	95.7	97.55

and

Table 3. Comparison of different methods on the VehicleId dataset. NR: not reported.

Sl. No.	Method	Rank-1			Rank-5
		Small (%)	Medium (%)	Large (%)	Small (%)	Medium (%)	Large (%)
1	SLSR [47]	75.10	71.80	68.70	89.70	86.10	83.10
2	VANet [48]	88.12	83.17	80.35	97.29	95.14	92.97
3	Batch Sample [49]	78.80	73.41	69.33	96.17	92.57	89.45
4	Part regularised near duplicate [25]	78.40	75.00	74.20	92.30	88.30	86.40
5	MRL + Softmax Loss [50]	84.80	80.90	78.40	96.9	94.1	92.1
6	MRM [23]	76.64	74.20	70.86	92.34	88.54	84.82
7	SAN [27]	79.7	78.4	75.6	94.3	91.3	88.3
8	TCPM [34]	81.96	78.82	74.58	96.38	94.29	90.71
9	DGPM [35]	88.56	87.63	86.31	94.69	94.29	93.34
10	Attribute Net [51]	86.0	81.9	79.6	97.4	95.1	92.7
11	PGAN [52]	NR	NR	77.8	NR	NR	92.1
12	TransReid [54]	83.6	NR	NR	97.1	NR	NR
13	VehicleGAN [55]	83.5	78.2	75.7	96.5	93.2	90.6
14	URRNet [39]	76.5,	73.7	68.2	96.5	92.0	89.6
15	MDFENet [42]	83.66	80.78	77.88	NR	NR	NR
16	MIMANet [16]	83.28	80.14	77.72	96.31	93.71	91.29
17	HCI-Net [56]	83.8	79.4	76.4	96.5	92.7	91.2
18	GLSIPNet + RR (Ours)	86.26	82.15	80.12	98.26	95.92	93.80

compare the results of our proposed method with those of other works on the VeRi and VehicleId datasets, respectively.

In Table 1, we can see that the proposed method consistently outperformed the baseline across all metrics. Re-ranking significantly improved accuracy on the VeRi dataset but resulted in only a slight improvement on the VehicleId dataset. This is due to the dataset structure of VehicleId, which contains only one image per vehicle identity in the gallery set, in contrast to VeRi, which contains many images per vehicle identity. Also, the VehicleId dataset includes only two camera viewpoints, whereas VeRi includes 20. The Euclidean distance is therefore sufficient for the VehicleId dataset, and nearest-neighbor extrapolation does not have any additional advantage. Also, it can be seen that the improvement due to re-ranking on the VeRi dataset was more pronounced for GLSIPNet than for the baseline

We present a comparison of the performance of our proposed method with that of several state-of-the-art methods for vehicle re-identification in Table 2 and Table 3.

The comparison includes different methods that focus on global and part/local feature-based learning, adopting various strategies. SLSR uses a DenseNet121 backbone for feature extraction. It also uses a generative network to generate fake images at different scales for better feature learning [48]. VANeT addresses the problem of different viewpoints [49]. The method learns the viewpoint of the image—front/rear—along with deep features using a GoogleNet backbone. The idea of sampling information data points was explored for vehicle re-identification in a batch sample through the introduction of a triplet sampling variant [50]. Part regularized near duplicate is a framework for combining local region-of-interest-based features and global features [25]. First, the different parts of the vehicle, like windows and lights, are extracted using a part extraction network. Then, a two-branch model is used, where the local branch learns deep features from the local parts generated by the earlier network, and a global branch learns the global features using the ResNet50 backbone. MRL divides the problem of re-identification into two groups on the basis of view—one of images with the same view and the other with different views. Further, on a ResNet 50 backbone, with global features, it uses two types of ranking loss to optimize the training process [50]. Local feature extraction is performed in detail in MRM, where a separate labeled dataset is created to train a model for local feature/attribute extraction [23]. The model is then used to extract the features from the re-identification dataset and then merge them with the global features generated using a ResNet backbone. A part-based network with horizontal partitions, along with a global branch, was introduced in SAN [27]. Here, the global branch has two fully connected layers—one for ID prediction and the other for attribute prediction (model/type)—with a DenseNet backbone. TCPM uses part-based horizontal and vertical partitioning in two branches, along with a triplet center loss and a softmax loss for optimization [35]. A three-branch network with a global branch and two part-based branches (one with horizontal parts and the other with vertical parts) is used in DGPM, along with an attention module to suppress background information interference in the part-based branches [36]. The method also uses spatio-temporal information (camera and time) to improve accuracy. However, our method generates results without spatio-temporal information for a better comparison. AttributeNet is a joint network that generates attribute-based and global re-identification features [51]. A constraint is used to encourage the joint feature. PGAN is a local feature/attribute aggregating method with a local feature branch and a global feature branch [52]. However, it uses an attention mechanism to assign weights to the local features prior to aggregation with the global feature. URRNet integrates global and local features. The global features are enhanced using a graph convolutional network, and then the keypoints, viewpoint, and multiscale characteristics of the vehicle are extracted as local features [39]. MDFE-Net includes a non-local attention block to enhance the long-range dependence of discriminative features [42]. A three-branch structure with one global and two local branches was proposed in [16]. The local branches include channel attention modules to generate subtle discriminative features. A two-stage feature generation method was proposed in [45], where the pixel-level feature map is first transformed into a patch-level map within a global context; then, the patches are further refined in a local context using self-attention.

Analysis on VeRi: In Table 2, it can be seen that our proposed approach outperformed the other methods in mean average precision. However, AttributeNet [51] and PGAN [52] outperformed our proposed approach in Rank-1 accuracy. AttributeNet uses separate networks to generate attributes, which adds to the complexity of the network. PGAN uses attribute-based features along with an attention model, which increases complexity. Our model outperformed SAN, which combines a global network with a horizontal part-based network [27]. This result is significant because it establishes the motivation for our proposed approach in that our unifying framework can effectively replace a global feature branch. Our approach also outperformed general transformer-based methods like TransReid and Swin Transformer, as well as recent state-of-the-art methods like URRNet, MIMANet, and CFSA.

Analysis on VehicleId: In Table 3 it can be seen that our proposed model outperformed the other models in Rank-5 accuracy. However, in Rank-1 accuracy, VANet [48] and DGPM [35] performed better due to their complex attention mechanism and partitioning in multiple dimensions. Again, our model outperformed SAN [27], thereby establishing the superiority of our induced global similarity learning over direct global feature learning. Our approach outperformed most recent state-of-the-art approaches on the VehicleId dataset.

Empirical analysis: The proposed GLSIPNet model rewards parts that are more similar to, or more representative of, the vehicle image while penalizing parts that are dissimilar. In Figure 6, the similarity scores are shown for three vehicle images. The scores are used as weights for the losses of the individual parts to arrive at the total loss over which the entire network is trained. This leads to higher attention on the more relevant parts. We can see in Figure 6 that the model accurately identified regions of more relevance in the vehicle images. This can help in cases of images with occlusion and other interference in patches.

The following conclusions can be drawn from the above experimental results:

• Our proposed model clearly outperforms the baseline, thereby establishing the significance of global–local similarity-induced part learning in addressing the VReID problem using local part-based features. Our method achieves improvements of 2.5% (mAP) on the VeRi dataset and 2.4%, 3.3%, and 2.8% (mAP) on the small, medium, and large variants of the VehicleId dataset, respectively.
• Our proposed model also performs better than the other comparison models in terms of mAP on the VeRi dataset and Rank-5 accuracy on the VehicleId dataset.
• The models proposed in [51,52], which outperform our model in Rank-1 and Rank-5 accuracies on the VeRi dataset, fail to do so on the VehicleId dataset. Similarly, the models proposed in [35,48] perform better in Rank-1 accuracy, but in Rank-5 accuracy on the VeRi dataset, our method performs better. Thus, considering its low complexity and consistent performance, our approach offers a better trade-off than attention-based models with higher complexity.
• Our proposed model successfully achieves better results in part-based vehicle re-identification by incorporating a global aspect without adding any additional complexity. This approach is found to be more effective than directly incorporating a global feature branch, as demonstrated by the comparison with SAN [27] in Table 2 and Table 3.

5. Conclusions

The current work proposes an improved part-feature extraction and aggregation strategy, named GLSIPNet, as a novel contribution to the deep learning CNN framework. Without increasing model complexity or adding trainable parameters, the effect of learning part-based features using global cues is incorporated through a similarity score computed between the global and partitioned features. We have thoroughly evaluated the proposed approach through extensive experimentation on two standard datasets, VehicleId and VeRi, demonstrating its superior accuracy compared to both the baseline and other related works in the literature. Our method outperforms the baseline by 2.5% (mAP) on the VeRi dataset and by 2.4%, 3.3%, and 2.8% (mAP) on the small, medium, and large variants of the VehicleId dataset, respectively. These results show potential for further extension using other distance metrics and other similarity scores and their combinations. Also, the framework may be extended to address other retrieval problems like person re-identification. The accuracy of the model needs to be increased further, which is a limitation of the current work, and it could be improved by incorporating attention modules and state-of-the-art vision transformers. Also, the effect of parameters, like the number of parts and the mode of partitioning, may be analysed as part of future work.