Cross-Domain Person Re-Identification Based on Feature Fusion Invariance

Abstract

Cross-domain person re-identification is a technique for identifying the same individual across different cameras or environments that necessitates the overcoming of challenges posed by scene variations, which is a primary challenge in person re-identification and a bottleneck for its practical applications. In this paper, we learn the invariance model of cross-domain feature fusion in a labeled source domain and an unlabeled target domain. First, our method learns the global and local fusion features of a person in the source domain by means of supervised learning with no component label and only person identification and obtains the fusion features of the person in the source and target domains by means of unsupervised learning. Based on person fusion features, this paper introduces feature memory to store the fused target features and designs a cross-domain invariance loss function to improve the cross-domain adaptability of the person. Finally, this paper carries out cross-domain person re-identification verification experiments between the Market-1501 and DukeMTMC-reID datasets; the experimental results show that the proposed method achieves significant performance improvement in cross-domain person re-identification.

1. Introduction

Person re-identification is a technique that uses computer vision to determine whether a specific person exists in an image or a video sequence; it has broad application prospects in the fields of public safety, intelligent transportation, and smart cities. The person re-identification task is challenging because person images are easily affected by light changes, various body postures, and different camera angles.

With the development of deep learning techniques, person re-identification methods have evolved from traditional feature engineering methods to the end-to-end utilization of deep neural networks, especially convolutional neural networks (CNNs), for end-to-end feature representation learning. The method in [1] designed various depth models to extract global features from entire person images. To mine local information from different body parts, some local information-based methods [2] are used to extract local features. Some research approaches [3] process person images with the help of human pose estimation or body-cutting techniques to locate accurate and meaningful body parts. Person re-identification is performed by drawing on the results of human pose estimation for person area segmentation. However, the dataset labeling bias between pose estimation and person re-identification remains an obstacle to the accurate delineation of person re-identification images. The use of direct segmentation strategies [4] to segment feature maps or person images into multiple horizontal grids without the need for training labels for each region of the person has received considerable attention. Among them, the visible part perception model (VPM) [5] focuses on learning fine-grained region properties on the regions that are jointly visible in two person images to solve the spatial location mismatch between person parts and the overall person image, which significantly improves the feature representation learning ability. Sun Y. et al. [6] proposed a part convolution pooling method based on person local features to enhance the semantic consistency of person part regions in person re-identification. The introduction of a deep learning-based attention mechanism [7] to a person rerecognition model to automatically assign a weight to each person image further boosts person rerecognition performance. Chen T. et al. [8] proposed a person re-identification model based on a dual attention mechanism through CNN feature channel aggregation and spatial location awareness, which integrates an attention module and diversity regularization to learn representative, robust, and more discriminative features. Li et al. [9] propose a Diverse and Compact Transformer (DC-Former) that achieves a similar effect by partitioning the embedding space into multiple diverse and compact subspaces. The compact embedding subspace facilitates the model in learning more robust and discriminative embeddings for identifying similar classes, while the fusion of these diverse embeddings containing finer-grained information further enhances the effectiveness of reID. Rao et al. [10] propose a generic Transformer-based Skeleton Graph prototype contrastive learning (TranSG) approach with structure–trajectory prompted reconstruction to comprehensively capture skeletal relations and valuable spatial–temporal semantics from skeleton graphs for person re-identification. Although the above approach considers local instance knowledge, it fails to achieve simultaneous feature discovery for samples at both the global distribution and local instance levels.

The above supervised learning-based person re-identification methods rely on rich labeled data and deep networks to achieve success on single-domain datasets. However, serious performance degradation occurs when this approach is applied to unknown datasets. In realistic surveillance scenarios, collecting and labeling a large quantity of training data for a specific scene is not feasible. Therefore, unsupervised domain adaptation (UDA) methods have become a popular research topic [11].

UDA methods are mainly classified into image-integration, feature distribution, and clustering-based adaptation methods. Image-integration-based UDA methods are usually based on generative adversarial networks (GANs), which convert the labeled images of the source domain into unlabeled target-domain image styles and attempt to preserve the person ID information; thus, they use the source ID labels for supervised learning on a synthetic gallery. Chen Y. et al. [12] proposed a pairwise conditional generative adversarial network that uses a large number of unlabeled target-domain person instances as a contextual guide for generating images, transferring source domain targets to the target-domain context, and enhancing the discriminative power of person re-identification by migrating rich contextual information. Rahman et al. [13] incorporate a correlation alignment module and adversarial learning to create a more domain-agnostic model that can effectively reduce domain discrepancy. Lee et al. [14] present a Camera-driven Curriculum Learning (CaCL) framework, which harnesses camera labels of person images to facilitate a progressive transfer of knowledge from source to target domains. To tackle the issue of camera bias, they further introduce a Camera-Diversity (CD) loss. This loss encourages the inclusion of person images with identical pseudo-labels but captured by diverse cameras, thereby enhancing discriminative feature learning.

However, this image synthesis method is based only on global image distribution alignment and does not consider fine-grained instance-level knowledge. The feature-based UDA method assumes that the discriminative ability of the model is related to the statistical information of the global feature distribution. Li et al. [15] introduced the concept of stream shape regularization, which achieves stream shape alignment and balanced distribution adaptation by weighting the distributional importance of the source and target domains and preserving the neighborhood structure. Moreover, a novel Meta Distribution Alignment (MDA) method [16] is proposed to enable them to share similar distribution in a test-time training fashion. Ni et al. [17] propose a pure Transformer model (termed Part-aware Transformer) for DG-ReID by designing a proxy task to mine local visual information shared by different IDs. Jiao et al. [18] develop a new normalization scheme called Dynamically Transformed Instance Normalization (DTIN) to alleviate the drawback of IN (i.e., the limitation of eliminating discriminative patterns). Pu et al. [19] design an Adaptive Knowledge Accumulation (AKA) framework that is endowed with two crucial abilities: knowledge representation and knowledge operation. However, the feature adaptive method is based on global feature representation distribution alignment, which also ignores local instance matching.

Recently, some clustering-based UDA methods have achieved good results. ACT utilizes an asymmetric coteaching framework, which reduces the noisy labels generated by the clustering algorithm [20]. Tang et al. [21] introduced a random discard strategy to mitigate the impact of noisy pseudo-labels on model performance. Their approach enhances both the model’s generalization capacity and its feature representation efficacy within the target domain through a mutual teaching framework. Lee et al. [14] put forth a camera-guided learning framework that exploits camera labels associated with pedestrian images to facilitate knowledge transfer from the source to the target domain. This framework incorporates a camera discrepancy loss, which prompts the model to learn discriminative pedestrian features. Li et al. [22] devised a novel intra-to-inter mechanism aimed at extracting more discriminative pedestrian features from images. This innovation addresses the challenges posed by camera-style discrepancies to the accurate prediction of pseudo-labels. Lastly, Ge et al. [23] presented an end-to-end Domain Adaptive Network (DAN) framework. It optimizes the feature encoder by dynamically modeling sample relationships in an online fashion and further refines the network leveraging both the genuine labels of source domain samples and the inferred pseudo-labels from the target domain, thereby advancing the adaptation process.

These methods ignore the differences between differently labeled pedestrians in the target domain and the visual differences that exist between the same labels in the target domain, which has important implications for the UDA problem.

In this paper, we simultaneously explore the sample feature characteristics of the source domain from both global distribution and local instances and fuse the two to train the network model to further improve the feature-matching accuracy from the source domain labeled training data to the target-domain unlabeled training data. Based on the feature fusion invariance, the feature representation of each marker in the target domain is then stored in the memory to impose invariance constraints on the person in the target domain to improve cross-domain feature invariance.

2. Materials and Methods

2.1. Framework Description

Figure 1 shows the framework diagram of the fusion feature invariant-based network proposed in this paper. The underlying backbone network uses the pretrained ResNet-50 [24] network on ImageNet [25], and the labeled source domain data and unlabeled target-domain data are fed into the network to obtain the feature representation of the person instances. In this paper, the network structure of ResNet-50 with Pooling-5 layers is retained, and deep feature extraction is divided into a global feature module and a local feature module. A global average pooling layer is added to the global feature module to obtain global features. An adaptive average pooling layer is added to the local feature module to divide the 3D tensor T into P parts, after which the dimensionality is reduced to 256 dimensions. After that, the global and local features are fused by the multiscale channel attention module (MSCAM) [26] for feature fusion, and the model is optimized using the cross-entropy loss function and the ternary loss function. In the target domain, feature memory is introduced to save the fused global and local features of each person sample, and by estimating the similarity between a small batch of target samples and the features in the feature memory, invariance loss occurs in the labeled target domain.

2.2. Integration Features

2.2.1. Global Feature

In this paper, the global average pooling (GAP) of the GLAD model [27] was used to extract global features after the Pooling-5 layer of ResNet-50. Global features can identify features macroscopically, but at the same time, they lack knowledge of the internal details of person images.

2.2.2. Local Feature

Employing local features aims to capture detailed information from key human body parts, compensating for the limitations of global features under varying poses, occlusions, or appearance changes. Focusing on distinct regions, such as the head, torso, and legs, local features enhance recognition accuracy in complex scenarios, thereby achieving more robust pedestrian matching capabilities. In this paper, local person feature extraction is based on the PCB model [6], and the person image is uniformly divided into six regions from top to bottom, as shown in Figure 2a. Considering the internal features of each person part, a linear classification softmax function is used as the person part classifier. After this classification, the classification of the parts belonging to each feature block can be obtained, and the effect is shown in Figure 2b.

To align person widget regions for two images that dynamically match localities from top to bottom to find the alignment of local features with minimum total distance, each pair of sample pairs calculates the minimum total distance and then forms a matrix of minimum total distances, which is defined as the shortest path between the two images. Given the local features of two images $F=f_1,f_2,f_3\dots$ and $G=g_1,g_2,g_3\dots$, first, the distance is normalized by element transformation to $[0,1]$ as follows:

$$d_{i,j}=exp\left({\Vert f_i-g_j\Vert }_2\right)-1/exp\left({\Vert f_i-g_j\Vert }_2\right)+1$$

(1)

In the formula, $d_{i,j}$ is the distance between the i component of the first person image and the j component of the second person image. A distance matrix is formed based on these distances, and the local distance between two images is defined as the total distance of the shortest path from $(1,1)$ to $(t,t)$ in the matrix. This distance can be calculated via dynamic programming as follows:

$$S_i,j=\left\{\begin{array}{cc}{d}_{i,j}\hfill & i=1,j=1\hfill \\ S_{i-1,j}+d_{i,j}\hfill & i\ne 1,j=1\hfill \\ S_{i,j-1}+d_{i,j}\hfill & i=1,j\ne 1\hfill \\ \min (S_{i-1,j},S_{i,j-1})+d_{i,j}\hfill & i\ne 1,j\ne 1\hfill \end{array}\right.$$

(2)

The division and semantic alignment of the local features of the person components are shown in Figure 2c. After semantic alignment, the local features are obtained, and each local feature is used as a node to construct a complete local graph; additionally, the relationships between parts and within parts are learned. These nodes are connected by local edges and local inner edges. For each node in the graph, we use an attention weight to weight all its neighbors and then aggregate them to obtain these two relationships. At the same time, we use the loss of attention regularization to constrain the attention weight of local edges to accurately describe the relationship between local edges and inject context information into the attention weight of local inner edges to consider structural information.

Taking the local feature $F_p$ as the node, a complete local graph $G=(F,E)$ is constructed, where e is the edge set, including the local intermediate edge connecting the corresponding parts of different person images and the adjacent nodes and the local inner edge connecting the same person image node. There are $N\ast P$ incompletely connected nodes in a complete local graph, and each node has several adjacent nodes, which are the connections of two kinds of edges. We divide these neighboring nodes into three types, as shown in Figure 3, namely, $N_{SP}$, $N_{AP}$, and $N_{SI}$. Among them, $N_{SP}$ refers to neighboring nodes of corresponding parts of different person images, $N_{AP}$ refers to neighboring nodes of neighboring parts of different person images, and $N_{SI}$ refers to neighboring nodes of different parts of the same person image.

The traditional GAT [28] is used to distinguish the importance of neighboring nodes and gather information. We distinguish the attention weight in the learning process between local relations and local internal relations. In the complete local graph, the attention weight $\alpha (f_p^i,f_q^j)$ between node $f_p^i$ and its neighboring node $f_q^j$ is calculated according to the cosine similarity. Specifically, we first transform $f_p^i$ and $f_q^j$ by using the transformation matrix W and then calculate the cosine similarity to measure their correlation. Finally, we use the softmax function to obtain the attention weight $\alpha (f_p^i,f_q^j)$.

$$\alpha (f_p^i,f_q^j)=\frac{exp\left(\varphi (W{f}_p^i,W{f}_q^j)\right)}{{\sum }_{f_q^k\in N\left(f_p^i\right)}exp\left(\varphi (W{f}_p^i,W{f}_q^k)\right)}\times K$$

(3)

where $\phi$(.,.) is the cosine similarity function, $N\left(f_p^i\right)$ is the neighbor node set of node $f_q^j$, and k is the adjustment coefficient considering different types of neighbor nodes. There are three types of neighboring nodes. When node $f_q^j$ belongs to different neighbor node sets, we define a different K.

When $i\ne j$ and $p=q$, both $f_p^i$ and $f_q^j$ come from the corresponding parts of different images, and $f_q^j$ belongs to the adjacent node set $N_{SP}$, that is, $N\left(f_p^i\right)=N_{SP}$. Let us set $K=1$ in our experiment.

When $i\ne j$, $p=q+1$, or $p=q-1$, $N\left(f_p^i\right)$ and $f_q^j$ come from adjacent parts of different images, and $f_q^j$ belongs to the adjacent node set $N_{AP}$, that is, $N\left(f_p^i\right)=N_{AP}$. We set k to a constant smaller than $N_{SP}$ because the correlation decreases with increasing spatial distance.

When $i=j$, $p\ne q$, $f_p^i$, and $f_q^j$ come from different parts of the same image, and $f_q^j$ belongs to the adjacent node set $N_{SI}$, that is, $N\left(f_p^i\right)=N_{SI}$. With decreasing spatial distance, the correlation between parts increases. Therefore, context information is injected into the attention weight to consider the structural information, and k is expressed as

$$K=\frac{1}{|p-q|\times s}$$

(4)

We learn the local relationship from the neighbor node sets $N_{SP}$ and $N_{AP}$, and we learn the local internal relationship from the neighbor node set $N_{SI}$. To integrate the interlocal relationship and the intralocal relationship, we aggregate these local features with the attention weight to obtain the feature $z_p^i$ as follows:

$$z_p^i=\sigma (f_p^i+\sum _{f_q^j\in N\left(f_p^i\right)}\alpha (f_p^i,f_q^j)V{f}_q^j)$$

(5)

2.2.3. Fusion Features

The global features and local features are fused by a multiscale channel attention module (MSCAM) [16].

$$\mathrm{Z}=\mathrm{M}(\mathrm{X}\uplus \mathrm{Y})\otimes \mathrm{X}+(1-\mathrm{M}(\mathrm{X}\uplus \mathrm{Y}\left)\right)\otimes \mathrm{Y}$$

(6)

where X is the local feature $p_i$, Y is the global feature, and then the fusion feature set $Z_i$ is obtained.

2.3. Feature Memory

Our method proposes enhancing the invariance learning of the network by estimating the similarity between target images to improve the generalization ability of the network on the target test set. Our method constructs a feature memory for storing the fusion features of all target images to achieve this goal. The memory is a key-value structure with a key (k) and a value (v).

In the memory, each slot stores the L2-normalized feature of FC-4096 in the key part and the tag in the value part. Given unlabeled target data containing Ni subimages, the feature memory has 6N feature storage slots. In this paper, the fusion features of each image are regarded as a separate category. In each slot, the features and tags of the target image are stored. In the initialization process, the values of all functions in the key are initialized to zero, the corresponding indices are designated as the labels of the target samples, and they are stored in the values. For example, the class of the i-th target image in the values is assigned to $v\left[i\right]=i$. During the whole training process, the labels in the values are fixed. In each training iteration, for the target training sample $X_{t,i}$, the 4096-dimensional feature $h(X_t,j)$ is obtained by the feature extraction module and L2 normalization. In the backpropagation process, the feature weights in the keys of the training samples $X_{t,j}$ are updated:

$$R\left[\begin{array}{c}i\end{array}\right]=aR\left[\begin{array}{c}i\end{array}\right]+\left(\begin{array}{c}1-a\end{array}\right)R\left[\begin{array}{c}i\end{array}\right]$$

(7)

where $R\left[i\right]$ is the key pool of the image in the first slot. The superparameter ∈[0, 1] controls the update rate.

2.4. Feature Invariant Learning

Because the depth reID model obtained only by supervised learning in the source domain is sensitive to changes in the target domain, this paper considers the difference changes in the target domain, and the feature invariance function is:

$$L_{\mathrm{tgt}}=-\frac{1}{n_t}\sum _{i=1}^{n_i}\sum _j{\omega }_{i,j}logp\left(\mathrm{j}\right|\overline{x_{t,i}})$$

(8)

First, the camera style characteristics of the source domain are trained and learned based on supervised learning. When it is applied to the target domain, there will be differences in camera style, so this paper narrows the distance between an image and the images converted by other cameras. StarGAN is used to train a camera style conversion model, and the learned camera style conversion model is used. Assuming that C is the number of camera styles in the target domain, a picture can contain $C-1$ images of other camera styles that belong to the same person identification ID. The input in the invariant feature $x_{(}t,i)$ is $X_{t,i}$ and a picture randomly selected from the set of generated pictures. If $i=j$, different images with the same identification ID are processed. At this stage, the distance between samples with the same ID decreases, and the distance between samples with different IDs increases. To achieve this goal, $N_t$ target images are regarded as $N_t$ different classes, and each image is regarded as a class. Given the target image $X_{t,j}$, the cosine similarity between it and the features already stored in the feature template pool is calculated, and then the prediction probability that $X_{t,j}$ belongs to the class is calculated by using the softmax function. The formula is as follows:

$$p\left(i|x_{t,i}\right)=e^{\left(R{\left[i\right]}^{T}h\left(x_{t,i}\right)/\beta \right)}/\sum _{j=1}^{n_t}{e}^{\left(R{\left[i\right]}^{T}h\left(x_{t,i}\right)/\beta \right)}$$

(9)

$\beta$∈(0,1) is the balance factor, which minimizes the negative logarithmic likelihood on the target training image, and the invariance function becomes:

$$L_{\mathrm{tgt}}=-\frac{1}{n_t}\sum _{i=1}^{n_t}\sum _{j}logp\left(j\right|\overline{x_{t,i}})$$

(10)

When $i\ne j$, the positive samples in the target domain are used to meet the changes in the target domain to improve the stability of the ReID model. First, the cosine similarity between $h\left(X_{t,j}\right)$ and the values stored in the feature template pool is calculated. Then, the k neighborhood of $\left(X_{t,j}\right)$ is found in the feature template pool. Therefore, the final invariance function is:

$$L_{\mathrm{tgt}}=-\frac{1}{n_t}\sum _{i=1}^{n_i}\sum _j\frac{1}{k}logp\left(j\right|\overline{x_{t,i}})$$

(11)

3. Experimental Results

3.1. Ablation Experiment

Ablation Study is an experimental approach used to evaluate the impact of individual components on a model’s overall performance. It involves systematically removing or disabling certain components or features of the model to observe the resultant effects on its performance, thereby illuminating the significance of key factors and components within the model.

We will analyze the independent effects of global feature module (GAP), local feature module (PAF), feature fusion (AFF), invariance loss (Inv), and feature memory on the cross-domain person re-identification model, the results are as follows: Figure 4 shows.

3.1.1. Invariance Analysis Based on Feature Fusion

Validation experiments were carried out on the Market-1501 [29] and DukeMTMC-reID [30] datasets to investigate the effectiveness of the proposed feature fusion and cross-domain feature invariance learning methods for the target domain. The experiment was divided into two groups. In one group, DukeMTMC-reID→Market-150 refers to DukeMTMC-reID as the source domain and Market-1501 as the target domain. The other set Market-1501→DukeMTMC-reID has the opposite effect. In this paper, global features are taken as the baseline of the experiment and recorded as GAP, and local features based on an attention mechanism are recorded as PAF (part-aligned feature). The features based on the fusion of the attention mechanism are denoted as attention fusion features (AFFs), and cross-domain feature invariance learning is denoted as Inv. GAP, FAP, AFF, Inv + GAP, Inv + FAP, Inv + GAP+FAP, and a combination of Inv + AFF are used to verify the validity of each part.

(1)

Effectiveness analysis of local features

As shown in

Table 1. Performance ablation experiments comparison of methods when tested on Market-1501 and DukeMTMC-reID. Rank-n metrics assess the proportion of correct matches within the top n results when querying pedestrian images, where rank-1 demands the correct match at the first position. Rank-5, rank-10, and rank-20 progressively relax this criterion, examining the top 5, 10, and 20 results, respectively. mAP (mean Average Precision) evaluates the average precision, reflecting the overall performance of a retrieval system by combining the true positive rate and the ranking of results across all queries.

Method	Market-1501						DukeMTMC-reID
	Src	R-1	R-5	R-10	R-20	mAP	Src	R-1	R-5	R-10	R-20	mAP
GAP	DukeMTMC-reID	65.7	81.5	86.0	89.9	31.9	Market-1501	52.7	64.7	69.7	73.4	27.0
PAF		75.7	89.4	92.0	94.3	49.5		64.1	76.5	80.3	83.8	43.0
AFF		77.6	89.7	92.0	95.2	57.0		66.5	78.6	82.1	86.5	50.8
Inv + GAP		81.2	92.1	93.7	96.6	65.4		71.6	81.0	84.2	88.3	59.0
Inv + PAF		86.4	93.1	95.5	97.8	76.6		78.5	84.7	90.3	91.7	68.4
Inv + GAP + PAF		87.4	94.9	96.0	98.5	78.1		80.1	86.0	92.1	94.6	70.3
Inv + AFF		90.6	96.4	98.3	99.3	82.2		82.5	87.7	93.6	95.4	71.6

, in the cross-domain experiment from the DukeMTMC-reID dataset to the Market-1501 dataset, the average accuracy mAP based on the local feature (PAF) increased by 17.6%, and the rank 1 increased by 10% compared with the global feature baseline (GAP). In the comparison of Inv + GAP + PAF and Inv + GAP, the two evaluation indices increased by 12.9% and 6.2%, respectively. In the cross-domain experiment from the Market-1501 dataset to the DukeMTMC-reID dataset, PAF increased by 16% and 11.4% compared with GAP, and Inv + GAP + PAF increased by 11.3% and 9.1% compared with Inv + GAP. The experimental results show that local features can focus on more fine-grained pedestrian features, and it is necessary to use local features to train and optimize the network model.

(2)

Effectiveness of feature fusion

In the two cross-domain experiments, the combination of invariant learning and attention mechanism feature fusion (Inv + AFF) improved by 2.1% and 3.2% and 1.3% and 2.4%, respectively, compared with the combination of Inv + GAP + PAF. The experimental results verify the effectiveness of using an attention mechanism for feature fusion.

(3)

Effectiveness of invariant learning

In two cross-domain experiments, Inv + GAP increased by 33.5% and 15.5% and 32% and 18.9% compared with GAP, respectively. Compared with those of PAF, those of Inv + PAF increased by 27.1%, 10.7%, 25.4%, and 14.4%, respectively. For both local features and local and global fusion features, the addition of cross-domain feature invariance learning improves the pedestrian rerecognition performance. The average accuracy of the pedestrian rerecognition mAP increases by 13.1% and 23.8%, respectively, and the performance of rank 1 increases by 20.6% and 29.8%, respectively. Finally, the model combining global features, local features, and feature invariance achieves the best pedestrian rerecognition performance, and its rank-1 and mAP reach 90.6% and 80.2%, respectively, on the target-domain Market-1501 data. The same consistency conclusion experiment also occurred in the cross-domain pedestrian rerecognition from the Market-1501 dataset to the DukeMTMC-reID dataset.

3.1.2. Analysis of the Importance of Feature Memory

To verify the effectiveness of the feature memory proposed in this paper for cross-domain pedestrians to reidentify gender, the function of feature memory is tested on the basis of fusion feature invariance. As shown in

Table 2. Performance comparison of memory and minibatch.

Method	DukeMTMC-reID → Market-1501
	R-1	R-5	R-10	R-20	mAP
Minibatch	76.4	79.5	82.3	85.6	66.2
Minibatch + Memory	$\mathbf{90}.\mathbf{6}$	$\mathbf{96}.\mathbf{4}$	$\mathbf{98}.\mathbf{3}$	$\mathbf{99}.\mathbf{3}$	$\mathbf{82}.\mathbf{2}$

, the method of adding feature memory is superior to the method based on a small batch, with improvements of nearly 14.2% and 16% in rank-1 and mAP, respectively. However, the disadvantage of using feature memory is that it adds limited additional training time (≈+1.3 min) and GPU memory (≈+260 MB) compared to using the mini-batch.

3.2. Comparison Experiment with the Current First-Class Algorithm

To compare the effectiveness of cross-domain pedestrian rerecognition based on local semantic feature invariance proposed in this paper, the experiments in this section conducted a comparative verification of cross-domain pedestrian rerecognition performance on the Market-1501 and DukeMTMC-reID datasets. As shown in

Table 3. Performance comparison with other advanced methods.

Method	Market-1501				DukeMTMC-relD
	R-1	R-5	R-10	mAP	R-1	R-5	R-10	mAP
SPGAN + LM [31]	57.7	75.8	82.4	26.7	46.4	62.3	68.0	26.2
TJ-AIDL [32]	58.2	74.8	81.1	26.5	44.3	59.6	65.0	23.0
CamStyle [33]	58.8	78.2	84.3	27.4	48.4	62.5	68.9	25.1
HHL [34]	62.2	78.8	84.0	31.4	46.9	61.0	66.7	27.2
ECN [35]	75.1	87.6	91.6	43.0	63.3	75.8	80.4	40.4
SSG [36]	80.0	90.0	92.4	58.3	73.0	80.6	83.2	53.4
MMT [37]	88.4	96.2	97.8	74.6	75.1	87.3	91.2	61.0
Ours	90.6	96.4	98.3	82.2	82.5	87.7	93.6	71.6

, in the cross-domain pedestrian rerecognition experiment of DukeMTMC-reID to Market-1501, the performance is as follows: rank-1 is 90.6%, rank-5 is 96.4%, rank-10 is 98.3%, and mAP is 82.2%. Compared with the current first-class cross-domain pedestrian rerecognition algorithms, the performance is significantly improved. Similarly, in the cross-dataset experiments of Market-1501 to DukeMTMC-reID, the proposed method outperforms the pedestrian rerecognition algorithm. Therefore, the cross-domain pedestrian rerecognition method based on local semantic feature invariance in this paper effectively improves the performance of cross-domain pedestrian rerecognition.

3.3. Parameters Analysis

(1)

The number of parts p.

Intuitively, p determines the granularity of the part feature. When p = 1, the learned feature is a global one. As p increases, retrieval accuracy improves at first. However, accuracy does not always increase with p, as illustrated in Figure 5. When $p=8$ or 12, the performance drops dramatically. A visualization of the refined parts offers insights into this phenomenon, as illustrated in Figure 6. When p increases to 8 or 12, some of the refined parts are very similar to others and some may collapse to an empty part. As a result, an over-increased p actually compromises the discriminative ability of the part features. In real-world applications, we would recommend using $p=6$ parts.

4. Conclusions

In response to the challenges posed by scarce annotation resources and pronounced domain discrepancies, we devise a cross-domain person re-identification model. Central to this model is a strategy rooted in the invariance of feature fusion, aimed at bridging the data domain gap and achieving efficient, robust person matching.

Our model dispenses with the need for intricate part annotations in the source domain, relying solely on person identity labels. It ingeniously integrates attention-guided extraction and fusion mechanisms for both global and local features. By constructing a local feature graph and employing Graph Attention Networks (GAT), it adeptly discriminates and aggregates neighboring node information. Leveraging multi-scale channel attention modules, it then deeply integrates these global and local features, capturing not only the overall structure of the person but also meticulously detailing their local characteristics, thereby realizing profound comprehension and precise depiction of person features.

To tackle the unlabeled data challenge in the target domain, we adopt an unsupervised learning strategy to effectively excavate and distill the essence of person features. Building upon this, we incorporate a feature memory module, which effectively stores and organizes the fused pedestrian features learned from the target domain, with a particular emphasis on ensuring that these features retain the distinctive distribution characteristics inherent to the target domain.

In pursuit of consistent performance across domains, we meticulously craft a cross-domain invariance loss function designed to minimize disparities between feature spaces of the source and target domains, significantly enhancing the model’s generalization capabilities and robustness under diverse environmental conditions and varying camera configurations.

Extensive experimental validations, conducted using the two cross-domain datasets—DukeMTMC-reID to Market-1501 and vice versa—conclusively demonstrate that our proposed method achieves substantial performance leaps in both cross-domain tasks. The marked improvements in key metrics such as identification accuracy and mean Average Precision (mAP) resoundingly validate the exceptional efficacy and practical value of our approach in addressing the complexities of cross-domain pedestrian re-identification.