Enhancing Human Pose Transfer with Convolutional Block Attention Module and Facial Loss Optimization

Abstract

Pose transfer methods often struggle to simultaneously preserve fine-grained clothing textures and facial details, especially under large pose variations. To address these limitations, we propose a model based on the Multi-scale attention guided pose transfer model, with modifications to its residual block by integrating the convolutional block attention module and changing the activation function from ReLU to Mish to capture more features related to clothing and skin color. Additionally, as the generated images had facial features differing from the original image, we propose two different facial feature loss functions to help the model learn more precise image features. According to the experimental results, the proposed method demonstrates superior performance compared to the Multi-scale Attention Guided Pose Transfer (MAGPT) on the DeepFashion dataset, achieving a 3.41% reduction in FID, a 0.65% improvement in SSIM, a 2% decrease in LPIPS, and a 2.7% decrease in LPIPS. Ultimately, only one reference image is required to enable users to transform into different action videos with the proposed system architecture.

1. Introduction

Pose transfer research aims to synthesize a realistic image of a person in a new pose while maintaining their identity and clothing details. It is motivated by several key factors, including advancements in computer vision and practical applications in various industries. The applications include fashion design and marketing, animation and game development, medical and rehabilitation, and data augmenting for machine learning.

Since the introduction of Generative Adversarial Networks (GANs) [1], various types of GANs [2,3,4,5,6] have made significant progress in the field of realistic image synthesis over the past few years. The initial idea of GANs was to use a generator and a discriminator to train the generative model through adversarial competition. Later, to generate images for specific domains, Conditional Generative Adversarial Networks (CGAN) [7,8,9,10,11,12] were developed. CGANs can be applied to various fields, including image translation [13,14,15], image inpainting [16], image super-resolution [17,18], and pose transfer tasks [7,19]. In the field of human image generation, generative models often incorporate the concept of CGANs, using images of people with target poses as input to generate realistic human images that match the target pose, as shown in Figure 1.

The research proposed by Lassner et al. [3] combines variational autoencoder [20] and GANs to generate images of people wearing different clothes based on 3D human models. The work proposed by Zhao et al. [7] uses a method that progresses from low frequency to high frequency to generate multi-view images of people or clothes from a single viewpoint observation. This method starts with a simplified model and gradually refines the details to achieve high-precision image reconstruction. It is effective for both human and clothing images, enabling detailed multi-view image reconstruction from a single image. Balakrishnan et al. [7] introduce a pose transfer scheme that decomposes the task of generating human images into separate foreground and background generation using GANs. This method first segments and generates the foreground and background independently and finally merges them to achieve high-quality image reconstruction. Ma et al. [9] propose a two-stage approach for pose transfer, generating human images that match the target pose from low resolution to high resolution. The first stage extracts rough features, preserving the texture details of the clothes. The second stage processes the results of the first stage, enhancing the texture features obtained in the first stage. In their subsequent work, Ma et al. [10] improve their previous work by decoupling and encoding the foreground, background, and pose of the input image into embedded features, then decoding these features back into an image. Siarohin et al. [21] use a U-Net-based generator in their research on deformable GANs, with deformable skip connections to alleviate pixel-to-pixel misalignment issues caused by pose differences. Esser et al. [22] utilize a variational autoencoder combined with a conditional U-Net to model inherent shapes and appearances. Zhu et al. [23] introduce multiple connected pose attention transfer blocks into the generator in their study, using attention mechanisms to gradually guide the deformable pose transfer process. In this method, pose transfer is handled in small local regions at each intermediate stage, helping to overcome various difficulties posed by the complex structures on the global manifold. Similarly, the work in [24] addresses the pose transfer task using an attention mechanism. They encode the source image and human pose separately at different resolutions, then combine them with an attention mechanism before decoding the image to obtain the pose-transferred image.

Recently, diffusion-based methods have gained attention in the field of pose transfer for their ability to generate high-fidelity images. For example, Lu et al. [24] propose a coarse-to-fine latent diffusion framework that integrates multi-scale attention to preserve appearance and pose consistency. Similarly, Shen et al. [25] introduce a progressive conditional diffusion model to gradually refine structure and detail throughout the generation process. Although these models achieve impressive results, they typically suffer from high computational and inference costs, making them less practical in resource-constrained settings.

Based on the above analysis, existing pose transfer methods generally face the following challenges:

(1)

The inability to simultaneously preserve both clothing textures and facial details.

(2)

Structural distortions or pixel misalignment under large pose variations.

(3)

Difficulty in maintaining identity consistency of the subject.

To address these issues, this paper improves the generative model architecture of [26]. We improve image detail generation by incorporating dual attention mechanisms within the residual blocks and leveraging the Mish activation function to enhance feature representation. Furthermore, we introduce two specialized facial loss functions to guide the network in accurately capturing the facial features of the original subjects.

2. Proposed Method

2.1. Network Architecture

We use Multi-scale Attention Guided Pose Transfer (MAGPT) [26] as our base model. In the experimental results of the MAGPT model, some generated images exhibit significant differences in clothing details compared to the original images, while others produce facial features that do not resemble the actual faces. This paper addresses these two issues to improve the results.

To address the significant differences in clothing details, we introduce the Convolutional Block Attention Module (CBAM) [27]. By incorporating dual attention mechanisms, we enhance the model’s ability to recognize and generate detailed clothing features. CBAM allows the model to focus more on clothing details by strengthening spatial and channel attention, helping the model to capture textures, colors, and shapes more accurately. Additionally, we propose a second method for extracting detailed textures from images. We replace the ReLU activation function with Mish [28], which enables the model to capture more features during training through nonlinear activation. The complete model architecture is shown in Figure 2.

In the proposed model, $I_A^j$ represents the source image, and $P_A^j$ is the keypoint map of the source image. The index j indicates the specific entry in the dataset. The pose $P_B^j$ is the keypoint map of the desired pose for the person in the image, and ${\widehat{I}}_B^j$ is the image generated by the model. Both $P_A^j$ and $P_B^j$ are produced using the pre-trained Human Pose Estimator (HPE) based on OpenPose [29] and are converted into heatmaps represented as $H_A^j$ and $H_B^j$, respectively. In addition, σ represents the sigmoid function, and ☉ represents element-wise multiplication.

The encoder consists of two parallel down-sampling branches. One branch specifically processes the conditional person image $I_A^j$, and the other handles the combined pose heatmaps $H_A^j$ and $H_B^j$. Initially, both branches convert the input data into a space with ${\mathrm{N}}_{\mathrm{f}}$ feature maps of dimensions ℎ × w through a two-dimensional 3 × 3 convolutional layer, followed by batch normalization and Mish activation function to obtain the mapped input features. These feature maps are then passed to N consecutive Down2xBlocks, where each block halves the input dimensions while doubling the number of feature maps.

The decoder consists of an up-sampling branch, which aims to generate the target pose image ${\widehat{I}}_B^j$. Starting from the feature space dimensions, it processes the feature maps through N consecutive 2 × up-sampling blocks. In each 2 × up-sampling block, the input size is doubled, and the number of feature maps is halved. After these 2 × up-samplings, we achieve a feature space with dimensions h × w $\times {\mathrm{N}}_{\mathrm{f}}$. Ultimately, these feature maps continuously pass through four residual blocks enhanced by CBAM, followed by a pointwise two-dimensional convolution operation that projects the feature maps into a three-channel output space. Finally, tanh is used to ensure a more uniform distribution across each channel of the model output, which helps reduce the issue of vanishing gradients during training, thus enhancing training stability and convergence speed, then to produce the generated image ${\widehat{I}}_B^j$.

The attention mechanism establishes attention connections between layers of different resolutions in the encoder and decoder. In each layer, an attention mask $M_k$, where k denotes the corresponding resolution level, is calculated using the sigmoid function to determine how much of the original image’s features should be retained in each pixel of the final generated image. This attention mask updates the image feature maps $I_k$ by element-wise multiplication. The updated feature maps, enriched by $M_k$, directly influence the final output image ${\widehat{I}}_B^j$ by contributing weighted feature information at various decoding stages. These feature maps are then fed into the decoder’s up-sampling module, where they are further processed to reconstruct the image. As these operations are repeated across all resolution levels, starting from the lowest level and connecting upwards progressively, the model builds a series of attention connections. This allows the model to focus on important features more effectively during the decoding process and more accurately reconstruct the target pose image.

To address the issue of generated facial features not resembling the actual faces, we introduced the head region loss to help the generated model learn the facial features. The head region loss utilizes the dataset segmented from DeepFashion from ADGAN [30]. However, since this method is only applicable to the DeepFashion dataset, we proposed another approach, Face Focused Loss, which uses OpenPose [29] to segment the generated image’s face. This demonstrates that extracting facial features can enhance the quality of generated facial images.

We adopted the same discriminator model as MAGPT, which is derived from the PatchGAN Discriminator [13]. In the implementation of the architecture, the discriminator evaluates the transformation of images, not just the generated images. By combining the conditional image $I_j^A$ with either the target image $I_j^B$ or the generated image ${\widehat{I}}_j^B$ in depth, and labeling ($I_j^A,I_j^B$) as real and ($I_j^A,{\widehat{I}}_j^B$) as fake, the discriminator can make judgments based on the differences between the conditional and target or generated images. This design allows the discriminator to not only identify the authenticity of individual images but also to evaluate whether the transformation from the conditional image to the target pose is reasonable. The architecture of the discriminator model is shown in Figure 3. It takes as input the result of combining the conditional image $I_j^A$ and either the target image $I_j^B$ or the generated image ${\widehat{I}}_j^B$, and processes them through a series of convolutional layers. Following the PatchGAN design, the discriminator operates with a 70 × 70 receptive field, which enables localized evaluation. This structure allows the model to effectively determine whether an image is real or fake and also assess whether the transformation from the conditional image to the target pose is plausible.

2.2. Residual Block with CBAM

In pose transfer tasks, we aim to extract more features from the source images. To achieve this, the MAGPT architecture already employs residual blocks to enhance the model’s feature extraction capabilities and maintain feature transmission in deep networks, thereby avoiding gradient vanishing issues. The introduction of residual blocks is crucial for preserving and enhancing the performance of deep learning models, especially when dealing with complex images. However, despite the use of residual blocks in MAGPT to maintain overall network stability, defects still occur in some clothing or human features. To more effectively capture the detailed features from the source images, we incorporated the Convolutional Block Attention Module (CBAM) into the residual blocks of MAGPT, as shown in Figure 4. The structure consists of a standard residual block enhanced with two attention modules: channel attention and spatial attention. These modules are applied sequentially to refine the intermediate features, allowing the network to focus on important textures and spatial regions. This design helps improve the generation of fine details, especially in clothing and human contours.

2.3. Mish Activation Function

During the training process of pose transfer, MAGPT initially used the ReLU activation function. ReLU is widely used in various deep learning architectures due to its ability to effectively introduce non-linearity and maintain consistent gradients when processing positive inputs. However, in handling complex pose transfer tasks, ReLU may cause the loss of important human features because it outputs all negative values as zero. Based on this observation, we chose to introduce the Mish activation function to optimize the MAGPT architecture. The Mish activation curve is shown in Figure 5.

2.4. Head Region Loss

To address the issue of facial features not matching the source images, we propose a loss function specifically for the head region, called Head Region Loss. We extract feature masks representing the head region from the segmentation dataset designed specifically for the DeepFashion dataset from ADGAN. This mask corresponds to labels 2 and 4 in the dataset, which represent the hair and face regions, respectively. By performing an element-wise multiplication of the target image $I_j^B$ and the generated image ${\widehat{I}}_j^B$ with the mask, we can isolate the head region’s features.

In our implementation, we first resize the head region features to 64 × 64 using bilinear interpolation to facilitate computation within the model. We then combine this with the perceptual loss. The cropped images are processed through the intermediate layers of VGG19 to obtain the feature differences between the target image and the generated image. The head region loss $L_H^G$ is defined as:

$${\mathrm{L}}_H^{\mathrm{G}}={\displaystyle \frac{1}{h_H{w}_H{c}_H}}{\sum }_{x=1}^{w_H}{\sum }_{y=1}^{h_H}{\sum }_{c=1}^{c_H}{‖{\varphi }_{\rho }\left({C_{head,x,y,c}(\widehat{I}}_j^B)\right)-{\varphi }_{\rho }\left({C_{head,x,y,c}(I}_j^B)\right)‖}_1$$

(1)

where $C_{head}$ represents the output of the head region features using the mask. In this paper, we use the 4th and 9th layers of VGG19 to calculate the perceptual loss, denoted as ${\varphi }_{\rho }$ in Equation (1). Also, the terms $h_H$, $w_H$, and $c_H$ denote the height, width, and channel number of the head region features, respectively.

2.5. Face-Focused Loss

Since the method in Section 2.4 can only be applied to the DeepFashion dataset, we propose an adaptable method that can be used with other datasets. Our alternative approach utilizes OpenPose. We use the nose, right eye, and left eye, denoted as p0, p1, and p2, respectively. Then, we flip the positions of the right and left eyes with the nose as the center point, denoted as $p{0}^{\prime }$ and $p{1}^{\prime }$. This forms a square to segment the face portion. The face-focused loss $L_F^G$ can be expressed as:

$${\mathrm{L}}_F^{\mathrm{G}}={\displaystyle \frac{1}{h_F{w}_F{c}_F}}{\sum }_{x=1}^{h_F}{\sum }_{y=1}^{w_F}{\sum }_{c=1}^{c_F}{‖{\varphi }_{\rho }\left({C_{face,x,y,c}(\widehat{I}}_j^B)\right)-{\varphi }_{\rho }\left({C_{face,x,y,c}(I}_j^B)\right)‖}_1$$

(2)

where $C_{face}$ represents the output of the facial features extracted using the keypoints. In Equation (2), the terms $h_F$, $w_F$, and $c_F$ denote the height, width, and channel number of the face region features, respectively.

2.6. Total Loss

To train the network, we define the total generator loss ${\mathrm{L}}_{\mathrm{G}}$ as a weighted combination of multiple components:

$${\mathrm{L}}_{\mathrm{G}}={\lambda }_1{\mathrm{L}}_1+{\lambda }_2{\mathrm{L}}_{\mathrm{B}\mathrm{C}\mathrm{E}}(D\left(I_j^A, {\widehat{I}}_j^B\right), 1)+{\lambda }_3{\mathrm{L}}_{\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{c}\mathrm{e}\mathrm{p}\mathrm{t}\mathrm{u}\mathrm{a}\mathrm{l}}+{\lambda }_4{\mathrm{L}}_{\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{o}\mathrm{n}},$$

(3)

where ${\mathrm{L}}_{\mathrm{B}\mathrm{C}\mathrm{E}}$ denotes the binary cross-entropy loss, computed using a PatchGAN discriminator. ${\mathrm{L}}_{\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{c}\mathrm{e}\mathrm{p}\mathrm{t}\mathrm{u}\mathrm{a}\mathrm{l}}$ is the perceptual loss computed from the 4th and 9th layers of VGG19, and ${\mathrm{L}}_{\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{o}\mathrm{n}}$ denotes either the head region loss or the face-focused loss, depending on the loss variant used. In our experiments, the weights ${\lambda }_1$ to ${\lambda }_4$ are set to 5, 1, 5 and 3, respectively.

The discriminator loss ${\mathrm{L}}_{\mathrm{D}}$ is defined as:

$${\mathrm{L}}_{\mathrm{D}}={\mathrm{L}}_{\mathrm{B}\mathrm{C}\mathrm{E}}\left(D\left(I_j^A, I_j^B\right), 1\right)+{\mathrm{L}}_{\mathrm{B}\mathrm{C}\mathrm{E}}(D\left(I_j^A, {\widehat{I}}_j^B\right), 0),$$

(4)

where $I_j^A$ and $I_j^B$ represent the source and target images, respectively, and ${\widehat{I}}_j^B$ is the image generated by the network.

3. Experiments

To evaluate our proposed architecture, we conducted comprehensive qualitative and quantitative analyses by comparing it with several established pose transfer techniques [9,22,23,26,31] using a consistent dataset [32]. Our approach surpasses previous methods in most evaluation metrics and also outperforms MAGPT.

3.1. Dataset

This paper utilizes the DeepFashion dataset [32] to perform experiments. We employ the In-shop Clothes Retrieval Benchmark subset from this dataset, which consists of 7982 items. Each item is displayed through real-person fitting photos, including front, side, back, and full-body shots. This subset exhibits a richer variety of human poses and clothing changes compared to other subsets. The images in this dataset have a resolution of 176 × 256 pixels. For standardization, we use bilinear interpolation to adjust these images to a resolution of 256 × 256 pixels, ensuring that the adjusted images are centered within a 256 × 256 square grid. For a fair comparison, we adopt the same training and testing dataset split as [26], where each data point represents a person before and after pose transformation. Additionally, we observed a scarcity of male data in DeepFashion, amounting to only 4149 entries. Hence, we match them into pairs and randomly select 20,000 data points from these to add to the original training dataset, bringing the total to 121,966 entries in the training set and 8570 in the testing set. We also ensure that there is no overlap between training and testing datasets.

3.2. Evaluation Metrics

We utilize evaluation metrics including Structural Similarity Index (SSIM) [33], Inception Score (IS) [34], PCKh [35], and LPIPS [36] to evaluate the quality of pose-transferred images.

SSIM measures the similarity between a generated image and a reference image in terms of luminance, contrast, and structure. It is defined as:

$$\mathrm{SSIM}\left(x,y\right)={\displaystyle \frac{\left(2{\mathrm{\mu }}_x{\mathrm{\mu }}_y+C_1\right)\left(2{\mathrm{\sigma }}_{xy}+C_2\right)}{\left({\mathrm{\mu }}_x^2+{\mathrm{\mu }}_y^2+C_1\right)\left({\mathrm{\sigma }}_x^2+{\mathrm{\sigma }}_y^2+C_2\right)}}$$

(5)

where ${\mu }_x$, ${\mu }_y$ are the means, ${\mathrm{\sigma }}_x^2$, ${\mathrm{\sigma }}_y^2$ are the variances, and ${\mathrm{\sigma }}_{xy}$ is the covariance of the two images $x$ and $y$. The Inception Score evaluates image quality and diversity by passing generated images through a pre-trained Inception network and is defined as:

$$\mathrm{IS}=\exp \left(E_x\left[D_{\mathrm{KL}}\left(p\left(y|x\right)\parallel p\left(y\right)\right)\right]\right)$$

(6)

where $p\left(y|x\right)$ is the conditional label distribution given image $x$, and $p\left(y\right)$ is the marginal distribution. PCKh (Percentage of Correct Keypoints head) measures keypoint localization accuracy, calculated as the proportion of predicted keypoints within a normalized distance from ground truth keypoints.

$$\mathrm{PCKh}={\displaystyle \frac{\mathrm{Number} \mathrm{of} \mathrm{correct} \mathrm{keypoints} }{\mathrm{Total} \mathrm{number} \mathrm{of} \mathrm{keypoints}}}$$

(7)

LPIPS (Learned Perceptual Image Patch Similarity) evaluates the perceptual similarity between generated and real images using deep features extracted from pre-trained networks. It is computed as:

$$\mathrm{LPIPS}\left(x,y\right)={\sum }_l{\displaystyle \frac{1}{H_l{W}_l}}{\sum }_{h,w}|w_l\odot \left(\widehat{f_l^x}\left(h,w\right)-\widehat{f_l^y}\left(h,w\right)\right){|}_2^2$$

(8)

where $\widehat{f_l^x}$ and $\widehat{f_l^y}$ are normalized feature activations from layer $l$ for images $x$ and $y$, $w_l$ are learned weights, and $H_l$ and $W_l$ are the spatial dimensions. We also evaluate Fréchet Inception Distance (FID) [37], which measures the distance between feature distributions of real and generated images in the embedding space of a pre-trained Inception model. It is computed as:

$$\mathrm{FID}=|{\mu }_r-{\mu }_g{|}_2^2+\mathrm{Tr}\left({\mathrm{\Sigma }}_r+{\mathrm{\Sigma }}_g-2{\left({\mathrm{\Sigma }}_r{\mathrm{\Sigma }}_g\right)}^{1/2}\right)$$

(9)

where (${\mu }_r$, ${\mathrm{\Sigma }}_r$) and (${\mu }_g$, ${\mathrm{\Sigma }}_g$) are the means and covariances of real and generated image features, respectively.

For all these metrics, we clarify the direction of desirable performance: higher SSIM, IS, and PCKh scores indicate better perceptual quality, structure alignment, and image diversity, respectively, while lower values of FID and LPIPS (for both VGG and SqueezeNet variants) indicate smaller perceptual and distributional gaps between generated and real images. Therefore, improvements in these metrics, such as increased SSIM and reduced FID and LPIPS, directly reflect the model’s effectiveness in generating high-quality pose transfer images with realism and structural consistency.

3.3. Qualitative Comparison

This section compares the original architecture with the method proposed in this paper. We refer to the original architecture as MAGPT, and we will call our modified paper structure CMF. Since we have two types of facial loss functions, we will distinguish these as CMF with Head Region loss and CMF with Face-Focused loss.

The qualitative comparison between our model and MAGPT is shown in Figure 6. We can observe that although MAGPT can successfully perform pose transfer with clothing and facial features, it still exhibits issues with fragmented parts in clothing details and mismatched skin colors for some individuals. In particular, when dealing with clothing that contains complex textures and patterns, our model produces significantly more coherent and realistic results. Compared to MAGPT, which tends to introduce noticeable artifacts and distortions in these scenarios, our method demonstrates superior preservation of fine-grained visual details and clothing consistency. As highlighted by the red box in Figure 6, MAGPT generates distorted textures and incomplete patterns in clothing with complex designs. In contrast, our method avoids such distortions by leveraging residual blocks with CBAM and the Mish activation function, which enhance spatial attention and feature expressiveness to better preserve the original garment structure. Additionally, in the third row, MAGPT fails to preserve the original sleeve length of the clothing, erroneously generating short sleeves instead of the long sleeves present in the target image. In contrast, both variants of our method correctly retain the garment’s structure, demonstrating improved spatial awareness and contour preservation. Furthermore, the images produced by MAGPT have issues with insufficient facial features, such as unequal eye sizes and irregular gaze directions, and even instances of facial disappearance. By incorporating head region loss or face-focused loss, the proposed model also achieves more detailed feature representation, and the aforementioned issues are addressed in the performance of our qualitative experiments.

To better understand the individual contributions of CBAM and Mish, we conduct an ablation study based on the MAGPT architecture. As shown in Figure 7, we compare the original MAGPT with its variants integrating CBAM or replacing ReLU with Mish. The CBAM-enhanced version demonstrates improved structural retention and spatial attention, particularly evident in clothing contours and folds. Meanwhile, the version with Mish activation produces cleaner local textures and fewer visual distortions. These findings confirm that both modules independently improve different aspects of the pose transfer process.

3.4. Quantitative Comparison

We analyze the performance of the proposed model in quantitative experiments across various metrics, compared to different methods including MAGPT [26], ${PG}^2$ [9], Deform [31], VUNet [22], and PATN [23]. The results of the quantitative comparisons are shown in

Table 1. Quantitative comparison among different pose transfer methods.

Model	FID ↓	SSIM ↑	IS ↑	PCKh ↑	LPIPS (VGG) ↓	LPIPS (SqzNet) ↓
${PG}^2$ [9]	-	0.773	3.163	0.89	0.523	0.416
Deform [31]	-	0.760	3.362	0.94	-	-
VUNet [22]	-	0.763	3.440	0.93	-	-
PATN [23]	20.73	0.773	3.209	0.96	0.299	0.170
MAGPT [26]	10.56	0.769	3.379	0.98	0.200	0.111
Proposed with Head Region Loss	10.40	0.775	3.070	0.98	0.196	0.108
Proposed with Face Focused Loss	10.20	0.774	3.411	0.96	0.196	0.108
Real data	7.68	1.000	3.864	1.00	0.000	0.000

. The arrows in Table 1 indicate the desired direction for each metric: ↑ means higher values are better (e.g., SSIM, IS, and PCKh), while ↓ means lower values are better (e.g., FID and LPIPS).

The proposed model with head region loss and with face-focused loss both outperform the MAGPT model in terms of SSIM and LPIPS scores. This demonstrates that the modifications with the CBAM attention module and Mish activation function can result in greater detail and structural closeness to the target image. The integration of CBAM helps the model focus more on key areas of the image, thereby enhancing the authenticity and detail of the generated image structure. In terms of LPIPS performance, the facial loss functions specifically optimize the facial details of the generated images. It improves the authenticity and naturalness of facial expressions and features, which is crucial for enhancing the overall perceptual image quality. Additionally, the Mish activation function improves the model’s learning efficiency and stability across different input values due to its non-monotonicity and smoothness, directly contributing to an enhancement in LPIPS scores. For the PCKh metric, both the proposed method with head region loss and the MAGPT model reach 0.98. Since the ideal value of PCKh computed using ground truth is 1, the space for improving this metric is quite limited. The proposed model also performs better compared to MAGPT in terms of FID. The FID metric measures the statistical similarity between generated and real images, reflecting the model’s strong ability to capture overall image distribution and detail authenticity. For the IS metric, the proposed model with face-focused loss slightly outperforms MAGPT. The IS metric evaluates the diversity of generated images and their recognizability in a pre-trained classification model, focusing more on the categorical attributes of images rather than the true reproduction of details. The proposed model focuses more on enhancing the authenticity of the image details. Therefore, the improvement of the proposed model on the IS score is not as obvious as that on the FID score.

3.5. Experiment on Handling Diverse Actions

In this subsection, we apply the proposed model on data with more diverse actions to demonstrate that the proposed model is not limited to the DeepFashion dataset. From the results shown in Figure 8, we can validate that the proposed model can handle diverse action transformations. For the jumping jack action, the model captures the details of the hand movements effectively. As for the stylized and rhythmically intense hip-hop dancing, the model demonstrates its ability to capture the movement details and maintain consistency in the style of the action.

4. Conclusions

This paper introduces a pose transformation model based on a dual attention mechanism and improved head or facial loss functions. We enhance clothing details and overall image edge features using CBAM and Mish and enhance personal feature learning through head region loss and face-focused loss. Head region loss is tailored for the DeepFashion dataset to yield better results, while face-focused loss uses Openpose for facial feature extraction and is applicable to any datasets. Despite some differences, both loss functions perform well in qualitative and quantitative experiments compared to the original MAGPT structure.

Based on our findings, future research could focus on the following improvements.

• Enhancing hand detail generation, as current models lack training on images with open hands and struggle with hand and finger details.
• Improving handling of complex backgrounds, since current experiments are trained on all-white backgrounds.
• Diversifying the training dataset to include non-Western features, as the current model is predominantly trained on Western faces. Adding a variety of ethnicities could improve performance.