PA-MSFormer: A Phase-Aware Multi-Scale Transformer Network for ISAR Image Enhancement
Abstract
1. Introduction
- 1.
- We propose a PA-MSA module that implements complex-domain feature modulation through learnable phase difference parameters. By embedding orthogonal phase encoding in query/key vectors, this module effectively separates spectral components of noise and weak scattering features via frequency-phase decomposition.
- 2.
- A DBGFN module is introduced to decouple feature selection from nonlinear mapping through channel-splitting strategies. Dynamic weight adjustment mechanisms distinguish noise suppression paths from weak component enhancement paths, enabling precise feature selection.
- 3.
- This MSDF module employs parallel multi-branch convolutions and max-pooling operations to establish frequency-complementary downsampling pathways. The design achieves joint optimization of dynamic noise suppression and weak scattering component preservation through band-selective sampling.
- 4.
- The PGF module implements deformable transposed convolution for progressive upsampling, combined with channel-attention gating for cross-level feature weighting. This mechanism achieves pixel-level alignment between high-frequency details and low-frequency features during decoding, overcoming the feature misalignment issues inherent in traditional skip connections.
2. Materials and Methods
2.1. Enhancement Network
2.2. Key Architectural Components
2.3. Loss Function Design
Algorithm 1 Training strategy of PA-MSformer |
|
3. Results
3.1. Experimental Configuration
3.2. Simulated Electromagnetic Data Enhancement Comparision Results
3.3. Measured Data Enhancement Comparision Results
3.4. Robustness Validation
3.5. Ablation Study
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
ISAR | inverse synthetic aperture radar |
PA-MSFormer | Phase-Aware Multi-Scale Transformer networ |
PA-MSA | Phase-Aware Multi-Head Self-Attention |
DBGFN | Dual-Branch Gated Fusion Network |
PGF | Progressive Gate Fuser |
SOTA | state-of-the-art |
LN | Layer Normalization |
Q | Queries |
K | Keys |
V | Values |
PSNR | peak signal-to-noise ratio |
SSIM | measurement coordinate system |
References
- Zhao, L.; Wang, J.; Su, J.; Luo, H. Spatial Feature-Based ISAR Image Registration for Space Targets. Remote Sens. 2024, 16, 3625. [Google Scholar] [CrossRef]
- Li, W.; Yuan, Y.; Zhang, Y.; Luo, Y. Unblurring ISAR Imaging for Maneuvering Target Based on UFGAN. Remote Sens. 2022, 14, 5270. [Google Scholar] [CrossRef]
- Liu, L.; Zhou, Z.; Li, C.; Zhou, F. An Enhanced Sequential ISAR Image Scatterer Trajectory Association Method Utilizing Modified Label Gaussian Mixture Probability Hypothesis Density Filter. Remote Sens. 2025, 17, 354. [Google Scholar] [CrossRef]
- Zhou, W.; Liu, L.; Du, R.; Wang, Z.; Shang, R.; Zhou, F. Three-Dimensional Reconstruction of Space Targets Utilizing Joint Optical-and-ISAR Co-Location Observation. Remote Sens. 2025, 17, 287. [Google Scholar] [CrossRef]
- Zhang, X.; Ye, P.; Leung, H.; Gong, K.; Xiao, G. Object fusion tracking based on visible and infrared images: A comprehensive review. Inf. Fusion 2020, 63, 166–187. [Google Scholar] [CrossRef]
- Yang, S.; Jiang, W.; Tian, B. ISAR Image Matching and 3D Reconstruction Based on Improved SIFT Method. In Proceedings of the 2019 International Conference on Electronic Engineering and Informatics (EEI), Nanjing, China, 8–10 November 2019; pp. 224–228. [Google Scholar] [CrossRef]
- Du, R.; Liu, L.; Bai, X.; Zhou, Z.; Zhou, F. Instantaneous Attitude Estimation of Spacecraft Utilizing Joint Optical-and-ISAR Observation. IEEE Trans. Geosci. Remote Sens. 2022, 60, 1–14. [Google Scholar] [CrossRef]
- Wong, A.; Orchard, J. A nonlocal-means approach to exemplar-based inpainting. In Proceedings of the 2008 15th IEEE International Conference on Image Processing, San Diego, CA, USA, 12–15 October 2008; pp. 2600–2603. [Google Scholar] [CrossRef]
- Halidou, A.; Mohamadou, Y.; Ari, A.A.A.; Zacko, E.J.G. Review of wavelet denoising algorithms. Multimed. Tools Appl. 2023, 82, 41539–41569. [Google Scholar] [CrossRef]
- Tian, C.; Zheng, M.; Zuo, W.; Zhang, B.; Zhang, Y.; Zhang, D. Multi-stage image denoising with the wavelet transform. Pattern Recognit. 2023, 134, 109050. [Google Scholar] [CrossRef]
- Paul, A.; Bhattacharya, P.; Maity, S.P. Histogram modification in adaptive bi-histogram equalization for contrast enhancement on digital images. Optik 2022, 259, 168899. [Google Scholar] [CrossRef]
- Zhang, F.; Shao, Y.; Sun, Y.; Gao, C.; Sang, N. Self-supervised low-light image enhancement via histogram equalization prior. In Proceedings of the Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Xiamen, China, 13–15 October 2023; pp. 63–75. [Google Scholar] [CrossRef]
- Sun, Z.-W.; Chen, Z.-Q.; Pei, Q.-Q.; Lai, J.-A. Low-Light Image Enhancement Network based on Gamma Correction and Multi-scale Attention Mechanism. In Proceedings of the 2023 IEEE 7th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 15–17 September 2023; Volume 7, p. 772. [Google Scholar] [CrossRef]
- Hashemi, S.; Beheshti, S. Adaptive image denoising by rigorous Bayesshrink thresholding. In Proceedings of the 2011 IEEE Statistical Signal Processing Workshop (SSP), Nice, France, 28–30 June 2011; pp. 713–716. [Google Scholar] [CrossRef]
- Jin, F.; Fieguth, P.; Winger, L.; Jernigan, E. Adaptive Wiener filtering of noisy images and image sequences. In Proceedings of the 2003 International Conference on Image Processing (Cat. No.03CH37429), Barcelona, Spain, 14–17 September 2003; Volume 3, p. III-349. [Google Scholar] [CrossRef]
- Salehi, H.; Vahidi, J.; Abdeljawad, T.; Khan, A.; Rad, S.Y.B. A SAR Image Despeckling Method Based on an Extended Adaptive Wiener Filter and Extended Guided Filter. Remote Sens. 2020, 12, 2371. [Google Scholar] [CrossRef]
- Limshuebchuey, A.; Duangsoithong, R.; Saejia, M. Comparison of Image Denoising using Traditional Filter and Deep Learning Methods. In Proceedings of the 2020 17th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Phuket, Thailand, 24–27 June 2020; pp. 193–196. [Google Scholar] [CrossRef]
- Liang, J.; Cao, J.; Sun, G.; Zhang, K.; Van Gool, L.; Timofte, R. SwinIR: Image Restoration Using Swin Transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Montreal, BC, Canada, 11–17 October 2021; pp. 1833–1844. [Google Scholar] [CrossRef]
- Pan, X.; Zhan, X.; Dai, B.; Lin, D.; Loy, C.C.; Luo, P. Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation. IEEE Trans. Pattern Anal. Mach. Intell. 2022, 44, 7474–7489. [Google Scholar] [CrossRef]
- Wang, X.; Dai, Y.; Song, S.; Jin, T.; Huang, X. Deep Learning-Based Enhanced ISAR-RID Imaging Method. Remote Sens. 2023, 15, 5166. [Google Scholar] [CrossRef]
- Qi, M.; Li, K.; Qi, X.; Hu, Y. ISAR image enhancement based on a Recursive Residual Network. In Proceedings of the 2023 3rd International Conference on Electronic Information Engineering and Computer Science (EIECS), Changchun, China, 22–24 September 2023; pp. 1391–1396. [Google Scholar] [CrossRef]
- Liu, F.; Huang, D.; Guo, X.; Feng, C. SR-DNnet: A Deep Network for Super-Resolution and De-Noising of ISAR Images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2025, 18, 6567–6583. [Google Scholar] [CrossRef]
- Liu, X.; Hu, J.; Chen, X.; Dong, C. UDC-UNet: Under-display camera image restoration via U-shape dynamic network. In Proceedings of the European Conference on Computer Vision. Springer, Tel Aviv, Israel, 23–27 October 2022; pp. 113–129. [Google Scholar] [CrossRef]
- Jin, Z.; Iqbal, M.Z.; Bobkov, D.; Zou, W.; Li, X.; Steinbach, E. A Flexible Deep CNN Framework for Image Restoration. IEEE Trans. Multimed. 2020, 22, 1055–1068. [Google Scholar] [CrossRef]
- Wang, Z.; Cun, X.; Bao, J.; Zhou, W.; Liu, J.; Li, H. Uformer: A General U-Shaped Transformer for Image Restoration. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 17662–17672. [Google Scholar] [CrossRef]
- Zhao, H.; Gou, Y.; Li, B.; Peng, D.; Lv, J.; Peng, X. Comprehensive and Delicate: An Efficient Transformer for Image Restoration. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 14122–14132. [Google Scholar] [CrossRef]
- Mai, Y.; Zhang, S.; Jiang, W.; Zhang, C.; Liu, Y.; Li, X. ISAR Imaging of Target Exhibiting Micro-Motion With Sparse Aperture via Model-Driven Deep Network. IEEE Trans. Geosci. Remote Sens. 2022, 60, 1–12. [Google Scholar] [CrossRef]
- He, X.; Cheng, J. Revisiting L1 loss in super-resolution: A probabilistic view and beyond. arXiv 2022, arXiv:2201.10084. [Google Scholar] [CrossRef]
- Wen, Y.; Xu, P.; Li, Z.; Xu, W. An illumination-guided dual attention vision transformer for low-light image enhancement. Pattern Recognit. 2025, 158, 111033. [Google Scholar] [CrossRef]
- Cai, Y.; Bian, H.; Lin, J.; Wang, H.; Timofte, R.; Zhang, Y. Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement. In Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 4–6 October 2023; pp. 12470–12479. [Google Scholar] [CrossRef]
- Zamir, S.W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F.S.; Yang, M. Restormer: Efficient Transformer for High-Resolution Image Restoration. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–22 June 2022; pp. 5718–5729. [Google Scholar] [CrossRef]
- Zamir, S.W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F.S.; Yang, M.H.; Shao, L. Learning Enriched Features for Fast Image Restoration and Enhancement. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 1934–1948. [Google Scholar] [CrossRef]
- Mehri, A.; Ardakani, P.B.; Sappa, A.D. MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution. In Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Virtual, 5–9 January 2021; pp. 2703–2712. [Google Scholar] [CrossRef]
- Chen, L.; Lu, X.; Zhang, J.; Chu, X.; Chen, C. HINet: Half Instance Normalization Network for Image Restoration. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Virtual, 19–25 June 2021; pp. 182–192. [Google Scholar] [CrossRef]
- Chen, L.; Chu, X.; Zhang, X.; Sun, J. Simple baselines for image restoration. In Proceedings of the European conference on computer vision, Tel Aviv, Israel, 23–27 October 2022; pp. 17–33. [Google Scholar] [CrossRef]
- Paszke, A. Pytorch: An imperative style, high-performance deep learning library. arXiv 2019, arXiv:1912.01703. [Google Scholar] [CrossRef]
- Kingma, D. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar] [CrossRef]
- Loshchilov, I.; Hutter, F. SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv 2017, arXiv:1608.03983. [Google Scholar] [CrossRef]
- Wang, Z.; Bovik, A.; Sheikh, H.; Simoncelli, E. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef]
Method | Complexity | Weak Degradation | Low-SNR Degradation | ||||
---|---|---|---|---|---|---|---|
FLOPS (G) | Params (M) | Time (s) | PSNR | SSIM | PSNR | SSIM | |
NAFNet | 14.84 | 17.11 | 1.25 | 66.14 | 0.9005 | 65.49 | 0.8467 |
HINet | 146.64 | 88.67 | 1.47 | 71.62 | 0.9286 | 66.90 | 0.5396 |
MIRNet_v2 | 130.49 | 5.86 | 1.33 | 72.07 | 0.9841 | 69.47 | 0.9390 |
Restormer | 104.75 | 19.93 | 1.45 | 73.49 | 0.9878 | 69.78 | 0.9414 |
MPRNet | 533.83 | 15.74 | 1.47 | 73.72 | 0.9869 | 70.33 | 0.9688 |
Retinexformer | 17.29 | 1.77 | 1.19 | 73.75 | 0.9887 | 70.20 | 0.9546 |
IGDFormer | 22.46 | 2.30 | 1.18 | 73.94 | 0.9881 | 70.04 | 0.9653 |
PA-MSFormer | 16.06 | 1.59 | 1.13 | 74.11 | 0.9892 | 70.93 | 0.9710 |
Method | Weak Degradation | Low-SNR Degradation | ||
---|---|---|---|---|
PSNR | SSIM | PSNR | SSIM | |
NAFNet | 48.77 | 0.9926 | 48.55 | 0.3797 |
HINet | 47.11 | 0.9903 | 46.77 | 0.6453 |
MIRNet_v2 | 49.78 | 0.9920 | 49.40 | 0.9861 |
Restormer | 51.05 | 0.9955 | 50.58 | 0.8943 |
MPRNet | 49.32 | 0.9917 | 49.10 | 0.9906 |
Retinexformer | 50.27 | 0.9942 | 50.23 | 0.9559 |
IGDFormer | 50.26 | 0.9936 | 50.06 | 0.9472 |
PA-MSFormer | 51.43 | 0.9956 | 51.07 | 0.9311 |
Configuration | PA-MSA | Modules | Electromagnetic Simulation | Measured Data | ||||||
---|---|---|---|---|---|---|---|---|---|---|
Learnable | Random | None | DBGFN | MSDF | PGF | PSNR | SSIM | PSNR | SSIM | |
Ours (Full Model) | 🗸 | × | × | 🗸 | 🗸 | 🗸 | 70.93 | 0.9653 | 51.07 | 0.9311 |
PA-MSA-Random | × | 🗸 | × | 🗸 | 🗸 | 🗸 | 68.49 | 0.8191 | 50.26 | 0.7754 |
PA-MSA-None | × | × | 🗸 | 🗸 | 🗸 | 🗸 | 68.69 | 0.8267 | 49.28 | 0.8431 |
Ours w/o DBGFN | 🗸 | × | × | × | 🗸 | 🗸 | 68.32 | 0.7252 | 49.61 | 0.6815 |
Ours w/o MSDF | 🗸 | × | × | 🗸 | × | 🗸 | 70.05 | 0.9626 | 49.18 | 0.7652 |
Ours w/o PGF | 🗸 | × | × | 🗸 | 🗸 | × | 69.49 | 0.9661 | 49.28 | 0.9534 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Huang, J.; Li, X.; Liu, L.; Shi, X.; Zhou, F. PA-MSFormer: A Phase-Aware Multi-Scale Transformer Network for ISAR Image Enhancement. Remote Sens. 2025, 17, 3047. https://doi.org/10.3390/rs17173047
Huang J, Li X, Liu L, Shi X, Zhou F. PA-MSFormer: A Phase-Aware Multi-Scale Transformer Network for ISAR Image Enhancement. Remote Sensing. 2025; 17(17):3047. https://doi.org/10.3390/rs17173047
Chicago/Turabian StyleHuang, Jiale, Xiaoyong Li, Lei Liu, Xiaoran Shi, and Feng Zhou. 2025. "PA-MSFormer: A Phase-Aware Multi-Scale Transformer Network for ISAR Image Enhancement" Remote Sensing 17, no. 17: 3047. https://doi.org/10.3390/rs17173047
APA StyleHuang, J., Li, X., Liu, L., Shi, X., & Zhou, F. (2025). PA-MSFormer: A Phase-Aware Multi-Scale Transformer Network for ISAR Image Enhancement. Remote Sensing, 17(17), 3047. https://doi.org/10.3390/rs17173047