Cross-Modal Supervised Human Body Pose Recognition Techniques for Through-Wall Radar
Abstract
:1. Introduction
2. Preprocessing of Target Signals behind Walls
2.1. Radar Antenna Parameter Design
2.2. Radar Array Structure Configuration
2.2.1. Antenna Array Configuration Platform
2.2.2. Switch Matrix Module Design
- (1)
- Operating frequency range: 1.9 GHz–2.9 GHz;
- (2)
- channel selection paths: 1 out of 10;
- (3)
- channel isolation: ≥45 dB.
2.3. Preprocessing and RF Front-End Design
2.3.1. Data Acquisition and Preprocessing Section
- (1)
- Sampling rate: ≥20 MHz;
- (2)
- intermediate frequency: 15 MHz;
- (3)
- A/D quantization bits: 16 bits.
- (1)
- Operating frequency range: covering 1.9 GHz–2.9 GHz;
- (2)
- transmitter output power: +20 dBm;
- (3)
- frequency step: 4 MHz/2 MHz;
- (4)
- receiver intermediate frequency: 15 MHz.
2.3.2. RF Front-End Module Section
2.4. Imaging System Design
- (1)
- Two-dimensional imaging processing:
- (2)
- Clutter suppression:
- (3)
- Height information extraction:
3. Modeling of Through-The-Wall Radar Imaging Systems
3.1. Imaging System Model
3.2. Through-Wall Radar Clutter Suppression Techniques
UWB Virtual Aperture Imaging Method
4. Methodology for Through-Wall Radar Human Body Pose Estimation
4.1. Cross-Modal Supervision Method
4.2. Human Body Pose Key Point Collection and Input
4.2.1. Spatiotemporal Graph Convolution
4.2.2. Space-Time Modeling
4.2.3. Implementation of Spatiotemporal Graph Convolutional Networks
4.3. Keypoint Association and Data Fusion
5. Experimental Design
5.1. Comparative Experiments on Clutter Suppression
5.1.1. Experimental Environment Setup
5.1.2. Comparative Experiments on Clutter Suppression
5.2. Cross-Modal Target Attitude Recognition Experiment
5.2.1. Data Creation
5.2.2. Experimental Environment
5.2.3. Experimental Results
6. Discussion
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Acknowledgments
Conflicts of Interest
References
- Papandreou, G.; Zhu, T.; Kanazawa, N.; Toshev, A.; Tompson, J.; Bregler, C.; Murphy, K. Towards Accurate Multi-person Pose Estimation in the Wild. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017. [Google Scholar] [CrossRef]
- Fang, H.S.; Xie, S.; Tai, Y.W.; Lu, C. RMPE: Regional Multi-Person Pose Estimation. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017. [Google Scholar] [CrossRef]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 386–397. [Google Scholar] [CrossRef]
- Kim, K.H.; Kim, Y.; Kim, H.K.; Nam, W.; Boo, S.; Sung, M.; Shin, D.; Yeo, D.; Wooju, R.Y.U.; Lee, M.C.; et al. Method and Device for Detecting Vehicle Occupancy Using Passenger’s Keypoint Detected through Image Analysis for Humans’ Status Recognition. U.S. Patent 10,635,917, 28 April 2020. [Google Scholar]
- Zhao, M.; Adib, F.; Katabi, D. Emotion recognition using wireless signals. Commun. ACM 2018, 61, 91–100. [Google Scholar] [CrossRef]
- Insafutdinov, E.; Pishchulin, L.; Andres, B.; Andriluka, M.; Schiele, B. DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model. arXiv 2016, arXiv:1605.03170v3. [Google Scholar]
- Pishchulin, L.; Insafutdinov, E.; Tang, S.; Andres, B.; Andriluka, M.; Gehler, P.V.; Schiele, B. DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar] [CrossRef]
- Tolentino, R.E.; Guinto, P.M.F.; Maypa, D.Y.B. Recognition of Different Emergency Situation through Body Gesture Using Microsoft Kinect Sensor. In Proceedings of the 2019 IEEE 11th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Laoag, Philippines, 29 November–1 December 2019. [Google Scholar] [CrossRef]
- Bronevich, A.G.; Rozenberg, I.N. Incoherence correction and decision making based on generalized credal sets. In Symbolic and Quantitative Approaches to Reasoning with Uncertainty, Proceedings of the 14th European Conference, ECSQARU 2017, Lugano, Switzerland, 10–14 July 2017; Proceedings 14; Springer International Publishing: Berlin/Heidelberg, Germany, 2017; pp. 271–281. [Google Scholar]
- Adib, F.; Hsu, C.Y.; Mao, H.; Katabi, D.; Durand, F. Capturing the human figure through a wall. ACM Trans. Graph. 2017, 34, 1–13. [Google Scholar] [CrossRef]
- Cardillo, E.; Li, C.; Caddemi, A. Radar-based monitoring of the worker activities by exploiting range-Doppler and micro-Doppler signatures. In Proceedings of the 2021 IEEE International Workshop on Metrology for Industry 4.0 & IoT (MetroInd4. 0&IoT), Rome, Italy, 7–9 June 2021; pp. 412–416. [Google Scholar]
- Zhou, Z.; Chen, J.; Huang, Z.; Lv, J.; Song, J.; Luo, H.; Wu, B.; Li, Y.; Diniz, P.S.R. HRLE-SARDet: A lightweight SAR target detection algorithm based on hybrid representation learning enhancement. IEEE Trans. Geosci. Remote Sens. 2023, 61, 1–22. [Google Scholar] [CrossRef]
- Chang, S.; Yuan, L.; Nie, X.; Huang, Z.; Zhou, Y.; Chen, Y.; Feng, J.; Yan, S. Towards Accurate Human Pose Estimation in Videos of Crowded Scenes. In Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA, 12–16 October 2020. [Google Scholar] [CrossRef]
- Meylan, A.; Kitazoe, M.; Tenny, N.E. Method and Apparatus for Channel Identification in a Wireless Communication System. U.S. Patent 12/356, 443.US2009190544A1, 30 July 2009. [Google Scholar]
- Feng, W.; Friedt, J.M.; Hu, Z.; Cherniak, G.; Sato, M. Wifi-Based Imaging for Gpr Applications: Fundamental Study and Experimental Results. In Proceedings of the IET International Radar Conference, Nanjing, China, 17–19 October 2018; pp. 1–5. [Google Scholar]
- Hsu, C.Y.; Liu, Y.; Kabelac, Z.; Hristov, R.; Katabi, D.; Liu, C. Extracting Gait Velocity and Stride Length from Surrounding Radio Signals. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017. [Google Scholar] [CrossRef]
- Srinivasan, S.; Huang, Z.; Kirchhoff, K. Representation learning through cross-modal conditional teacher-student training for speech emotion recognition. arXiv 2021, arXiv:2112.00158. [Google Scholar]
- Denisov, P.; Vu, N.T. Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning. arXiv 2020, arXiv:2007.01836. [Google Scholar]
- Li, J.; Zhang, X.; Li, J.; Liu, Y.; Wang, J. Building and optimization of 3D semantic map based on Lidar and camera fusion. Neurocomputing 2020, 409, 394–407. [Google Scholar] [CrossRef]
- Dhawan, R.; Moyal, K.; Choudhary, A. Optical Filter-Less Photonic FMCW Radar for Multi-Target Detection. IEEE Photonics Technol. Lett. 2023, 35, 81–84. [Google Scholar] [CrossRef]
- Tian, X.; Bai, X.; Zhou, F. Recognition of Micro-Motion Space Targets Based on Attention-Augmented Cross-Modal Feature Fusion Recognition Network. IEEE Trans. Geosci. Remote Sens. 2023, 61, 1–9. [Google Scholar] [CrossRef]
- Newell, A.; Huang, Z.; Deng, J. Associative Embedding: End-to-End Learning for Joint Detection and Grouping. In Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016. [Google Scholar] [CrossRef]
- Zhao, M.; Yue, S.; Katabi, D.; Jaakkola, T.S.; Bianchi, M.T. Learning Sleep Stages from Radio Signals: A Conditional Adversarial Architecture. In Proceedings of the International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017. [Google Scholar]
- Brooks, F., Jr.; Burbeck, C.; Durlach, N.; Ellis, M.S.; Lackner, J.; Robinett, W. Research directions in virtual environments. ACM SIGGRAPH Comput. Graph. 1992, 26, 153–177. [Google Scholar] [CrossRef]
- Cao, Z.; Simon, T.; Wei, S.E.; Sheikh, Y. Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017. [Google Scholar] [CrossRef]
- Tang, H.; Li, B.; Wang, G.; Xu, H.; Li, C.; Barr, A.; Cappellaro, P.; Li, J. Communication-efficient quantum algorithm for distributed machine learning. Phys. Rev. Lett. 2023, 130, 150602. [Google Scholar] [CrossRef]
- Xu, D.; Liu, Y.; Wang, Q. Random forest-based human pose detection system for through-the-wall radar. J. Phys. Conf. Ser. 2021, 1966, 012040. [Google Scholar] [CrossRef]
- Hu, J.W.; Yu, J.; Han, Y.C. Multi-path effect in population transfer dynamics of the photoassociation of hot Mg atoms by a femtosecond laser pulse. Chem. Phys. Lett. 2022, 792, 139405. [Google Scholar] [CrossRef]
- Song, J.; Wang, L.; Van Gool, L.; Hilliges, O. Thin-Slicing Network: A Deep Structured Model for Pose Estimation in Videos. In Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 5563–5572. [Google Scholar] [CrossRef]
Test Set | 1 | 2 | 3 | 4 | 5 | Average |
---|---|---|---|---|---|---|
Accuracy | 0.95 | 0.98 | 0.92 | 0.98 | 0.90 | 0.94 |
Test Set | 1 | 2 | 3 | 4 | 5 | Average |
---|---|---|---|---|---|---|
Accuracy | 0.93 | 0.91 | 0.92 | 0.95 | 0.91 | 0.92 |
Test Set | 1 | 2 | 3 | 4 | 5 | Average |
---|---|---|---|---|---|---|
Accuracy | 0.99 | 0.98 | 0.99 | 1.00 | 0.92 | 0.98 |
Test Set | 1 | 2 | 3 | 4 | 5 | Average |
---|---|---|---|---|---|---|
Accuracy | 0.98 | 0.76 | 0.94 | 0.74 | 0.72 | 0.83 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xu, D.; Liu, Y.; Wang, Q.; Wang, L.; Shen, Q. Cross-Modal Supervised Human Body Pose Recognition Techniques for Through-Wall Radar. Sensors 2024, 24, 2207. https://doi.org/10.3390/s24072207
Xu D, Liu Y, Wang Q, Wang L, Shen Q. Cross-Modal Supervised Human Body Pose Recognition Techniques for Through-Wall Radar. Sensors. 2024; 24(7):2207. https://doi.org/10.3390/s24072207
Chicago/Turabian StyleXu, Dongpo, Yunqing Liu, Qian Wang, Liang Wang, and Qiuping Shen. 2024. "Cross-Modal Supervised Human Body Pose Recognition Techniques for Through-Wall Radar" Sensors 24, no. 7: 2207. https://doi.org/10.3390/s24072207
APA StyleXu, D., Liu, Y., Wang, Q., Wang, L., & Shen, Q. (2024). Cross-Modal Supervised Human Body Pose Recognition Techniques for Through-Wall Radar. Sensors, 24(7), 2207. https://doi.org/10.3390/s24072207