Decision Fusion-Based Deep Learning for Channel State Information Channel-Aware Human Action Recognition
Abstract
:1. Introduction
1.1. Contributions
1.2. Structure of This Paper
2. Preliminaries on WiFi CSI
- 1.
- How much the signal’s amplitude changes (attenuation);
- 2.
- How much the signal’s phase shifts;
- 3.
- These changes for each frequency (f) at each time point (t).
3. Related Work
4. Materials and Methods
4.1. Materials
4.2. Methods
- 1.
- Normalization of the signal. The 1D signal x must be scaled into the range [−1, 1]. This is essential because the GADF uses the arccosine function, which requires input within this range. Formally, we can write:
- 2.
- Transformation to polar coordinates. Each normalized value has to be converted into its angular representation () using the arccosine function:This maps the time-series data to angles in the range [0, ].
- 3.
- Creation of the angular difference matrix. Pairwise angular differences between all time points are computed as follows:
- 4.
- Generation of GADF. Finally, the GADF is determined as the cosine of the angular differences:This results in a 2D matrix.
5. Results
5.1. Evaluation Metrics and Protocol
5.2. Numerical Results
6. Discussion
7. Conclusions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
BLSTM | bi-directional long short-term memory |
CNN | convolutional neural network |
CSI | channel state information |
DF | decision fusion |
GAF | Gramian angular field |
GADF | Gramian angular difference field |
IEEE | Insitute of Electrical and Electronics Engineers |
LoG | Laplacian of Gaussian |
LSTM | long short-term memory |
NISE | N-iteration signal enhancement |
OFDM | orthogonal frequency division multiplexing |
PSE | P-signal enhancement |
RNN | recurrent neural network |
RSSI | received signal strength indicator |
SVM | support vector machine |
References
- Jiang, H.; Cai, C.; Ma, X.; Yang, Y.; Liu, J. Smart home based on WiFi sensing: A survey. IEEE Access 2018, 6, 13317–13325. [Google Scholar] [CrossRef]
- Khan, U.M.; Kabir, Z.; Hassan, S.A. Wireless health monitoring using passive WiFi sensing. In Proceedings of the 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC), Valencia, Spain, 26–30 June 2017; pp. 1771–1776. [Google Scholar]
- Tan, B.; Chen, Q.; Chetty, K.; Woodbridge, K.; Li, W.; Piechocki, R. Exploiting WiFi channel state information for residential healthcare informatics. IEEE Commun. Mag. 2018, 56, 130–137. [Google Scholar] [CrossRef]
- Guo, L.; Lu, Z.; Zhou, S.; Wen, X.; He, Z. When healthcare meets off-the-shelf WiFi: A non-wearable and low-costs approach for in-home monitoring. arXiv 2020, arXiv:2009.09715. [Google Scholar]
- Sruthy, S.; George, S.N. WiFi enabled home security surveillance system using Raspberry Pi and IoT module. In Proceedings of the 2017 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES), Kollam, Kerala, India, 16–18 August 2017; pp. 1–6. [Google Scholar]
- Cheng, L.; Wang, J. Walls have no ears: A non-intrusive WiFi-based user identification system for mobile devices. IEEE/ACM Trans. Netw. 2019, 27, 245–257. [Google Scholar] [CrossRef]
- Liu, J.; He, Y.; Xiao, C.; Han, J.; Ren, K. Time to think the security of WiFi-based behavior recognition systems. IEEE Trans. Dependable Secur. Comput. 2023, 21, 449–462. [Google Scholar] [CrossRef]
- Institute of Electrical and Electronics Engineers. IEEE Standard for Information Technology—Local and Metropolitan Area Networks—Specific Requirements—Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications—Amendment 5: Enhancements for Higher Throughput; IEEE Std 802.11n-2009 (Amendment to IEEE Std 802.11-2007 as amended by IEEE Std 802.11k-2008, IEEE Std 802.11r-2008, IEEE Std 802.11y-2008, and IEEE Std 802.11w-2009); IEEE: Piscataway, NJ, USA, 2009; pp. 1–565. [Google Scholar] [CrossRef]
- Strohmayer, J.; Kampel, M. Wifi csi-based long-range through-wall human activity recognition with the esp32. In Proceedings of the International Conference on Computer Vision Systems, Vienna, Austria, 27–29 September 2023; Springer: Berlin/Heidelberg, Germany, 2023; pp. 41–50. [Google Scholar]
- Mosharaf, M.; Kwak, J.B.; Choi, W. WiFi-Based Human Identification with Machine Learning: A Comprehensive Survey. Sensors 2024, 24, 6413. [Google Scholar] [CrossRef]
- Alazrai, R.; Hababeh, M.; Baha’A, A.; Ali, M.Z.; Daoud, M.I. An end-to-end deep learning framework for recognizing human-to-human interactions using Wi-Fi signals. IEEE Access 2020, 8, 197695–197710. [Google Scholar]
- Cheng, X.; Huang, B.; Zong, J. Device-free human activity recognition based on GMM-HMM using channel state information. IEEE Access 2021, 9, 76592–76601. [Google Scholar] [CrossRef]
- Ding, J.; Wang, Y. WiFi CSI-based human activity recognition using deep recurrent neural network. IEEE Access 2019, 7, 174257–174269. [Google Scholar] [CrossRef]
- Yousefi, S.; Narui, H.; Dayal, S.; Ermon, S.; Valaee, S. A survey on behavior recognition using WiFi channel state information. IEEE Commun. Mag. 2017, 55, 98–104. [Google Scholar] [CrossRef]
- Schmidhuber, J.; Hochreiter, S.; Bengio, Y.; Frasconi, P.; Ricci, F. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar]
- Yang, X.; Cao, R.; Zhou, M.; Xie, L. Temporal-frequency attention-based human activity recognition using commercial WiFi devices. IEEE Access 2020, 8, 137758–137769. [Google Scholar] [CrossRef]
- Soydaner, D. Attention mechanism in neural networks: Where it comes and where it goes. Neural Comput. Appl. 2022, 34, 13371–13385. [Google Scholar] [CrossRef]
- Schäfer, J.; Barrsiwal, B.R.; Kokhkharova, M.; Adil, H.; Liebehenschel, J. Human activity recognition using CSI information with nexmon. Appl. Sci. 2021, 11, 8860. [Google Scholar] [CrossRef]
- Kecman, V. Support vector machines—An introduction. In Support vector Machines: Theory and Applications; Springer: Berlin/Heidelberg, Germany, 2005; pp. 1–47. [Google Scholar]
- Zhou, Q.; Xing, J.; Li, J.; Yang, Q. A device-free number gesture recognition approach based on deep learning. In Proceedings of the 2016 12th International Conference on Computational Intelligence and Security (CIS), Wuxi, China, 16–19 December 2016; pp. 57–63. [Google Scholar]
- Huang, S.; Wang, D.; Zhao, R.; Zhang, Q. Wiga: A wifi-based contactless activity sequence recognition system based on deep learning. In Proceedings of the 2019 15th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN), Shenzhen, China, 11–13 December 2019; pp. 69–74. [Google Scholar]
- Sheng, B.; Xiao, F.; Sha, L.; Sun, L. Deep spatial–temporal model based cross-scene action recognition using commodity WiFi. IEEE Internet Things J. 2020, 7, 3592–3601. [Google Scholar] [CrossRef]
- Elkelany, A.; Ross, R.; Mckeever, S. WiFi-Based Human Activity Recognition Using Attention-Based BiLSTM. In Proceedings of the Irish Conference on Artificial Intelligence and Cognitive Science, Munster, Ireland, 8–9 December 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 121–133. [Google Scholar]
- Ge, F.; Dai, Z.; Yang, Z.; Wu, F.; Tan, L. VBCNet: A Hybird Network for Human Activity Recognition. Sensors 2024, 24, 7793. [Google Scholar] [CrossRef]
- Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
- Yang, J.; Liu, Y.; Liu, Z.; Wu, Y.; Li, T.; Yang, Y. A framework for human activity recognition based on WiFi CSI signal enhancement. Int. J. Antennas Propag. 2021, 2021, 6654752. [Google Scholar] [CrossRef]
- Zhou, Q.; Xing, J.; Chen, W.; Zhang, X.; Yang, Q. From signal to image: Enabling fine-grained gesture recognition with commercial Wi-Fi devices. Sensors 2018, 18, 3142. [Google Scholar] [CrossRef]
- Jiao, W.; Zhang, C. An Efficient Human Activity Recognition System Using WiFi Channel State Information. IEEE Syst. J. 2023, 7, 6687–6690. [Google Scholar] [CrossRef]
- Zhang, C.; Jiao, W. Imgfi: A high accuracy and lightweight human activity recognition framework using csi image. IEEE Sens. J. 2023, 23, 21966–21977. [Google Scholar] [CrossRef]
- Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
- Zhang, J.; Wu, F.; Wei, B.; Zhang, Q.; Huang, H.; Shah, S.W.; Cheng, J. Data augmentation and dense-LSTM for human activity recognition using WiFi signal. IEEE Internet Things J. 2020, 8, 4628–4641. [Google Scholar] [CrossRef]
- Li, T.; Shi, C.; Li, P.; Chen, P. A novel gesture recognition system based on CSI extracted from a smartphone with nexmon firmware. Sensors 2020, 21, 222. [Google Scholar] [CrossRef] [PubMed]
- Zou, H.; Zhou, Y.; Yang, J.; Gu, W.; Xie, L.; Spanos, C. Wifi-based human identification via convex tensor shapelet learning. In Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018; Volume 32. [Google Scholar]
- Li, H.; He, X.; Chen, X.; Fang, Y.; Fang, Q. Wi-motion: A robust human activity recognition using WiFi signals. IEEE Access 2019, 7, 153287–153299. [Google Scholar] [CrossRef]
- Guo, L.; Wang, L.; Liu, J.; Zhou, W.; Lu, B. HuAc: Human activity recognition using crowdsourced WiFi signals and skeleton data. Wirel. Commun. Mob. Comput. 2018, 2018, 1–15. [Google Scholar] [CrossRef]
- Chen, D.; Chen, J.; Tong, Y.; Zheng, S. Skeleton-Based Human Activity Recognition of Low-voltage Distribution Room Based on MQTT Protocol and Wireless Sensor Network. Proc. J. Phys. Conf. Ser. 2023, 2488, 012032. [Google Scholar] [CrossRef]
- Yang, J.; Chen, X.; Wang, D.; Zou, H.; Lu, C.X.; Sun, S.; Xie, L. Deep learning and its applications to WiFi human sensing: A benchmark and a tutorial. arXiv 2022, arXiv:2207.07859. [Google Scholar]
- Zhang, W.; Wang, Z.; Wu, X. WiFi signal-based gesture recognition using federated parameter-matched aggregation. Sensors 2022, 22, 2349. [Google Scholar] [CrossRef]
- Boudlal, H.; Serrhini, M.; Tahiri, A. Exploiting channel state information of WiFi signal for human activity detection: An experimental study. Bull. Electr. Eng. Inform. 2024, 13, 1031–1041. [Google Scholar] [CrossRef]
- Moshiri, P.F.; Shahbazian, R.; Nabati, M.; Ghorashi, S.A. A CSI-based human activity recognition using deep learning. Sensors 2021, 21, 7225. [Google Scholar] [CrossRef] [PubMed]
- Forbes, G.; Massie, S.; Craw, S. Wifi-based human activity recognition using Raspberry Pi. In Proceedings of the 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI), Baltimore, MD, USA, 9–11 November 2020; pp. 722–730. [Google Scholar]
- Gringoli, F.; Schulz, M.; Link, J.; Hollick, M. Free your CSI: A channel state information extraction platform for modern Wi-Fi chipsets. In Proceedings of the 13th International Workshop on Wireless Network Testbeds, Experimental Evaluation & Characterization, Los Cabos, Mexico, 25 October 2019; pp. 21–28. [Google Scholar]
- Moghaddam, M.G.; Shirehjini, A.A.N.; Shirmohammadi, S. Device-Free Fine-Grained Dining Activity Sensing. In Proceedings of the 2023 IEEE Sensors Applications Symposium (SAS), Ottawa, ON, Canada, 18–20 July 2023; pp. 1–6. [Google Scholar]
- Babiuch, M.; Foltỳnek, P.; Smutnỳ, P. Using the ESP32 microcontroller for data processing. In Proceedings of the 2019 20th International Carpathian Control Conference (ICCC), Kraków, Poland, 26–29 May 2019; pp. 1–6. [Google Scholar]
- Horn, R.A.; Johnson, C.R. Matrix Analysis; Cambridge University Press: Cambridge, UK, 2012. [Google Scholar]
- Xu, Z.; Wang, Y.; Feng, X.; Wang, Y.; Li, Y.; Lin, H. Quantum-enhanced forecasting: Leveraging quantum gramian angular field and CNNs for stock return predictions. Financ. Res. Lett. 2024, 67, 105840. [Google Scholar] [CrossRef]
- Wang, Z.; Oates, T. Imaging time-series to improve classification and imputation. arXiv 2015, arXiv:1506.00327. [Google Scholar]
- Xu, Z.; Wang, Y.; Feng, X.; Wang, Y.; Li, Y.; Lin, H. Quantum-Enhanced Forecasting: Leveraging Quantum Gramian Angular Field and CNNs for Stock Return Predictions. arXiv 2023, arXiv:2310.07427. [Google Scholar]
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L.C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 4510–4520. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Reaño, C.; Silla, F. Performance evaluation of the NVIDIA pascal GPU architecture: Early experiences. In Proceedings of the 2016 IEEE 18th International Conference on High Performance Computing and Communications, IEEE 14th International Conference on Smart City, IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Sydney, NSW, Australia, 12–14 December 2016; pp. 1234–1235. [Google Scholar]
- Heydarian, M.; Doyle, T.E.; Samavi, R. MLCM: Multi-label confusion matrix. IEEE Access 2022, 10, 19083–19095. [Google Scholar] [CrossRef]
- Shahverdi, H.; Nabati, M.; Fard Moshiri, P.; Asvadi, R.; Ghorashi, S.A. Enhancing CSI-based human activity recognition by edge detection techniques. Information 2023, 14, 404. [Google Scholar] [CrossRef]
- Jawad, S.K.; Alaziz, M. Human Activity and Gesture Recognition Based on WiFi Using Deep Convolutional Neural Networks. Iraqi J. Electr. Electron. Eng. 2022, 18, 110–116. [Google Scholar] [CrossRef]
- Varga, D. Mitigating Data Leakage in a WiFi CSI Benchmark for Human Action Recognition. Sensors 2024, 24, 8201. [Google Scholar] [CrossRef] [PubMed]
- Saupe, D.; Hahn, F.; Hosu, V.; Zingman, I.; Rana, M.; Li, S. Crowd workers proven useful: A comparative study of subjective video quality assessment. In Proceedings of the QoMEX 2016: 8th International Conference on Quality of Multimedia Experience, Lisbon, Portugal, 6–8 June 2016. [Google Scholar]
Parameter | Value |
---|---|
Loss function | Cross-entropy |
Optimizer | Adam [54] |
Learning rate | 0.0003 |
Decay rate | 0.8 |
Batch size | 32 |
Epochs | 10 |
Computer model | STRIX Z270H Gaming |
Operating system | Windows |
CPU | Intel(R) Core(TM) i7-7700K CPU 4.20 GHz (8 cores) |
Memory | 15 GB |
GPU | Nvidia GeForce GTX 1080 |
Method | Accuracy |
---|---|
2D-CNN [40] | 66.4% |
1D-CNN [40] | 55.0% |
LSTM [40] | 61.8% |
BLSTM [40] | 62.2% |
CNN-Plain [57] | 58.6% |
CNN-Canny [57] | 61.4% |
CNN-Sobel [57] | 60.4% |
CNN-Prewitt [57] | 59.7% |
CNN-LoG [57] | 61.0% |
Jawad et al. [58] | 77.1% |
CNN [28] | 77.5% |
ImgFi [29] | 79.1% |
DF-CNN (proposed) | 90.7% |
Method | Accuracy |
---|---|
2D-CNN [40] | 26.67% |
1D-CNN [40] | 23.33% |
LSTM [40] | 23.33% |
BLSTM [40] | 23.33% |
CNN-Plain [57] | 26.67% |
CNN-Canny [57] | 28.33% |
CNN-Sobel [57] | 30.00% |
CNN-Prewitt [57] | 30.00% |
CNN-LoG [57] | 28.33% |
Jawad et al. [58] | 28.33% |
CNN [28] | 26.67% |
ImgFi [29] | 26.67% |
DF-CNN (proposed) | 33.3% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Varga, D. Decision Fusion-Based Deep Learning for Channel State Information Channel-Aware Human Action Recognition. Sensors 2025, 25, 1061. https://doi.org/10.3390/s25041061
Varga D. Decision Fusion-Based Deep Learning for Channel State Information Channel-Aware Human Action Recognition. Sensors. 2025; 25(4):1061. https://doi.org/10.3390/s25041061
Chicago/Turabian StyleVarga, Domonkos. 2025. "Decision Fusion-Based Deep Learning for Channel State Information Channel-Aware Human Action Recognition" Sensors 25, no. 4: 1061. https://doi.org/10.3390/s25041061
APA StyleVarga, D. (2025). Decision Fusion-Based Deep Learning for Channel State Information Channel-Aware Human Action Recognition. Sensors, 25(4), 1061. https://doi.org/10.3390/s25041061