Energy-Efficient PPG-Based Respiratory Rate Estimation Using Spiking Neural Networks
Abstract
:1. Introduction
- We designed an end-to-end SNN architecture using a feedback-based neuronal model. To the best of our knowledge, this is the first regression study that applies end-to-end SNN to real-world PPG data.
- We employed a direct encoding method to convert real-valued PPG segments into spatial–temporal spike trains. We generated explainable spike trains for RR estimation via trainable convolution blocks with a biological neuron model.
- We compared the proposed model with other deep learning methods and demonstrated that the proposed model had an accuracy comparable to that of existing DNN models while being more energy efficient.
2. Materials and Method
2.1. Data and Preprocessing
2.2. Spiking Neuron Model
2.3. Spike Encoding
2.4. Surrogate Gradient Learning
2.5. Network Structure
2.6. Model Evaluation
3. Experimental Results
3.1. Model Accuracy
3.2. Computational Cost and Energy Consumption
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Comroe, J.H. Physiology of respiration. Acad. Med. 1965, 40, 887. [Google Scholar]
- Fieselmann, J.F.; Hendryx, M.S.; Helms, C.M.; Wakefield, D.S. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J. Gen. Intern. Med. 1993, 8, 354–360. [Google Scholar] [CrossRef] [PubMed]
- Lim, W.; Carty, S.; Macfarlane, J.; Anthony, R.; Christian, J.; Dakin, K.; Dennis, P. Respiratory rate measurement in adults—How reliable is it? Respir. Med. 2002, 96, 31–33. [Google Scholar] [CrossRef] [PubMed]
- Nam, S.; Bautista, J.L.; Hahm, C.; Shin, H. Recognition of Respiratory Instability using a Photoplethysmography of Wrist-watch typeWearable Device. IEIE Trans. Smart Process. Comput. 2002, 11, 97–104. [Google Scholar]
- Teng, X.; Zhang, Y. Continuous and noninvasive estimation of arterial blood pressure using a photoplethysmographic approach. In Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No. 03CH37439), Cancun, Mexico, 17–21 September 2003; Volume 4, pp. 3153–3156. [Google Scholar]
- Rodrigues, E.M.; Godina, R.; Cabrita, C.M.; Catalão, J.P. Experimental low cost reflective type oximeter for wearable health systems. Biomed. Signal Process. Control 2017, 31, 419–433. [Google Scholar] [CrossRef]
- Allen, J. Photoplethysmography and its application in clinical physiological measurement. Physiol. Meas. 2007, 28, R1. [Google Scholar] [CrossRef] [PubMed]
- Haddad, S.; Boukhayma, A.; Caizzone, A. Continuous PPG-based blood pressure monitoring using multi-linear regression. IEEE J. Biomed. Health Inform. 2021, 26, 2096–2105. [Google Scholar] [CrossRef] [PubMed]
- Islam, M.T.; Zabir, I.; Ahamed, S.T.; Yasar, M.T.; Shahnaz, C.; Fattah, S.A. A time-frequency domain approach of heart rate estimation from photoplethysmographic (PPG) signal. Biomed. Signal Process. Control 2017, 36, 146–154. [Google Scholar] [CrossRef]
- McCance, K.L.; Huether, S.E. Pathophysiology: The Biologic Basis for Disease in Adults and Children; Elsevier Health Sciences: Amsterdam, The Netherlands, 2014. [Google Scholar]
- Flenady, T.; Dwyer, T.; Applegarth, J. Accurate respiratory rates count: So should you! Australas. Emerg. Nurs. J. 2017, 20, 45–47. [Google Scholar] [CrossRef]
- Charlton, P.H.; Birrenkott, D.A.; Bonnici, T.; Pimentel, M.A.; Johnson, A.E.; Alastruey, J.; Tarassenko, L.; Watkinson, P.J.; Beale, R.; Clifton, D.A. Breathing rate estimation from the electrocardiogram and photoplethysmogram: A review. IEEE Rev. Biomed. Eng. 2017, 11, 2–20. [Google Scholar] [CrossRef]
- Bian, D.; Mehta, P.; Selvaraj, N. Respiratory rate estimation using PPG: A deep learning approach. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; pp. 5948–5952. [Google Scholar]
- Madhav, K.V.; Ram, M.R.; Krishna, E.H.; Komalla, N.R.; Reddy, K.A. Estimation of respiration rate from ECG, BP and PPG signals using empirical mode decomposition. In Proceedings of the 2011 IEEE International Instrumentation and Measurement Technology Conference, Hangzhou, China, 10–12 May 2011; pp. 1–4. [Google Scholar]
- Garde, A.; Karlen, W.; Dehkordi, P.; Ansermino, J.M.; Dumont, G.A. Empirical mode decomposition for respiratory and heart rate estimation from the photoplethysmogram. In Proceedings of the Computing in Cardiology 2013, Zaragoza, Spain, 22–25 September 2013; pp. 799–802. [Google Scholar]
- Lazazzera, R.; Carrault, G. Breathing rate estimation methods from PPG signals, on CAPNOBASE database. In Proceedings of the 2020 Computing in Cardiology, Rimini, Italy, 13–16 September 2020; pp. 1–4. [Google Scholar]
- Pankaj; Kumar, A.; Kumar, M.; Komaragiri, R. Optimized deep neural network models for blood pressure classification using Fourier analysis-based time–frequency spectrogram of photoplethysmography signal. Biomed. Eng. Lett. 2023, 13, 739–750. [Google Scholar] [CrossRef] [PubMed]
- Nilsson, L.M. Respiration signals from photoplethysmography. Anesth. Analg. 2013, 117, 859–865. [Google Scholar] [CrossRef] [PubMed]
- Osathitporn, P.; Sawadwuthikul, G.; Thuwajit, P.; Ueafuea, K.; Mateepithaktham, T.; Kunaseth, N.; Choksatchawathi, T.; Punyabukkana, P.; Mignot, E.; Wilaiprasitporn, T. RRWaveNet: A Compact End-to-End Multi-Scale Residual CNN for Robust PPG Respiratory Rate Estimation. IEEE Internet Things J. 2023, 10, 15943–15952. [Google Scholar] [CrossRef]
- Chowdhury, M.H.; Shuzan, M.N.I.; Chowdhury, M.E.; Reaz, M.B.I.; Mahmud, S.; Al Emadi, N.; Ayari, M.A.; Ali, S.H.M.; Bakar, A.A.A.; Rahman, S.M.; et al. Lightweight End-to-End Deep Learning Solution for Estimating the Respiration Rate from Photoplethysmogram Signal. Bioengineering 2022, 9, 558. [Google Scholar] [CrossRef] [PubMed]
- Chollet, F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1251–1258. [Google Scholar]
- Yamazaki, K.; Vo-Ho, V.K.; Bulsara, D.; Le, N. Spiking neural networks and their applications: A Review. Brain Sci. 2022, 12, 863. [Google Scholar] [CrossRef] [PubMed]
- Xing, Y.; Zhang, L.; Hou, Z.; Li, X.; Shi, Y.; Yuan, Y.; Zhang, F.; Liang, S.; Li, Z.; Yan, L. Accurate ECG classification based on spiking neural network and attentional mechanism for real-time implementation on personal portable devices. Electronics 2022, 11, 1889. [Google Scholar] [CrossRef]
- Yang, J. Accurate Prediction and Analysis of College Studentsfrom Online Learning Behavior Data. IEIE Trans. Smart Process. Comput. 2023, 12, 404–411. [Google Scholar] [CrossRef]
- Rajagopal, R.; Karthick, R.; Meenalochini, P.; Kalaichelvi, T. Deep Convolutional Spiking Neural Network optimized with Arithmetic optimization algorithm for lung disease detection using chest X-ray images. Biomed. Signal Process. Control 2023, 79, 104197. [Google Scholar] [CrossRef]
- Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural Netw. 1997, 10, 1659–1671. [Google Scholar] [CrossRef]
- Theunissen, F.; Miller, J.P. Temporal encoding in nervous systems: A rigorous definition. J. Comput. Neurosci. 1995, 2, 149–162. [Google Scholar] [CrossRef]
- Victor, J.D. Spike train metrics. Curr. Opin. Neurobiol. 2005, 15, 585–592. [Google Scholar] [CrossRef] [PubMed]
- Auge, D.; Hille, J.; Mueller, E.; Knoll, A. A survey of encoding techniques for signal processing in spiking neural networks. Neural Process. Lett. 2021, 53, 4693–4710. [Google Scholar] [CrossRef]
- Wu, J.; Chua, Y.; Zhang, M.; Li, H.; Tan, K.C. A spiking neural network framework for robust sound classification. Front. Neurosci. 2018, 12, 836. [Google Scholar] [CrossRef] [PubMed]
- Yan, Z.; Zhou, J.; Wong, W.F. Energy efficient ECG classification with spiking neural network. Biomed. Signal Process. Control 2021, 63, 102170. [Google Scholar] [CrossRef]
- Balakrishnan, P.; Baskaran, B.; Vivekanan, S.; Gokul, P. Binarized Spiking Neural Networks Optimized with Color Harmony Algorithm for Liver Cancer Classification. IEIE Trans. Smart Process. Comput. 2023, 12, 502–510. [Google Scholar] [CrossRef]
- Sengupta, A.; Ye, Y.; Wang, R.; Liu, C.; Roy, K. Going deeper in spiking neural networks: VGG and residual architectures. Front. Neurosci. 2019, 13, 95. [Google Scholar] [CrossRef] [PubMed]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Dora, S.; Subramanian, K.; Suresh, S.; Sundararajan, N. Development of a self-regulating evolving spiking neural network for classification problem. Neurocomputing 2016, 171, 1216–1229. [Google Scholar] [CrossRef]
- Pimentel, M.A.; Johnson, A.E.; Charlton, P.H.; Birrenkott, D.; Watkinson, P.J.; Tarassenko, L.; Clifton, D.A. Toward a robust estimation of respiratory rate from pulse oximeters. IEEE Trans. Biomed. Eng. 2016, 64, 1914–1923. [Google Scholar] [CrossRef]
- Goldberger, A.L.; Amaral, L.A.; Glass, L.; Hausdorff, J.M.; Ivanov, P.C.; Mark, R.G.; Mietus, J.E.; Moody, G.B.; Peng, C.K.; Stanley, H.E. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation 2000, 101, e215–e220. [Google Scholar] [CrossRef]
- Lee, J.; Scott, D.J.; Villarroel, M.; Clifford, G.D.; Saeed, M.; Mark, R.G. Open-access MIMIC-II database for intensive care research. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2011; pp. 8315–8318. [Google Scholar]
- Xiang, S.; Jiang, S.; Liu, X.; Zhang, T.; Yu, L. Spiking vgg7: Deep convolutional spiking neural network with direct training for object recognition. Electronics 2022, 11, 2097. [Google Scholar] [CrossRef]
- Hodgkin, A.L.; Huxley, A.F. A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 1952, 117, 500. [Google Scholar] [CrossRef] [PubMed]
- Izhikevich, E.M. Simple model of spiking neurons. IEEE Trans. Neural Netw. 2003, 14, 1569–1572. [Google Scholar] [CrossRef] [PubMed]
- Wu, Y.; Deng, L.; Li, G.; Zhu, J.; Xie, Y.; Shi, L. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; Volume 33, pp. 1311–1318. [Google Scholar]
- Datta, G.; Beerel, P.A. Can deep neural networks be converted to ultra low-latency spiking neural networks? In Proceedings of the 2022 Design, Automation & Test in Europe Conference & Exhibition, Antwerp, Belgium, 14–23 March 2022; pp. 718–723. [Google Scholar]
- Horowitz, M. 1.1 computing’s energy problem (and what we can do about it). In Proceedings of the 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), San Francisco, CA, USA, 9–13 February 2014; pp. 10–14. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed]
- Medsker, L.R.; Jain, L. Recurrent neural networks. Des. Appl. 2001, 5, 2. [Google Scholar]
- Fang, W.; Yu, Z.; Chen, Y.; Huang, T.; Masquelier, T.; Tian, Y. Deep residual learning in spiking neural networks. Adv. Neural Inf. Process. Syst. 2021, 34, 21056–21069. [Google Scholar]
- Duan, C.; Ding, J.; Chen, S.; Yu, Z.; Huang, T. Temporal effective batch normalization in spiking neural networks. Adv. Neural Inf. Process. Syst. 2022, 35, 34377–34390. [Google Scholar]
- Kim, Y.; Li, Y.; Park, H.; Venkatesha, Y.; Panda, P. Neural architecture search for spiking neural networks. In Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel, 23–27 October 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 36–56. [Google Scholar]
- Rathi, N.; Roy, K. Diet-snn: Direct input encoding with leakage and threshold optimization in deep spiking neural networks. arXiv 2020, arXiv:2008.03658. [Google Scholar]
Model Components | Proposed Model | CNN-LSTM | CNN-RNN | VGG-8 | Osathitporn et al. [19] |
---|---|---|---|---|---|
CNN Layers | 3 | 3 | 3 | 5 | 5 |
LSTM Layers | - | 1 | - | - | - |
RNN Layers | - | - | 1 | - | - |
Dense Layers | 2 | 1 | 1 | 3 | 3 |
CNN Filter Size | 20/8/8 | 10/5/5 | 10/5/5 | 3/3/3/3/3 | 16/32/64/3/3 |
Activation Functions | IF/RLIF | Leaky-ReLU | Leaky-ReLU | Leaky-ReLU | Leaky-ReLU |
Time Steps (T) | Window Size (s) = 16 | Window Size (s) = 32 | Window Size (s) = 64 | |||
---|---|---|---|---|---|---|
PCC | MAE (bpm) | PCC | MAE (bpm) | PCC | MAE (bpm) | |
T = 4 | 0.4980 ± 0.0287 | 1.5247 ± 0.0332 | 0.6074 ± 0.0064 | 1.3642 ± 0.0129 | 0.6219 ± 0.0111 | 1.3153 ± 0.0134 |
T = 8 | 0.5695 ± 0.0319 | 1.3710 ± 0.0481 | 0.6360 ± 0.0070 | 1.2234 ± 0.0372 | 0.6615 ± 0.0473 | 1.1518 ± 0.0697 |
T = 16 | 0.5383 ± 0.0343 | 1.4671 ± 0.0157 | 0.5447 ± 0.0117 | 1.3646 ± 0.0352 | 0.6630 ± 0.0199 | 1.2204 ± 0.0493 |
Model | Window Size (s) = 16 | Window Size (s) = 32 | Window Size (s) = 64 | |||
---|---|---|---|---|---|---|
PCC | MAE (bpm) | PCC | MAE (bpm) | PCC | MAE (bpm) | |
CNN-LSTM | 0.5926 ± 0.0355 | 1.3681 ± 0.0685 | 0.6864 ± 0.0342 | 1.1169 ± 0.0705 | 0.7077 ± 0.0245 | 1.1116 ± 0.0343 |
CNN-RNN | 0.5233 ± 0.0113 | 1.4757 ± 0.0348 | 0.6980 ± 0.0270 | 1.1605 ± 0.0675 | 0.7489 ± 0.0298 | 1.1537 ± 0.0448 |
VGG-8 | 0.4577 ± 0.0329 | 1.4721 ± 0.1795 | 0.5305 ± 0.0312 | 1.4434 ± 0.0705 | 0.5007 ± 0.1199 | 1.4053 ± 0.0964 |
Osathitporn et al. [19] | 0.5945 ± 0.0142 | 1.3460 ± 0.0128 | 0.6705 ± 0.0045 | 1.1121 ± 0.0108 | 0.6643 ± 0.0103 | 1.1321 ± 0.0150 |
Proposed model | 0.5695 ± 0.0319 | 1.3710 ± 0.0481 | 0.6360 ± 0.0070 | 1.2240 ± 0.0372 | 0.6615 ± 0.0473 | 1.1518 ± 0.0697 |
Model | Window Size (s) = 16 | Window Size (s) = 32 | Window Size (s) = 64 | |||
---|---|---|---|---|---|---|
FLOPs (M) | Energy Cost (μJ) | FLOPs (M) | Energy Cost (μJ) | FLOPs (M) | Energy Cost (μJ) | |
CNN-LSTM | 16.44 | 0.5260 | 44.38 | 1.4200 | 84.75 | 2.7120 |
CNN-RNN | 31.99 | 1.0230 | 35.04 | 1.1210 | 69.26 | 2.2160 |
VGG-8 | 16.01 | 0.5523 | 28.19 | 1.0760 | 55.99 | 2.1250 |
Osathitporn et al. [19] | 6.98 | 0.2233 | 13.44 | 0.4301 | 26.34 | 0.8428 |
Proposed model | 6.59 | 0.0120 | 12.21 | 0.0230 | 21.32 | 0.0420 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yang, G.; Kang, Y.; Charlton, P.H.; Kyriacou, P.A.; Kim, K.K.; Li, L.; Park, C. Energy-Efficient PPG-Based Respiratory Rate Estimation Using Spiking Neural Networks. Sensors 2024, 24, 3980. https://doi.org/10.3390/s24123980
Yang G, Kang Y, Charlton PH, Kyriacou PA, Kim KK, Li L, Park C. Energy-Efficient PPG-Based Respiratory Rate Estimation Using Spiking Neural Networks. Sensors. 2024; 24(12):3980. https://doi.org/10.3390/s24123980
Chicago/Turabian StyleYang, Geunbo, Youngshin Kang, Peter H. Charlton, Panayiotis A. Kyriacou, Ko Keun Kim, Ling Li, and Cheolsoo Park. 2024. "Energy-Efficient PPG-Based Respiratory Rate Estimation Using Spiking Neural Networks" Sensors 24, no. 12: 3980. https://doi.org/10.3390/s24123980
APA StyleYang, G., Kang, Y., Charlton, P. H., Kyriacou, P. A., Kim, K. K., Li, L., & Park, C. (2024). Energy-Efficient PPG-Based Respiratory Rate Estimation Using Spiking Neural Networks. Sensors, 24(12), 3980. https://doi.org/10.3390/s24123980