# Attention Mechanism-Based Convolutional Long Short-Term Memory Neural Networks to Electrocardiogram-Based Blood Pressure Estimation

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Blood Pressure Estimation Methods

#### 2.1. Signal Preprocessing

_{f}samples and each adjacent frame is separated by L

_{k}samples. For each frame, the Hamming window is applied to reduce leakage, and then the fast Fourier transform is applied to estimate the corresponding frequency responses. Then, the frequency responses of ${s}_{i}^{e}$ and ${s}_{i}^{p}$ are denoted as ${f}_{i}^{e}$ and ${f}_{i}^{p}$. Therefore, the corresponding frequency input sequence for ECG and PPG signals can be found and denoted as ${F}^{e}=\left\{{f}_{1}^{e},{f}_{2}^{e},\cdots ,{f}_{{L}_{s}}^{e}\right\}$ and ${F}^{p}=\left\{{f}_{1}^{p},{f}_{2}^{p},\cdots ,{f}_{{L}_{s}}^{p}\right\}$. Finally, the inputs of convolutional neural networks are ${X}^{s}={S}^{e}\oplus {S}^{P}$ and ${X}^{f}={F}^{e}\oplus {F}^{P}$ for input vectors in the time domain and frequency domain, respectively.

#### 2.2. Convolutional Neural Networks

_{n}, w

_{n}, and b are the outputs of the previous layer, the weight of the convolutional kernel, and the bias value, respectively. f(•) is the rectified linear unit, which is a non-linear activation function. The rectified linear unit is used to activate only certain neurons. Thus, the neurons can be activated if the output of the linear transform is greater than or equal to 0.

#### 2.3. Attention Neural Networks

_{i}. The a

_{i}is multiplied by three weights matrix w

^{q}, w

^{k}, and w

^{v}that are trained in the training process, and then the query vector Q

_{i}, the key vector K

_{i}, and the value vector V

_{i}can be obtained as follows.

_{i}, K

_{i}, and V

_{i}are obtained, the self-attention operation Attention(•), which is modeled as dot-production attention, is used to find the weighted self-attention outputs SA

_{i}. Attention(•) is defined as

#### 2.4. LSTM Neural Networks

_{i}and the outputs of previous LSTM units, respectively. In time slot t, the input and the output of the LSTM unit are y

_{t}and h

_{t}, respectively. For an LSTM unit, it composes an input gate g

^{i}, a forget gate g

^{f}, an output gate g

^{o}, a mapping function m

_{t}, and a memory cell c

_{t}.

^{i}, W

^{f}, W

^{o}, and W

^{m}are the weighted matrices, and b

^{i}, b

^{f}, b

^{o}, and b

^{m}are bias vectors of LSTM. These parameters are learned during training. Three gates have their weights and then each LSTM unit works like a state machine. Therefore, the LSTM neural network can deal with sequence problems.

#### 2.5. Fully Connected Neural Networks

## 3. Experimental Results

_{f}and L

_{k}are 512 and 256, respectively. Moreover, the length of the hamming window and the size of FFT are 512. Therefore, the size of inputs to neural networks for the time and frequency domain are 512 and 256, respectively. N-fold cross-validation (N = 10) was used to evaluate the proposed approaches by subject cross-validation and the results are detailed in the following subsections.

#### 3.1. Experimental Setup

#### 3.2. Results of Feature Evaluation and Selection

#### 3.3. Analysis of Network Structure

#### 3.4. Experimental Results Compared with AAMI Standard

## 4. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Acknowledgments

## Conflicts of Interest

## References

- Coelho, G.L.L.M. Worldwide trends in blood pressure from 1975 to 2015: A pooled analysis of 1479 population-based measurement studies with 19 1 million participants. Lancet
**2017**, 389, 37–55. [Google Scholar] - Miao, F.; Liu, Z.-D.; Liu, J.-K.; Wen, B.; He, Q.-Y.; Li, Y. Multi-Sensor fusion approach for cuff-less blood pressure measurement. IEEE J. Biomed. Health Inf.
**2020**, 24, 79–91. [Google Scholar] [CrossRef] [PubMed] - Sagirova, Z.; Kuznetsova, N.; Gogiberidze, N.; Gognieva, D.; Suvorov, A.; Chomakhidze, P.; Omboni, S.; Saner, H.; Kopylov, P. Cuffless blood pressure measurement using a smartphone-case based ECG monitor with photoplethysmography in hypertensive patients. Sensors
**2021**, 21, 3525. [Google Scholar] [CrossRef] - Yan, W.; Peng, R.; Zhang, Y.; Ho, D. Cuffless continuous blood pressure estimation from pulse morphology of photoplethysmograms. IEEE Access
**2019**, 7, 141970–141977. [Google Scholar] [CrossRef] - Liu, Z.-D.; Liu, J.-K.; Wen, B.; He, Q.-Y.; Li, Y.; Miao, F. Cuffless blood pressure estimation using pressure pulse wave signals. Sensors
**2018**, 18, 4227. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Ding, X.; Yan, B.P.; Zhang, Y.-T.; Liu, J.; Zhao, N.; Tsang, H.K. Pulse transit time based continuous cuffless blood pressure estimation: A new extension and a comprehensive evaluation. Sci. Rep.
**2017**, 7, 11554. [Google Scholar] [CrossRef] [PubMed] - Khalid, S.G.; Zhang, J.; Chen, F.; Zheng, D. Blood pressure estimation using photoplethysmography only: Comparison between different machine learning approaches. J. Healthc. Eng.
**2018**, 2018, 1548647. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Wang, L.; Zhou, W.; Xing, Y.; Zhou, X. A novel neural network model for blood pressure estimation using photoplethesmography without electrocardiogram. J. Healthc. Eng.
**2018**, 2018, 7804243. [Google Scholar] [CrossRef] [PubMed] - Chen, X.; Yu, S.; Zhang, Y.; Chu, F.; Sun, B. Machine learning method for continuous noninvasive blood pressure detection based on random forest. IEEE Access
**2021**, 9, 34112–34118. [Google Scholar] [CrossRef] - Wu, T.H.; Pang, G.K.-H.; Kwong, E.W.-Y. Predicting systolic blood pressure using machine learning. In Proceedings of the 7th International Conference on Information Automation for Sustainability, Colombo, Sri Lanka, 22–24 December 2014; pp. 1–6. [Google Scholar]
- Wu, D. Research on Continuous Non-Invasive Blood Pressure Detection and Its Application Based on Deep Neural Network. Ph.D. Thesis, Department of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China, 2017. [Google Scholar]
- Baek, S.; Jang, J.; Yoon, S. End-to-end blood pressure prediction via fully convolutional networks. IEEE Access
**2019**, 7, 185458–185468. [Google Scholar] [CrossRef] - Argha, A.; Celler, B.G. Blood pressure estimation from time-domain features of oscillometric waveforms using long short-term memory recurrent neural networks. IEEE Trans. Instrum. Meas.
**2020**, 69, 3614–3622. [Google Scholar] [CrossRef] - Miao, F.; Wen, B.; Hu, Z.; Fortino, G.; Li, Y. Continuous blood pressure measurement from one-channel electrocardiogram signal using deep-learning techniques. Artif. Intell. Med.
**2020**, 108, 101919. [Google Scholar] [CrossRef] [PubMed] - Mou, H.; Yu, J. CNN-LSTM prediction method for blood pressure based on pulse wave. Electronics
**2021**, 10, 1664. [Google Scholar] [CrossRef] - Tahir, G.A.; Loo, C.K. Progressive kernel extreme learning machine for food image analysis via optimal features from quality resilient CNN. Appl. Sci.
**2021**, 11, 9562. [Google Scholar] [CrossRef] - Guo, J.; Liu, H.; Li, X.; Xu, D.; Zhang, Y. An attention enhanced spatial–temporal graph convolutional LSTM network for action recognition in Karate. Appl. Sci.
**2021**, 11, 8641. [Google Scholar] [CrossRef] - Lázaro, J.; Gil, E.; Bailón, R.; Mincholé, A.; Laguna, P. Deriving respiration from photoplethysmographic pulse width. Med. Biol. Eng. Comput.
**2013**, 51, 233–242. [Google Scholar] [CrossRef] [PubMed] - Nemati, E.; Deen, M.J.; Mondal, T. A wireless wearable ECG sensor for long-term applications. IEEE Commun. Mag.
**2012**, 50, 36–43. [Google Scholar] [CrossRef] - Chu, Y.; Yue, X.; Yu, L.; Sergei, M.; Wang, Z. Automatic image captioning based on ResNet50 and LSTM with soft attention. Wirel. Commun. Mob. Comput.
**2020**, 2020, 1–7. [Google Scholar] [CrossRef] - Moody, G.B.; Mark, R.G. A database to support development and evaluation of intelligent intensive care monitoring. Comput. Cardiol.
**1996**, 1996, 657–660. [Google Scholar] - Noninvasive Sphygmomanometers—Part 2: Clinical Investigation of Automated Measurement Type ANSI/AAMI, Standard ISO 81060-2:2018(E)/ANSI-AAMI; Association for the Advancement of Medical Instrumentation: Arlington, VA, USA, 2018.
- Song, K.; Chung, K.; Chang, J. Cuffless Deep Learning-Based Blood Pressure Estimation for Smart Wristwatches. IEEE Trans. Instrum. Meas.
**2020**, 69, 4292–4302. [Google Scholar] [CrossRef]

Type of Neurons | Parameters | Selected Value |
---|---|---|

Convolutional layer 1 and 2 | Number of kernel size | 64 |

Number of filter size | 10 | |

Number of pooling size | 2 | |

Convolutional layer 3 | Number of kernel size | 32 |

Number of filter size | 10 | |

Number of pooling size | 2 | |

LSTM layer 1 and 2 | Number of hidden units | 64 |

Fully connected layer 1,2, and 3 | Number of hidden units | 64, 32, 16 |

RMSE | MAPE | |||
---|---|---|---|---|

SBP | DBP | SBP | DBP | |

I(ECG) | 4.87 | 2.62 | 2.51 | 1.40 |

I(PPG) | 4.39 | 2.68 | 2.28 | 1.44 |

I(ECG, PPG) | 3.90 | 2.31 | 2.04 | 1.25 |

RMSE | MAPE | |||
---|---|---|---|---|

SBP | DBP | SBP | DBP | |

Time | 3.96 | 2.48 | 2.07 | 1.25 |

Frequency | 4.69 | 2.57 | 2.43 | 1.37 |

Time + Frequency | 3.90 | 2.31 | 2.04 | 1.25 |

RMSE | MAPE | |||
---|---|---|---|---|

SBP | DBP | SBP | DBP | |

CNN | 5.54 | 4.32 | 2.85 | 2.24 |

CNN + LSTM | 4.39 | 3.02 | 2.28 | 1.60 |

CNN + AN | 4.60 | 3.32 | 2.38 | 1.75 |

Proposed approach | 3.90 | 2.31 | 2.04 | 1.25 |

SBP | DBP | |
---|---|---|

CNN | 3.88 ± 4.55 | 3.18 ± 3.94 |

CNN + LSTM | 3.22 ± 4.69 | 2.42 ± 3.48 |

CNN + AN | 3.34 ± 4.91 | 2.60 ± 3.64 |

Proposed approach | 2.94 ± 4.65 | 2.02 ± 3.81 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chuang, C.-C.; Lee, C.-C.; Yeng, C.-H.; So, E.-C.; Chen, Y.-J.
Attention Mechanism-Based Convolutional Long Short-Term Memory Neural Networks to Electrocardiogram-Based Blood Pressure Estimation. *Appl. Sci.* **2021**, *11*, 12019.
https://doi.org/10.3390/app112412019

**AMA Style**

Chuang C-C, Lee C-C, Yeng C-H, So E-C, Chen Y-J.
Attention Mechanism-Based Convolutional Long Short-Term Memory Neural Networks to Electrocardiogram-Based Blood Pressure Estimation. *Applied Sciences*. 2021; 11(24):12019.
https://doi.org/10.3390/app112412019

**Chicago/Turabian Style**

Chuang, Chia-Chun, Chien-Ching Lee, Chia-Hong Yeng, Edmund-Cheung So, and Yeou-Jiunn Chen.
2021. "Attention Mechanism-Based Convolutional Long Short-Term Memory Neural Networks to Electrocardiogram-Based Blood Pressure Estimation" *Applied Sciences* 11, no. 24: 12019.
https://doi.org/10.3390/app112412019