Intra- and Interrater Reliability of Short-Term Measurement of Heart Rate Variability on Rest in Individuals Post-COVID-19

Individuals affected by COVID-19 have an alteration in autonomic balance, associated with impaired cardiac parasympathetic modulation and, consequently, a decrease in heart rate variability (HRV). This study examines the inter- and intrarater reliability of HRV) parameters derived from short-term recordings in individuals post-COVID. Sixty-nine participants of both genders post-COVID were included. The RR interval, the time elapsed between two successive R-waves of the QRS signal on the electrocardiogram (RRi), were recorded during a 10 min period in a supine position using a portable heart rate monitor (Polar® V800 model). The data were transferred into Kubios® HRV standard analysis software and analyzed within the stable sessions containing 256 sequential RRi. The intraclass correlation coefficient (ICC) ranged from 0.920 to 1.000 according to the intrarater analysis by Researcher 01 and 0.959 to 0.999 according to the intrarater by Researcher 02. The interrater ICC ranged from 0.912 to 0.998. The coefficient of variation was up to 9.23 for Researcher 01 intrarater analysis, 6.96 for Researcher 02 intrarater analysis and 8.83 for interrater analysis. The measurement of HRV in post-COVID-19 individuals is reliable and presents a small amount of error inherent to the method, supporting its use in the clinical environment and in scientific research.


Introduction
In addition to directly infecting cells of the respiratory tract, the inflammatory response from SARS-CoV-2 infection can lead to systemic inflammation and affect almost all organ systems, including the cardiovascular system and the autonomic nervous system (ANS). Damage to these systems is associated with multiple factors and may result from an imbalance between high metabolic demand, low cardiac reserve, thromboembolism, and direct tissue damage caused by the virus [1][2][3]. Physiologically, the persistent inflammatory process in this population is mediated by the ANS through a rapid reflex action involving a central set of structures, especially the vagus nerve, which is considered the primary interface between inflammation and action in the central nervous system, in addition to being an important neuroimmunomodulator in the adaptive regulation of the inflammatory process [4,5].
Considering the close relationship between the vagus nerve and cardiac autonomic control, the heart rate variability (HRV), a simple, non-invasive, objective and validated physiological metric for the assessment of the autonomic nervous system, has been used in the most diverse scientific and clinical contexts, whether for diagnostic or prognostic purposes [4,6]. As the ANS is sensitive to physiological changes and capable of quickly generating adaptive responses, it can offer signals, in the form of an HRV, that can quickly warn of an imminent health risk [5]. Recent studies have shown that individuals affected by COVID- 19 have an alteration in autonomic balance, associated with impaired cardiac parasympathetic modulation and, consequently, a decrease in HRV, which may persist for at least six months after hospital discharge [7][8][9]. Furthermore, in those individuals who required hospitalization, a lower HRV was associated with the prediction of admission to the intensive care unit in the first week of infection, whereas those who had a higher HRV had a higher chance of survival [10].
Methodologically, the acquisition of the biological signal for the processing of HRV can be done through non-invasive tools, such as an electrocardiogram or a heart rate monitor [6]. Although the acquisition of metrics related to heart rate and HRV are obtained with greater precision using an electrocardiogram of multiple variations, the use of this type of equipment is unfeasible due to its availability in most cases only in large research centers and hospitals. There is a high level of knowledge required for data collection and processing, which leads the industry to produce and expand valid handheld devices that are easy to manage, collect and analyze [11]. Despite the large scientific collection available on the subject, the processing and analysis of the HRV signal continues to be a challenge for researchers. This is because the choice of the best stationary stretch, in addition to being manually and visually selected, depends on the subjective decisions of the evaluator. This can generate biases in the results and differences between and among researchers capable of underestimating or overestimating the values found and influencing the clinical interpretation [6,[12][13][14].
Although the measurement of HRV can easily become subject to bias in the face of inadequate collection and equivocal analysis, few studies have been directed to measure its reliability [15][16][17]. Above all, in individuals who are victims of COVID-19, where the severity of the disease can influence the levels of inflammatory cytokines and impair the autonomic and cardiovascular nervous system, the evaluation of the HRV measurement provides subsidies that can influence the decision-making process evaluation and treatment decisions. However, the technical aspects involving the collection and analysis can affect the reproducibility of this measure and generate dubious results and considerable clinical impacts [18,19].
Considering this context and the growing investigations involving HRV in individuals affected by COVID-19, this study aims to evaluate the intra-and interrater reliability of the short-term HRV measurement in post-COVID-19 individuals. The main contributions of this research are: 1. Even though the best stationary parts of the HRV are selected for analysis, they are evaluator-dependent, that is, prone to errors; 2. Researchers who analyze HRV signals must have previous experience and training, to reduce the chance of bias in the analysis; 3. In post-COVID-19 patients, excellent intra-and inter-researcher reliability values were found, supporting its clinical applicability.

Study Design
This is a reliability study conducted according to the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) [20]. The research was conducted at the Universidade Ceuma (São Luís, MA, Brazil) after the study procedures were approved by the Research Ethics Committee of the institution (protocol number: 4.179.747) and conducted according to the Declaration of Helsinki. All participants were informed of the purpose of the study and informed consent was obtained. Study design is graphically presented in Figure 1. This is a reliability study conducted according to the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) [20]. The research was conducted at the Universidade Ceuma (São Luís, MA, Brazil) after the study procedures were approved by the Research Ethics Committee of the institution (protocol number: 4.179.747) and conducted according to the Declaration of Helsinki. All participants were informed of the purpose of the study and informed consent was obtained. Study design is graphically presented in Figure 1.

Participants
Individuals of both sexes aged ≥18 years, positive for SARS-CoV-2 according to the RT-PCR test, were included. The stratification followed the following classifications according to the COVID-19 Treatment Guidelines [21] produced by the National Institute of Health: mild cases (individuals who presented signs and symptoms of the disease, such as fever, cough, sore throat, malaise, headache pain, muscle pain, nausea, vomiting, diarrhea, loss of taste and smell, but without the need for additional oxygen therapy and/or mechanical ventilation); moderate cases (individuals who showed evidence of respiratory disease or imaging with oxygen saturation ≥ 94%); severe cases (individuals with SpO2 < 94% on room air, with a ratio of arterial oxygen partial pressure to inspired oxygen fraction (PaO2/FiO2) < 300 mmHg, respiratory rate > 30 breaths/min, or lung infiltrates > 50%); critical cases (individuals who have respiratory failure, need for highflow oxygen therapy, invasive or non-invasive mechanical ventilation).
Volunteers who had myocardial infarction (within six months from the start of data collection), an implanted pacemaker or any metallic synthesis, a history of heart disease, unstable angina, uncontrolled hypertension, uncontrolled diabetes mellitus and/or insulin-dependent, chronic obstructive pulmonary disease, neoplasms, cognitive deficit, declared users of illicit drugs, and pregnant were not included in the study.
The sample size calculation was performed according to the study carried out by Fleiss. We considered the following criteria for the calculation: minimum acceptable ICC value of 0.40, expected ICC of 0.75 (moderate reliability), alpha error of 5%, power of 80%, sample loss rate of 15%. Therefore, a minimum sample size of 33 volunteers was estimated.

Habitual Physical Activity
The Baecke Habitual Physical Activity Questionnaire is a self-applicable, self-report instrument that assesses physical activity over the past 12 months. It consists of 16 items, divided into 3 domains: physical activity in occupation (items 1-8), physical activity in sports in free time (items 9-12), and physical activity during leisure other than sport (items [13][14][15][16]. There are five Likert scale (1-5) response possibilities. For each domain, the final

Participants
Individuals of both sexes aged ≥18 years, positive for SARS-CoV-2 according to the RT-PCR test, were included. The stratification followed the following classifications according to the COVID-19 Treatment Guidelines [21] produced by the National Institute of Health: mild cases (individuals who presented signs and symptoms of the disease, such as fever, cough, sore throat, malaise, headache pain, muscle pain, nausea, vomiting, diarrhea, loss of taste and smell, but without the need for additional oxygen therapy and/or mechanical ventilation); moderate cases (individuals who showed evidence of respiratory disease or imaging with oxygen saturation ≥ 94%); severe cases (individuals with SpO 2 < 94% on room air, with a ratio of arterial oxygen partial pressure to inspired oxygen fraction (PaO 2 /FiO 2 ) < 300 mmHg, respiratory rate > 30 breaths/min, or lung infiltrates > 50%); critical cases (individuals who have respiratory failure, need for high-flow oxygen therapy, invasive or non-invasive mechanical ventilation).
Volunteers who had myocardial infarction (within six months from the start of data collection), an implanted pacemaker or any metallic synthesis, a history of heart disease, unstable angina, uncontrolled hypertension, uncontrolled diabetes mellitus and/or insulindependent, chronic obstructive pulmonary disease, neoplasms, cognitive deficit, declared users of illicit drugs, and pregnant were not included in the study.
The sample size calculation was performed according to the study carried out by Fleiss. We considered the following criteria for the calculation: minimum acceptable ICC value of 0.40, expected ICC of 0.75 (moderate reliability), alpha error of 5%, power of 80%, sample loss rate of 15%. Therefore, a minimum sample size of 33 volunteers was estimated.

Habitual Physical Activity
The Baecke Habitual Physical Activity Questionnaire is a self-applicable, self-report instrument that assesses physical activity over the past 12 months. It consists of 16 items, divided into 3 domains: physical activity in occupation (items 1-8), physical activity in sports in free time (items 9-12), and physical activity during leisure other than sport (items [13][14][15][16]. There are five Likert scale (1-5) response possibilities. For each domain, the final score ranges from 1 to 5; the higher the score, the higher the level of physical activity [22,23].

Heart Rate Variability Assessment
Volunteers were positioned supine in a calm environment with controlled temperature between 22-24 • C and relative humidity between 50-60% in the morning and were encouraged to remain relaxed, breathe normally, not talk, fidget or sleep during the session. They were instructed not to ingest any stimulant substance, not to perform any strenuous exercise, and to have a good night's sleep the day before and the day of the exam. Before the collection, the volunteers remained relaxed for 10 min to stabilize the heart rate and after this period of time the collection started [6].

Heart Rate Collection and RR Intervals
Heart rate and RR intervals (iR-R) (time interval between complex peaks (QRS) which are determined by modulation of the sympathetic and parasympathetic nervous system, were obtained by a heart rate monitor (Polar V800, Polar Electro Inc., Bethpage, NY, USA), sampling frequency of 1000 Hz, recorded continuously for a period of 10 min with a chest strap and the volunteer in the supine position. Data were saved as iR-R data files, with intervals in ms. The data were exported from the Polar Flow web service as a space delimited .txt file [24].

Data Processing and Analysis
Data were transferred to Kubios ® HRV Standard analysis software (MATLAB, version 3.5, Kuopio, Finland). O software Kubios is an easy-to-use HRV analysis tool including a wide variety of time-domain, frequency-domain and nonlinear analysis options. Another important fact is that the software is available free of charge for Windows, Linux and iOS operating systems [14].
After transferred to Kubios ® the data were analyzed within stable sessions containing 256 sequential RRi. Initially, a visual inspection of the collected signals was performed. If there were 10% of ectopic beats in relation to the total pure sinus beat, the data were discarded. For the selection of the best stationary period, the following criteria were considered: (1) no large R-R interval outliers (i.e., a R-R interval much higher or lower than the whole R-R signal, based on the visual inspection of the HRV recording performed by the researcher), (2) R-R intervals equidistance and (3) Gaussian R-R intervals and heart rate distribution graphics [17].
Following the guidelines of the Task Force of the European Society of Cardiology [13], we derived variables from linear analysis (time and frequency-domain) and nonlinear analysis. For time-domain analysis, the following variables were analyzed: the mean of intervals RR in milliseconds (ms) (Mean RR), the standard deviation of all normal R-R intervals (SDNN) in ms, the mean of heart rate in beats per minute (bpm) (Mean HR), the squared root of the mean of the sum of the squares of successive normal R-R interval differences (RMSSD) in ms and the integral of the density of the RR interval histogram divided by its height (RR Tri). For frequency-domain the following variables were analyzed: low-frequency band (0.04-0.15 Hz) in normal units and milliseconds (L.F.) and high-frequency band in normal units and milliseconds (0.15-0.4 Hz) (H.F.) [25].
Finally, for non-linear analysis, the following variables were used: approximate entropy (ApEn) and sample entropy (SampEn), which measures the regularity and complexity of a time series and were proposed to quantify the entropy rate of short to medium-length NN series, detrended fluctuation analysis, which describes short-term fluctuations (DFA α1) and detrended fluctuation analysis, which describes long-term fluctuations (DFA α2) [25].

Statistical Analysis
The intra-and interrater reliability of HRV parameters was analyzed using the following statistical procedures: intraclass correlation coefficient (ICC) 2,1 [26], confidence interval (95% IC), standard error of measurement (SEM) and minimum detectable change (MDC), coefficient of variation (CV) and Bland-Altman plots (mean difference [bias] and limits of agreement). The interpretation of the ICC 2,1 value was based on the study by Fleiss [27]: for values below 0.40, reliability was considered low; between 0.40 and 0.75, moderate; between 0.75 and 0.90, substantial; and finally, for values greater than 0.90, reliability was rated as excellent.
SEM, MDC and CV values were added in order to complement the interpretation of the error of the HRV measurement method. SEM can be defined as an estimate of the expected random variation in scores when no real change has taken place and was calcu-lated using the formula: standard deviation o f means × √ (1 − ICC) [14,28]. MDC can be defined as the minimal change that falls outside the measurement error in the result of an instrument used to measure a clinical characteristic and was calculated using the formula: 1.96 × SEM × √ 2 [14,28]. The CV is the ratio of the standard deviation to the mean value and represents the extent of variability of an assay. It is expressed as a percentage of deviation from the mean; the larger the CV, the greater the error in the assay. The formula for computation of CV is straightforward: CV(%) = standard deviation mean × 100 [29]. The Bland Altman Plots is an alternative analysis, based on the quantification of the agreement between two quantitative measurements by studying the mean difference and constructing limits of agreement. The midline represents the mean systematic difference between inter and intrarater scores (d) and can be interpreted as being bias estimated by the mean difference between the two measures (X1 − X2)/n, where "n" represents the number of individuals included in the sample. The two dotted lines above and below the line mean represent the limits of agreement, and these are drawn at d ± 1.96 × s, where "s" represents standard deviation of the differences [30,31]. All analyses were performed using the Statistical Package for the Social Sciences software (SPSS version 20, SPSS Inc., Chicago, IL, USA). Table 1 describes personal, clinical and anthropometric data. We included 69 participants with a confirmed diagnosis of post-COVID-19 in this study. Of these, 43 (62.3%) were women and 26 (37.7%) were men. The mean and dispersion values of linear and nonlinear HRV indices in the supine position evaluated by Researcher 01 and Researcher 02 are described in Table 2.

Results
Regarding the intrarater reliability of Researcher 01, in Table 3, the ICCs of HRV parameters in time-domain ranged from 0.992 to 0.999, in frequency-domain ranged 0.951 to 0.986, and in the nonlinear methods ranged 0.920 to 0.972. The SEM ranged from 0 to 4.29% in the time-domain, 7.93 to 15.04% in the frequency-domain, and 2.54 to 16.76% in the nonlinear methods. The MDC ranged from 0 to 11.90% in the time-domain, 21.99 to 41.68 in the frequency-domain, and 7.04 to 46.46% in the nonlinear methods.        Overall, the CV for the HRV parameters ranged from 0.11 to 4.46% in the time-domain, 7.66 to 15.92% in the frequency-domain, and 1.43 to 9.23% in the nonlinear methods of intrarater reliability by Researcher 01 (Table 3); 0.76 to 3.05% in the time-domain, 3.08 to 4.60% in the frequency-domain, and 0.86 to 6.86% in the nonlinear methods in intrarater reliability by Researcher 02 (Table 4); and 0.86 to 3.97 in the time-domain, 6.02 to 14.38% in the frequencydomain, and 1.45 to 8.83% in the nonlinear methods in interrater reliability (Table 5).

Discussion
The main findings of this study are: (1)

Discussion
The main findings of this study are: (1) the results on the influence of selection of the best stationary stretch may reflect on the results of HRV indices when evaluated by the same researcher and by different researchers at different times in post-COVID individuals; (2) intra-and inter-researcher analyzes show excellent reliability values in patients post-COVID-19 represented by ICC values greater than 0.90; (3) the analysis variation coefficients were less than 10% in the intrarater analysis and less than 9% in the interrrater analysis; (4) the Bland-Altman plots showed wider limits of agreement in the inter-rater HRV signal data processing in comparison with the intrarater.
First, it is important to clarify that the reliability of the Polar V800 heart rate monitor has already been evaluated and validated [24,32]. However, the biggest issue would be the reliability of the analysis of these signals referring to HRV, especially in the post-COVID-19 population.
A recent systematic review investigated studies that evaluated autonomic dysfunction in patients with COVID-19 and identified seven studies that used HRV as an instrument to assess dysautonomia [7]. However, to the best of our knowledge, there is no study in the literature that has investigated the amount of error related to HRV measurement in patients with COVID-19 or post-COVID-19. Thus, our results stand out for providing clinical support for the use of HRV in these individuals, given that the error inherent in an assessment instrument is population-dependent [20].
The results of the present study are similar to research that investigated the reliability of HRV in other populations. Bassi et al. [15] identified ICC values ranging from 0.73 to 0.99 in the analysis of the intra-and inter-rater reliability of the HRV at rest in diabetic type 2 patients; Zizzo et al. [33] observed values ≥ 0.80 in the measurement of fetal HRV; and Sima et al. [34] identified ICC values ranging from 0.72 to 0.93 in chronic obstructive pulmonary disease patients.
Our study has limitations that must be considered. We included post-COVID-19 individuals, and the reliability of patients with ongoing infection was not evaluated. Our study investigated the reliability of a heart rate monitor. Thus, we recommend that future studies measure the reliability of other HRV capture devices. The reliability presented in this study is based on the HRV analysis method (comprising the identification of the best stationary stretch of the signal and measurement of parameters), according to a previously published study [15] we recommended that future studies consider measuring the reliability of the entire evaluation procedure, including patient positioning, orientations, placement of the strap or signal capture electrodes and capture of the signal.

Conclusions
This study provides a quantification of possible differences between researchers and intra-researchers, such as the influence of the selection of the iR-R by different researchers (interrater) and by the same researcher at different times in the quantification of HRV parameters (intrarater). This is extremely relevant in the context of qualitative biological variables. In this sense, the measurement of HRV in post-COVID-19 individuals is reliable and presents a small amount of error inherent to the method, supporting its use in the clinical environment and in scientific research.