Validation of an Index for Functionally Important Respiratory Symptoms among Adults in the Nationally Representative Population Assessment of Tobacco and Health Study, 2014–2016

The purpose of this study is to validate the seven-item wheezing module from the International Study of Asthma and Allergies in Children (ISAAC) in the nationally representative Population Assessment of Tobacco and Health Study. Adult participants with complete Wave 2–3 data were selected, including those with asthma but excluding those with COPD and other respiratory diseases (n = 16,295). We created a nine-point respiratory symptom index from the ISAAC questions, assessed the reliability of the index, and examined associations with self-reported asthma diagnosis. Threshold values were assessed for association with functional outcomes. The weighted prevalence for one or more respiratory symptom was 18.0% (SE = 0.5) for adults without asthma, 70.1% (SE = 1.3) for those with lifetime asthma, 75.7% (SE = 3.7) for adults with past-year asthma not on medications, and 92.6% (SE = 1.6) for those on medications. Cronbach’s alpha for the respiratory symptom index was 0.86. Index scores of ≥2 or ≥3 yielded functionally important respiratory symptom prevalence of 7–10%, adequate sensitivity and specificity for identifying asthma, and consistent independent associations with all functional outcomes and tobacco use variables. Respiratory symptom index scores of ≥2 or ≥3 are indicative of functionally important respiratory symptoms and could be used to assess the relationship between tobacco use and respiratory health.


Introduction
The focus of this report is respiratory symptoms (wheezing and dry cough) indicative of airway obstruction. Wheezing is an important respiratory symptom suggesting the existence of asthma and/or chronic bronchitis [1]. Its association with cigarette smoke exposure, especially among young people [2], makes this an important endpoint in any study of tobacco use and disease. There are several validated questionnaires for assessing wheezing in children [3,4] or adults [5] but not typically for both populations.
The Population Assessment of Tobacco and Health (PATH) Study is an ongoing, nationally representative, longitudinal cohort study that obtains detailed information on tobacco use in the U.S. population. The PATH Study aims to assess the relationship between tobacco product use and health outcomes. Respondents are asked questions about their health, including self-reports of chronic diseases such as diabetes, heart attacks, stroke, cancer, respiratory disease, disease symptoms, and functional outcomes [6].
The PATH Study adult interview includes a core wheezing assessment from the International Study of Asthma and Allergies in Children (ISAAC) that was validated and used widely in children [7][8][9], and modified for use with adults [10,11]. The goal of this report is to create a validated respiratory symptom index from the included ISAAC questions in a sample of adult asthmatics, as well as in a large representative sample of the adult general population (including asthmatics).
We also sought to assist researchers in assigning a respiratory symptoms index cutoff value that would serve as a binary indicator of functionally important respiratory symptoms, which was validated against self-reported asthma diagnosis and functional outcomes. By exploring how various respiratory symptom index cut-off values relate to asthma diagnoses and functional outcomes such as fatigue, researchers can choose a dichotomous outcome indicative of functional impairment. The respiratory symptoms index and potential cut-off values should provide estimates of impairment or disease that align with existing literature. Since approximately 10-15% of the adult population has significant obstructive impairment [12] and about 8% of the adult population has been diagnosed with asthma [13,14], we will evaluate if our index yields comparable estimates of impairment.
Finally, the association between current cigarette smoking, cigarette smoking history, and second-hand smoke exposure with the respiratory symptom index was examined using PATH Study data. Our hope is that researchers will be able to confidently use this index to study the effects of tobacco use on respiratory health.
The validation sample included in these analyses corresponds to the sample selected for a separate manuscript, which utilized a complete case analysis to evaluate the association between tobacco use and changes in respiratory symptoms in the general population of adults, including adults with asthma [15], which will be referred to as the general population moving forward (see Figure 1). At W2, there were 24,798 adults without self-reported health professional diagnosis of chronic obstructive pulmonary disease (COPD) or other non-asthma respiratory diseases by Wave 3. Adults with COPD or other non-asthma respiratory disease were excluded because those are non-reversible (or unknown) lung diseases and might skew the ability to validate the respiratory symptom scale for the general population. The exclusion criteria outlined in Figure 1 include (1) participants lost to follow-up at W3 (n = 2837, 11%) and (2) those with missing data on any analytic variables (n = 5666, 23%) used in the aforementioned manuscript and described more in the Figure 1 footnotes, for a final analytic sample of 16,295. Validation results in this report did not change when using the full sample (i.e., not excluding those with missing data on any analytic variables; n = 21,961). All analytic variables in this report come from W2 (2014-2015) except for when evaluating test-retest reliability, as described below. PATH Study design and methods [6,16,17], interviewing procedures, questionnaires, sampling, weighting, and response rates are in the PATH Study Restricted Use Files User Guide [18]. The study was conducted by Westat and approved by the Westat Institutional Review Board. All respondents provided informed consent. Main analytic sample determination. This figure illustrates the sample determination from the Population Assessment of Tobacco and Health Study for the main analyses that this validation will be used as described in the Materials and Methods of the manuscript. Some functional outcome analyses have additional variables that reduce the sample for those analyses further due to missingness. a Chronic obstructive pulmonary disease. b Other non-asthma respiratory diseases. c Weights adjust for non-response. d Dependent variables include Respiratory Symptom Index at Waves 2 and 3. Independent variables include tobacco product use at Waves 2 and 3 along with Wave 2 determinations of asthma, cigarette pack years, age, sex, ethnicity/race, education, income, congestive heart failure, diabetes, cancer, heart attack, body mass index, marijuana use, past week second-hand smoke exposure, regular use of Beta Blockers, Angiotensin Receptor Blockers, or Ace Inhibitors, and Wave 1 determination of urbanicity.

Measures for Respiratory Symptoms in the PATH Study
The core wheezing module from the ISAAC [7] is used to assess wheezing and nighttime cough in children and adults in the PATH Study. The seven ISAAC questions (Table 1) have been validated in children and widely implemented across many countries [8,9].

Item-Level Validation against Similar NHANES Items
Five of the ISAAC items are similar to those used to assess respiratory symptoms in the National Health and Nutrition Examination Survey (NHANES). Weighted prevalence estimates were calculated for ISAAC items in W2 (2013-2014) of the PATH Study and compared with similar items from the 2011-12 NHANES. No" response to question R02_AX0046 (ever wheezing) and/or R02_AX0047 (past 12-month wheezing). e None/no for these three questions include the "No" option from RDQ070 (past 12-month wheezing).

Creating a Respiratory Symptoms Index
Since combining similar items in an index increases its reliability [19], responses to the seven ISAAC core wheezing module questions were used to create a respiratory symptoms index. Three wheezing variables (ever wheezing, past 12-month (P12M) wheezing, and P12M wheezing attack frequency) were combined to create one variable (0 = never wheezing; 1 = ever wheezing, but no P12M wheezing OR P12M wheezing but no wheezing attacks; 2 = P12M wheezing and 1-3 attacks; 3 = P12M wheezing and 4-12 attacks; 4 = P12M wheezing and more than 12 attacks). This new variable was summed with remaining four questions (three of which are yes = 1/no = 0, and one of which has three answer choices scored as 0, 1, or 2) to create the index ranging from 0 to 9, where 0 represented no respiratory symptoms, and 9 represents the highest level of symptoms (i.e., a "yes" to all dichotomous outcomes and the highest level for questions with multiple responses).
Internal consistency of the index was examined using Cronbach's alpha. Test-retest reliability was assessed by pairwise correlation between respiratory symptoms index scores at W2 with scores approximately one year later at W3.

Validating the Respiratory Symptoms Index against Asthma
Then, we examined the association between index scores and self-reported health professional-diagnosed asthma. Since wheezing is a key feature of asthma, using the index to predict asthma diagnosis is useful to support construct validity and the possibility for utilizing the index within the general population as well. Lifetime asthma diagnosis was indicated by a response of asthma to the following question, "Has a doctor, nurse or other health professional ever told you that you had any of the following lung or respiratory conditions?" Individuals with lifetime asthma were asked if they had asthma in the past 12 months and whether they had taken any medication for asthma in the past 12 months. Those with P12M asthma were divided into two groups based on whether they had taken any asthma treatment medications. The weighted proportion with each type of asthma was examined as a function of their respiratory symptom index score.

Determining Cut-Off Values for Functionally Important Respiratory Symptoms among Those with Asthma
Weighted logistic regression and unweighted receiver operator characteristic (ROC) curves were used to examine the association between the respiratory symptoms index at various cut-off levels with the three asthma outcomes (ever, P12M without medications, P12M with medications), reporting prevalence, sensitivity, the false positive rate, and the unadjusted association with asthma for cut-off values of ≥1, ≥2, ≥3, and ≥4. ROC curves allow researchers interested in defining thresholds for disease to understand the tradeoff between sensitivity (i.e., the ability to correctly include all with the disease outcome of interest) and the false positive rate (i.e., those without the disease of interest who are included in the test positive subpopulation) [20].

Association between Respiratory Symptoms and Functional Outcomes in the General Population
PATH Study respondents were also questioned about functional outcomes (e.g., physical limitations, fatigue, and general health) with items from the Patient-Reported Outcomes Measurement Information System (PROMIS) physical health question bank [21]. The questions captured activity limitations (e.g., health limits walking 3 blocks), fatigue in the past 7 days, and general physical health, and they have high reliability [22].
Weighted logistic regression or multinomial logistic regression was used to assess the relationship between respiratory symptom scores (as a continuous outcome and at cut-off values ≥1, ≥2, ≥3, and ≥4) and functional outcomes. First, we inspected the association between higher scores on the respiratory symptoms index and the proportion with functional limitations, looking for nonlinearity. Next, we tested the independent relationship of both the continuous index and dichotomous measures of functionally important respiratory symptoms with functional outcomes using weighted multivariable logistic or multinomial regression, adjusting for demographics and other diseases (diabetes, cancer, congestive heart failure, and heart attack) that could confound the association.
Finally, weighted multivariable logistic regression was used to test the independent associations between both the continuous respiratory symptom index and functionally important symptoms at the four cut-off levels and three tobacco outcomes that have been previously associated with wheezing: current cigarette smoking (never, former, past 30-day nondaily, or past 30-day daily), pack-years of cigarette smoking (never smokers were assigned a value of 0; entered as a continuous measure with each 1-point increase indicative of an additional 5 pack-years of smoking), and second-hand smoke exposure (with each 1-point increase indicative of an additional hour per week of exposure).

Analytical Approach
All main analyses were weighted using the W3 longitudinal (all-waves) full-sample and replicate weights to adjust for the complex sample design and loss to follow up. Variances were estimated using the BRR method [23] with Fay's adjustment set to 0.3 to increase estimate stability [24]. Pack-years of cigarette smoking and second-hand smoke exposure were Winsorized at the 95th and 99th percentiles respectively to address outliers [25]. All analyses used Stata survey data procedures, version 16

Descriptive Statistics, Internal Consistency, and Test-Retest Reliability
The weighted prevalence for one or more index respiratory symptoms was 18.0% (standard error (SE) = 0.5) for adults without asthma, 70.1% (SE = 1.3) for those with lifetime asthma, 75.7% (SE = 3.7) for adults with asthma in the past year but not on medications, and 92.6% (SE = 1.6) for past-year asthmatics on medications. For the same four categories, the weighted mean (SE) for the respiratory symptoms index was 0.3 (0.0), 2.0 (0.1), 2.2 (0.1), and 3.6 (0.1) respectively. The respiratory symptom index had acceptable internal consistency (Cronbach's alpha = 0.86) and one-year test-retest reliability (r = 0.72 for pairwise correlation between W2 and W3 results). Table 1 reports weighted prevalence for respiratory symptoms based on W2 of the PATH Study compared to NHANES 2011-2012 among adults 18+ years old. The table shows similar-to-higher prevalence of past-year wheezing based on PATH Study W2 compared to NHANES. For example, the prevalence of disturbed sleep due to wheezing one or more nights per week was ≈3% in both studies, whereas wheezing in the past 12 months was higher in the PATH Study than NHANES (16.8% vs. 13.1%). The prevalence of dry cough at night was much higher in the PATH Study, which was most likely due to differences in time frame (at least 14 days of persistent cough for the NHANES question compared to no timeframe for the PATH Study question). Figure 2 illustrates the association between higher scores on the respiratory symptoms index and self-reports of asthma diagnosis (respiratory symptom scores of 7-9 were collapsed because of small numbers in those categories). Less than 4% of persons with no symptoms had any asthma diagnosis. For lifetime and past-year asthma without medications, the association was nonlinear, increasing up to an index score of 3 with less consistent change thereafter (suggesting a cut-off threshold of ≥3); in contrast, the association was roughly linear for past year asthma with medications, increasing from 0.2% (95% confidence interval (CI) 0.1, 0.3) for those with no symptoms to >45% for those with scores of 6 or more.

Association of Various Respiratory Symptom Score Cut-Off Values with Self-Reported Asthma
Results from the ROC analysis (Table 2) indicated that the respiratory symptoms index was strongly associated with asthma (area under the ROC curve = 0.75, 0.80, and 0.90 for lifetime asthma, past-year asthma on no medications, and past-year asthma on medications, respectively). Symptom prevalence was upwards of 20% for a threshold of ≥1, 10.7% for ≥2, 7.2% for ≥3, and less than 5% for a threshold of ≥4. Sensitivity was best for more severe asthma; for example, for a threshold of ≥3, sensitivity was 35.2%, 45.8%, and 67.7% respectively for lifetime asthma, past-year asthma on no medications, and past-year asthma on medications. In classifying past-year asthma on medications, thresholds of ≥1 through ≥4 were associated with sensitivities of 91.9%, 80.1%, 67.7%, and 54.2%, respectively. The false positive rates were approximately 22% for a threshold of ≥1 and 10% and 7% for thresholds of ≥2 and ≥3, respectively. Unadjusted odds ratios (OR) for the association with asthma indicated strong associations that increased as expected with recency and severity of asthma diagnosis (e.g., for a threshold of ≥3, the ORs were 12.37 (95% CI 10.55, 14.49), 17.85 (95% CI 13.13, 24.28), and 43.62 (95% CI 31.24, 60.90) for lifetime asthma, past-year asthma on no medications, and past-year asthma on medications, respectively).

Association of Respiratory Symptom Score Cut-Off Values with Functional Outcomes in the General Population
The functional outcome questions and their weighted prevalence in the adult population are shown in Table 3. Whereas 4.2% experienced health limits for walking up 10 steps, 9.4% experienced limits for walking a mile. Only 23.8% of adults experienced no fatigue, but only 4.0% and 0.1% respectively experienced severe or very severe fatigue. Similarly, only 17.4% were in excellent health, but only 9.3% and 1.1% respectively experienced fair or poor health. The associations between higher scores on the respiratory symptoms index and three functional outcomes-health limits walking three blocks, severe/very severe fatigue, and fair/poor health-are illustrated in Figure 3. The percentage of people with health limits walking three blocks increased from 4.7% (95% CI 4.1, 5.5) for adults with no respiratory symptoms to 21.1% (95% CI 13.9, 30.9) for those with scores of ≥7. The percentage with severe to very severe fatigue increased from 3.6% (95% CI 3.2, 4.1) for those with no symptoms to 27.0% (95% CI 18.3, 37.9) with scores of ≥7, and fair to poor health increased from 8.2% (95% CI 7.5, 9.0) to 44.1% (95% CI 33.7, 55.0). Finally, there was no visual evidence of a respiratory symptom threshold for functional impairment; instead, increases in functional impairment were seen across the spectrum of the respiratory symptoms index. Table 4 shows the independent association between the continuous index or various respiratory symptom index cut-off levels and functional outcomes. Regardless of cut-off level chosen, persons with scores above the cut-off level had a higher risk of all functional outcomes. For the continuous index, all relationships were significant, but with lower estimates (ORs or risk ratios) than when using any of the cut-points. For the physical outcomes such as walking three blocks, the adjusted ORs increased from 1.25 (95% CI 1.18, 1.33) as a continuous index to 1.77 (95% CI 1.41, 2.22) among those with a score of ≥1 and to 2.89 (95% CI 2.13, 3.93) among those with a score of ≥4. Estimates tended to be higher with higher cut-off levels. For example, the OR for difficulty in walking a mile increased from 2.00 (95% CI 1.65, 2.43) to 3.13 (95% CI 2.39, 4.10) as the cut-off value increased from ≥1 to ≥4. Higher ORs were also observed for more difficult function tasks at a given cut-off value; for example, at a cut-off value of ≥3, the ORs for difficulty walking up 10 steps was 2.00 (95% CI 1.45, 2.76) compared to 2.71 (95% CI 2.14, 3.43) for difficulty walking a mile.  Similar patterns were found in the multinomial regressions that examined the independent associations between various cut-off levels and more fatigue and poorer physical health. In no case was the independent association between respiratory symptoms and functional outcomes not statistically significant, and in many cases, the relative risks were above 2.00. For very severe fatigue vs. none, the relative risks were 1.74 (95% CI 1.54, 1.96), 3.82 (95% CI 2.37, 6.17), 6.31 (95% CI 3.94, 10.11), 7.69 (95% CI 4.73, 12.49), and 10.00 (95% CI 5.52, 18.11) for continuous and then cut-off levels of ≥1, ≥2, ≥3, and ≥4 respectively, and for a cut-off of ≥3, the relative risks increased from 1 Table 5 shows the association between various indicators of cigarette smoking exposure and respiratory symptom index as continuous and at the four cut-off levels. Important associations are found consistently for all measures of former and current cigarette use as well as pack-years of cigarette smoking and second-hand smoke exposure. The associations changed little as a function of which cut-off level was chosen, with the exception of current daily smoking, for which the relative risk is 2.29 (95% CI 2.08, 2.53) at a threshold of ≥1 and almost double that (4.31 (95% CI 3.36, 5.52)) at a threshold of ≥4.  Figure 1 Footnote e. b Adjusted for the other variables in the table. c Never tobacco user category includes non-established (e.g., lifetime use of <100 cigarettes) users; former established user category user includes all established users (e.g., lifetime use of 100 or more cigarettes) who did not smoke a cigarette in the past 30 days; both current categories include established users (e.g., lifetime use of 100 or more cigarettes) who did smoke a cigarette in the past 30 days.

Discussion
In this large nationally representative sample of adults without COPD, respiratory symptoms (wheezing and dry cough) indicative of airway obstruction were strongly associated with an asthma diagnosis and also related to functional impairments in the general population-including difficulty with physical tasks such as walking up stairs, increased levels of fatigue, and poorer perceptions of physical health. The association between higher scores on the respiratory symptoms index and functional impairment was linear, with those having more symptoms being more likely to be impaired, and the association was also independent of sociodemographics and other conditions predictive of functional impairment. We conclude that this index is a reliable and compact measure of respiratory symptoms in epidemiologic samples of adults without COPD.
With regard to guiding researchers toward a cut-off level indicative of functional impairment, relative risks for the association ranged from 2 to 11, depending on the functional outcome and the cut-off value chosen. Our results did not suggest a specific cut-off value for this index indicative of functionally important respiratory symptoms; indeed, all cut-off values from ≥1 to ≥4 were associated with both asthma and functional impairment. However, there are considerations that might narrow the choice for a cut-off threshold. Specifically, for the association between the respiratory symptoms index and lifetime and current asthma, a cut-off of ≥3 was suggested by the nonlinear association such that symptom scores began to level off after a score of ≥3.
Considering the prevalence of functionally important respiratory symptoms, a cut-off value of ≥1 seemed too inclusive, because it returned a prevalence of over 20%, whereas prevalence studies find that significant obstructive respiratory impairment is between 10 and 15% of the adult population [12]. One study using NHANES, where spirometry was used to diagnose obstructive lung disease, found that severe or very severe airway obstruction (FEV1% predicted <50%) was present in 3-5% of adults without asthma/COPD and 14-23% of those with the diagnosis of asthma/COPD [27]. Prevalence considerations suggest a cut-off level of ≥2 or ≥3, which result in prevalence consistent with the prevalence of asthma, which is about 8% of the adult population [13,14]. Another consideration is that higher cut-off levels identify more severe symptoms but increase the chances of missing significant disease, for example current asthma diagnosis with use of medications. ROC analysis confirmed that a cut-off of ≥2 or ≥3 offers the best tradeoff between sensitivity and specificity for the diagnosis of asthma. This study demonstrates that all cut-off levels show robust, independent associations with functional outcomes.
Additional analyses showed a significant relationship between the respiratory symptoms index and cigarette smoking exposure, pack-years of cigarette smoking, and secondhand smoke exposure. This index could provide researchers with a validated respiratory outcome in cross-sectional and longitudinal studies of tobacco use when more clinical measures are not available. Results showed that researchers could use the index as a continuous measure or use any of the four cut-off values presented.
Strengths of this study are the inclusion of a large, nationally representative sample of adults for the validation process, along with an extensive analysis that includes examination of index reliability, its association with self-reported asthma diagnosis, and its association with functional outcomes. As a large population study, there are weaknesses in that the study relies solely on self-report data. In addition, there was no medical record confirmation of the asthma diagnosis, and we were unable to make associations between self-reported wheezing symptoms and more objective measures such as spirometry. These limitations were addressed by comparing PATH Study data to that of NHANES and using questions already validated for use in capturing respiratory illness in other populations.

Conclusions
This respiratory symptoms index, based on questions that have been extensively developed and tested in children, serves as an adequate population-level instrument for assessing functionally important respiratory symptoms, indicative of airway obstruction, in this large nationally representative sample of adults without COPD. Additionally, possible thresholds for determining "functionally important respiratory symptoms" could be used to assess the relationship between exposures, such as tobacco product use, and respiratory illness. Analyses confirmed that a cut-off of ≥2 or ≥3 offers the best tradeoff between sensitivity and specificity for the diagnosis of asthma, although researchers may wish to use this data to create a different cut-off value depending on their research questions.