The Impact of E-Cigarette Warnings, Warning Themes and Inclusion of Relative Harm Statements on Young Adults’ E-Cigarette Perceptions and Use Intentions

Although e-cigarettes in the United States are required to carry one nicotine addiction warning, little is known about the impact of other potential e-cigarette warning themes, nor about pairing warnings with messages that communicate e-cigarettes’ reduced-harm potential relative to cigarettes. We randomly assigned 876 young adults (ages 18–29) to view e-cigarette ads in a 3 × 2 plus control online experiment that varied by warning theme (i.e., nicotine addiction; nicotine’s impact on adolescent brain development; presence of harmful chemicals) and warning type—i.e., the presence (“relative harm warning”) or absence (“standard warning”) of a relative harm (RH) statement in the warning label (“e-cigarettes may cause harm to health but are less harmful than cigarettes”). Warning believability, informativeness, understandability and support were high across conditions and there were no significant differences by warning theme on e-cigarette harm perceptions or use intentions nor on nicotine (mis)perceptions. Perceived warning effectiveness for discouraging youth initiation was higher for the “brain” and “chemicals” warnings compared to the addiction warning. Warnings with the included RH statement were perceived as less believable and credible and were less frequently correctly recalled. Research should continue to investigate the impact of different e-cigarette warning themes and formats with priority audiences.


Introduction
Electronic cigarettes ("e-cigarettes") likely pose substantially fewer health risks to individuals than combustible cigarettes, and might be relevant for use in harm-reduction by smokers, but they are not risk free and their initiation by non-smokers could constitute public health harm [1,2]. Further, research suggests consumers want to know about the potential risks of e-cigarettess [3]. As such, risk and addiction perceptions, and e-cigarette use intentions compared to e-cigarette ads with no or minimal unstandardized warnings. We also aimed to examine whether exposure to such ads would unintentionally increase misperceptions that e-cigarette and cigarettes are equally harmful, increase nicotine misperceptions, and reduce smokers' intentions to use e-cigarettes for harm reduction.
Second, we aimed to examine whether warning themes other than nicotine addiction would have a different impact on these same outcomes, and whether other themes would be perceived by young adults to be as effective or more effective than the nicotine addiction warning.
Finally, we also aimed to explore the potential effects of pairing e-cigarette warnings with a relative harm statement. We hypothesized that those exposed to such warnings would have more accurate e-cigarette/cigarette relative harm perceptions and higher intentions to use e-cigarettes for harm-reduction among smokers. However, we also hypothesized that these warnings would be perceived as less effective and credible, more difficult to understand and recall, and would encourage use among non-smokers. We focused on young adults given their high prevalence of e-cigarette experimentation and priority as a target audience for such warnings.

Participants and Procedures
An online experiment was conducted between January-February 2018 with 876 young adults recruited through Amazon's Mechanical Turk (mTurk; Table 1). Eligibility was limited to registered mTurk participants between the ages of 18-29 located in the United States who had an 80% approval rating from previous mTurk human intelligence tasks (HITs). After a large number of non-smokers initially completed the survey (with approximately 80 non-smokers per condition), we limited eligibility and access to the full survey to current smokers only. Smoking status was determined in the first few questions of the survey, which included distractor questions about other health behaviors (e.g., soda and alcohol consumption, exercise) to mask the focus of the survey. The study was advertised on mTurk as a Health Behavior Survey and the study description did not reveal the focus on e-cigarettes.
Eligible participants were randomly assigned to view e-cigarette ads in one of seven conditions in a 3 (warning theme) by 2 (warning type) between group factorial design with a control group. In the experimental groups, three "warning themes" i.e., (a) nicotine addiction; (b) nicotine's impact on adolescent brain development; (c) presence of harmful chemicals), were each crossed with "warning type", in which a relative-harm (RH) statement ("E-cigarettes may cause harm to health but are less harmful than cigarettes") was either included ("RH warning") or not included ("standard warning") in the warning label. Participants in the control group viewed an unaltered set of control ads (see Figure A1).
Regarding themes, the addiction warning was the FDA's pending nicotine addiction warning. The "brain" warning stated, "WARNING: This product contains nicotine. Nicotine can harm adolescent brain development, which continues into the early to mid-20s." The "chemicals" warning stated: "WARNING: E-cigarette aerosol may contain chemicals that are harmful to the lungs and are known to cause cancer." Message themes were based on formative research [6,14,27] and wording was adapted from the CDC's e-cigarette fact sheet page at the time of the study [16].
Within each experimental group, participants viewed four different e-cigarette ads representing different leading brands (JUUL, Vuse, Blu, MarkTen), which were each manipulated to include the same warning message per their assigned condition. Warnings were sized at approximately 20% of the ad and placed in the upper portion of the ad, per new FDA e-cigarette warning requirements. Ads were presented in random order. Participants in the control groups viewed the same ads in their original un-manipulated form (i.e., as found in the marketplace) ( Figure A1, Appendix A). Participants could view each ad for as long as they wished but each ad was displayed for a minimum of seven seconds. After exposure to all four ads, participants completed outcome measures and then viewed a screen with debriefing information corresponding to their condition. The median study completion time was ten minutes and participants were compensated with a $1.75 payment.

Measures
Before viewing the stimuli ads, participants answered questions about their demographics and tobacco use (Table 1). Ever e-cigarette users were defined as those who indicated ever trying an e-cigarette, even one or two times. Current smokers and current e-cigarette users were defined as those indicating they now smoke cigarettes/use e-cigarettes every day or some days, respectively. Immediately after viewing the stimuli, the main dependent variables were measured. Our outcomes of focus were informed by our research objectives and by the Message Impact Framework for tobacco warnings, which suggests that characteristics of warnings, and audiences' perceptions about warnings may impact whether warnings are noticed and recalled, and may influence warnings' impact on tobacco product beliefs and use intentions [30]. Question wording for the measures below is available in Supplemental Table S1.
E-cigarette Use Intentions. All respondents were asked about their likelihood of buying an e-cigarette in the next 6 months (1 = not at all-5 = extremely likely). We also measured smokers' likelihood to use e-cigarettes for harm reduction by creating an average scale of three items asked of current smokers about their likelihood to use e-cigarettes to cut down on smoking, quit smoking completely, and likelihood of switching to e-cigarettes completely (α = 0.91) [31].
E-cigarette Beliefs. We assessed perceived absolute harm of e-cigarettes by asking respondents how harmful they believed daily e-cigarette use to be to a user's health (1 = not at all-5 = extremely harmful) [12]. Next, we created a sum score of the perceived likelihood of disease (lung cancer, other lung disease, heart disease, oral cancer) (1 = not at all-5 = extremely likely) from daily e-cigarette use over a lifetime (α = 0.92) [32], where higher total scores represented higher perceived risk. Perceived harm of e-cigarettes compared to cigarettes was measured by asking how harmful to health daily e-cigarette use is compared to daily cigarette smoking (1 = a lot less harmful-5 = a lot more harmful) [12,33]. We also measured perceived likelihood of addiction to e-cigarettes (1 = not at all-5 = extremely likely).
Nicotine Beliefs. We measured the extent to which participants agreed that nicotine causes smoking related cancer as an indicator of nicotine misperceptions (1 = not at all-4 = very much) and also asked participants to indicate their overall perceived harmfulness of nicotine to health (see Table 2) [20].
Warning Recall. Participants in the experimental groups were asked to type what they could remember of the warning in the ads they viewed in as much detail as possible (unaided recall). Consistent with previous research [34,35], two reviewers independently coded the responses for reference to keywords or concepts that were present in the warnings viewed (Table 3; average Kappa = 0.9). Respondents were then also exposed to the text of all six warnings and asked to select which they recalled viewing on their stimulus ads (aided recall).
Perceived Warning Effectiveness, Credibility and Support: Participants in the experimental groups viewed the warning per their condition again and were asked the extent to which they agreed (1 = strongly disagreed-4 = strongly agreed) it was believable, understandable, informative and discouraged e-cigarette use among young people and smokers [36][37][38]. Perceived credibility of the warning was measured adapting Meyer's credibility scale (5-point semantic differential items: e.g., accurate/inaccurate, biased/unbiased; α = 0.88) [12,39]. Finally participants were asked about the extent to which they agreed (1 = strongly disagreed-4 = strongly agreed) that e-cigarettes should carry warning labels about their potential risks. Results were dichotomized as agreed/disagreed.
Analysis: Because of the unbalanced experimental design (3 × 2 plus control), we first tested for mean difference across all seven warning conditions (including control) in the first set of outcomes, and for any interaction effects with smoking status (Model 1, Table 2). We then removed the control group and tested for any effects of warning theme or warning type when the experimental conditions were collapsed across all participants (Model 2, Table 2). For the perceived warning effectiveness outcomes (which were not assessed for the control group), we assessed the effects of warning theme and type, theme x type interactions and interactions with smoking status (Table 4). Any significant interaction effects across all analyses are noted in the results. We used ANOVA tests for semi-continuous outcomes (using Tukey's post-hoc tests for pairwise comparisons and error-rate control) and Chi-square tests for categorical outcomes (e.g., message recall, warning support). To minimize the likelihood of Type 1 error from multiple testing, we used a more conservative significance level of 0.01 for all analyses. Analyses were completed using SPSS version 25 (SPSS Inc., Chicago, IL, USA).

Results
Sample description. Conditions did not significantly differ by participant demographics, cigarette smoking and e-cigarette use, indicating that randomization was successful. Overall, the sample was evenly split by sex, was predominantly non-Hispanic white (64.3%), and most were employed (75.1%) ( Table 1). Participant average age was 25% and 51.1% had at least a college degree.
Over one-third (35.7%) were current smokers and over half of current smokers (54.5%) were current e-cigarette users. About 41% of all participants and 46.0% of smokers had ever noticed a warning on an e-cigarette ad or product package before. E-cigarette use intentions. There was no significant difference in likelihood of buying e-cigarettes across all conditions (see Table 2, Model 1), nor by warning theme or type among the collapsed experimental conditions (Model 2). Among current smokers (n = 313), we also found no significant difference in mean likelihood of using e-cigarettes for harm reduction across all conditions (p = 0.27) ( Table 2, Model 1). However, among the experimental conditions, the main effect for warning type approached significance (p = 0.012), with a higher likelihood of using e-cigarettes for harm reduction among smokers exposed to the ads with RH warnings (M = 2.93, SD = 1.1) relative to the standard warnings (M = 2.58, SD = 1.1) (data not in table). (1 = not at all-5 = extremely likely), 2 (1 = not at all-5 = extremely harmful), 3 (1 = a lot less harmful-5 = a lot more harmful), 4 (1 = not at all-4 = very much).

E-cigarette Beliefs.
There was no effect of warning condition (nor interaction with smoking status) on perceived absolute harm of e-cigarette use nor on perceived likelihood of disease from e-cigarette use (Table 2). For both outcomes, there were no differences by warning theme or type among the experimental conditions. Perceived harm of e-cigarettes compared to cigarettes was also not significantly associated with warning conditions, nor with warning theme or type when experimental conditions were collapsed (Table 2).
There was no effect of warning condition (nor interaction with smoking status) on perceived likelihood of addiction (Table 2, Model 1). When experimental conditions were collapsed by theme and type, the main effect of warning theme approached significance (p = 0.013, Table 2, Model 2), with scores higher for the FDA addiction warning (M = 3.65, SD = 1.0) relative to the "chemicals" warning (M = 3.38, SD = 0.9, p = 0.01), but not relative to the nicotine brain warning (M = 3.54, SD = 1.0).
Nicotine Beliefs. Over half of all participants (56.2%) incorrectly believed nicotine to be the chemical largely responsible for cancer caused by smoking and 54.4% rated nicotine as being very or extremely harmful to health. There was no significant mean differences for either of the nicotine beliefs by warning condition (Table 2) (nor interactions between condition and smoking status), nor by warning theme or type among the experimental conditions.
Warning Recall. Unaided recall of specific warning keywords/concepts was significantly lower in the RH warnings versus the standard versions of the nicotine addiction and brain warning themes (see Table 3). Among those exposed to the standard version of the FDA warning, most recalled the keywords/concepts of "nicotine" (84.1%) and "addiction" (66.7%). For the brain warning (standard version), recall of some reference to age (78%) was higher than for the keywords/concepts "nicotine" (69.3%) and "brain" (65.4%). For the "chemicals" warning, recall of "cancer" was more prevalent than "chemicals" and "lungs" (Table 3).
When aided recall/recognition was measured, there was no significant association between correct recall and warning theme but correct warning recall/recognition was lower among those who viewed the RH warnings (69.5%) versus the standard warnings (84.1%, p =< 0.001).
Perceived warning effectiveness, credibility and support. Most participants across conditions agreed that the warning they viewed was believable (89.3%), however mean believability was significantly higher for the standard versus the RH warnings (see Table 4). Agreement that the warning was understandable was also high (95.5%) and was not significantly associated with warning type nor theme. Overall, 82.7% of respondents agreed that the warning they viewed would make people more informed about e-cigarette health risks. Mean perceived informativeness did not significantly differ by theme or type (Table 4).  Note: F = F statistic, η 2 = partial eta squared, p = p-value for F statistic. f = significantly different than FDA nicotine theme, b = significantly different than Brain theme, c = significantly different than Chemicals theme, with significance level set at p = 0.01.
About 61% agreed that the warning they viewed could discourage young people from starting to use e-cigarettes. Mean agreement was significantly associated with warning type and was higher for the standard (M = 2.88) versus RH warnings (M = 2.45). Agreement was also associated with warning theme, and was lowest for the FDA addiction warning (Table 4). In contrast, fewer respondents overall (37.1%) agreed that the warning they viewed would discourage current smokers from using e-cigarettes, with significant main effects by warning theme and type (Table 4). Smoker discouragement was rated as higher for the standard warnings relative to the RH warnings, and was lowest for the addiction theme relative to the brain and chemicals themes.
For perceived warning credibility, only the main effect of warning type was significant (F = 23.92, R 2 = 0.03, p < 0.001), with participants rating the standard warnings as more credible (M = 3.88, SD = 1.4) than the RH warnings (M = 3.34, SD = 1.5) (data not in table).
Warning Support. Finally, the vast majority (93.2%) agreed that e-cigarettes should carry warning labels about their potential risks. Support was slightly higher among non-smokers (95.0%) versus current smokers (90.0%) (p < 0.01) but did not significantly vary by warning condition.

Discussion
This study tested the impact of exposure to text warnings on e-cigarette ads in a new standardized format, the impact of different warning themes and the inclusion of a message comparing the harm of e-cigarettes relative to cigarettes. Overall, young adults expressed strong support for e-cigarette warnings and perceived them to be believable, understandable and informative about e-cigarette risks. However, we found few experimental effects of warning exposure on e-cigarette harm and risk perceptions and use intentions.
Consistent with a previous study with young adult non-smokers [12], we found no significant effect of exposure to the experimental warnings on participants' e-cigarette harm and addiction beliefs, nor on decreasing e-cigarette use intentions among non-smokers when compared to ads with no or small unstandardized warnings. These null findings may suggest that text-only warnings (even large ones) may have limited impact with young adults when placed in otherwise colorful and interesting e-cigarette advertisements [12]. These findings, along with previous research [8,12,14,34], may suggest that the use of visual elements, such as colored warnings or some relevant pictorials, may be worth exploring given that they might enhance the noticeability, salience and recall of these warnings.
On the other hand, some of the null findings might be viewed as favorable outcomes. Exposure to the test warnings did not increase misperceptions about nicotine's role in cancer or harmfulness, nor that e-cigarettes are as harmful as cigarettes. It also did not appear to reduce smokers' intentions to use e-cigarettes for harm reduction. Together these findings may suggest that the new warning requirements may have minimal unintended consequences in terms of increasing product misperceptions or discouraging use of e-cigarettes for harm reduction.
Notably, this study also found that most participants perceived the experimental warnings, including the FDA's addiction warning, to be informative about e-cigarette health risks. This is important given that this is the primary purpose of such warnings [8]. In this study unaided recall of the word "nicotine" was high among those exposed to the FDA's nicotine addiction warning. This too is important given that research has continued to show that some young people remain unsure of whether the e-cigarettes they use contain nicotine or misperceive their products to be nicotine-free [40][41][42], underscoring a basic need for such clear and conspicuous labeling. In this group of young adults, the other warning themes tested (presence of chemicals and nicotine impact on adolescent brain development) did not significantly outperform the FDA's addiction warning on any of the main experimental outcome measures including e-cigarette harm/risk perceptions and buying likelihood. Together, these findings support the FDA's use of the nicotine addiction theme as a starting point for e-cigarette warnings.
However, the nicotine addiction warning trailed behind the other warning themes in terms of its perceived ability to discourage e-cigarette use among young people. Participants also found the "brain" and "chemicals" warnings to be as believable, informative and credible as the nicotine addiction warning. These findings suggest that these warning themes may deserve additional research attention as potential future warnings, particularly for youth audiences. Indeed, a recent e-cigarette study with adolescents found that while 83% knew that e-cigarettes usually contain nicotine, far fewer knew that e-cigarettes contain harmful chemicals (67%) and might harm teen brain development (49%) [43].
This study also explored the impact of including a relative harm statement within the warning label, as has previously been proposed by smokeless tobacco companies and could similarly be proposed for e-cigarettes in the future. Given the calls for increased communications about the relative harms of different tobacco products, it was encouraging to find that these messages were not perceived as being less understandable than the standard warnings, and were not associated with higher e-cigarette use intentions among non-smokers, in contrast to our hypotheses. However, consistent with prior research on this topic [25][26][27][28][29] and our hypotheses, these warnings were perceived as less believable and credible, and also as less likely to discourage e-cigarette use among young people. In line with our expectation, inclusion of the relative harm statements in the warning label also appeared to reduce recall of the risk-related part of the message, potentially because of the increased length of the final overall message and a recency effect of the relative harm information at the end of the message. Future research should continue to examine the effects of various relative harm/modified risk tobacco messages, including appropriate wording, locations and formats that maximize potential public health benefits and minimize unintended consequences. Future research on this topic, as well as on e-cigarette warning labels more generally, should also be examined with more representative samples of participants, as participants in mTurk convenience samples have shown to generally be younger, more educated, have above average cognitive aptitude and be more likely to be white and Asian compared to the general population [44].

Conclusions
Findings from this study suggest that warning labels with messages about harms beyond nicotine's addictiveness are perceived as believable, informative, understandable and credible among young adults, may provide novel information, and may discourage e-cigarette use among young people. While inclusion of a relative harm statement in e-cigarette warnings may not unintentionally increase use by non-smokers, it may reduce recall of warning messages in young adults. Further research should continue to identify warning themes, types and execution formats that accurately inform audiences about e-cigarettes.  Funding: This work was supported by a grant (K01CA189301) from the National Cancer Institute (NCI) and the Food and Drug Administration Center for Tobacco Products. Contributions by OAW, ROJ and CDD were also supported in part by R37CA222002 from NCI. ACV was supported by the NCI under Award Number R03CA212694 and the Centers of Biomedical Research Excellence P20GM103644 award from the National Institute of General Medical Sciences. Figure A1. Manipulation of stimuli ads, by condition. NOTE: All four e-cigarette brand ads were viewed in each condition, each containing the same warning, respectively. Figure A1. Manipulation of stimuli ads, by condition. NOTE: All four e-cigarette brand ads were viewed in each condition, each containing the same warning, respectively.