Next Article in Journal
Inhaler Technique and Adherence to Inhaled Medications among Patients with Acute Exacerbation of Chronic Obstructive Pulmonary Disease in Vietnam
Previous Article in Journal
Self-Efficacy as a Moderator between Stress and Professional Burnout in Firefighters
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

The Impact of E-Cigarette Warnings, Warning Themes and Inclusion of Relative Harm Statements on Young Adults’ E-Cigarette Perceptions and Use Intentions

Center for Tobacco Studies, Rutgers School of Public Health, New Brunswick, NJ 08901, USA
School of Public Health and Health Systems, University of Waterloo, Waterloo, ON N2L 3G1, Canada
Department of Health Behavior, Roswell Park Cancer Institute, Buffalo, NY 14263, USA
Department of Epidemiology & Biostatistics, Rutgers School of Public Health, Piscataway, NJ 08854, USA
Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
Department of Psychiatry, Vermont Center on Behavior and Health, University of Vermont, Burlington, VT 05405, USA
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2019, 16(2), 184;
Original submission received: 21 September 2018 / Revised: 21 December 2018 / Accepted: 7 January 2019 / Published: 10 January 2019


Although e-cigarettes in the United States are required to carry one nicotine addiction warning, little is known about the impact of other potential e-cigarette warning themes, nor about pairing warnings with messages that communicate e-cigarettes’ reduced-harm potential relative to cigarettes. We randomly assigned 876 young adults (ages 18–29) to view e-cigarette ads in a 3 × 2 plus control online experiment that varied by warning theme (i.e., nicotine addiction; nicotine’s impact on adolescent brain development; presence of harmful chemicals) and warning type—i.e., the presence (“relative harm warning”) or absence (“standard warning”) of a relative harm (RH) statement in the warning label (“e-cigarettes may cause harm to health but are less harmful than cigarettes”). Warning believability, informativeness, understandability and support were high across conditions and there were no significant differences by warning theme on e-cigarette harm perceptions or use intentions nor on nicotine (mis)perceptions. Perceived warning effectiveness for discouraging youth initiation was higher for the “brain” and “chemicals” warnings compared to the addiction warning. Warnings with the included RH statement were perceived as less believable and credible and were less frequently correctly recalled. Research should continue to investigate the impact of different e-cigarette warning themes and formats with priority audiences.

1. Introduction

Electronic cigarettes (“e-cigarettes”) likely pose substantially fewer health risks to individuals than combustible cigarettes, and might be relevant for use in harm-reduction by smokers, but they are not risk free and their initiation by non-smokers could constitute public health harm [1,2]. Further, research suggests consumers want to know about the potential risks of e-cigarettess [3]. As such, inclusion of warning labels on e-cigarette packaging and advertising is one strategy for informing the public of e-cigarettes’ potential risks, as is done for combustible cigarettes and smokeless tobacco [4].
In August 2018, the U.S. Food & Drug Administration (FDA) began requiring that e-cigarette packaging and advertising carry a warning that the products contain nicotine, an addictive chemical. Although some e-cigarette brands previously carried voluntary warnings [5], their contents and format have varied greatly. For example, MarkTen brand ads have included long warnings in small font size measuring around 10% of the ad, while others (e.g., Vuse, Blu) have included messages in small font with low contrast unrecognizable as traditional warnings [6,7]. Warning format is now standardized under new FDA requirements to enhance their noticeability: warnings need to occupy at least 20% of ad space, appear in the upper portion of the ad, include a rectangular border, and use a font size that occupies most of the warning area [8]. They also need to occupy 30% of the front and back of the principal panels of e-cigarette packaging. This is important given that previous research has shown strengthening of warnings to be associated with improvements in warning attention, recall, perceived effectiveness and tobacco risk-related knowledge [9,10].
With respect to warning content, research on the efficacy of nicotine addiction messages has been mixed. One study found that smokers and e-cigarette users exposed to e-cigarette ads with an addiction warning had higher e-cigarette risk beliefs and lower willingness to try e-cigarettes relative to an unexposed group [11]. However, the tested warning stated that “electronic cigarettes are addictive” in contrast to the more indirect wording of the FDA’s warning [“WARNING: This product contains nicotine. Nicotine is an addictive chemical”]. Another study found no impact of e-cigarette warnings, including those similar to the FDA’s, embedded in advertisements on young adult non-smokers’ e-cigarette harm perceptions and use intentions [12].
Research suggests that other warning themes might be relevant and potentially more impactful than an addiction warning [6,13], and the FDA could propose additional warnings if appropriate for public health. Some tobacco control experts have noted that e-cigarette warnings could describe negative nicotine effects, such as its harm on adolescent brain development, a message which might resonate more with young people, who are a priority audience for the addiction warning [14]. Indeed, this risk has been highlighted in the 2016 Surgeon General’s Report on e-cigarettes [15] and in the CDC’s public e-cigarette informational materials [16]. Research has also suggested that the presence of harmful chemicals found in e-cigarettes could be a relevant warning theme [6,14,17], with one experimental study finding an effect of ingredient-themed warnings on reducing e-cigarette cravings [17].
On the other hand, the use of strong e-cigarette warnings could have unintended consequences if they contribute to misperceptions that e-cigarettes are equally harmful as cigarettes, and discourage smokers from using them to reduce or quit smoking cigarettes. Nicotine-focused warnings could also potentially contribute to existing nicotine misperceptions (e.g., that nicotine causes cancer) [18,19,20]. Ideally, e-cigarette warnings should inform about risks and discourage initiation by non-smokers and young people without simultaneously increasing misperceptions and discouraging smokers from using them for smoking cessation and harm-reduction [14,21,22]. Following this, research evaluating the effectiveness of e-cigarette warnings should consider their impact on these different audiences.
One potential strategy for minimizing unintended effects could be to pair e-cigarette warnings with a relative harm (RH) statement, ultimately communicating that e-cigarettes may cause harm but are less harmful then combusted cigarettes. Limited research on this topic (mostly on smokeless tobacco) suggests that warning labels that include relative harm statements may improve relative harm perceptions [23,24], but also be viewed as less believable, credible and more ambiguous than standard/traditional warnings [25,26,27,28,29]. Another concern is that the inclusion of relative harm statements might unintentionally encourage use among non-smokers, which would constitute public health harm [24].
Because e-cigarette warnings are a new research area with potential intended and unintended effects, this study had several objectives. First we aimed to examine whether e-cigarette ads with the new larger standardized warnings would have greater impact on young adults’ e-cigarette harm, risk and addiction perceptions, and e-cigarette use intentions compared to e-cigarette ads with no or minimal unstandardized warnings. We also aimed to examine whether exposure to such ads would unintentionally increase misperceptions that e-cigarette and cigarettes are equally harmful, increase nicotine misperceptions, and reduce smokers’ intentions to use e-cigarettes for harm reduction.
Second, we aimed to examine whether warning themes other than nicotine addiction would have a different impact on these same outcomes, and whether other themes would be perceived by young adults to be as effective or more effective than the nicotine addiction warning.
Finally, we also aimed to explore the potential effects of pairing e-cigarette warnings with a relative harm statement. We hypothesized that those exposed to such warnings would have more accurate e-cigarette/cigarette relative harm perceptions and higher intentions to use e-cigarettes for harm-reduction among smokers. However, we also hypothesized that these warnings would be perceived as less effective and credible, more difficult to understand and recall, and would encourage use among non-smokers. We focused on young adults given their high prevalence of e-cigarette experimentation and priority as a target audience for such warnings.

2. Materials and Methods

2.1. Participants and Procedures

An online experiment was conducted between January–February 2018 with 876 young adults recruited through Amazon’s Mechanical Turk (mTurk; Table 1). Eligibility was limited to registered mTurk participants between the ages of 18–29 located in the United States who had an 80% approval rating from previous mTurk human intelligence tasks (HITs). After a large number of non-smokers initially completed the survey (with approximately 80 non-smokers per condition), we limited eligibility and access to the full survey to current smokers only. Smoking status was determined in the first few questions of the survey, which included distractor questions about other health behaviors (e.g., soda and alcohol consumption, exercise) to mask the focus of the survey. The study was advertised on mTurk as a Health Behavior Survey and the study description did not reveal the focus on e-cigarettes.
Eligible participants were randomly assigned to view e-cigarette ads in one of seven conditions in a 3 (warning theme) by 2 (warning type) between group factorial design with a control group. In the experimental groups, three “warning themes” i.e., (a) nicotine addiction; (b) nicotine’s impact on adolescent brain development; (c) presence of harmful chemicals), were each crossed with “warning type”, in which a relative-harm (RH) statement (“E-cigarettes may cause harm to health but are less harmful than cigarettes”) was either included (“RH warning”) or not included (“standard warning”) in the warning label. Participants in the control group viewed an unaltered set of control ads (see Figure A1).
Regarding themes, the addiction warning was the FDA’s pending nicotine addiction warning. The “brain” warning stated, “WARNING: This product contains nicotine. Nicotine can harm adolescent brain development, which continues into the early to mid-20s.” The “chemicals” warning stated: “WARNING: E-cigarette aerosol may contain chemicals that are harmful to the lungs and are known to cause cancer.” Message themes were based on formative research [6,14,27] and wording was adapted from the CDC’s e-cigarette fact sheet page at the time of the study [16].
Within each experimental group, participants viewed four different e-cigarette ads representing different leading brands (JUUL, Vuse, Blu, MarkTen), which were each manipulated to include the same warning message per their assigned condition. Warnings were sized at approximately 20% of the ad and placed in the upper portion of the ad, per new FDA e-cigarette warning requirements. Ads were presented in random order. Participants in the control groups viewed the same ads in their original un-manipulated form (i.e., as found in the marketplace) (Figure A1, Appendix A). Participants could view each ad for as long as they wished but each ad was displayed for a minimum of seven seconds. After exposure to all four ads, participants completed outcome measures and then viewed a screen with debriefing information corresponding to their condition. The median study completion time was ten minutes and participants were compensated with a $1.75 payment.

2.2. Measures

Before viewing the stimuli ads, participants answered questions about their demographics and tobacco use (Table 1). Ever e-cigarette users were defined as those who indicated ever trying an e-cigarette, even one or two times. Current smokers and current e-cigarette users were defined as those indicating they now smoke cigarettes/use e-cigarettes every day or some days, respectively. Immediately after viewing the stimuli, the main dependent variables were measured. Our outcomes of focus were informed by our research objectives and by the Message Impact Framework for tobacco warnings, which suggests that characteristics of warnings, and audiences’ perceptions about warnings may impact whether warnings are noticed and recalled, and may influence warnings’ impact on tobacco product beliefs and use intentions [30]. Question wording for the measures below is available in Supplemental Table S1.
E-cigarette Use Intentions. All respondents were asked about their likelihood of buying an e-cigarette in the next 6 months (1 = not at all–5 = extremely likely). We also measured smokers’ likelihood to use e-cigarettes for harm reduction by creating an average scale of three items asked of current smokers about their likelihood to use e-cigarettes to cut down on smoking, quit smoking completely, and likelihood of switching to e-cigarettes completely (α = 0.91) [31].
E-cigarette Beliefs. We assessed perceived absolute harm of e-cigarettes by asking respondents how harmful they believed daily e-cigarette use to be to a user’s health (1 = not at all–5 = extremely harmful) [12]. Next, we created a sum score of the perceived likelihood of disease (lung cancer, other lung disease, heart disease, oral cancer) (1 = not at all–5 = extremely likely) from daily e-cigarette use over a lifetime (α = 0.92) [32], where higher total scores represented higher perceived risk. Perceived harm of e-cigarettes compared to cigarettes was measured by asking how harmful to health daily e-cigarette use is compared to daily cigarette smoking (1 = a lot less harmful–5 = a lot more harmful) [12,33]. We also measured perceived likelihood of addiction to e-cigarettes (1 = not at all–5 = extremely likely).
Nicotine Beliefs. We measured the extent to which participants agreed that nicotine causes smoking related cancer as an indicator of nicotine misperceptions (1 = not at all–4 = very much) and also asked participants to indicate their overall perceived harmfulness of nicotine to health (see Table 2) [20].
Warning Recall. Participants in the experimental groups were asked to type what they could remember of the warning in the ads they viewed in as much detail as possible (unaided recall). Consistent with previous research [34,35], two reviewers independently coded the responses for reference to keywords or concepts that were present in the warnings viewed (Table 3; average Kappa = 0.9). Respondents were then also exposed to the text of all six warnings and asked to select which they recalled viewing on their stimulus ads (aided recall).
Perceived Warning Effectiveness, Credibility and Support: Participants in the experimental groups viewed the warning per their condition again and were asked the extent to which they agreed (1 = strongly disagreed–4 = strongly agreed) it was believable, understandable, informative and discouraged e-cigarette use among young people and smokers [36,37,38]. Perceived credibility of the warning was measured adapting Meyer’s credibility scale (5-point semantic differential items: e.g., accurate/inaccurate, biased/unbiased; α = 0.88) [12,39]. Finally participants were asked about the extent to which they agreed (1 = strongly disagreed–4 = strongly agreed) that e-cigarettes should carry warning labels about their potential risks. Results were dichotomized as agreed/disagreed.
Analysis: Because of the unbalanced experimental design (3 × 2 plus control), we first tested for mean difference across all seven warning conditions (including control) in the first set of outcomes, and for any interaction effects with smoking status (Model 1, Table 2). We then removed the control group and tested for any effects of warning theme or warning type when the experimental conditions were collapsed across all participants (Model 2, Table 2). For the perceived warning effectiveness outcomes (which were not assessed for the control group), we assessed the effects of warning theme and type, theme x type interactions and interactions with smoking status (Table 4). Any significant interaction effects across all analyses are noted in the results. We used ANOVA tests for semi-continuous outcomes (using Tukey’s post-hoc tests for pairwise comparisons and error-rate control) and Chi-square tests for categorical outcomes (e.g., message recall, warning support). To minimize the likelihood of Type 1 error from multiple testing, we used a more conservative significance level of 0.01 for all analyses. Analyses were completed using SPSS version 25 (SPSS Inc., Chicago, IL, USA).

3. Results

Sample description. Conditions did not significantly differ by participant demographics, cigarette smoking and e-cigarette use, indicating that randomization was successful. Overall, the sample was evenly split by sex, was predominantly non-Hispanic white (64.3%), and most were employed (75.1%) (Table 1). Participant average age was 25% and 51.1% had at least a college degree.
Over one-third (35.7%) were current smokers and over half of current smokers (54.5%) were current e-cigarette users. About 41% of all participants and 46.0% of smokers had ever noticed a warning on an e-cigarette ad or product package before.
E-cigarette use intentions. There was no significant difference in likelihood of buying e-cigarettes across all conditions (see Table 2, Model 1), nor by warning theme or type among the collapsed experimental conditions (Model 2). Among current smokers (n = 313), we also found no significant difference in mean likelihood of using e-cigarettes for harm reduction across all conditions (p = 0.27) (Table 2, Model 1). However, among the experimental conditions, the main effect for warning type approached significance (p = 0.012), with a higher likelihood of using e-cigarettes for harm reduction among smokers exposed to the ads with RH warnings (M = 2.93, SD = 1.1) relative to the standard warnings (M = 2.58, SD = 1.1) (data not in table).
E-cigarette Beliefs. There was no effect of warning condition (nor interaction with smoking status) on perceived absolute harm of e-cigarette use nor on perceived likelihood of disease from e-cigarette use (Table 2). For both outcomes, there were no differences by warning theme or type among the experimental conditions. Perceived harm of e-cigarettes compared to cigarettes was also not significantly associated with warning conditions, nor with warning theme or type when experimental conditions were collapsed (Table 2).
There was no effect of warning condition (nor interaction with smoking status) on perceived likelihood of addiction (Table 2, Model 1). When experimental conditions were collapsed by theme and type, the main effect of warning theme approached significance (p = 0.013, Table 2, Model 2), with scores higher for the FDA addiction warning (M = 3.65, SD = 1.0) relative to the “chemicals” warning (M = 3.38, SD = 0.9, p = 0.01), but not relative to the nicotine brain warning (M = 3.54, SD = 1.0).
Nicotine Beliefs. Over half of all participants (56.2%) incorrectly believed nicotine to be the chemical largely responsible for cancer caused by smoking and 54.4% rated nicotine as being very or extremely harmful to health. There was no significant mean differences for either of the nicotine beliefs by warning condition (Table 2) (nor interactions between condition and smoking status), nor by warning theme or type among the experimental conditions.
Warning Recall. Unaided recall of specific warning keywords/concepts was significantly lower in the RH warnings versus the standard versions of the nicotine addiction and brain warning themes (see Table 3). Among those exposed to the standard version of the FDA warning, most recalled the keywords/concepts of “nicotine” (84.1%) and “addiction” (66.7%). For the brain warning (standard version), recall of some reference to age (78%) was higher than for the keywords/concepts “nicotine” (69.3%) and “brain” (65.4%). For the “chemicals” warning, recall of “cancer” was more prevalent than “chemicals” and “lungs” (Table 3).
When aided recall/recognition was measured, there was no significant association between correct recall and warning theme but correct warning recall/recognition was lower among those who viewed the RH warnings (69.5%) versus the standard warnings (84.1%, p =< 0.001).
Perceived warning effectiveness, credibility and support. Most participants across conditions agreed that the warning they viewed was believable (89.3%), however mean believability was significantly higher for the standard versus the RH warnings (see Table 4). Agreement that the warning was understandable was also high (95.5%) and was not significantly associated with warning type nor theme. Overall, 82.7% of respondents agreed that the warning they viewed would make people more informed about e-cigarette health risks. Mean perceived informativeness did not significantly differ by theme or type (Table 4).
About 61% agreed that the warning they viewed could discourage young people from starting to use e-cigarettes. Mean agreement was significantly associated with warning type and was higher for the standard (M = 2.88) versus RH warnings (M = 2.45). Agreement was also associated with warning theme, and was lowest for the FDA addiction warning (Table 4). In contrast, fewer respondents overall (37.1%) agreed that the warning they viewed would discourage current smokers from using e-cigarettes, with significant main effects by warning theme and type (Table 4). Smoker discouragement was rated as higher for the standard warnings relative to the RH warnings, and was lowest for the addiction theme relative to the brain and chemicals themes.
For perceived warning credibility, only the main effect of warning type was significant (F = 23.92, R2 = 0.03, p < 0.001), with participants rating the standard warnings as more credible (M = 3.88, SD = 1.4) than the RH warnings (M = 3.34, SD = 1.5) (data not in table).
Warning Support. Finally, the vast majority (93.2%) agreed that e-cigarettes should carry warning labels about their potential risks. Support was slightly higher among non-smokers (95.0%) versus current smokers (90.0%) (p < 0.01) but did not significantly vary by warning condition.

4. Discussion

This study tested the impact of exposure to text warnings on e-cigarette ads in a new standardized format, the impact of different warning themes and the inclusion of a message comparing the harm of e-cigarettes relative to cigarettes. Overall, young adults expressed strong support for e-cigarette warnings and perceived them to be believable, understandable and informative about e-cigarette risks. However, we found few experimental effects of warning exposure on e-cigarette harm and risk perceptions and use intentions.
Consistent with a previous study with young adult non-smokers [12], we found no significant effect of exposure to the experimental warnings on participants’ e-cigarette harm and addiction beliefs, nor on decreasing e-cigarette use intentions among non-smokers when compared to ads with no or small unstandardized warnings. These null findings may suggest that text-only warnings (even large ones) may have limited impact with young adults when placed in otherwise colorful and interesting e-cigarette advertisements [12]. These findings, along with previous research [8,12,14,34], may suggest that the use of visual elements, such as colored warnings or some relevant pictorials, may be worth exploring given that they might enhance the noticeability, salience and recall of these warnings.
On the other hand, some of the null findings might be viewed as favorable outcomes. Exposure to the test warnings did not increase misperceptions about nicotine’s role in cancer or harmfulness, nor that e-cigarettes are as harmful as cigarettes. It also did not appear to reduce smokers’ intentions to use e-cigarettes for harm reduction. Together these findings may suggest that the new warning requirements may have minimal unintended consequences in terms of increasing product misperceptions or discouraging use of e-cigarettes for harm reduction.
Notably, this study also found that most participants perceived the experimental warnings, including the FDA’s addiction warning, to be informative about e-cigarette health risks. This is important given that this is the primary purpose of such warnings [8]. In this study unaided recall of the word “nicotine” was high among those exposed to the FDA’s nicotine addiction warning. This too is important given that research has continued to show that some young people remain unsure of whether the e-cigarettes they use contain nicotine or misperceive their products to be nicotine-free [40,41,42], underscoring a basic need for such clear and conspicuous labeling. In this group of young adults, the other warning themes tested (presence of chemicals and nicotine impact on adolescent brain development) did not significantly outperform the FDA’s addiction warning on any of the main experimental outcome measures including e-cigarette harm/risk perceptions and buying likelihood. Together, these findings support the FDA’s use of the nicotine addiction theme as a starting point for e-cigarette warnings.
However, the nicotine addiction warning trailed behind the other warning themes in terms of its perceived ability to discourage e-cigarette use among young people. Participants also found the “brain” and “chemicals” warnings to be as believable, informative and credible as the nicotine addiction warning. These findings suggest that these warning themes may deserve additional research attention as potential future warnings, particularly for youth audiences. Indeed, a recent e-cigarette study with adolescents found that while 83% knew that e-cigarettes usually contain nicotine, far fewer knew that e-cigarettes contain harmful chemicals (67%) and might harm teen brain development (49%) [43].
This study also explored the impact of including a relative harm statement within the warning label, as has previously been proposed by smokeless tobacco companies and could similarly be proposed for e-cigarettes in the future. Given the calls for increased communications about the relative harms of different tobacco products, it was encouraging to find that these messages were not perceived as being less understandable than the standard warnings, and were not associated with higher e-cigarette use intentions among non-smokers, in contrast to our hypotheses. However, consistent with prior research on this topic [25,26,27,28,29] and our hypotheses, these warnings were perceived as less believable and credible, and also as less likely to discourage e-cigarette use among young people. In line with our expectation, inclusion of the relative harm statements in the warning label also appeared to reduce recall of the risk-related part of the message, potentially because of the increased length of the final overall message and a recency effect of the relative harm information at the end of the message. Future research should continue to examine the effects of various relative harm/modified risk tobacco messages, including appropriate wording, locations and formats that maximize potential public health benefits and minimize unintended consequences. Future research on this topic, as well as on e-cigarette warning labels more generally, should also be examined with more representative samples of participants, as participants in mTurk convenience samples have shown to generally be younger, more educated, have above average cognitive aptitude and be more likely to be white and Asian compared to the general population [44].

5. Conclusions

Findings from this study suggest that warning labels with messages about harms beyond nicotine’s addictiveness are perceived as believable, informative, understandable and credible among young adults, may provide novel information, and may discourage e-cigarette use among young people. While inclusion of a relative harm statement in e-cigarette warnings may not unintentionally increase use by non-smokers, it may reduce recall of warning messages in young adults. Further research should continue to identify warning themes, types and execution formats that accurately inform audiences about e-cigarettes.

Supplementary Materials

The following are available online at, Table S1: Appendix of Study Measures.

Author Contributions

O.A.W. designed and obtained funding for the study and led study implementation, data collection, data analysis and manuscript writing. J.M.S. assisted in design and implementation of the study, data collection and data analysis. P.A.O.-S. assisted with data analysis and results presentation. D.H., R.J.O., A.A.S. and C.D.D. provided substantive input into the study design, experimental instrument and analysis plan, and A.C.V. contributed to results interpretation and manuscript editing. All authors contributed to reviewing and editing drafts of the manuscript and approved the final manuscript.


This work was supported by a grant (K01CA189301) from the National Cancer Institute (NCI) and the Food and Drug Administration Center for Tobacco Products. Contributions by OAW, ROJ and CDD were also supported in part by R37CA222002 from NCI. ACV was supported by the NCI under Award Number R03CA212694 and the Centers of Biomedical Research Excellence P20GM103644 award from the National Institute of General Medical Sciences.


Thank you to Erin Miller Lo for help with data collection activities.

Conflicts of Interest

The authors declare no conflict of interest. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations.

Appendix A

Figure A1. Manipulation of stimuli ads, by condition. NOTE: All four e-cigarette brand ads were viewed in each condition, each containing the same warning, respectively.
Figure A1. Manipulation of stimuli ads, by condition. NOTE: All four e-cigarette brand ads were viewed in each condition, each containing the same warning, respectively.
Ijerph 16 00184 g0a1


  1. National Academies of Sciences, Engineering, and Medicine. Public Health Consequences of E-Cigarettes; The National Academies Press: Washington, DC, USA, 2018; Available online: (accessed on 22 December 2018). [CrossRef]
  2. McNeill, A.; Brose, L.S.; Calder, R.; Hitchman, S.C.; Hajek, P.; McRobbie, H. E-Cigarettes: An Evidence Update (A Report Commissioned by Public Health England). Available online: (accessed on 31 May 2017).
  3. Wackowski, O.A.; Bover Manderski, M.T.; Delnevo, C.D. Smokers’ sources of e-cigarette awareness and risk information. Prev. Med. Rep. 2015, 2, 906–910. [Google Scholar] [CrossRef] [PubMed]
  4. Thrasher, J.F.; Brewer, N.T.; Niederdeppe, J.; Peters, E.; Strasser, A.A.; Grana, R.; Kaufman, A.R. Advancing Tobacco Product Warning Labels Research Methods and Theory: A Summary of a Grantee Meeting Held by the US National Cancer Institute. Nicot. Tob. Res. 2018. [Google Scholar] [CrossRef] [PubMed]
  5. Shang, C.; Chaloupka, F.J. The Trend of Voluntary Warnings in Electronic Nicotine Delivery System Magazine Advertisements. Int. J. Environ. Res. Public Health 2017, 14, 62. [Google Scholar] [CrossRef] [PubMed]
  6. Wackowski, O.A.; Hammond, D.; O’Connor, R.J.; Strasser, A.A.; Delnevo, C.D. Smokers’ and E-Cigarette Users’ Perceptions about E-Cigarette Warning Statements. Int. J. Environ. Res. Public Health 2016, 13, 655. [Google Scholar] [CrossRef]
  7. Wackowski, O.A.; Lewis, M.J. E-cigarette brand mocks tobacco control warning labels. Tob. Control 2017. [Google Scholar] [CrossRef]
  8. Food and Drug Administration. Deeming Tobacco Products To Be Subject to the Federal Food, Drug, and Cosmetic Act, as Amended by the Family Smoking Prevention and Tobacco Control Act; Regulations on the Sale and Distribution of Tobacco Products and Required Warning Statements for Tobacco Products. Final rule. Federal Register; 10 May 2016. Available online: (accessed on 22 May 2017).
  9. Noar, S.M.; Francis, D.B.; Bridges, C.; Sontag, J.M.; Brewer, N.T.; Ribisl, K.M. Effects of Strengthening Cigarette Pack Warnings on Attention and Message Processing: A Systematic Review. J. Mass Commun. Q. 2017, 94, 416–442. [Google Scholar] [CrossRef] [PubMed]
  10. Noar, S.M.; Francis, D.B.; Bridges, C.; Sontag, J.M.; Ribisl, K.M.; Brewer, N.T. The impact of strengthening cigarette pack warnings: Systematic review of longitudinal observational studies. Soc. Sci. Med. 2016, 164, 118–129. [Google Scholar] [CrossRef][Green Version]
  11. Berry, C.; Burton, S.; Howlett, E. Are Cigarette Smokers’, E-Cigarette Users’, and Dual Users’ Health-Risk Beliefs and Responses to Advertising Influenced by Addiction Warnings and Product Type? Nicot. Tob. Res. 2017, 19, 1185–1191. [Google Scholar] [CrossRef]
  12. Mays, D.; Smith, C.; Johnson, A.C.; Tercyak, K.P.; Niaura, R.S. An experimental study of the effects of electronic cigarette warnings on young adult nonsmokers’ perceptions and behavioral intentions. Tob. Induc. Dis. 2016, 14, 17. [Google Scholar] [CrossRef]
  13. Shang, C.; Huang, J.; Chaloupka, F.J.; Emery, S.L. The impact of flavour, device type and warning messages on youth preferences for electronic nicotine delivery systems: Evidence from an online discrete choice experiment. Tob. Control 2017. [Google Scholar] [CrossRef]
  14. Wackowski, O.A.; Hammond, D.; O’Connor, R.J.; Strasser, A.A.; Delnevo, C.D. Considerations and Future Research Directions for E-Cigarette Warnings-Findings from Expert Interviews. Int. J. Environ. Res. Public Health 2017, 14, 781. [Google Scholar] [CrossRef] [PubMed]
  15. US Department of Health and Human Services. E-Cigarette Use among Youth and Young Adults: A Report of the Surgeon General; US Department of Health and Human Services, CDC: Atlanta, GA, USA, 2016.
  16. Centers for Disease Control and Prevention. E-Cigarettes. Fact Sheet. Available online: (accessed on 22 December 2018).
  17. Sanders-Jackson, A.; Schleicher, N.C.; Fortmann, S.P.; Henriksen, L. Effect of warning statements in e-cigarette advertisements: An experiment with young adults in the United States. Addiction 2015, 110, 2015–2024. [Google Scholar] [CrossRef] [PubMed]
  18. Bansal, M.A.; Cummings, K.M.; Hyland, A.; Giovino, G.A. Stop-smoking medications: Who uses them, who misuses them, and who is misinformed about them? Nicot. Tob. Res. 2004, 6, 303–310. [Google Scholar] [CrossRef]
  19. Mooney, M.E.; Leventhal, A.M.; Hatsukami, D.K. Attitudes and knowledge about nicotine and nicotine replacement therapy. Nicot. Tob. Res. 2006, 8, 435–446. [Google Scholar] [CrossRef] [PubMed]
  20. O’Brien, E.K.; Nguyen, A.B.; Persoskie, A.; Hoffman, A.C. US adults’ addiction and harm beliefs about nicotine and low nicotine cigarettes. Prev. Med. 2017, 96, 94–100. [Google Scholar] [CrossRef] [PubMed]
  21. Cox, S.; Frings, D.; Ahmed, R.; Dawkins, L. Messages matter: The Tobacco Products Directive nicotine addiction health warning versus an alternative relative risk message on smokers’ willingness to use and purchase an electronic cigarette. Addict. Behav. Rep. 2018, 8, 136–139. [Google Scholar] [CrossRef]
  22. Mendel, J.R.; Hall, M.G.; Baig, S.A.; Jeong, M.; Brewer, N.T. Placing health warnings on e-cigarettes: A standardized protocol. Int. J. Environ. Res. Public Health 2018, 15, 1578. [Google Scholar] [CrossRef]
  23. Callery, W.E.; Hammond, D.; O’Connor, R.J.; Fong, G.T. The appeal of smokeless tobacco products among young Canadian smokers: The impact of pictorial health warnings and relative risk messages. Nicot. Tob. Res. 2011, 13, 373–383. [Google Scholar] [CrossRef]
  24. Mays, D.; Moran, M.B.; Levy, D.T.; Niaura, R.S. The Impact of Health Warning Labels for Swedish Snus Advertisements on Young Adults’ Snus Perceptions and Behavioral Intentions. Nicot. Tob. Res. 2016, 18, 1371–1375. [Google Scholar] [CrossRef]
  25. Katz, S.J.; Lindgren, B.; Hatsukami, D. E-cigarette warning labels and modified risk statements: Tests of messages to reduce recreational use. Tob. Regul. Sci. 2017, 3, 445–458. [Google Scholar] [CrossRef]
  26. Rodu, B.; Plurphanswat, N.; Hughes, J.R.; Fagerstrom, K. Associations of Proposed Relative-Risk Warning Labels for Snus With Perceptions and Behavioral Intentions Among Tobacco Users and Nonusers. Nicot. Tob. Res. 2016, 18, 809–816. [Google Scholar] [CrossRef] [PubMed]
  27. Wackowski, O.A.; O’Connor, R.J.; Strasser, A.A.; Hammond, D.; Villanti, A.C.; Delnevo, C.D. Smokers’ and e-cigarette users’ perceptions of modified risk warnings for e-cigarettes. Prev. Med. Rep. 2016, 4, 309–312. [Google Scholar] [CrossRef]
  28. Fix, B.V.; Adkison, S.E.; O’Connor, R.J.; Bansal-Travers, M.; Cummings, K.M.; Rees, V.W.; Hatsukami, D.K. Evaluation of modified risk claim advertising formats for Camel Snus. Health Educ. J. 2017, 76, 971–985. [Google Scholar] [CrossRef]
  29. Berry, C.; Burton, S. Reduced-Risk Warnings versus the U.S. FDA-Mandated Addiction Warning: The Effects of E-Cigarette Warning Variations on Health Risk Perceptions. Nicot. Tob. Res. 2018. [Google Scholar] [CrossRef]
  30. Noar, S.M.; Hall, M.G.; Francis, D.B.; Ribisl, K.M.; Pepper, J.K.; Brewer, N.T. Pictorial cigarette pack warnings: A meta-analysis of experimental studies. Tob. Control 2016, 25, 341–354. [Google Scholar] [CrossRef]
  31. Popova, L.; Neilands, T.B.; Ling, P.M. Testing messages to reduce smokers’ openness to using novel smokeless tobacco products. Tob. Control 2014, 23, 313–321. [Google Scholar] [CrossRef] [PubMed]
  32. Pepper, J.K.; Emery, S.L.; Ribisl, K.M.; Rini, C.M.; Brewer, N.T. How risky is it to use e-cigarettes? Smokers’ beliefs about their health risks from using novel and traditional tobacco products. J. Behav. Med. 2015, 38, 318–326. [Google Scholar] [CrossRef]
  33. Persoskie, A.; Nguyen, A.B.; Kaufman, A.R.; Tworek, C. Criterion validity of measures of perceived relative harm of e-cigarettes and smokeless tobacco compared to cigarettes. Addict. Behav. 2017, 67, 100–105. [Google Scholar] [CrossRef][Green Version]
  34. Mays, D.; Villanti, A.; Niaura, R.S.; Lindblom, E.N.; Strasser, A.A. The Effects of Varying Electronic Cigarette Warning Label Design Features On Attention, Recall, and Product Perceptions Among Young Adults. Health Commun. 2017, 1–8. [Google Scholar] [CrossRef] [PubMed]
  35. Strasser, A.A.; Tang, K.Z.; Romer, D.; Jepson, C.; Cappella, J.N. Graphic warning labels in cigarette advertisements: Recall and viewing patterns. Am. J. Prev. Med. 2012, 43, 41–47. [Google Scholar] [CrossRef]
  36. Lee, H.Y.; Lin, H.C.; Seo, D.C.; Lohrmann, D.K. The effect of e-cigarette warning labels on college students’ perception of e-cigarettes and intention to use e-cigarettes. Addict. Behav. 2018, 76, 106–112. [Google Scholar] [CrossRef] [PubMed]
  37. Lee, Y.O.; Shafer, P.R.; Eggers, M.E.; Kim, A.E.; Parvanta, S.A.; Nonnemaker, J.M. Effect of a voluntary e-cigarette warning label on risk perceptions. Tob. Regul. Sci. 2016, 2, 82–93. [Google Scholar] [CrossRef]
  38. Thrasher, J.F.; Arillo-Santillan, E.; Villalobos, V.; Perez-Hernandez, R.; Hammond, D.; Carter, J.; Sebrie, E.; Sansores, R.; Regalado-Pineda, J. Can pictorial warning labels on cigarette packages address smoking-related health disparities? Field experiments in Mexico to assess pictorial warning label content. Cancer Cause Control 2012, 23, 69–80. [Google Scholar] [CrossRef] [PubMed][Green Version]
  39. Meyer, P. Defining and measuring credibility of newspapers: Developing an index. Journal. Q. 1988, 65, 567–588. [Google Scholar] [CrossRef]
  40. Truth Initiative. A Lot of Youth Think They Aren’t Vaping Nicotine, But Is That True? 2017. Available online: (accessed on 22 December 2018).
  41. Pepper, J.K.; Farrelly, M.C.; Watson, K.A. Adolescents’ understanding and use of nicotine in e-cigarettes. Addict. Behav. 2018, 82, 109–113. [Google Scholar] [CrossRef] [PubMed]
  42. Willett, J.G.; Bennett, M.; Hair, E.C.; Xiao, H.; Greenberg, M.S.; Harvey, E.; Cantrell, J.; Vallone, D. Recognition, use and perceptions of JUUL among youth and young adults. Tob. Control 2018. [Google Scholar] [CrossRef] [PubMed]
  43. Rohde, J.A.; Noar, S.M.; Horvitz, C.; Lazard, A.J.; Cornacchione Ross, J.; Sutfin, E.L. The Role of Knowledge and Risk Beliefs in Adolescent E-Cigarette Use: A Pilot Study. Int. J. Environ. Res. Public Health 2018, 15, 830. [Google Scholar] [CrossRef]
  44. Chandler, J.; Shapiro, D. Conducting clinical research using crowdsourced convenience samples. Annu. Rev. Clin. Psychol. 2016, 12, 53–81. [Google Scholar] [CrossRef]
Table 1. Participant demographics and tobacco use (n = 876).
Table 1. Participant demographics and tobacco use (n = 876).
 Non-Hispanic white56364.3
 Non-Hispanic black9310.6
 Other/mixed race414.7
 High school of less11513.1
 At least some college31235.6
 College degree or more44851.1
 Employed for wages51458.9
 Out of work697.9
Average Age 25.0
E-cigarette Status
 Ever e-cigarette user53461.0
 Current e-cigarette user22926.2
Smoking Status
 Current smoker 31335.7
 Non-smoker 56364.3
Table 2. Impact of Experimental Conditions on E-cigarette Use Intentions, E-cigarette Beliefs and Nicotine Beliefs (n = 876).
Table 2. Impact of Experimental Conditions on E-cigarette Use Intentions, E-cigarette Beliefs and Nicotine Beliefs (n = 876).
Model 1
(Across All Conditions Including Control) a
Model 2
(Experimental Groups Only) b
Main EffectsInteraction Main EffectsInteraction
Warning Condition x Smoker Warning ThemeWarning
Warning Theme x Type
F, η2, pF, η2, pF, η2, p F, η2, pF, η2, pF, η2, p
Likelihood of buying e-cigarettes 11.47, 0.010, 0.19284.46, 0.25, <0.0012.76, 0.019, 0.012 0.31, 0.001, 0.730.99, 0.001, 0.321.55, 0.004, 0.21
Likelihood of using e-cigarettes for harm reduction 1
(among current smokers only)
1.27, 0.024, 0.27 0.45,0 0.003, 0.646.35, 0.024, 0.0120.06, <0.001, 0.95
Perceived absolute harm of e-cigarettes 20.45, 0.003, 0.8438.21, 0.042, <0.0011.43, 0.010, 0.201 0.093, <0.001, 0.911.28, 0.002, 0.260.02, 0.000, 0.98
Perceived likelihood of disease from e-cigarette use 11.09 0.007, 0.3710.62, 0.012, 0.0011.05, 0.007, 0.39 0.36, 0.001, 0.700.57, 0.001, 0.461.41, 0.004, 0.25
Perceived harm of e-cigarettes compared to cigarettes 30.82, 0.006, 0.550.015, <0.001, 0.921.36, 0.009, 0.23 0.39, 0.001, 0.681.10, 0.001, 0.300.94, 0.003, 0.39
Perceived likelihood of addiction to e-cigarettes 12.17, 0.015, 0.04310.99, 0.013, 0.0010.87, 0.006, 0.51 4.39, 0.012, 0.0130.16, <0.001, 0.690.89, 0.002, 0.41
Agreement that nicotine causes smoking related cancer 40.83, 0.006, 0.553.18, 0.004, 0.0750.36, 0.003, 0.91 1.69, 0.005, 0.190.09, <0.001, 0.760.18, 0.000, 0.84
Perceived harmfulness of nicotine to health 20.16, 0.001, 0.9919.31, 0.022, <0.0011.06, 0.007, 0.39 0.046, <0.001, 0.960.008, <0.001, 0.930.30, 0.001, 0.74
Mean (Standard Deviation), by condition
AddictionAddiction + RHBrainBrain + RHChemicalsChem. + RHControl
E-cigarette Use Intentions
Likelihood of buying e-cigarettes 11.83 (1.1)1.84 (1.1)1.77 (1.1)2.06 (1.2)1.89 (1.2)1.83 (1.2)1.84 (1.0)
Likelihood of using e-cigarettes for harm reduction (among current smokers) 12.52 (1.2)2.81 (1.1)2.61 (1.2)2.98 (1.1)2.61 (1.1)3.0 (1.1)2.89 (1.3)
E-cigarette and Nicotine Beliefs
Perceived absolute harm of e-cigarettes 22.29 (1.0)2.20 (0.9)2.2 (0.9)2.26 (0.9)2.26 (0.9)2.10 (0.8)2.13 (0.9)
Perceived likelihood of disease from e-cigarette use 112.2 (3.9)12.5 (3.8)12.1 (3.6)12.0 (3.9)12.6 (3.9)11.8 (3.7)11.6 (4.1)
Perceived harm of e-cigarettes compared to cigarettes 32.29 (1.0)2.20 (0.9)2.20 (0.9)2.26 (1.0)2.26 (0.9)2.10 (0.8)2.13 (0.9)
Perceived likelihood of addiction to e-cigarettes 13.64 (1.1)3.65 (1.0)3.5 (1.0)3.57 (1.0)3.46 (1.0)3.30 (1.0)3.4 (1.0)
Agreement that nicotine causes smoking related cancer 42.51 (1.0)2.54 (1.0)2.68 (1.0)2.65 (1.0)2.71 (1.0)2.63 (1.0)2.72 (0.9)
Perceived harmfulness of nicotine to health 23.57 (1.0)3.63 (1.0)3.62 (1.1)3.62 (1.0)3.66 (1.0)3.58 (1.1)3.57 (1.1)
a Model 1 based on Anova analysis for mean differences across all seven warning conditions (including control), with test for interaction with smoking status; b Model 2 based on two-way Anova analysis for warning theme, warning type and warning theme x type interaction among experimental conditions (control group excluded). F = F statistic η2—eta squared, p = p-value from Anova test. 1 (1 = not at all–5 = extremely likely), 2 (1 = not at all–5 = extremely harmful), 3 (1 = a lot less harmful–5 = a lot more harmful), 4 (1 = not at all—4 = very much).
Table 3. Unaided recall of warning keywords and concepts by warning theme and type.
Table 3. Unaided recall of warning keywords and concepts by warning theme and type.
% Correctly Recalling Keywords/Concepts for: Warning Typep-Value *
StandardRH Warning
FDA Nicotine Addiction Warning
 E-cigarettes are less harmful than cigarettes 58.1
Nicotine Brain Development Warning
 E-cigarettes are less harmful than cigarettes 36.0
Harmful Chemicals Warning
 E-cigarettes are less harmful than cigarettes 56.1
* p-values based on Chi-square tests between warning types and recall of each keyword/concept.
Table 4. Perceived effectiveness of e-cigarette warnings.
Table 4. Perceived effectiveness of e-cigarette warnings.
Focal Statistics for Anova Tests
Young People
Discourage Smokers
F, η2, pF, η2, pF, η2, pF, η2, pF, η2, p
Main Effect
 Warning type24.32, 0.032, <0.0015.00, 0.007, 0.0266.05, 0.008, 0.01440.44, 0.051, <0.00120.07, 0.026, <0.001
 Warning theme3.43, 0.009, 0.0331.36, 0.004, 0.264.14, 0.011, 0.01613.30, 0.034, <0.0016.06, 0.016, 0.002
 Smoking status0.87, 0.001, 0.351.47, 0.002, 0.230.19, <0.001, 0.660.022, <0.001, 0.8812.07, 0.016, 0.001
Interaction Effects
 Warning type x theme4.01, 0.011, 0.0183.58, 0.010, 0.0280.68, 0.002, 0.5080.63, 0.002, 0.5331.08, 0.003, 0.339
 Warning type x smoking status 0, <0.001, 0.9890.05, <0.001, 0.8260.16, <0.001, 0.6930.36, <0.001, 0.5490.01, <0.001, 0.927
 Warning theme x smoking status3.11, 0.008, 0.0453.6, 0.010, 0.0280.88, 0.002, 0.4150.19, 0.001, 0.8280.17, <0.001, 0.844
 Warning type x warning theme x smoking status 2.25, 0.006, 0.1063.49, 0.009, 0.0310.5, 0.001, 0.6040.03, <0.001, 0.9750.01, <0.001, 0.995
Overall Mean Agreement
(Across Groups)
Mean Agreement
(St. Deviation) by
Warning Theme
Mean Agreement
(St. Deviation) by
Warning Type
Agreed the warning they viewed: FDABrainChemicalsStandardRH
 Is believable3.20 (0.7)3.31 (0.7) 3.17 (0.7) 3.17 (0.8) 3.35 (0.7)3.10 (0.7)
 Easy to understand3.41 (0.6)3.45 (0.6)3.36 (0.6)3.40 (0.6)3.46 (0.6)3.36 (0.6)
 Would inform people about e-cigarette risks3.01 (0.7)2.92 (0.8)3.00 (0.7)3.1 (0.7)3.07 (0.7)2.94 (0.7)
 Would discourage young people from starting e-cigarettes2.69 (0.9)2.47 (0.9) b,c2.86 (0.8) f2.73 (0.8) f2.88 (0.8)2.45 (0.9)
 Would discourage current smokers from using e-cigarettes2.27 (0.9) 2.12 (0.9) b2.36 (0.9) f2.33 (0.9)2.41 (0.9)2.12 (0.9)
Note: F = F statistic, η2 = partial eta squared, p = p-value for F statistic. f = significantly different than FDA nicotine theme, b = significantly different than Brain theme, c = significantly different than Chemicals theme, with significance level set at p = 0.01.

Share and Cite

MDPI and ACS Style

Wackowski, O.A.; Sontag, J.M.; Hammond, D.; O’Connor, R.J.; Ohman-Strickland, P.A.; Strasser, A.A.; Villanti, A.C.; Delnevo, C.D. The Impact of E-Cigarette Warnings, Warning Themes and Inclusion of Relative Harm Statements on Young Adults’ E-Cigarette Perceptions and Use Intentions. Int. J. Environ. Res. Public Health 2019, 16, 184.

AMA Style

Wackowski OA, Sontag JM, Hammond D, O’Connor RJ, Ohman-Strickland PA, Strasser AA, Villanti AC, Delnevo CD. The Impact of E-Cigarette Warnings, Warning Themes and Inclusion of Relative Harm Statements on Young Adults’ E-Cigarette Perceptions and Use Intentions. International Journal of Environmental Research and Public Health. 2019; 16(2):184.

Chicago/Turabian Style

Wackowski, Olivia A., Jennah M. Sontag, David Hammond, Richard J. O’Connor, Pamela A. Ohman-Strickland, Andrew A. Strasser, Andrea C. Villanti, and Cristine D. Delnevo. 2019. "The Impact of E-Cigarette Warnings, Warning Themes and Inclusion of Relative Harm Statements on Young Adults’ E-Cigarette Perceptions and Use Intentions" International Journal of Environmental Research and Public Health 16, no. 2: 184.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop