Next Article in Journal
Post-Traumatic Stress Disorder (PTSD) Symptoms in Secondary Stalked Children of Danish Stalking Survivors—A Pilot Study
Next Article in Special Issue
Electronic Cigarette Use and Smoking Initiation in Taiwan: Evidence from the First Prospective Study in Asia
Previous Article in Journal
When Less is More: Vaping Low-Nicotine vs. High-Nicotine E-Liquid is Compensated by Increased Wattage and Higher Liquid Consumption
Previous Article in Special Issue
Where Do Vapers Buy Their Vaping Supplies? Findings from the International Tobacco Control (ITC) 4 Country Smoking and Vaping Survey
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 16
Issue 5
10.3390/ijerph16050724
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Comparison of E-Cigarette Use Patterns and Smoking Cessation Behavior among Vapers by Primary Place of Purchase

by
Greta Hsu
1,
Anthony C. Gamst
2,3,
Yue-Lin Zhuang
4,
Tanya Wolfson
3,4 and
Shu-Hong Zhu
4,5,*
1
Graduate School of Management, University of California, Davis, Davis, CA 95616, USA
2
Department of Mathematics, University of California San Diego, La Jolla, CA 92093-0112, USA
3
San Diego Supercomputer Center, University of California San Diego, La Jolla, CA 92093-0505, USA
4
Moores Cancer Center, University of California San Diego, La Jolla, CA 92093-0905, USA
5
Department of Family Medicine and Public Health, University of California San Diego, La Jolla, CA 92093-0905, USA
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2019, 16(5), 724; https://doi.org/10.3390/ijerph16050724
Submission received: 12 January 2019 / Revised: 22 February 2019 / Accepted: 25 February 2019 / Published: 28 February 2019
(This article belongs to the Special Issue Electronic Cigarettes: Good and Bad Impacts)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background: E-cigarettes are purchased through multiple channels, including general retail, online, and specialty smoke and vape shops. We examine how e-cigarette users’ primary purchase place relates to e-cigarette use and smoking cessation behaviors. Methods: Probability-based samples of the U.S. population who were current e-cigarette users were surveyed in 2014 (N = 879) and 2016 (N = 743), with responses combined for most analyses. E-cigarette use and smoking cessation behaviors were compared across users’ primary purchase place. Results: Higher percentages of vape shop (59.1%) and internet (42.9%) customers were current daily users of e-cigarettes compared to retail (19.7%) and smoke shop (23.2%) customers (p-values < 0.001). Higher percentages of vape shop (40.2%) and internet (35.1%) customers were also former smokers, compared to 17.7% of retail and 19.3% of smoke shop customers (p’s < 0.001). Among those smoking 12 months prior to survey, smoking cessation rates were higher for vape shop (22.2%) and internet customers (22.5%) than for retail customers (10.7%, p = 0.010 and p = 0.022, respectively), even though retail customers were more likely to use FDA-approved smoking cessation aids. The percentage of customers purchasing from vape shops increased from 20.4% in 2014 to 37.6% in 2016, surpassing general retail (27.7%) as the most likely channel in 2016. Conclusions: E-cigarette customers differed in significant ways by channels of purchase, most notably in their smoking cessation behaviors. Previous population studies have relied mostly on retail channel data, which accounted for less than 30% of all products sold by 2016. Future studies of e-cigarette use should consider a broader set of channels.

1. Introduction

E-cigarettes can be purchased through several channels in the U.S., including general retail locations (e.g., supermarkets, convenience stores), online stores, and specialty smoke and vape shops. Each appear to be major outlets for e-cigarettes, although sales through internet and specialty shops have generally not been well-tracked and channels are often combined differently within different industry surveys [1,2]. For example, Coleman et al. estimated that, in 2013–2014, 29% of established e-cigarette users purchased e-cigarette products from convenience stores/gas stations, while 56% purchased from a tobacco/smoke or vape shop, and 14% from the internet [3]. Braak et al. found that in 2016, roughly 43.3% of U.S. e-cigarette users purchased vaping products through a vape shop, 26.8% purchased online, and 29.9% purchased through “other retail” channels (including tobacco specialty shops, supermarkets, pharmacy, and convenience stores) [4]. In recent years, a proliferation of vape shops in the U.S. further indicates their increasing position as a source of e-cigarette purchases [5,6].
Gaining an accurate picture of e-cigarette sales across channels is important for understanding this young industry’s evolution. It is also important, from a public health viewpoint, to understand how each channel relates to e-cigarette users’ behavior with regards to cigarette smoking and intentions to quit. Qualitative evidence suggests different purchase channels expose e-cigarette consumers to different types of information about electronic nicotine delivery systems (ENDS). Vape shops appear particularly distinctive in this regard. Many vape shop owners and staff are former cigarette smokers who—despite lack of formal training—provide testimonials, guidance, and informal counseling to customers interested in smoking cessation [5,7,8,9,10]. Vape shops also encourage the development of a community of e-cigarette enthusiasts by hosting social events and providing a venue for users to socialize and share information [7,10,11,12]. The different roles vape shops and other outlets play in tobacco control, whether positive or negative, deserves study.
This study has two objectives. First, it aims to develop understanding of whether e-cigarette users’ primary place of purchase is related to their e-cigarette use patterns, smoking quit attempts, and likelihood of smoking cessation. To date, there has been little research comparing user behaviors across multiple e-cigarette purchase channels. Importantly, the present study distinguishes between vape shops and smoke shops that carry vaping products. While industry surveys commonly combine the two, qualitative studies suggest that vape shops often present different beliefs about cigarettes (such as anti-smoking views) compared to smoke shops that carry both vaping and smoking products [5,9,13]. The two forms may thus be associated with different types of smoking behavior.
A secondary aim is to obtain a more complete picture of the distribution of e-cigarette sales across the different major channels and how this evolved from 2014 to 2016. Market research firms such as Nielsen focus on traditional retail channels such as convenience stores, g’as stations, and drug stores and do not track e-cigarette sales conducted online or through specialty tobacco and vape shops [2]. Given that these appear to be major channels for e-cigarette purchases in the U.S. [3,4], this study instead tries to obtain relative estimates for sales directly from a representative sample of e-cigarette users.
This study is based on two surveys of e-cigarette users—the first conducted in 2014 and the second in 2016. Both surveys are based on a nationally representative sample of U.S. adults.

2. Materials and Methods

2.1. Data Source

The 2014 and 2016 surveys were both drawn from GfK’s KnowledgePanel, an ongoing probability-based panel that relies on address-based sampling from a sample frame of residential addresses that covers approximately 97% of U.S. households [14]. This panel was first developed in 1999 by Knowledge Networks, an online consumer product and service research firm acquired by the market research organization GfK. This sample includes individuals with unlisted phone numbers and without landlines or Internet access; the latter were provided a computer with WiFi to complete the survey. We increased sampling efficiency by oversampling ever smokers (because the majority of e-cigarettes users are ever smokers [15,16]). Out of 9334 invited adults, 8619 completed the 2014 survey, yielding a 92.3% response rate. Out of 11,509 people, 10,314 completed the 2016 survey—an 89.5% response rate. Because our aim was to gather information on current e-cigarette users’ purchase and smoking-related behaviors, we only included in our sample those respondents who reported their primary place of purchase. Our analyses focus on 879 current e-cigarette users from the 2014 survey and 743 from the 2016 survey.
This research was approved by the University of California, San Diego’s Institutional Review Board (IRB #111664). Participants provided informed consent.

2.2. Measures

‘E-cigarette users’ were defined as respondents who answered using e-cigarettes “every day” or “some days” to the question: “Do you currently use e-cigarettes every day, some days or not at all?” To determine e-cigarette users’ ‘primary purchase channel’, they were asked, “Where do you usually buy your e-cigarettes or e-liquid?” Responses to the question are: “Internet,” “Pharmacy,” “Gas station/convenience store,” “Grocery store/supermarket,” “Smoke shop,” “E-cigarette/vape shop,” “Other,” and “I have never bought e-cigarettes.” Retail locations include “Pharmacy,” “Gas station/convenience store,” and “Grocery story/supermarket.” Those who answered “other” or “I have never bought e-cigarettes” were excluded from the analysis.
We defined an ‘ever smoker’ as having smoked ≥100 cigarettes in one’s lifetime. Ever smokers were further broken into two groups: ‘current smokers’ (ever smoker, smoked every day or some days at time of survey) and ‘former smokers’ (did not smoke at time of survey). All ever smokers were asked whether they were smoking every day or some days 12 months prior to the time of survey.
To determine e-cigarette users’ current e-cigarette model in 2014, we presented respondents with photos of different systems and asked a series of questions. Each question was accompanied by photos with descriptive captions to illustrate the types of e-cigarette systems referred to. ‘Closed system users’ were those who answered “Yes” to one of the following two questions:
1. “Do you currently use disposable e-cigarettes?”; caption: “Looks like a cigarette; one piece.”
2. “Do you currently use a 2-piece (or 3-piece) e-cigarette similar to the ones shown below (not (personal vaporizer) PV or (advanced personal vaporizer) APV)?”; caption: “2 or 3 piece; consists of battery and cartomizer (or cartridge and atomizer).”
Open system users were those who answered “Yes” to one of the following two questions:
1. “Do you currently use a PV or APV e-cigarette (e.g., eGo) similar to the ones shown below?”; caption: “Personal vaporizer. Customizable. May be used with tanks, cartomizers, clearomizers, or drip tips.”
2. “Do you currently use a MOD or handmade e-cigarette?”; caption: “Personal vaporizer. Endlessly customizable. May be used with tank devices, cartomizers, clearomizers, or drip tips.”
In 2016, the questions for e-cigarette model use were more straightforward. E-cigarette users were presented with two sets of photos and captions:
1. “Closed System—any disposable or pre-filled cartridge system. You do not add your own e-liquid.”
2. “Open system—any system you refill using bottles of e-liquid.”
Then, they were asked, “Currently, which type of e-cigarette do you use MOST OFTEN?” Their e-cigarette model used was determined by their response: “Closed system”, Open system”, or “Use both closed and open systems equally.”
E-cigarette users who reported being smokers 12 months before the survey were asked whether they attempted to quit smoking in the past 12 months. A quit attempt was defined as having quit smoking for ≥24 hours. Among those who reported a quit attempt in the past 12 months, we asked, “When you quit smoking the LAST TIME, did you use any of the following to help you quit? Nicotine patches, nicotine gum, nicotine lozenges, nicotine spray or inhaler, Zyban/Wellbutrin (Bupropion), Chantix (Varenicline), e-cigarettes (vapes, vape pen, hookah pen, e-hookah), Smokeless tobacco (chew, dip, snus).” Respondents indicated yes or no to each. The first four options were collapsed into NRT (nicotine replacement therapy) use. We defined users who quit smoking as respondents who were smoking 12 months ago but reported quitting by the time of survey.

2.3. Statistical Analysis

In most analyses, we combined the two surveys when summarizing and comparing responses to survey questions. While 58.5% of included respondents were exclusive to either the 2014 or 2016 survey, 41.5% participated in both the 2014 and 2016 surveys. Given this substantial overlap, the surveys could not be considered independent, with negligible error due to serial dependence. Our approach deals with the serial dependence of respondents who participate in both surveys by transforming the data into two independent surveys via random splitting of the dependent sample, averaging over the large number of such splits and accounting for the resultant additional variance. This approach produces unbiased estimates.
More specifically, we sampled from the binomial distribution to place each respondent at random into exactly one of the surveys to which they responded. This respondent’s weight in the other survey was set to 0. Survey weights were then recomputed so that the total weight of each survey equaled the total number of respondents for that year. Overall weighted proportions for each survey item of interest were computed, as well as the differences in proportions for the items to be compared and the standard errors associated with each proportion and each difference. We then repeated this procedure 2000 times to obtain 2000 sets of weighted proportions, differences, and associated standard errors. Those data were used to compute averaged weighted proportions and both conditional and unconditional variance components. Confidence intervals and p-values for differences in proportions were computed around each weighted statistic using those variance components.

3. Results

Table 1 shows demographics of survey respondents. Overall, the differences across these four purchase channels were modest. The biggest difference was between vape and smoke shop customers, in terms of education. Vape shops had a higher percentage of high-education customers compared to smoke shops (50.1% versus 39.2%, p < 0.05).
Vape shop e-cigarette users had the lowest percentage of users in the youngest age bracket (18–24), 3.2%, compared with 12.0% for retail channels (p < 0.01), with 12.9% for smoke shops (p < 0.01), and 15.5% for internet (p < 0.001) users. They had higher percentage in the 25–34 age bracket, 39.3%, compared with retail, 24.5% (p < 0.01) and smoke shop, 24.0% (p < 0.01)
Table 2 is divided into two halves. The top half includes all e-cigarette users while the bottom includes only those e-cigarette users who reported that they were smoking 12 months before the survey.
The top half of Table 2 shows that most current e-cigarette users were ever smokers. Vape shop customers had the highest percentage of ever smokers (94.7% = 54.5% current smokers + 40.2% former smokers). This percentage was higher than that of internet (83.8% = 48.7% + 35.1%; p < 0.001) but did not differ from retail customers (91.4% = 73.7% current smokers + 17.7% former smokers; p = 0.36) and smoke shop customers (89.7% = 70.4 current smokers + 19.3% former smokers; p = 0.12). Vape shop and internet customers were more likely to be former smokers, 40.2% and 35.1%, respectively—each a significantly higher percentage when compared to the 17.7% of retail and the 19.3% of smoke shop customers who were former smokers (p’s < 0.01).
E-cigarette usage pattern differed significantly across groups. A significantly higher percentage of vape shop (59.1%) and internet (42.9%) customers reported daily use of e-cigarettes relative to retail (19.7%) and smoke shop (23.2%) customers (p’s < 0.001).
The sharpest difference across these groups was in their choice of e-cigarette system. Vape shop customers overwhelmingly preferred open-system designs, with 92.8% of them using open systems exclusively. Retail customers were the least likely to use open systems only (17.3%), with smoke shop and internet customers in between (47.6% and 51.3%, respectively). P-values for differences between retail and each of the other three channels were all <0.001. The majority of retail customers used closed systems exclusively—76.4%, compared to 41.1% of smoke shop, 36.1% of internet, and 3.2% of vape shop customers (p-values of differences relative to retail all <0.001).
The bottom half of Table 2 presents the results for those who reported being smokers 12 months before the survey. Similar differences emerged in terms of the percentage of daily e-cigarette use and choice of models across the four purchase channels. For example, vape shop customers were more likely to be daily e-cigarette users (53.6% compared to 14.2% of retail customers). Vape shop customers were also more likely to use open systems (92.7% compared to 13.7% of retail customers).
Among those who reported being a smoker 12 months before the survey, there was no significant difference in the likelihood of being daily smokers when they were smoking 12 months ago. Approximately three-quarters of each group were daily smokers 12 months before the survey. This is important for the analysis presented below, which focuses on the quitting activities of these e-cigarette users.
Figure 1 presents rates of quit attempt and of smoking cessation for those who were smokers 12 months before the survey. As the left panel shows, the majority of these e-cigarette users made a quit attempt in the last 12 months. The percentage of users making a quit attempt was lowest among smoke shop customers (50.6%) compared to other groups, although only the difference between smoke shops and vape shops (66.4%) is statistically significant (p < 0.05). Quit attempt percentages were not statistically different among the three other groups (retail, internet, and vape shop customers).
The right panel shows the percentage of smokers who reported they quit smoking by the time of the survey. Vape shop and internet customers quit smoking at higher rates (22.2% and 22.5%, respectively) relative to retail and smoke shop customers (10.7% and 9.4%, respectively). Using the quit rate for retail customers as the reference point, the difference was statistically significant when comparing internet against retail (p = 0.025) and when comparing vape shop against retail (p = 0.014). Adjusting for demographics of users does not change the results, with p-values for the two comparisons at 0.022 and 0.010, respectively. The difference between smoke shop and retail customer quit rates was insignificant (p = 0.97).
Table 3 presents data on usage of FDA-approved quitting aids and of e-cigarettes in the last quit attempt reported in Figure 1. Retail customers were most likely to have used an FDA-approved quit aid, 46.9%, compared to smoke shop (23.5%, p < 0.01), and vape shop (21.8%, p < 0.001) customers. Vape shop customers, on the other hand, were the most likely to have used an e-cigarette—93.8%, followed by internet customers (89.6%), smoke shop (80.2%), and then retail (72.3%) customers (p-values for the differences between retail and internet, and retail and vape shop, were <0.05; p-value for the difference between retail and smoke shop customers was 0.30).
NRT was the most popular quitting aid. Some users used more than one quitting aid, but the difference across these four groups was mainly due to the difference in likelihood of using NRT.
It is worth noting that the 2016 survey asked those who smoked 12 months ago whether they had ever used FDA-approved quitting aids to help them with quitting smoking, regardless of whether the quit attempt occurred in the last 12 months. A majority (59.0%) of these respondents reported ever use of a FDA-approved quitting aid. The ever use rates of quitting aids among those who smoked 12 months ago were 62.0%, 58.4%, 58.0%, and 56.8%, for retail store, smoke shop, internet, and vape shop customers, respectively. There was no statistically significant difference among these four groups of customers (p = 0.94).
Finally, we compared the primary purchase channel for respondents in 2014 versus 2016 in Figure 2. In 2014, the most likely channel for e-cigarette purchase was general retail store (35.0%), followed by smoke shops (23.1%), internet (21.5%), and vape shops (20.4%). In 2016, the percentages for the first three channels dropped—general retail store to 27.7%, smoke shops to 17.1%, and internet to 17.5%. Meanwhile, vape shops grew substantially as a purchase channel (to 37.6%) and replaced general retail stores as the most likely channel for e-cigarette purchases in 2016.

4. Discussion

The study found e-cigarette users differed in vaping and smoking cessation patterns, depending on where they primarily purchased e-cigarette products. There are multiple potential reasons for these differences which our study cannot distinguish between. For example, differences could be the result of a self-selection process such that serious users tend to purchase e-cigarettes from vape shops, while less serious users tend to purchase from retail stores. The differences could also be due to interaction between users and products such that users who purchased certain types of products may have primarily purchased from certain types of outlets, and the products in turn affected their e-cigarette usage patterns.
Vape shop and retail store customers exhibited the sharpest differences in e-cigarette use patterns. Preference for e-cigarette device was strikingly different: 92.8% of vape shop customers used open-system models, compared to only 17.3% of retail customers. This preference for open systems also correlated with the frequency of vaping: more than half of vape shop customers used e-cigarettes daily, while only 1 of 5 retail customers did. Daily use of e-cigarettes has been found to be associated with smoking cessation [17,18,19], as has use of open designs [20,21,22]. It is thus expected that these groups would likely differ in their smoking cessation rate. That is, because vape shop customers were more likely to use open system e-cigarettes and more likely to vape on a daily basis relative to retail store customers, these differences could have contributed to differences in smoking cessation outcomes across purchase channels.
Vape shop and retail store customers indeed differed in smoking cessation outcomes, with the former more likely to have quit smoking in a 12-month period. One potential explanation for this difference in smoking cessation is that vape shop customers were simply more motivated to quit smoking than retail store customers. However, data shown in Figure 1 and Table 3 suggest that may not be the whole story. If “making a quit attempt” is used as a measure of motivation, vape shop smokers were not significantly more motivated than retail store customers (Figure 1). If using FDA-approved cessation aids in their most recent quit attempt is considered an indication of motivation to stop smoking cigarettes, then retail customers were more motivated than vape shop customers: retail store customers were twice as likely to use FDA-approved quitting aids as vape shop customers (Table 3). In other words, neither measure suggests the difference in smoking cessation between these two groups was entirely the result of differences in users’ initial motivation to quit smoking.
The fact that the vape shop and retail store customers did not differ in their rate of “having ever used FDA-approved quitting aids” also suggests that the vape shop customers were not inherently biased against using FDA-approved quitting aids. However, these two groups differed in their preference for using e-cigarettes rather than traditional quitting aids in their most recent quit attempt. Prior research has found higher quit rates through the use of e-cigarettes, as compared to rates for smokers using NRT or smoking cessation medications [23,24]. This suggests differences in use of e-cigarettes as a quit aid could have contributed to differences in smoking cessation outcomes by purchase channel. Relatedly, differences in the level of guidance and social support provided through vape shops versus other channels may also be contributing factors to differences found [5,7,8,9,10,11,12,25]. Customers of vape shops report professional guidance and support as reasons for purchasing from vape shops rather than through other channels [26]. Further, a pilot study found evidence of high success rates in quitting when first-time e-cigarette users were provided professional and technical support through vape shops [27]. It is possible that vape shop customers may be more likely to seek personalized guidance in the use of e-cigarettes in quit attempts than customers of other purchase channels, contributing to higher quit rates.
Another possibility for the higher smoking cessation rate for vape shop customers is that these customers were less addicted to cigarette smoking than retail store customers before they attempted to quit. However, these two groups did not differ significantly in their rate of daily cigarette usage 12 months prior to the survey. In other words, any potential difference in nicotine addiction level was not sufficiently large to explain the significant difference in cessation outcomes.
To some extent, these explanations can also be applied to internet customers. Internet customers were similar to vape shop customers in many ways—a finding in line with other research studying usage patterns of online e-cigarette customers [28]. In contrast, smoke shop customers were more like those who purchased e-cigarettes primarily from retail stores (Figure 1).
It is important to note that vape shops could present public health dangers. Researchers have raised the possibilities that vape shops encourage youths who have never smoked to experiment with e-cigarettes [29,30,31]. While there is general agreement about the lower toxicity of e-cigarettes relative to cigarettes, e-cigarettes still contain harmful substances [32]. There is also concern that e-cigarettes could be a “gateway” to cigarettes [33,34,35]. Interestingly, this study found vape shop e-cigarette users had the lowest percentage of never smokers as well as the lowest percentage of young adults age 18–24. We also found that vape shop users were more likely to (i) be former smokers and (ii) to report using an e-cigarette in their most recent quit attempt than retail and smoke shop customers, suggesting that vape shops may be particularly attractive to smokers who are interested in e-cigarettes as a method for smoking cessation. Overall, our study points to the need for further investigation of the role vape shops may play in tobacco control.
Finally, this study found there was a general market shift from retail stores to vape shops from 2014 to 2016. This is consistent with Braak et al., who found vape shops to be the most common purchase channel among U.S. vapers in 2016 [4]. We do not study the factors driving this change in demand towards vape shops. We speculate that one potential driver may be shifting consumer preferences from closed-system disposable ENDS to more customizable open-systems vaping devices [36]. From 2013–2014 to 2015–2017, e-cigarette brand websites increasingly offered open-systems designs, with increasing range in flavors and nicotine options [37]. Research also finds that, as e-cigarettes users become increasingly experienced, they are likely to transition from simple closed-system to newer generation advanced designs [21]. Thus, as the U.S. market has matured, both new and existing customers may be moving towards open-systems. This may have contributed to a shift in sales towards vape shops, which tend to focus on the sale of newer generation open-system product models [37].
There are two implications of this trend. First, it suggests limitations to reliance on Nielsen data about market trends in the e-cigarette industry, since Nielsen does not track sales through non-traditional channels such as the internet and specialty vape shops [2]. Focusing purely on Nielsen-covered channels may limit researchers’ ability to accurately describe vaper behavior. By 2016, less than 30% of vapers reported their primary place of purchase as retail stores, the only category covered by Nielsen. Gaining a more accurate description of market trends requires supplemental information about sales through channels not tracked by Nielsen as well as clear distinctions between different types of channels (such as smoke versus vape shops).
Second, given the association between higher quit rates and vape shops (relative to retail stores), this shift from retail store to vape shops could be viewed as a positive trend. Moreover, vape shops tend to be independently operated, while retail stores tend to be part of a larger network that is controlled by large tobacco companies [38]. However, the full implementation of the Deeming Rule may change the trajectory of current market trends. The Deeming Rule requires e-cigarette manufacturers to complete a labor- and resource-intensive application process that includes detailed ingredient, manufacturing, and product labeling/marketing information [39]. This could constrain vape shops’ ability to appeal to vaping enthusiasts by offering a variety of open-system designs and e-liquids. Vape shops are also themselves subject to new retailer requirements [40]. The findings from this study suggest that consideration of how regulations may ultimately shape e-cigarette users’ access to different purchase channels is warranted.
Finally, a number of vape shop retailers have demonstrated gaps in awareness of as well as weak compliance with impending FDA regulations. Yu et al. find a heavy reliance among vape shop employees on social media and e-cigarette company sales representatives for information about e-cigarette products [41]. The rising prominence of vape shops as a channel for e-cigarette purchases suggests they will play an increasingly important role in the extent to which FDA policies will be upheld within the e-cigarette marketplace. They also will increasingly influence the type and quality of information conveyed to e-cigarette users. Our study suggests the need for stronger FDA communication efforts about e-cigarette rules and policies as well as greater training and monitoring for compliance.

5. Limitations

There are several limitations for the estimates presented in this study. First, they were based on users’ self-report, which may be biased. Relatedly, respondents were asked where they usually purchased their products, although respondents may still have purchased e-cigarette products through other channels. What we present should thus be viewed as rough estimates of the relative proportion of e-cigarette users’ primary place of purchase. These data may be viewed as a first step towards complementing the data available about e-cigarette sales through general retail stores such as Nielsen datasets. A third limitation is that the results about smoking cessation and place of purchase were correlational, as purchase behavior is based on self-selection, which is not readily subjected to controlled trials.

6. Conclusions

Our study points to a need for policymakers to consider how regulatory implementation may relate to the different channels through which consumers purchase electronic nicotine delivery systems. Recently, the FDA has proposed a policy framework for sale of flavored ENDS products that restricts sales of most flavors to channels that only sell to adults, such as vape shops and online retailers, or to general retail stores that segregate ENDS flavors to an adults-only section [42]. This regulatory move could shift the distribution of sales across channels, perhaps increasing vape shop shares further. Better understanding of how such regulatory changes may affect overall e-cigarette user behaviors requires better understanding of how user behaviors relate to e-cigarette channel of purchase.

Author Contributions

Conceptualization, S.-H.Z.; Formal analysis, A.C.G., Y.-L.Z. and T.W.; Funding acquisition, S.-H.Z.; Methodology, A.C.G. and S.-H.Z.; Supervision, S.-H.Z.; Writing—original draft, G.H.; Writing—review and editing, A.C.G., Y.-L.Z., T.W. and S.-H.Z.

Funding

This research was supported by the National Cancer Institute of the National Institutes of Health under the State and Community Tobacco Control Initiative (grant number U01CA154280). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funder had no role in the decision to submit for publication, or in review or approval of the manuscript for publication.

Conflicts of Interest

The authors do not have any financial disclosures to report. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. King, B.A.; Gammon, D.G.; Marynak, K.L.; Rogers, T. Electronic Cigarette Sales in the United States, 2013-2017. JAMA 2018, 320, 1379–1380. [Google Scholar] [CrossRef] [PubMed]
  2. U.S. Department of Health and Human Services. E-Cigarette Use among Youth and Young Adults: A Report of the Surgeon General; Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health: Atlanta, GA, USA, 2016.
  3. Coleman, B.N.; Rostron, B.; Johnson, S.E.; Ambrose, B.K.; Pearson, J.; Stanton, C.A.; Wang, B.; Delnevo, C.; Bansal-Travers, M.; Kimmel, H.L.; et al. Electronic cigarette use among US adults in the Population Assessment of Tobacco and Health (PATH) Study, 2013–2014. Tob. Control 2017, 26, e117–e126. [Google Scholar] [CrossRef] [PubMed]
  4. Braak, D.C.; Cummings, K.M.; Nahhas, G.J.; Heckman, B.W.; Borland, R.; Fong, G.T.; Hammond, D.; Boudreau, C.; McNeill, A.; Levy, D.T.; et al. Where Do Vapers Buy Their Vaping Supplies? Findings from the International Tobacco Control (ITC) 4 Country Smoking and Vaping Survey. Int. J. Environ. Res. Public Health 2019, 16, 338. [Google Scholar] [CrossRef] [PubMed]
  5. Sussman, S.; Baezconde-Garbanati, L.; Garcia, R.; Barker, D.C.; Samet, J.M.; Leventhal, A.; Unger, J.B. Commentary: Forces That Drive the Vape Shop Industry and Implications for the Health Professions. Eval. Health Prof. 2016, 39, 379–388. [Google Scholar] [CrossRef] [PubMed]
  6. Dai, H.; Hao, J. Geographic density and proximity of vape shops to colleges in the USA. Tob. Control 2017, 26, 379–385. [Google Scholar] [CrossRef] [PubMed]
  7. Cheney, M.K.; Gowin, M.; Wann, T.F. Vapor Store Owner Beliefs about Electronic Cigarette Regulation. Tob. Regul. Sci. 2015, 1, 227–235. [Google Scholar] [CrossRef]
  8. Allem, J.-P.; Unger, J.B.; Garcia, R.; Baezconde-Garbanati, L.; Sussman, S. Tobacco Attitudes and Behaviors of Vape Shop Retailers in Los Angeles. Am. J. Health Behav. 2015, 39, 794–798. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Burbank, A.D.; Thrul, J.; Ling, P.M. A Pilot Study of Retail ‘Vape Shops’ in the San Francisco Bay Area. Tob. Prev. Cessat. 2016, 2. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Tsai, J.Y.; Bluthenthal, R.; Allem, J.-P.; Garcia, R.; Garcia, J.; Unger, J.; Baezconde-Garbanati, L.; Sussman, S.Y. Vape shop retailers’ perceptions of their customers, products and services: A content analysis. Tob. Prev. Cessat. 2016, 2. [Google Scholar] [CrossRef] [PubMed]
  11. Lee, Y.O.; Kim, A.E. ‘Vape shops’ and ‘E-Cigarette lounges’ open across the USA to promote ENDS. Tob. Control 2015, 24, 410–412. [Google Scholar] [CrossRef] [PubMed]
  12. Hart, J.L.; Walker, K.L.; Sears, C.G.; Lee, A.S.; Smith, C.; Siu, A.; Keith, R.; Ridner, S.L. Vape Shop Employees: Public Health Advocates? Tob. Prev. Cessat. 2016, 2. [Google Scholar] [CrossRef] [PubMed]
  13. Giovenco, D.P. Smoke Shop Misclassification May Cloud Studies on Vape Shop Density. Nicotine Tob. Res. 2018, 20, 1025–1026. [Google Scholar] [CrossRef] [PubMed]
  14. KnowledgePanel(R) Design Summary. GfK. 2013. Available online: http://www.knowledgenetworks.com/knpanel/docs/knowledgepanel(R)-design-summary-description.pdf (accessed on 8 November 2018).
  15. Delnevo, C.D.; Giovenco, D.P.; Steinberg, M.B.; Villanti, A.C.; Pearson, J.L.; Niaura, R.S.; Abrams, D.B. Patterns of Electronic Cigarette Use among Adults in the United States. Nicotine Tob. Res. 2016, 18, 715–719. [Google Scholar] [CrossRef] [PubMed]
  16. Etter, J.-F.; Bullen, C. A Longitudinal Study of Electronic Cigarette Users. Addict. Behav. 2014, 39, 491–494. [Google Scholar] [CrossRef]
  17. Biener, L.; Hargraves, J.L. A Longitudinal Study of Electronic Cigarette Use among a Population-Based Sample of Adult Smokers: Association with Smoking Cessation and Motivation to Quit. Nicotine Tob. Res. 2015, 17, 127–133. [Google Scholar] [CrossRef] [PubMed]
  18. Zhuang, Y.-L.; Cummins, S.E.; Sun, J.Y.; Zhu, S.-H. Long-term e-cigarette use and smoking cessation: A longitudinal study with US population. Tob. Control 2016, 25, i90–i95. [Google Scholar] [CrossRef] [PubMed]
  19. Zhu, S.-H.; Zhuang, Y.-L.; Wong, S.; Cummins, S.E.; Tedeschi, G.J. E-cigarette use and associated changes in population smoking cessation: Evidence from US current population surveys. BMJ 2017, 358, j3262. [Google Scholar] [CrossRef] [PubMed]
  20. Hitchman, S.C.; Brose, L.S.; Brown, J.; Robson, D.; McNeill, A. Associations Between E-Cigarette Type, Frequency of Use, and Quitting Smoking: Findings From a Longitudinal Online Panel Survey in Great Britain. Nicotine Tob. Res. 2015, 17, 1187–1194. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Yingst, J.M.; Veldheer, S.; Hrabovsky, S.; Nichols, T.T.; Wilson, S.J.; Foulds, J. Factors Associated With Electronic Cigarette Users’ Device Preferences and Transition From First Generation to Advanced Generation Devices. Nicotine Tob. Res. 2015, 17, 1242–1246. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Chen, C.; Zhuang, Y.-L.; Zhu, S.-H. E-Cigarette Design Preference and Smoking Cessation. Am. J. Prev. Med. 2016, 51, 356–363. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Adriaens, K.; Van Gucht, D.; Baeyens, F. About One in Five Notice Vapers Buying Their First E-cigarette In a Vape Shop Are Smoking Abstinent after Six Months. Int. J. Environ. Res. Public Health 2018, 15, 1886. [Google Scholar] [CrossRef] [PubMed]
  24. Hajek, P.; Phillips-Waller, A.; Przulj, D.; Pesola, F.; Smith, K.; Bisal, N.; Li, J.; Parrott, S.; Sasieni, P.; Dawkins, L.; et al. A Randomized Trial of E-Cigarettes versus Nicotine-Replacement Therapy. N. Engl. J. Med. 2019, 380, 629–637. [Google Scholar] [CrossRef] [PubMed]
  25. Ward, E.; Cox, S.; Dawkins, L.; Jakes, S.; Holland, R.; Notley, C. A Qualitative Exploration of the Role of Vape Shop Environments in Supporting Smoking Abstinence. Int. J. Environ. Res. Public Health 2018, 15, 297. [Google Scholar] [CrossRef] [PubMed]
  26. Wagener, T.L.; Shaikh, R.A.; Meier, E.; Tackett, A.P.; Tahirkheli, N.N.; Leavens, E.L.; Driskill, L. Examining the Smoking and Vaping Behaviors and Preferences of Vape Shop Customers. Tob. Prev. Cessat. 2016, 2, 5. [Google Scholar] [CrossRef]
  27. Polosa, R.; Caponnetto, P.; Cibella, F.; Le-Houezec, J. Quit and Smoking Reduction Rates in Vape Shop Consumers: A Prospective 12-Month Survey. Int. J. Environ. Res. Public Health 2015, 12, 3428–3438. [Google Scholar] [CrossRef] [Green Version]
  28. Van Gucht, D.; Adriaens, K.; Baeyens, F. Online Vape Shop Customers Who Use E-cigarettes Report Abstinence from Smoking and Improved Quality of Life, But a Substantial Minority Still Have Vaping-Related Health Concerns. Int. J. Environ. Res. Public Health 2017, 14, 798. [Google Scholar] [CrossRef] [PubMed]
  29. Sussman, S.; Allem, J.-P.; Garcia, J.; Unger, J.B.; Cruz, T.B.; Garcia, R.; Baezconde-Garbanati, L. Who walks into vape shops in Southern California?: A naturalistic observation of customers. Tob. Induc. Dis. 2016, 14. [Google Scholar] [CrossRef] [PubMed]
  30. Bostean, G.; Crespi, C.M.; Vorapharuek, P.; McCarthy, W.J. E-cigarette use among students and e-cigarette specialty retailer presence near schools. Health Place 2016, 42, 129–136. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Giovenco, D.P.; Casseus, M.; Duncan, D.T.; Coups, E.J.; Lewis, M.J.; Delnevo, C.D. Association Between Electronic Cigarette Marketing Near Schools and E-cigarette Use Among Youth. J. Adolesc. Health 2016, 59, 627–634. [Google Scholar] [CrossRef] [PubMed]
  32. National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Population Health and Public Health Practice, Committee on the Review of the Health Effects of Electronic Nicotine Delivery Systems. Public Health Consequences of E-Cigarettes; Eaton, D.L., Kwan, L.Y., Stratton, K., Eds.; National Academies Press (US): Washington, DC, USA, 2018; ISBN 978-0-309-46834-3. Available online: http://www.ncbi.nlm.nih.gov/books/NBK507171/ (accessed on 24 July 2018).
  33. Leventhal, A.M.; Strong, D.R.; Kirkpatrick, M.G.; Unger, J.B.; Sussman, S.; Riggs, N.R.; Stone, M.D.; Khoddam, R.; Samet, J.M.; Audrain-McGovern, J. Association of Electronic Cigarette Use With Initiation of Combustible Tobacco Product Smoking in Early Adolescence. JAMA 2015, 314, 700. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Primack, B.A.; Soneji, S.; Stoolmiller, M.; Fine, M.J.; Sargent, J.D. Progression to Traditional Cigarette Smoking After Electronic Cigarette Use Among US Adolescents and Young Adults. JAMA Pediatr. 2015, 169, 1018. [Google Scholar] [CrossRef] [PubMed]
  35. Soneji, S.; Barrington-Trimis, J.L.; Wills, T.A.; Leventhal, A.M.; Unger, J.B.; Gibson, L.A.; Yang, J.; Primack, B.A.; Andrews, J.A.; Miech, R.A.; et al. Association Between Initial Use of e-Cigarettes and Subsequent Cigarette Smoking Among Adolescents and Young Adults: A Systematic Review and Meta-analysis. JAMA Pediatr. 2017, 171, 788. [Google Scholar] [CrossRef]
  36. Cantrell, J.; Huang, J.; Greenberg, M.; Willett, J.; Hair, E.; Vallone, D. History and Current Trends in the Electronic Cigarette Retail Marketplace in the United States: 2010–2016. Nicotine Tob. Res. 2018. [Google Scholar] [CrossRef] [PubMed]
  37. Hsu, G.; Sun, J.Y.; Zhu, S.-H. Evolution of Electronic Cigarette Brands from 2013–2014 to 2016–2017: Analysis of Brand Websites. J. Med. Internet Res. 2018, 20, e80. [Google Scholar] [CrossRef] [PubMed]
  38. Campaign for Tobacco-Free Kids. Deadly Alliance: How Big Tobacco and Convenience Stores Partner to Market Tobacco Products and Fight Life-Saving Policies. Available online: https://www.tobaccofreekids.org/what-we-do/industry-watch/deadly-alliance (accessed on 24 July 2018).
  39. Effective and Compliance Dates Applicable to Retailers, Manufacturers, Importers, and Distributors of Newly Deemed Tobacco Products. 2017. Available online: https://www.fda.gov/downloads/TobaccoProducts/GuidanceComplianceRegulatoryInformation/UCM501016.pdf (accessed on 18 September 2017).
  40. The “Deeming Rule”: Vape Shops. 2016. Available online: https://www.fda.gov/downloads/TobaccoProducts/GuidanceComplianceRegulatoryInformation/UCM501234.pdf (accessed on 18 September 2017).
  41. Yu, S.; Escobedo, P.; Garcia, R.; Cruz, T.B.; Unger, J.B.; Baezconde-Garbanati, L.; Meza, L.; Sussman, S. A Descriptive Longitudinal Study of Changes in Vape Shop Characteristics and Store Policies in Anticipation of the 2016 FDA Regulations of Tobacco Products, Including E-Cigarettes. Int. J. Environ. Res. Public Health 2018, 15, 313. [Google Scholar] [CrossRef] [PubMed]
  42. U.S. Food & Drug Administration. Statement from FDA Commissioner Scott Gottlieb, M.D., on Proposed New Steps to Protect Youth by Preventing Access to Flavored Tobacco Products and Banning Menthol in Cigarettes. 2018. Available online: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm625884.htm (accessed on 30 November 2018).
Ijerph 16 00724 g001 550
Figure 1. Quit attempt and cessation rate by place of purchase, among those who were smokers 12 months before the survey. (A) Quit attempts; (B) quit rate.
Figure 1. Quit attempt and cessation rate by place of purchase, among those who were smokers 12 months before the survey. (A) Quit attempts; (B) quit rate.
Ijerph 16 00724 g001
Ijerph 16 00724 g002 550
Figure 2. The changing market share, based on e-cigarette users’ self-report, from 2014 to 2016.
Figure 2. The changing market share, based on e-cigarette users’ self-report, from 2014 to 2016.
Ijerph 16 00724 g002
Table
Table 1. Demographics of e-cigarette users by place of purchase (N = 1622, with 2014 and 2016 surveys combined).
Table 1. Demographics of e-cigarette users by place of purchase (N = 1622, with 2014 and 2016 surveys combined).
Retail
% (95% CI)		Smoke Shop
% (95% CI)		Internet
% (95% CI)		Vape Shop
% (95% CI)
GenderN = 532N = 339N = 308N = 443
  Male43.9 (37.0–51.0)52.2 (43.8–60.6)43.5 (35.4–51.9)47.8 (41.1–54.6)
  Female56.1 (49.0–63.0)47.8 (39.4–56.2)56.5 (48.1–64.6)52.2 (45.4–58.9)
Age
  18–2412.0 (7.6–18.6)12.9 (7.4–21.5)15.5 (10.5–22.1)3.2 (1.5–6.7)
  25–3424.5 (18.8–31.3)24.0 (17.6–31.8)29.4 (22.0–38.1)39.3 (32.8–46.1)
  35–4422.6 (17.3–28.9)13.6 (8.6–20.8)18.5 (12.9–25.9)21.8 (16.5–28.1)
  45–5420.4 (15.5–26.5)21.7 (15.7–29.2)15.5 (10.1–23.0)15.9 (11.4–21.7)
  55–6412.6 (8.8–17.7)19.5 (13.9–26.7)14.4 (9.5–21.1)14.4 (10.3–19.9)
  65+7.5 (4.7–11.9)7.9 (4.6–13.2)6.6 (3.5–11.8)5.3 (3.0–9.1)
Education
  Low51.7 (44.6–58.8)60.8 (52.3–68.6)55.4 (47.0–63.5)49.9 (43.1–56.7)
  High48.3 (41.2–55.4)39.2 (31.4–47.7)44.6 (36.5–53.0)50.1 (43.3–56.9)
Race
  NH–White71.8 (65.0–77.8)66.7 (58.2–74.2)67.4 (59.0–74.8)75.1 (68.5–80.6)
  NH–Black8.3 (5.2–12.9)10.0 (6.2–15.8)1.7 (0.3–10.4)8.9 (5.8–13.5)
  Hispanic11.6 (7.6–17.4)18.0 (12.4–25.4)20.7 (14.9–28.0)8.0 (4.9–12.8)
  NH–Other8.1 (4.9–13.1)5.1 (2.1–11.7)9.7 (5.4–16.7)7.9 (4.7–13.0)
Table
Table 2. Cigarette smoking behavior and the use of e-cigarette systems (2014 and 2016 surveys combined).
Table 2. Cigarette smoking behavior and the use of e-cigarette systems (2014 and 2016 surveys combined).
RetailSmoke ShopInternetVape Shop
All E-cigarette Users at the Time of SurveyN = 532
% (95% CI)		N = 339
% (95% CI)		N = 308
% (95% CI)		N = 443
% (95% CI)
Smoking Status
  Never smokers8.2 (3.8–16.8)10.1 (5.5–17.8)16.1 (11.0–23.0) 5.3 (3.1–9.1)
  Current smokers73.7 (66.5–79.8)70.4 (61.4–78.0)48.7 (40.3–57.1)54.5 (47.4–61.3)
  Former smokers17.7 (12.5–24.4)19.3 (13.0–27.7)35.1 (27.5–43.7)40.2 (33.5–47.3)
Frequency of Current E-cigarette Use
  Daily use19.7 (14.4–26.4)23.2 (16.8–31.3)42.9 (34.8–51.4)59.1 (52.2–65.7)
  Non–daily Use80.3 (73.6–85.6)76.8 (68.7–83.2)57.1 (48.6–65.2)40.9 (34.3–47.8)
E-cigarette Models
  Open system17.3 (12.7–23.1)47.6 (39.2–56.1)51.3 (42.7–59.8)92.8 (87.8–95.8)
  Closed system76.4 (69.8–81.9)41.1 (33.1–49.5)36.1 (28.4–44.7)3.2 (1.5–6.7)
  Both6.2 (3.3–11.3)11.3 (7.0–17.8)12.5 (8.1 – 18.9)3.9 (1.7–8.6)
E-cigarette Users Who were Smoking Cigarettes 12 Months AgoN = 459
% (95% CI)		N = 302
% (95% CI)		N = 238
% (95% CI)		N = 314
% (95% CI)
Smoking Behavior 12 Months Ago
  Daily smokers75.2 (67.4–81.6)75.0 (65.9–82.3)72.8 (62.0–81.5)78.4 (70.0–85.0)
  Occasional smokers24.8 (18.4–32.6)25.0 (17.7–34.1)27.2 (18.5–38.0)21.6 (15.0–30.0)
Frequency of Current E-cigarette Use
  Daily Use14.2 (9.3 – 21.2)19.8 (13.4–28.2)33.0 (23.6–43.9)53.6 (45.0–62.0)
  Non–daily Use85.8 (78.8 – 90.7)80.2 (71.8–86.6)67.0 (56.1–76.4)46.4 (38.0–55.0)
E-cigarette Models
  Open system 13.7 (9.4–19.6)46.9 (37.9–56.1)54.7 (43.7–65.2)92.7 (86.8–96.1)
  Closed system79.0 (71.9–84.7)45.0 (36.1–54.2)38.8 (29.0–49.7)3.7 (1.6–8.3)
  Both7.1 (3.7–13.4)8.0 (4.1–15.2)6.4 (2.7–14.2)3.5 (1.4–8.9)
Table
Table 3. Use of FDA-approved quitting aids or E-cigarettes in the last quit attempt, among those who smoked 12 months ago.
Table 3. Use of FDA-approved quitting aids or E-cigarettes in the last quit attempt, among those who smoked 12 months ago.
For Those Who Made a Quit AttemptRetail
N = 259
% (95%CI)		Smoke Shop
N = 136
% (95%CI)		Internet
N = 129
% (95%CI)		Vape Shop
N = 189
% (95%CI)
Any Quitting Aid46.9 (37.0–57.1)23.5 (14.0–36.5)30.2 (19.2–44.2)21.8 (14.3–31.7)
  NRT42.5 (32.7–52.9)17.3 (9.2–30.2)22.9 (13.4–36.2)18.0 (11.2–27.7)
  Zyban12.6 (6.9–21.8)5.5 (1.9–15.2)12.0 (5.8–23.3)3.9 (1.5–10.0)
  Chantix10.2 (5.2–19.1)6.0 (1.9–17.3)14.8 (7.4–27.3)6.5 (3.0–13.4)
E-cigarettes72.3 (61.9–80.7)80.2 (67.1–88.9)89.6 (78.0–95.4)93.8 (83.4–97.8)

Share and Cite

MDPI and ACS Style

Hsu, G.; Gamst, A.C.; Zhuang, Y.-L.; Wolfson, T.; Zhu, S.-H. A Comparison of E-Cigarette Use Patterns and Smoking Cessation Behavior among Vapers by Primary Place of Purchase. Int. J. Environ. Res. Public Health 2019, 16, 724. https://doi.org/10.3390/ijerph16050724

AMA Style

Hsu G, Gamst AC, Zhuang Y-L, Wolfson T, Zhu S-H. A Comparison of E-Cigarette Use Patterns and Smoking Cessation Behavior among Vapers by Primary Place of Purchase. International Journal of Environmental Research and Public Health. 2019; 16(5):724. https://doi.org/10.3390/ijerph16050724

Chicago/Turabian Style

Hsu, Greta, Anthony C. Gamst, Yue-Lin Zhuang, Tanya Wolfson, and Shu-Hong Zhu. 2019. "A Comparison of E-Cigarette Use Patterns and Smoking Cessation Behavior among Vapers by Primary Place of Purchase" International Journal of Environmental Research and Public Health 16, no. 5: 724. https://doi.org/10.3390/ijerph16050724

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop