Health Effects of Electronic Cigarettes: An Umbrella Review and Methodological Considerations

E-cigarettes are often marketed as a safer alternative to combustible cigarettes. However, their health effects, especially those associated with long-term use, remain largely uncertain. We conducted an umbrella review of the cardiopulmonary and carcinogenic risks of e-cigarette use, distinguishing between short-term and long-term health effects. The search for systematic reviews was conducted across four electronic databases through 25 January 2022. Methodological quality was assessed using the AMSTAR-2 quality appraisal tool. Seventeen systematic reviews, including five meta-analyses, were included in our umbrella review. There was a clear underreporting of e-cigarette devices and e-liquid types, e-cigarette and cigarette exposure, and the health and smoking status of study participants. Overall, the findings suggest that short-term use of e-cigarettes may be associated with acute cardiopulmonary risks, although to a lesser extent than cigarette use. Long-term e-cigarette use may have pulmonary/respiratory benefits in those who switch from chronic cigarette smoking, particularly in individuals with asthma and chronic obstructive pulmonary disease (COPD). Evidence on intermediate and long-term carcinogenic effects is lacking. This umbrella review underscores the urgent need for systematic reviews with better adherence to established reporting guidelines, consistent definitions of duration of e-cigarette use, a focus on newer devices, and accounting for the impacts of former or current smoking.


Introduction
The electronic cigarette (EC) is a novel nicotine delivery system designed to produce an aerosol by heating a liquid that usually contains nicotine along with other additives. Unlike traditional cigarettes (TCs), ECs do not combust tobacco and do not produce "tar" or carbon monoxide, two of the most harmful elements in tobacco smoke [1]. Despite often being marketed as a safer alternative to combustible tobacco products, the harms of ECs, especially those associated with long-term use, remain largely uncertain. In addition, the potential health benefits of EC use relative to smoking are not very clear. A substantial amount of research has been conducted in the past decade to examine the health effects of EC use, often providing conflicting evidence and claims [2][3][4][5].
Many reviews have attempted to summarize the evidence by assessing studies that often vary in their methods, the population considered, and the health outcome measures

Materials and Methods
The peer-reviewed umbrella review protocol was developed a priori in adherence with the guidelines for umbrella reviews [9] and registered with PROSPERO (Registration Number: CRD42021237878). Initially, the protocol also planned to include evidence on the chemical constituents of ECs and their toxic health effects. However, due to the large volume of identified literature, we evaluated these outcomes in a separate umbrella review. We conducted the umbrella review in adherence with the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) [10].

Search Strategy
Two reviewers (CJC and NT) designed and performed a structured literature search in PubMed, Web of Science, Embase, and Cochrane Database of Systematic Reviews with no restrictions on the publication date. The following search terms were included: "electronic cigarette", "e-cigarette", "electronic nicotine delivery system", "personal vaporiser", "personal vaporizer", "e-liquid", "nicotine content", "systematic review", "meta-analysis" and "review". Gray literature searches were conducted in OpenGrey and Google custom search limited to U.S. government and non-profit organization domains (i.e., gov, .org). The initial search was conducted on 27 May 2020, and was updated on 25 January 2022. The complete search strategy employed for PubMed is presented in Supplementary File S1.
Two reviewers (NT and CJC in the initial search; and NT and MK in the updated search) independently extracted and screened titles and abstracts. The reviewers were blinded to author names and affiliations as well as the journals of publication. Two reviewers (NT and MK) independently screened the full-text articles for eligibility criteria. If the full-text articles were not accessible, the first authors were contacted with inquiries. Finally, we searched reference lists in the bibliographies of eligible papers for additional sources. Any disagreements between reviewers were discussed and resolved either by consensus or by consulting a third reviewer (DTL).

Eligibility
This review included systematic reviews and meta-analyses published in English that investigated any of three outcome categories: (1) cardiovascular effects, (2) respiratory or pulmonary effects, and (3) carcinogenic effects. A literature review was considered systematic if it explicitly identified itself as a systematic review in the title, abstract, keyword, or methods and was peer-reviewed.
Systematic reviews were included if they examined at least one outcome of interest. Evidence from in vitro, animal, and human studies was included. Human studies included healthy adult participants and adults with underlying cardiovascular, pulmonary, or oncological conditions (self-reported or physician-diagnosed) to assess the potential effects on disease improvement or progression. There was no limitation on the types of EC devices and e-liquids considered.
Since our focus was on the clinical/laboratory-measured physiological and pathophysiological health effects of ECs, self-reported adverse events (e.g., dry cough, throat irritation, and anxiety) were excluded. As our study focused on health outcomes from the intended use of ECs, we excluded reviews on health effects resulting from the misuse of ECs, such as poisoning, or outcomes associated with vaping illicit substances, as occurred in the instance of the e-cigarette or vaping product use-associated lung injury (EVALI). Safety outcomes related to device malfunction, such as burns and injuries, were also excluded. Finally, because we focused on direct health effects, we excluded reviews on the health effects of passive vaping.

Data Extraction
A standardized data extraction tool was developed and employed independently by two reviewers (NT and MK) to extract the following information from each included review: citation, author financial disclosures, review objectives, details of the literature search, designs of included studies, description of exposure (e.g., duration of use, EC device, and e-liquid types), health outcome variables, sample types (for preclinical studies), details on the population smoking status and history of tobacco use (for human outcomes), quality appraisal tool used, main results, authors' conclusions along with recommendations, and study limitations. When extracting evidence, we attempted to differentiate between acute and long-term effects as defined by the authors of the included reviews.

Assessment of Quality and Risk of Bias
Reviews that were eligible for inclusion were appraised for quality using the AMSTAR-2 checklist [11]. While several other quality appraisal tools for systematic reviews exist, AMSTAR-2 is the most commonly used tool to evaluate the methodological quality of systematic reviews (with or without a meta-analysis) of both randomized and non-randomized studies [12,13]. The checklist is comprised of 16 questions and includes seven domains that can critically influence the validity of a review and its conclusions. Critical domains include an a priori registered review protocol, adequacy of the literature search, justification for the exclusion of individual studies, risk of bias assessment, appropriate statistical methods for combining results and investigating publication bias (for meta-analysis), and consideration of potential biases when interpreting the results of individual studies [11].
Given the heterogeneity in the reviews and meta-analyses we included, and concerns about the potentially limited discriminative power of the tool due to the previously observed abundance of critically low ratings [14], the overall confidence score for systematic reviews was not derived. Instead, we address the limitations of each systematic review based on the individual items of the tool. Two reviewers (NT and MK) independently performed the assessment. Any disagreements were discussed and resolved by consensus.

Results
Our literature search identified 1880 unique records. The titles and abstracts were screened for eligibility, excluding 1526 records. Full-text articles were accessed for the remaining 354 records and thoroughly assessed for exclusion criteria (Figure 1).
The most common reason for excluding full texts was the unfulfilled criterion of being a systematic review or meta-analysis (n = 238). Potentially relevant literature reviews that were excluded as non-systematic are presented in Supplementary Table S1. Seventeen systematic reviews, including five meta-analyses, were ultimately included in our study. Eleven reviews summarized evidence on cardiovascular effects, thirteen on pulmonary/respiratory effects, and five on carcinogenic effects of ECs. An overview of the health domains included in systematic reviews is presented in Supplementary Table S2. Information on the studied EC device types was provided in eight reviews, and on e-liquid type (e.g., containing nicotine) in seven reviews. Among reviews on human health effects, the smoking status of the studied individuals was reported in ten, and the health status was reported in seven reviews. Tables 1-3 summarize the main characteristics of the included reviews and meta-analyses.
The most common reason for excluding full texts was the unfulfilled criterion of being a systematic review or meta-analysis (n = 238). Potentially relevant literature reviews that were excluded as non-systematic are presented in Supplementary Table S1. Seventeen systematic reviews, including five meta-analyses, were ultimately included in our study. Eleven reviews summarized evidence on cardiovascular effects, thirteen on pulmonary/respiratory effects, and five on carcinogenic effects of ECs. An overview of the health domains included in systematic reviews is presented in Supplementary Table S2. Information on the studied EC device types was provided in eight reviews, and on e-liquid type (e.g., containing nicotine) in seven reviews. Among reviews on human health effects, the smoking status of the studied individuals was reported in ten, and the health status was reported in seven reviews. Tables 1-3 summarize the main characteristics of the included reviews and meta-analyses.

AMSTAR 2 Assessment of Quality and Risk of Bias
The quality assessment of the included reviews is presented in Table 4. Thirteen [15][16][17][18][19][20][21][22][23][24][25][26][27] of the seventeen reviews failed to report whether their methods were established in a written protocol prior to conducting their review. Two [28,29] published a peer-reviewed protocol but did not provide sufficient details or justifications for deviations from the protocol, and one [30] briefly mentioned a review protocol without any details on its peer-review or registration status.
All five [22,23,25,29,31] meta-analyses used appropriate methods for the statistical combination of results. One [23] failed to apply a satisfactory technique to assess the quality and risk of bias of the included studies, while two [22,25] did not assess the potential impact of the risk of bias in individual studies on the overall findings. The risk of publication bias was assessed in four [23,25,29,31] meta-analyses.  The authors declare no conflict of interest.
(1) Systematic and critical review of the existing literature on the health consequences of ECs and discuss the implications of our findings for public health; (2) to investigate how many of the published articles have a conflict of interest.

Through 24
August 2014 Human HR, blood pressure, oxygen saturation, and cardiac function.
Eight studies on cardiovascular health effects.
(1) To provide a comprehensive update of data on the potential health effects of ECs since the NASEM report. (2) To provide a focused discussion of the scientific literature that will help inform the general public, health-care practitioners, and policy makers of the effects of EC use on health. 13 randomized trials and one prospective study.
EC use with outcome assessment up to 4 h post use.
Information was collected and presented in a tabular format. No stratified analysis by device type or discussion of the impact of device type on the outcome was provided. § Meta-analysis. COI = conflict of interest; EC = electronic cigarettes; TC = traditional cigarettes; SBP = systolic blood pressure; DBP = diastolic blood pressure; HR = heart rate; HRV = heart rate variation; and N/A = not available. * Chronic effects were described as those measured in chronic EC users (non-TC smokers) and in TC smokers after switching to chronic EC. Authors report no conflict of interest.
(1) Systematic and critical review of the existing literature on the health consequences of ECs and discuss the implications of our findings for public health; and (2)  Not specified.
14 studies on respiratory/pulmonary health effects.
Short-term and long-term EC use or exposure to EC vapor (no data on the duration).
To explore the relationship between ECs and the risk of asthma.  15 studies on asthma and nine for COPD (seven cross-sectional and two prospective).
EC use (current use, 30-day use, and long-term use).
N/A § Meta-analysis. COI = conflict of interest; EC = electronic cigarettes; TC = traditional cigarettes; COPD = chronic obstructive pulmonary disease; ROS = reactive oxygen species; FeNo = fraction of exhaled nitric oxide; and N/A = not available. * Duration of exposure varied according to the followed intervention protocols. Table 3. Characteristics of systematic reviews on carcinogenic risks of e-cigarettes.
(1) Systematic and critical review of the existing literature on the health consequences of ECs and discuss the implications of our findings for public health.  The authors report no conflicts of interests. One co-author reports to be receiving research support from a pharmaceutical company, which is also a manufacturer of a stop smoking medication.
To have a summary of the current, relevant literature on EC safety and efficacy.

Acute Effects
The report of the National Academies of Science, Engineering and Medicine (NASEM) [16] found, based on their strength of evidence framework for conclusions, that there is substantial evidence of an associated between acute nicotine-containing EC use and increases in HR, DBP, and limited evidence (i.e., supportive findings from fair quality studies or mixed findings with most favoring one conclusion) of an association with the increase in SBP. Further, the report found limited evidence of increases in endothelial dysfunction, arterial stiffness, or biomarkers of oxidative stress.
Two reviews [15,30] reported increased HR, SBP, and DBP in participants after nicotinecontaining EC use. However, one [15] did not stratify these findings by participants' TC smoking status. One meta-analysis [22] found an increase in HR, SBP, and DBP after the use of nicotine ECs; however, the effects were not examined by study participants' TC smoking status. In another meta-analysis [29], the use of nicotine ECs was associated with increases in HR, SPB, and DBP, comparable to the effects seen after TC use, with no significant changes observed after nicotine-free EC use.
In contrast, one review found that the use of ECs without nicotine also resulted in increased SBP and DBP, although less compared to nicotine containing ECs [26]. Other reviews similarly reported acute increases in HR [19,20,30], SBP [20,30], and DBP [20,30] after EC use (unknown nicotine content) but of smaller magnitude than TC use. There were indications of worse endothelial function and greater oxidative stress [29] and no significant immediate effect on myocardial function compared to TC use [19,20].

Long-Term Effects
The NASEM report [16] found insufficient evidence that chronic EC use is associated with long-term changes in HR, blood pressure, and cardiac geometry and function. Another review [22] reported that the adjusted risk of myocardial infarction was increased in EC users compared to non-users, although to a lesser extent than in TC smokers. However, the authors noted that these findings were sensitive to non-random misclassification bias regarding EC users and smokers with prior myocardial infarction diagnosis.

Effects of EC Use by Smoking Status
• Tobacco-naïve EC users Short-term use of both nicotine and nicotine-free ECs elevated SBP and DBP in nonsmokers [26]. In long-term (days or longer) exclusive nicotine EC users, only HRV, and not blood pressure or HR, was chronically elevated compared to non-users [30].

•
Former TC smokers who switch to ECs The NASEM report [16] indicated substantial evidence that completely switching from regular use of TCs to ECs reduces the short-term adverse health outcomes in several organ systems. Other reviews suggested that switching from TC use to chronic EC use may reduce cardiovascular harm [18] and regulate blood pressure [22,30] with no effect on HR [22,30]. In contrast, one review [24] found no difference in the risk of myocardial infarction or coronary heart disease for former smokers who transitioned to ECs compared to current exclusive TC smokers. •

• Dual (EC and TC) users
The NASEM report [16] found no evidence that long-term dual EC and TC use changed morbidity or mortality compared with exclusive TC use. One review [18] suggested a possible increased risk of cardiovascular disease for dual users of TCs and ECs compared to the exclusive use of either product.

In Vivo Animal Studies
Murine models exposed to EC vapor displayed vascular inflammatory markers and angiogenesis as well as increased development of atherosclerotic plaque, suggestive of endothelial dysfunction and increased atherosclerotic risk [15]. Some physiological effects (i.e., increased oxidative stress, elevated blood pressure, and decreased HR) were reported in animal models after EC vapor exposure but much less than from TC exposure [18,28].

In Vitro Cell Studies
One review [15] suggested an increase in reactive oxygen species (ROS) production, a decrease in antioxidant levels, and disorder in endothelial cellular culture function and interactions after short-term (4-72 h) exposure of cardiovascular cell cultures to EC aerosol extract. The NASEM report [16] similarly stated that there was substantial evidence that EC aerosols can induce acute endothelial cell dysfunction and increase oxidative stress. However, these effects were generally lower than from TC smoke. Other reviews found no gene expression alterations in human coronary artery endothelial cells [22] but reported increases in anti-inflammatory processes, oxidative stress, cell apoptosis and necrosis [28] after exposure to EC aerosol.

Respiratory/Pulmonary Health Effects
The effects of EC use on pulmonary and respiratory health were reported in thirteen systematic reviews, including four meta-analyses. Seven reviews examined acute effects (minutes to hours post EC exposure but often undefined) [16,17,[19][20][21]27,29], one examined sub-acute effects (two hours a day, five days a week for 30 days) [27], and two considered chronic effects (not further defined) [16,27]. Six reviews explored the association between EC use status and prevalence of chronic respiratory conditions [18,[23][24][25]28,31].

Acute Effects
The NASEM report [16] found that ECs containing nicotine, but not nicotine-free ECs, have short-term effects on the lung defense mechanism. Two reviews reported a decrease in FeNO [20,29] in response to EC use, while a meta-analysis [29] found that the decrease is likely related to the nicotine content. Other reviews reported an airway obstruction comparable to smoking, with a smaller impact on lung function [21]; and reduced lung function, especially from nicotine-containing ECs, although less than from TC smoke [19]. In contrast, a meta-analysis found no significant changes in spirometry measures following EC use, regardless of the nicotine content [29]. An increased respiratory resistance [17], upper respiratory tract irritation, and allergic airway inflammation [20] were also reported following EC use.

Effects in Individuals with COPD and Asthma
The NASEM report [16] indicated limited evidence of improved lung function and respiratory symptoms among people with asthma and of a reduction in exacerbations in COPD patients after switching completely or partly (dual use) from smoking to ECs. The report also found moderate evidence for EC use and increased asthma exacerbations. One review [28] also reported improved lung function and disease symptoms in individuals with asthma and COPD after switching from smoking to EC use. Three meta-analyses found an association between current, ever, and former EC use and the prevalence of asthma [23,25,31] and COPD [23]. The authors underlined the limited interpretation of their findings due to the cross-sectional nature of studies, observed heterogeneity, and likely publication bias.

•
Tobacco-naïve EC users One review [18] reported impaired lung function, measured by spirometry, in tobacconaïve EC users compared to never users.

• EC-naïve smokers
One review reported that short-term EC use in smokers was associated with an increase in impedance and overall peripheral airway resistance, and a concomitant decrease in specific airway conductance, similar to TC use [21].

•
Former TC smokers who switched to ECs Some positive health changes (e.g., reduced carboxyhemoglobin levels) [19] and improvements in respiratory symptoms in former smokers who switched to regular EC use [21] were reported, although the authors recognized that the latter findings were flawed by selection bias and conflict of interest. One review [24] showed 40% lower odds of respiratory outcomes (including asthma, COPD, chronic bronchitis, and emphysema) for former smokers who switched to EC use compared to current exclusive smokers, based on a limited number of observational studies and unknown time of switching to ECs among former smokers.

• Dual (EC and TC) users
A positive association between dual use and the prevalence of asthma was reported in one systematic review [31], while a meta-analysis [25] found that the association with asthma prevalence was even stronger for dual use than for TC use alone. Temporality and a dose-response relationship were not determined due to the cross-sectional nature of the associations and lack of information on the intensity and duration of use. Another systematic review [23] concluded that EC use contributes independently to respiratory risk from TC use.

In-Vivo Animal Studies
The NASEM report [16] found that exposure to EC aerosol and intratracheal exposure to e-liquids was associated with increased airway hyper reactivity, abnormalities in pulmonary function, significant increase in interleukin-1β (IL-1β), changes in immunomodulatory cytokines in the airways, and increases in markers of inflammation in mice. The authors noted substantial methodological differences, such as variations in EC devices, pumps, solutions, and exposures, which limit the comparability of results. Moreover, the reviewed studies failed to consider confounding factors, such as aerosol temperature and particle size.
One review [27] found that acute exposure to nicotine-containing EC aerosols induced higher inflammatory cell influx and release of pro-inflammatory cytokines, while a subacute exposure showed impairment of inflammatory response in mice. Chronic inhalation resulted in the downregulation of innate immunity against viral pathogens. In another review [18], long-term EC aerosol exposure was reported to have led to reduced small airway function, emphysematous lung destruction, and transient bronchoconstriction in animal models.

In Vitro Cell Studies
Reviews suggest that EC aerosol exposure may cause changes in bronchial gene expression in human bronchial epithelial cells, similar to the effects from tobacco smoke [20,21].

Carcinogenic Effects
Five reviews considered carcinogenic effects of EC use [16,[18][19][20][21]. The duration of EC exposure for carcinogenic outcomes was not clearly defined. The NASEM report [16] found no evidence that EC use is associated with intermediate cancer endpoints in humans, both when compared with TC use and no use of tobacco products. Further, the report found limited evidence that the EC aerosol can have mutagenic effects or damage DNA in human and animal in vitro cell models. Other reviews indicated some potentially carcinogenic effects based on compounds found in human urine samples of EC users [18,21] and likely increased the risk of lung cancer in high-risk individuals based on in vitro evidence [19,20].

Discussion
Our umbrella review synthesized the evidence from systematic reviews examining the cardiovascular, respiratory/pulmonary, and carcinogenic effects of EC use from human, animal, and in vitro studies. Reviewed evidence from human studies suggests the potential for cardiovascular harm of EC use through acute increases in heart rate, systolic and diastolic blood pressure, endothelial dysfunction, arterial stiffness, and biomarkers of oxidative stress. The effects were seen both in TC smokers and non-smokers. Evidence of the association of long-term EC use and chronic changes in heart rate, blood pressure, cardiac geometry and function, and increased risk of cardiovascular events compared to non-use was insufficient.
For pulmonary health outcomes, short-term use of ECs was found to reduce lung defense mechanisms and impact lung function and overall peripheral airway resistance both in smokers and non-smokers, while long-term EC use was suggested to increase exacerbations in individuals with asthma. Switching from chronic TC smoking to longterm EC use showed a potential reduction in cardiovascular and pulmonary/respiratory harm, especially in individuals with asthma and COPD. Evidence on the effects of dual EC and TC use compared to using one product alone is limited but suggests that EC use could be an independent risk factor contributing to respiratory harm. For carcinogenic outcomes, there is limited evidence of mutagenic effects or DNA damage in humans, and no evidence of an association between EC use and intermediate or long-term cancer endpoints.
Nicotine constituents of ECs were often found to be responsible for the cardiovascular [15,16,18,22,29,30], respiratory [16,19,29], and potentially carcinogenic effects [21]. Our findings are consistent with a previous umbrella review [6], which concluded that EC use is associated with increased cardiovascular risk, although to a lesser extent than combustible cigarettes. In addition, our umbrella review found that while EC use is associated with respiratory harm, there is evidence that switching from chronic TC to EC use likely reduces the harmful impact on lung function and improves disease symptoms in individuals with asthma and COPD.
Our study highlights important methodological weaknesses of the existing systematic reviews. First, the assessment of methodological quality and risk of bias using the AMSTAR-2 checklist [11] revealed several limitations in the methodology or reporting of existing systematic reviews, particularly in the items considered by AMSTAR-2 as critically important. Only about 20% of the included systematic reviews reported having developed a written protocol prior to conducting the review, despite recommendations by Cochrane Collaboration [32], the National Academy of Medicine (formerly known as the Institute of Medicine) [33], and PRISMA guidelines [34] since 2009. An appropriate technique for systematic assessment of the quality and risk of bias, as recommended by the PRISMA Guidelines [10], was applied in only about 50% of reviews. Finally, only 10% of reviews provided a list of excluded studies. Notably, this recommendation was only added to the PRISMA Checklist in 2020. These findings are largely in line with existing evidence that despite several guidelines for conducting and reporting systematic reviews, less than a third of systematic reviews report having adhered to these guidelines [35]. The overall poor reporting across systematic reviews presents an important limitation in the current research on EC health effects. Systematic reviews should follow a structured and pre-defined process with rigorous methods to produce reliable findings that can help guide regulatory decisions.
Significant variations were also discovered in the timeframe of the examined health outcomes across the reviews. While some categorized health outcomes into acute and chronic, depending on the time-point of the health outcome assessment after EC exposure, others focused on the duration of EC use. For acute effects, if specified, the range spanned from minutes to hours, while for chronic effects, a much wider time range spanning from several days to several years was used, raising concerns regarding whether health effects assessed in a short span of several days can even be regarded as chronic.
Often, results were categorized into short-term and long-term outcomes without clear definitions, thereby, limiting the comparability of results across reviews. Moreover, considering that the ECs are relatively new to the tobacco marketplace, their true long-term health effects may not be observable until the middle of this century [36]. Perhaps the most important limitation of research in human health effects is that few systematic reviews distinguished health effects of EC use in smokers or former smokers, while many did not report the smoking status of study participants.
Since many EC users continue to smoke TCs [7] and may be at increased risk of cardiovascular disease compared to only cigarette smokers [37,38], the smoking status of study participants is needed to distinguish the impacts of EC from that of TC or dual use. Moreover, accounting not only for smoking status but also for smoking exposure with measures, such as intensity, duration, or pack-years, is needed in studies of EC health effects to properly estimate their independent risk [39]. Further, reviews generally did not account for the health status of human subjects. This presents an important issue, especially when estimating the health effects in those who transitioned from regular TC to EC use, as switching from combustible to EC use is often driven by poor health and existing health conditions.
A rapidly changing landscape of available EC devices and e-liquids imposes another major limitation of current research [40], particularly regarding the assessment of longerterm health effects. Underreporting EC device and e-liquid types was a critical limitation across most reviews, limiting the ability to compare health effects across different generations of EC devices and their e-liquid characteristics. Some reviews collected the data but did not utilize it for data analysis and interpretation. No reviews reported the effects of the more popular latest generation EC devices, such as JUUL, which appear to deliver nicotine in similar concentrations as combustible tobacco products [41].
In addition, each study only considered a specific set of EC brands, device types, and e-liquids and one mode of operation of the particular device, thereby, limiting the generalizability of their findings to other brands, e-liquids, and devices. Future systematic reviews should focus on clinical trials to minimize the variability in product devices, eliquids, individual product use patterns, and study designs across the included studies. We expect that clinical trials will provide more potential for developing well-defined outcomes.
In addition to the shortcomings of the systematic reviews identified in this umbrella review, some common limitations of the primary studies were reported in the included reviews. Human experimental studies that compared the health effects of ECs to those of traditional smoking often used inconsistent measurements of pre/post plasma nicotine levels in the two groups. In addition, those less experienced with vaping may have inhaled a lower volume of aerosol and thus experienced a lesser acute health impact.
In studies that assessed the health impact in smokers who switched to ECs, the study population often included those aiming to quit smoking along with those with no intention to quit, which could have influenced compliance in abstaining from smoking during the studies. Long-term smokers switching to EC use were often reported to have continued smoking during the study period, which may have confounded the outcomes of these studies. Further, potential and apparent conflicts of interest (COI) were reported in a substantial number of original studies due to funding by EC manufacturers. Such studies tended to draw conclusions supportive of EC use compared to studies without COI [15,21].
Our umbrella review is subject to limitations. First, our analysis included only those literature reviews that explicitly self-identified as systematic in the title, abstract, keyword, or methods. This method was selected as the most straightforward way to detect a systematic review. Nevertheless, it led to the exclusion of some widely-cited reviews [42][43][44] that did not claim to be systematic, while our umbrella review suggests that many of the included reviews should not be considered systematic due to their methodological shortcomings indicated by the AMSTAR-2 checklist.
However, although AMSTAR-2 was considered most suitable among existing quality appraisal tools for our review [12], it has been previously found to set very high standards for its quality classifications [14,[45][46][47]. A different quality appraisal tool may have yielded different results. The insufficient reporting of the effect sizes across the included systematic reviews precluded us from summarizing the evidence in quantitative terms. Further, our umbrella review included systematic reviews without imposing a restriction on publication date.
Although most reviews were published in the past four years, three were published prior to 2017, when evidence of health effects of ECs was more limited and dependent on earlier-generation devices. In addition, our review included evidence from in vitro and animal studies, which should be extrapolated with caution since laboratory exposure to EC aerosol may not accurately reflect real-life exposure in humans. Finally, original studies included in examined systematic reviews may have overlapped, thereby, overrepresenting findings of such studies in our umbrella review.

Conclusions
This umbrella review highlights the need for future systematic reviews with better adherence to established reporting guidelines. Such guidelines include developing and registering a peer-reviewed review protocol before the commencement of the review, adequately assessing the methodological quality and risk of bias in individual studies, and carefully examining the impact that potential conflicts of interest in individual studies may have on the outcomes. In addition, future reviews should adhere to a consistent definition of the duration of EC exposure (i.e., explicitly defined acute and long-term use) and of the device and e-liquid type, and should focus on the health effects of newer generation EC devices.
In reviews of human studies, EC use and the TC smoking status of participants should be systematically reported to distinguish the risks of vaping from those of smoking. When possible, adjustment for health status and cumulative history of smoking should be conducted. Meeting each of these needs will ensure that the evidence of the health consequences of EC use is clear and reliable and can thereby be more directly useful in developing effective and proportionate tobacco regulatory and policy interventions designed to protect public health.
Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/ijerph19159054/s1, File S1: The search strategy for the PubMed database; Table S1: List of excluded non-systematic literature reviews with relevant health outcomes; Table S2: Systematic reviews, meta-analyses, and included health outcome domains; Table S3: Included systematic reviews-extended table with results, conclusions and limitations.