Next Article in Journal
Estimation of the Nutrient and Chlorophyll a Reference Conditions in Taihu Lake Based on A New Method with Extreme–Markov Theory
Next Article in Special Issue
Infections in Healthcare Workers in Germany—22-Year Time Trends
Previous Article in Journal
Stress Testing the Capacity of Health Systems to Manage Climate Change-Related Shocks and Stresses
Previous Article in Special Issue
Use of Non-Steroidal Anti-Inflammatory Drugs and Associated Gastroprotection in a Cohort of Workers
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 15
Issue 11
10.3390/ijerph15112371
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Amyotrophic Lateral Sclerosis and Occupational Exposures: A Systematic Literature Review and Meta-Analyses

by
Lars-Gunnar Gunnarsson
1,2,* and
Lennart Bodin
3,4
1
School of Medical Sciences, Örebro University, SE 701 82 Örebro, Sweden
2
Department of Occupational and Environmental Medicine, Örebro University, SE 701 82 Örebro, Sweden
3
Department of Statistics, Örebro University, SE 701 82 Örebro, Sweden
4
Institute of Environmental Medicine, Karolinska Institute, SE 177 77 Stockholm, Sweden
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2018, 15(11), 2371; https://doi.org/10.3390/ijerph15112371
Submission received: 16 September 2018 / Revised: 16 October 2018 / Accepted: 24 October 2018 / Published: 26 October 2018
(This article belongs to the Special Issue Occupational Epidemiology)
Browse Figures
Versions Notes

Abstract

:
Objectives: We conducted a systematic literature review to identify studies fulfilling good scientific epidemiological standards for use in meta-analyses of occupational risk factors for amyotrophic lateral sclerosis (ALS). Methods: We identified 79 original publications on associations between work and ALS. The MOOSE (Meta-analysis Of Observational Studies in Epidemiology) and GRADE (Grading of Recommendations, Assessment, Development and Evaluations) guidelines were used to ensure high scientific quality, and reliable protocols were applied to classify the articles. Thirty-seven articles fulfilled good scientific standards, while 42 were methodologically deficient and thus were excluded from our meta-analyses. Results: The weighted relative risks for the various occupational exposures were respectively; 1.29 (95% confidence interval (CI): 0.97–1.72; six articles) for heavy physical work, 3.98 (95% CI: 2.04–7.77; three articles) for professional sports, 1.45 (95% CI: 1.07–1.96; six articles) for metals, 1.19 (95% CI: 1.07–1.33; 10 articles) for chemicals, 1.18 (95% CI: 1.07–1.31; 16 articles) for electromagnetic fields or working with electricity, and 1.18 (95% CI: 1.05–1.34; four articles) for working as a nurse or physician. Conclusions: Meta-analyses based only on epidemiologic publications of good scientific quality show that the risk of ALS is statistically significantly elevated for occupational exposures to excessive physical work, chemicals (especially pesticides), metals (especially lead), and possibly also to electromagnetic fields and health care work. These results are not explained by publication bias.

1. Introduction

The average onset of amyotrophic lateral sclerosis (ALS) is between 58 and 60 years of age and the incidence is around two per 100,000 persons [1]. The disease is characterized by an adult-onset progressive degeneration of the motor neurons. The resulting paresis begins focally, spreads contiguously and causes death from respiratory failure when breathing muscles become involved [2].
Fifty years ago, an observational study indicated that heavy work load was associated with ALS [3]. Since then many epidemiological studies have been performed on diverse risk factors. Eight systematic literature reviews with meta-analyses have been published regarding associations between ALS and occupational exposures. Exposure to electromagnetic fields was the focus of four studies [4,5,6,7], out of which two were meta-analyses [5,6]. They concluded that there might be a moderate association with ALS but the results were hampered by the heterogeneity between the included studies. Two meta-analyses showed that exposure to pesticides resulted in an almost doubled risk for ALS [8,9]. Another meta-analysis based on nine case-control studies showed that exposure to lead also involved an almost doubled risk for ALS [10]. The last meta-analysis focused on physical activity as a potential risk factor, but the results were inconclusive [11].
In two studies, attempts were made only to include publications of acceptable scientific quality [5,11]. All the other meta-analyses included all relevant publications, irrespective of whether the studies suffered from major weaknesses in design or ascertainment of diagnosis, were biased by missing data, or used insufficient statistical methods. Such faultiness might have hampered the weighted risk ratios, and so there is a need for meta-analyses based only on publications fulfilling good scientific standards.
One of the biggest public insurance companies in the Swedish labour market (AFA Insurance) needed a scientifically based standard for evaluating work-related disease and commissioned us to perform an updated foundation for decisions regarding both the prevention and compensation of damages. We conducted a systematic review of the published epidemiological studies on work-related exposures with regard to ALS. Meta-analyses were carried out on exposures/occupations identified in three or more studies that fulfilled good scientific standards. Our first report was published in 2014 in a Swedish peer-review scrutinized series of publications https://gupea.ub.gu.se/handle/2077/37930. The aim of the present publication is to update our previous review and meta-analyses and present the results of summary measures for an international audience.

2. Methods

2.1. Literature Search

We identified relevant published articles using bibliographic search engines in PubMed, Embase, and Arbline prior to 15 February 2017. Our search criteria were the MeSH terms (Medical Subject Headings) for study design (cohort, epidemiol*, epidemiologic studies) in combination with the MeSH terms for exposure (employment, workplace, professions, career, career choice, job, occupations, employment, occupational health, occupational medicine, occupational exposure, occupational injuries, occupational diseases, electromagnetic field) in combination with any of the MeSH terms for the disease of interest (motor neuron disease, amyotrophic lateral sclerosis, primary lateral sclerosis). This search produced 372 articles. After we had scrutinized the titles and/or abstracts, excluded a few duplicates and included only articles based on original data on exposures related to occupation, 135 articles remained. The full papers were read and 79 were found to be relevant.

2.2. Quality Classification

We scrutinized all relevant publications according to the checklist proposed by the MOOSE group [12]. According to the GRADE guidelines we considered selection bias and ‘falling off’ [13], as well as the occurrence of dose-response effects [14]. We also used a system for grading the degree of evidence in observational epidemiologic articles into a global class ranging from I–V, as proposed by Armon [15]. Based on these documents, we constructed a decision protocol (shown in the Appendix A) involving the categories Diagnosis, Exposure, Study Group (selection, controls, missing data), Methods and Analysis, Armon Global Score, and Funding and Exposures; see Table 1 and Table 2. The checklist by Armon and our decision protocol was also presented in a recent publication [16]. A slightly updated version is presented in the Appendix A of the present publication. The quality of the diagnosis was graded as: 1 = the El Escorial criteria [17] were applied, 2 = diagnosis from a hospital (in-patients), 3 = diagnosis from a general practitioner (GP) (also including mortality registers), and 4 = ALS was not separated from other types of muscular atrophies. The other categories were graded 1 = good, 2 = sufficient, 3 = uncertain/insufficient, or 4 = unacceptable. Sometimes a category was graded in between, and thus was given an interval, for example 2–3. The reason for this was usually lack of sufficient information for carrying out a more specific categorization.
Prerequisites for accepting a publication as fulfilling good scientific standards (Armon global score II or III) were that diagnosis had to achieve a score of 1–3 and all the other categories should be scored as 1–2 as a single score or be included in the interval 2–3. Articles not qualifying for global score II–III were impaired by serious weaknesses (Armon score IV) or should not be paid attention to (Armon score V). None of the publications fulfilled Armon global score I, which almost requires an experimental design.
Only studies [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54] fulfilling good scientific standards (Armon class II or III) were used in our meta-analyses; see Table 1. Another 46 relevant publications [3,9,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98] not fulfilling good scientific standards regarding the exposures under study are summarized in Table 2.

2.3. Statistical Analysis

Risk estimates from the selected studies are reported as relative risks (RR), as the outcome is rare, and so odds ratios (OR) and hazard ratios (HR) can be considered equivalent to the RR. When unadjusted as well as multivariable adjusted risk estimates were reported, we only considered the adjusted estimates. Studies which reported stratified estimates for sex were considered as separate studies and included with both estimates. When exposure was categorized into different levels, the risk rate for the highest level was used according to the principle of dose-response [99] provided a sufficient number of exposed cases was observed, usually around 30 or more. Estimates based on an extremely small number of individuals were not included, as their effect on the combined estimate would be negligible.
We examined the fixed effects model as well as the random effects model by considering statistical heterogeneity. To this end, we used the I2 statistic where the recommended cut-offs of 25%, 50%, and 75% degrees of heterogeneity were considered. We stratified on study characteristics (gender, source of funding, exposure characteristics, previous versus recent studies), selected a priori, and used meta-regression to evaluate the significance of the stratification variable. Based on the I2 criterion and the meta-regression, a random effects model was found to be the most appropriate choice in all of our analyses; hence the results are reported only with random effects estimates. The weights used for pooling the risk estimates were equal to the inverse-variance weighting. Pooled risk estimates are presented with 95% confidence intervals (CI). We also performed leave-one-out analysis for each study, to check its influence on the combined estimate as well as the degree of heterogeneity. To illustrate the progress in accumulated information about the influence of occupational exposure on ALS, we used a cumulative meta-analysis where the pooled estimate is updated each time the results of a new study have been published—a meta-cumulative approach.
Publication bias was analysed by inspection of the funnel plot, in which the estimates of RR should be distributed symmetrically around the weighted RR unless the publication was affected by bias. The rank correlation test proposed by Begg and Mazumdar [100] was used to supplement the interpretation of the funnel plot. Statistical analyses were conducted using procedures for different aspects of meta-analysis available in STATA software (version 14.2, StataCorp, College Station, TX, USA), and described in articles from the STATA journal [101].

3. Results

3.1. Exposure to Heavy Physical Activities at Work

Since the object of our study was restricted to occupational exposures, only risk rates directly related to physical activities at work were considered. Published reviews and meta-analyses have shown that the RR is not elevated for moderate physical activity and thus we only included the results from studies on occupational exposure to heavy physical work [18,25,30,37,50,53] and likewise work as a professional athlete [20,36,50]. The weighted RRs were 1.29 (95% CI 0.97–1.72) and 3.98 (95% CI 2.04–7.77) respectively and the overall weighted RR was 1.89 (95% CI 1.27–2.82) (Figure 1). The difference in RR between the two exposure groups was significant according to the meta-regression, p = 0.017. There was a slight publication bias indicated in the funnel plot for heavy physical work, also shown by Begg’s test. The heterogeneity in each subgroup was lower than in the combined group and leave-one-out of Vanacore’s [50] RR for sports completely eliminated the heterogeneity for the stratum of professional sports. In the stratified analysis for gender, those studies that included both men and women had an RR of 1.45 (95% CI 1.06–2.00) and a comparably low I2 = 39.7%, and those with only men had an RR of 2.58 (95% CI 0.85–7.81) and a high heterogeneity I2 = 93.8%. However, the difference in RR was not statistically significant, p = 0.38.

3.2. Exposure to Chemicals

The relevant studies are presented in Figure 2 and the weighted RR was slightly elevated; 1.19 (95% CI 1.07–1.33). Pesticides are designed to disturb biological systems and involve a significant risk for Parkinson’s disease [16]. Thus, we made a separate sub-analysis on the risk for ALS when occupationally exposed to agricultural work and/or pesticides, which was addressed in five publications [19,26,38,40,51]. The weighted RR was 1.35 (95% CI 1.02–1.79). Exposure to other chemicals was considered in eight studies [24,27,38,40,42,43,49,52] and the weighted RR was 1.14 (95% CI 1.02–1.28). The two exposure strata had similar heterogeneity as the combined data and their RRs were not different in the meta-regression test, p = 0.44.
The funnel plot (Figure 3) indicated no publication bias, supported by Begg’s test. Regarding exposure to agricultural chemicals, the heterogeneity was considerable. In the group “Other chemicals”, the heterogeneity was 41% but was reduced to 19% if the RR for solvents in the study by Peters was left out.
Stratified meta-analysis regarding funding resulted in a weighted RR of 2.11 for those three studies that had not received public funding [19,24,49] compared to an RR of 1.14 for all the other studies. This difference was statistically significant, p = 0.044. Other stratifications had only a small effect on RR and heterogeneity.

3.3. Exposure to Metals

The weighted RR for exposure to lead [21,35,42] was 1.51 (95% CI 0.96–2.38) and exposure to metals from different sources (plants, shipyards, grinding) [24,27,38] resulted in a weighted RR of 1.43 (95% CI 0.95–2.14). The overall weighted RR was statistically significantly elevated at 1.45 (95% CI 1.07–1.96) (Figure 4). Taking into consideration that there were only six studies, there was an evident publication bias seen in the funnel plots and also by Begg’s test. The leave-one-out analysis showed that the heterogeneity vanished if the studies by Peters and Gait respectively were left out. The analysis of strata showed that heterogeneity was not present for the exposure to other metals, I2 = 0.0%; and was low for non-public funding, I2 = 17.7%, but the number of studies is very small in this stratified analysis.
Welding involves both exposure to fumes of vaporized metals and high electromagnetic fields. Focusing only on the first aspect (vaporized metals), we used risk rates based on years at work with welding and the weighted RR was 0.95 (95% CI 0.70–1.29), (Figure 5) [22,23,27,32,40,48,51,54]. The heterogeneity was apparent and leaving out the big study by Feychting [22], it was reduced from 82% to 77%.

3.4. Exposure to Electromagnetic Fields and Work with Electricity

Work with electricity in relation to ALS has been extensively studied (Table 1). The exposures can be separated into electrical accidents/trauma and electromagnetic fields (EMF) per se. The level of EMF exposure was estimated through a job exposure matrix (JEM) established by occupational hygienists [22,31,32,33,39,40,41,44,47,54], and the weighted RR was 1.23 (95% CI 1.04–1.45), which is statistically significant (Figure 6).
When the focus is on electrical accidents/trauma, the exposure is based on job titles and years of work with electricity [22,23,26,31,34,39,45,46,51,54], resulting in the slightly elevated weighted RR = 1.16 (95% CI 1.00–1.35). Meta-regression showed that there was no statistically significant difference between the two types of exposure, p = 0.57, and the heterogeneity was almost the same. Stratification on gender gave RR = 1.38 for studies with men only, RR = 1.10 for both men and women and RR = 0.90 for women only. The difference in RR for these three groups was, however, not statistically significant, p = 0.74. The heterogeneity was substantially lower for studies with men only and with women only (I2 = 36.0% and 38.8% respectively).
A slight publication bias indicated by the funnel plot was supported by Begg’s test. The leave-one-out test showed that the RR for EMF differed between 1.16 and 1.32 and for occupations working with electricity between 1.12 and 1.21, and had thus no significant impact on the results. The highest impact on heterogeneity was the exclusion of the study by Johansen [33], where heterogeneity for studies with EMF decreased from 48.7% to 33.5%, but the impact on the value of RR was more modest, from 1.23 to 1.16.
A meta-analysis over the time period 1991–2015 in which the results of earlier studies were gradually cumulated showed a successive reduction of RR (Figure 7). For the period 1991–2005, the weighted RR was 1.33 (95% CI 1.11–1.58), compared to the period 2006–2015 with a weighted RR of 1.18 (95% CI 1.07–1.31).

3.5. Nursing and Medical Treatment

Previous literature reviews have not indicated that exposure to medical work might involve increased risk for ALS. However, in four publications, the risk posed by working as a nurse or physician has been addressed [26,40,42,51]. The heterogeneity was low and thus the weighted risk estimate was based on the fixed model resulting in an RR of 1.18 (95% CI 1.05–1.34).

4. Discussion

Meta-analyses based only on epidemiologic publications of good scientific quality showed that the risk of amyotrophic lateral sclerosis was statistically significantly elevated for occupational exposures to heavy physical work, metals (especially lead) and chemicals (especially pesticides).
In the old study by Breland [3], elevated risk of ALS was stated for heavy work load involving the occupations farmers/farmworkers, construction workers and professional athletes. Our results support that these groups have a higher risk of being afflicted by ALS.
The risk for ALS in professional athletes was four times higher than that of the reference population, while high work load in other occupations only involved on average a 30% increased risk. The Italian study [20] included 7000 professional soccer players together with 1900 basketball players and 1700 cyclists. The mortality from ALS was not elevated among the latter two groups of athletes. The possibility that other exposures (for example chemicals) related to playing soccer may also be risk factors for ALS is also discussed. Since our meta-analysis showed that exposure to pesticides or agricultural chemicals resulted in a 35% increased risk, one mechanism might be that a combination of sports-related muscle trauma and pesticides associated with the pitch contribute to the four-fold greater risk for ALS.
A recent case-control study (175 patients and 317 controls) of good scientific quality using a validated questionnaire for historical physical activity showed that casual exercise-related physical activity was associated with an elevated risk of ALS (adjusted RR 1.28 with 95% CI 1.06–1.55), while work-related physical activity did not involve an increased risk (adjusted RR 1.02 with 95% CI 0.97–1.08) [28]. That study did not present risk rates for occupationally heavy physical work and could thus not be included in our meta-analyses. However, the risk rate for casual exercise almost equalled the weighted RR for heavy occupational physical work in our meta-analysis.
In a study among 2.5 million military personnel, 107 cases of ALS were identified. Those who were deployed to the Gulf Region during the Gulf War had an almost two-fold greater risk of ALS compared to those who did not participate in the Gulf War [102]. Risk factors discussed are both physical trauma and chemicals. Confounders such as head trauma (in the case of Gulf veterans and professional athletes) and increased metabolism [103] caused by heavy physical work and agricultural work also might have an impact on the risk for ALS.
Construction workers are exposed to both muscle trauma and metals. The latter exposure involved an almost 50% increased risk according to our meta-analysis. Exposure to lead was assessed in three studies, of which two fulfilled the best scientific standards (global class II) where the diagnosis was assessed by a neurologist [21,35]. One study was based on ALS patients and information on exposure was gathered from a questionnaire [35], and the other study only involved US veterans exposed to lead from firing practice [21]. In both studies, the patients afflicted with ALS also had elevated levels of lead in their blood compared to controls. The third study (global score III) was based on hospital registers for patients with ALS compared to population controls [42]. Exposure was estimated from the Swedish census and a job exposure matrix; thus, the quality of both the diagnosis and exposure was less valid than in the other two studies. To sum up, the RR of 1.90 for lead exposure from the first two studies is considerably more valid than the RR of 1.07 from the last study.
In comparison, there is a study of the levels of lead in the blood of patients with ALS in an Italian agricultural area [104]. This study was not included in our meta-analysis since data on exposure were missing. However, the patients had significantly higher blood levels of lead than population controls.
Occupational exposure to pesticides involved a statistically significant elevated risk (RR 1.35). The highest impact on the RR was shown in the Italian study by Bovinci [19] involving 41 cases and 82 controls. Although this study was funded by a patient organization, the study design was scientifically appropriate and fulfilled the criteria for Armon class II. Thus, in this case, the lack of public funding did not bias the study.
During the turn of the millennium, some studies indicated a doubled risk of ALS after exposure to electromagnetic fields or work with electricity. This stimulated further studies of different design and the cumulative risk rate has gradually diminished (Figure 7), probably because publication bias was more common before 2005. Possibly harmful exposures might be associated with EMF, such as electric shocks causing muscle trauma. In selecting appropriate risk rates from the publications, we tried to differentiate between exposures to EMF or work with electricity, but the weighted risk rates were almost identical. Welding involves exposures to strong electromagnetic fields and welders are included in the job exposure matrices used to estimate exposure to EMF. However, a separate meta-analysis of welders showed no risk (RR = 0.95), which might indicate that exposures other than EMF might involve risk for ALS in this population.
Some studies indicate a 50% increased risk of ALS among health care personnel [26,40,51]. A nation-wide nested case-control study [42] of ALS in Sweden from 1991–2010 resulted in an RR of 1.13 (95% CI 0.99–1.29) and the weighted cumulated RR over time is only slightly elevated.
General limitations of meta-analyses are that the calculations can only be based on published data and will reflect any inherent weaknesses of design in the studies included. Furthermore, all previously published meta-analyses on ALS have been based on all relevant publications identified, irrespective of the quality of the study design.
One strength of our study is that our meta-analyses were based on a systematic literature review, only including studies that fulfilled scientifically high-quality standards. Based on the detailed checklist proposed by Armon [15], we used an elaborated protocol for scrutinizing publications [16]. Another strength of our meta-analyses is that we focused heavily on finding all possible sources of bias, using stratification of data with regard to possible confounders such as gender and funding. We also looked for publication bias using both funnel plots and a test for publication bias.
In 25 out of the 37 studies, the diagnosis of ALS was made by a general practitioner or a mortality register [20,22,23,24,25,26,28,31,32,34,36,39,40,41,42,43,44,45,46,48,49,50,51,52,54]. In these studies, true cases of ALS (fulfilling the common diagnostic criteria) might have been mixed with a few cases of motor neuron disease mimics. However, such non-differential misdiagnoses are not expected to bias the risk estimates, but only cause wider confidence intervals.
Regarding exposure [15], we made sure that only exposures occurring a few years before the start of overt disease were considered and that methods of gathering information on exposure were the same for both affected and healthy individuals. In some studies, exposure was assessed by occupational hygienists with a job exposure matrix or established by a specifically exposed cohort. In the case of questionnaires/interviews, it is also more likely that occupational hygienists were involved (it was not always clear from the text). The heterogeneity of results between studies could be explained by the use of different measures regarding exposure, comparing results from JEM with questionnaires/interviews or exposed cohorts. The leave-one-out analyses verified that this was one of the main reasons for heterogeneity. Another factor that could lead to increased heterogeneity is the variation in geography, that is, nations where the exposure took place. Working conditions are certainly not the same in different countries, but the data at our disposal is too sparse to analyse this in depth. Several study groups were hampered by bias caused by missing data (low response rate) from cases, controls or inappropriate matching of controls. Regarding methods and analysis, attention was paid to sources of bias and confounding as well as the use of appropriate statistical methods. Based on all these parameters, both authors individually examined all relevant publications using a structured template summarizing different aspects of design, exposure, methods and results. The results of our examinations were condensed into the graded scores in Table 1. If our scores were divergent, we re-examined the publication and, after discussion, reached consensus. After a ‘run-in period’, we got used to the protocol for grading different aspects of studies and the consensus between us became almost complete; there were only very few cases where a consensus discussion was necessary. However, there is always room for a reader’s own discretion when judging a publication.

5. Conclusions

Our meta-analyses suggest positive associations between ALS and occupational exposures to excessive physical work, pesticides and possibly also to electromagnetic fields and health care work, and the results are not explained by publication bias. A positive association was also found for metals, but the few available studies indicated some degree of publication bias, making the results somewhat less valid. Further epidemiological studies should also consider confounders such as head trauma and increased metabolism, which is associated with heavy physical work and agricultural work.

Author Contributions

Both authors take full responsibility for the data, the analyses, and interpretation. We have both scrutinized all publications included in the systematic literature review and meta-analyses and have interpreted all results only on the basis of scientific grounds.

Funding

This research received no external funding.

Acknowledgments

The study is based on a Swedish report published in 2015 that was supported by a grant from AFA Insurance (which is owned by Sweden’s labour market parties) and the Department of Occupational and Environmental Medicine, Sahlgrenska Academy, University of Gothenburg, Sweden.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Appendix A.1. Decision Protocol

A modified system for categorisation of observational epidemiologic articles into classes I–V (based on Armon [15]) was applied for cohort studies with parallel controls and case-control studies, respectively. This appendix is based on a recent publication [16] and has been further adjusted.

Appendix A.2. Diagnosis

Cohort studies:
  • Diagnosis of the studied disease made by applying uniform efforts and criteria to exposed and unexposed cohorts.
Case-control studies:
  • Demonstration that ascertainment of patients is complete in the given population.
  • Diagnosis of the studied disease made by applying established criteria.

Appendix A.3. Exposure

Cohort studies:
  • Exposure, and hence assignment to the ‘exposed’ cohort, established before knowledge of diagnostic status, or without knowledge of diagnostic status, or confirmable independently of the knowledge of diagnostic status. Consideration of and accounting for possible misclassification.
  • Unexposed cohort is appropriate to the risk factor in question, is well-matched to the exposed cohort on factors other than the exposure, and is otherwise representative of the general population.
  • Exposure quantified, where possible, to permit assessment of dose-response relationships.
Case-control studies:
  • Putative risk factor or exposure occurred before probable biologic onset of disease.
  • Uniform effort to gather information equally from affected and unaffected individuals.
  • Blinding of the information-gathering method to individuals’ disease status ideal; if not done—adequate justification as to why this does not affect the assessment of the risk factor in question.
  • Blinding of subjects and individuals gathering the data as to the hypotheses being tested. If not done—adequate justification as to why this does not affect the assessment of the risk factor in question.
  • Meticulous attention to avoiding recall bias or, if not possible, to evaluating its impact, estimating the magnitude of its impact and controlling for it.
  • Exposure quantified, where possible, to permit assessment of dose-response.

Appendix A.4. Study Group

Cohort studies:
  • Loss to follow-up low, and comparable in exposed and unexposed cohorts. Possible roles of competing causes of mortality accounted for. Preferably, all mortality data available for both cohorts.
Case-control studies:
  • Appropriate choice of controls to assure they are matched to the patients and are also representative of the general population (assure adequate matching, avoid ‘overmatching’).
  • High response rates from patients and controls.

Appendix A.5. Methods and Analysis

Cohort studies:
  • Sources of biases and confounding identified and accounted for.
  • Conclusion based on large numbers. Appropriate statistical analysis.
Case-control studies:
  • Sources of biases and confounding identified and accounted for.
  • Conclusions based on large numbers. Appropriate statistical analysis. Methods state if hypotheses were selected a priori for confirmatory analysis.

Appendix A.6. Criteria for Categorization into Armon Class

In order to form the basis of the Armon classes I–V, we constructed a protocol with standardised criteria for grading the categories Diagnosis, Exposure, Study Group and Methods and Analysis. The protocol was graded into scores of 1 to 4 as defined below. Because of a lack of information, sometimes a category had to be graded in between, and thus given an interval, for example 2–3.
Diagnosis: 1 = the specific disease diagnosis was established by a specialist, 2 = diagnosis obtained from a hospital register (in-patients), 3 = diagnosis from a GP (also including mortality registers), and 4 = the specific disease diagnosis was not separated from other diagnoses with similar phenotype.
Exposure, Study Group and Methods and Analysis: 1 = good, 2 = sufficient, 3 = uncertain/insufficient, or 4 = unacceptable.
  • Class I: All criteria were met within the respective design, i.e., a single score of 1 in every category.
  • Class II: Single score of 1 or 2 in every category.
  • Class III: Single score of 3 allowed for Diagnosis and 1 or 2 in the other categories (interval 2–3 is allowed).
  • Class IV: Single score of 3 in categories other than Diagnosis.
  • Class V: Single score of 4 in any category.

References

  1. Talbott, E.O.; Malek, A.M.; Lacomis, D. The epidemiology of amyotrophic lateral sclerosis. In Handbook of Clinical Neurology; Elsevier: Amsterdam, The Netherlands, 2016; pp. 225–238. [Google Scholar]
  2. Bidhendi, E.E.; Bergh, J.; Zetterstrom, P.; Andersen, P.M.; Marklund, S.L.; Brannstrom, T. Two superoxide dismutase prion strains transmit amyotrophic lateral sclerosis-like disease. J. Clin. Investig. 2016, 126, 2249–2253. [Google Scholar] [CrossRef] [PubMed]
  3. Breland, A.E.; Currier, R.D. Multiple sclerosis and amyotrophic lateral sclerosis in Mississippi. Neurology 1967, 17, 1011–1016. [Google Scholar] [CrossRef] [PubMed]
  4. Ahlbom, I.C.; Cardis, E.; Green, A.; Linet, M.; Savitz, D.; Swerdlow, A. Review of the epidemiologic literature on EMF and Health. Environ. Health Perspect. 2001, 109, 911–933. [Google Scholar] [CrossRef] [PubMed]
  5. Capozzella, A.; Sacco, C.; Chighine, A.; Loreti, B.; Scala, B.; Casale, T.; Sinibaldi, F.; Tomei, G.; Giubilati, R.; Tomei, F.; et al. Work related etiology of amyotrophic lateral sclerosis (ALS): A meta-analysis. Ann. Igiene 2014, 26, 456–472. [Google Scholar] [CrossRef]
  6. Vergara, X.; Kheifets, L.; Greenland, S.; Oksuzyan, S.; Cho, Y.S.; Mezei, G. Occupational exposure to extremely low-frequency magnetic fields and neurodegenerative disease: A meta-analysis. J. Occup. Environ. Med. 2013, 55, 135–146. [Google Scholar] [CrossRef] [PubMed]
  7. Zhou, H.; Chen, G.; Chen, C.; Yu, Y.; Xu, Z. Association between extremely low-frequency electromagnetic fields occupations and amyotrophic lateral sclerosis: A meta-analysis. PLoS ONE 2012, 7, e48354. [Google Scholar] [CrossRef] [PubMed]
  8. Malek, A.M.; Barchowsky, A.; Bowser, R.; Youk, A.; Talbott, E.O. Pesticide exposure as a risk factor for amyotrophic lateral sclerosis: A meta-analysis of epidemiological studies: Pesticide exposure as a risk factor for ALS. Environ. Res. 2012, 117, 112–119. [Google Scholar] [CrossRef] [PubMed]
  9. Kamel, F.; Umbach, D.M.; Bedlack, R.S.; Richards, M.; Watson, M.; Alavanja, M.C.; Blair, A.; Hoppin, J.A.; Schmidt, S.; Sandler, D.P. Pesticide exposure and amyotrophic lateral sclerosis. Neurotoxicology 2012, 33, 457–462. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Wang, M.D.; Gomes, J.; Cashman, N.R.; Little, J.; Krewski, D. A meta-analysis of observational studies of the association between chronic occupational exposure to lead and amyotrophic lateral sclerosis. J. Occup. Environ. Med. 2014, 56, 1235–1242. [Google Scholar] [CrossRef] [PubMed]
  11. Lacorte, E.; Ferrigno, L.; Leoncini, E.; Corbo, M.; Boccia, S.; Vanacore, N. Physical activity, and physical activity related to sports, leisure and occupational activity as risk factors for ALS: A systematic review. Neurosci. Biobehav. Rev. 2016, 66, 61–79. [Google Scholar] [CrossRef] [PubMed]
  12. Stroup, D.F.; Berlin, J.A.; Morton, S.C.; Olkin, I.; Williamson, G.D.; Rennie, D.; Moher, D.; Becker, B.J.; Sipe, T.A.; Thacker, S.B. Meta-analysis of observational studies in epidemiology: A proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000, 283, 2008–2012. [Google Scholar] [CrossRef] [PubMed]
  13. Guyatt, G.H.; Oxman, A.D.; Vist, G.; Kunz, R.; Brozek, J.; Alonso-Coello, P.; Montori, V.; Akl, E.A.; Djulbegovic, B.; Falck-Ytter, Y.; et al. GRADE guidelines: 4. Rating the quality of evidence—Study limitations (risk of bias). J. Clin. Epidemiol. 2011, 64, 407–415. [Google Scholar] [CrossRef] [PubMed]
  14. Guyatt, G.H.; Oxman, A.D.; Sultan, S.; Glasziou, P.; Akl, E.A.; Alonso-Coello, P.; Atkins, D.; Kunz, R.; Brozek, J.; Montori, V.; et al. GRADE guidelines: 9. Rating up the quality of evidence. J. Clin. Epidemiol. 2011, 64, 1311–1316. [Google Scholar] [CrossRef] [PubMed]
  15. Armon, C. An evidence-based medicine approach to the evaluation of the role of exogenous risk factors in sporadic amyotrophic lateral sclerosis. Neuroepidemiology 2003, 22, 217–228. [Google Scholar] [CrossRef] [PubMed]
  16. Gunnarsson, L.G.; Bodin, L. Parkinson’s disease and occupational exposures: A systematic literature review and meta-analyses. Scand. J. Work. Environ. Health 2017, 43, 197–209. [Google Scholar] [CrossRef] [PubMed]
  17. Brooks, B.R.; Miller, R.G.; Swash, M.; Munsat, T.L. El Escorial revisited: Revised criteria for the diagnosis of amyotrophic lateral sclerosis. Amyotroph. Lateral Scler. Other Motor Neuron Disord. 2000, 1, 293–299. [Google Scholar] [CrossRef] [PubMed]
  18. Beghi, E.; Logroscino, G.; Chio, A.; Hardiman, O.; Millul, A.; Mitchell, D.; Swingler, R.; Traynor, B.J. Amyotrophic lateral sclerosis, physical exercise, trauma and sports: Results of a population-based pilot case-control study. Amyotroph. Lateral Scler. 2010, 11, 289–292. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Bonvicini, F.; Marcello, N.; Mandrioli, J.; Pietrini, V.; Vinceti, M. Exposure to pesticides and risk of amyotrophic lateral sclerosis: A population-based case-control study. Ann. Ist. Super. Sanita 2010, 46, 284–287. [Google Scholar] [CrossRef] [PubMed]
  20. Chio, A.; Calvo, A.; Dossena, M.; Ghiglione, P.; Mutani, R.; Mora, G. ALS in Italian professional soccer players: The risk is still present and could be soccer-specific. Amyotroph. Lateral Scler. 2009, 10, 205–209. [Google Scholar] [CrossRef] [PubMed]
  21. Fang, F.; Kwee, L.C.; Allen, K.D.; Umbach, D.M.; Ye, W.; Watson, M.; Keller, J.; Oddone, E.Z.; Sandler, D.P.; Schmidt, S.; et al. Association between blood lead and the risk of amyotrophic lateral sclerosis. Am. J. Epidemiol. 2010, 171, 1126–1133. [Google Scholar] [CrossRef] [PubMed]
  22. Feychting, M.; Jonsson, F.; Pedersen, N.L.; Ahlbom, A. Occupational magnetic field exposure and neurodegenerative disease. Epidemiology 2003, 14, 413–419. [Google Scholar] [CrossRef] [PubMed]
  23. Fischer, H.; Kheifets, L.; Huss, A.; Peters, T.L.; Vermeulen, R.; Ye, W.; Fang, F.; Wiebert, P.; Vergara, X.P.; Feychting, M. Occupational Exposure to Electric Shocks and Magnetic Fields and Amyotrophic Lateral Sclerosis in Sweden. Epidemiology 2015, 26, 824–830. [Google Scholar] [CrossRef] [PubMed]
  24. Gait, R.; Maginnis, C.; Lewis, S.; Pickering, N.; Antoniak, M.; Hubbard, R.; Lawson, I.; Britton, J. Occupational exposure to metals and solvents and the risk of motor neuron disease. A case-control study. Neuroepidemiology 2003, 22, 353–356. [Google Scholar] [CrossRef] [PubMed]
  25. Gallo, V.; Vanacore, N.; Bueno-de-Mesquita, H.B.; Vermeulen, R.; Brayne, C.; Pearce, N.; Wark, P.A.; Ward, H.A.; Ferrari, P.; Jenab, M.; et al. Physical activity and risk of Amyotrophic Lateral Sclerosis in a prospective cohort study. Eur. J. Epidemiol. 2016, 31, 255–266. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Gunnarsson, L.G.; Lindberg, G.; Soderfeldt, B.; Axelson, O. Amyotrophic lateral sclerosis in Sweden in relation to occupation. Acta Neurol. Scand. 1991, 83, 394–398. [Google Scholar] [CrossRef] [PubMed]
  27. Gunnarsson, L.G.; Bodin, L.; Soderfeldt, B.; Axelson, O. A case-control study of motor neurone disease: Its relation to heritability, and occupational exposures, particularly to solvents. Br. J. Ind. Med. 1992, 49, 791–798. [Google Scholar] [CrossRef] [PubMed]
  28. Harwood, C.A.; Westgate, K.; Gunstone, S.; Brage, S.; Wareham, N.J.; McDermott, C.J.; Shaw, P.J. Long-term physical activity: An exogenous risk factor for sporadic amyotrophic lateral sclerosis? Amyotroph. Lateral Scler. Front. Degener. 2016, 17, 377–384. [Google Scholar] [CrossRef] [PubMed]
  29. Horner, R.D.; Grambow, S.C.; Coffman, C.J.; Lindquist, J.H.; Oddone, E.Z.; Allen, K.D.; Kasarskis, E.J. Amyotrophic lateral sclerosis among 1991 Gulf War veterans: Evidence for a time-limited outbreak. Neuroepidemiology 2008, 31, 28–32. [Google Scholar] [CrossRef] [PubMed]
  30. Huisman, M.H.; Seelen, M.; de Jong, S.W.; Dorresteijn, K.R.; van Doormaal, P.T.; van der Kooi, A.J.; de Visser, M.; Schelhaas, H.J.; van den Berg, L.H.; Veldink, J.H. Lifetime physical activity and the risk of amyotrophic lateral sclerosis. J. Neurol. Neurosurg. Psychiatry 2013, 84, 976–981. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Huss, A.; Spoerri, A.; Egger, M.; Kromhout, H.; Vermeulen, R. Occupational exposure to magnetic fields and electric shocks and risk of ALS: The Swiss National Cohort. Amyotroph. Lateral Scler. Front. Degener. 2015, 16, 80–85. [Google Scholar] [CrossRef] [PubMed]
  32. Hakansson, N.; Gustavsson, P.; Johansen, C.; Floderus, B. Neurodegenerative diseases in welders and other workers exposed to high levels of magnetic fields. Epidemiology 2003, 14, 420–426, discussion 427–8. [Google Scholar] [CrossRef] [PubMed]
  33. Johansen, C.; Olsen, J.H. Mortality from amyotrophic lateral sclerosis, other chronic disorders, and electric shocks among utility workers. Am. J. Epidemiol. 1998, 148, 362–368. [Google Scholar] [CrossRef] [PubMed]
  34. Johansen, C. Exposure to electromagnetic fields and risk of central nervous system disease in utility workers. Epidemiology 2000, 11, 539–543. [Google Scholar] [CrossRef] [PubMed]
  35. Kamel, F.; Umbach, D.M.; Munsat, T.L.; Shefner, J.M.; Hu, H.; Sandler, D.P. Lead exposure and amyotrophic lateral sclerosis. Epidemiology 2002, 13, 311–319. [Google Scholar] [CrossRef] [PubMed]
  36. Lehman, E.J.; Hein, M.J.; Baron, S.L.; Gersic, C.M. Neurodegenerative causes of death among retired National Football League players. Neurology 2012, 79, 1970–1974. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Longstreth, W.T.; McGuire, V.; Koepsell, T.D.; Wang, Y.; van Belle, G. Risk of amyotrophic lateral sclerosis and history of physical activity: A population-based case-control study. Arch. Neurol. 1998, 55, 201–206. [Google Scholar] [CrossRef] [PubMed]
  38. McGuire, V.; Longstreth, W.T., Jr.; Nelson, L.M.; Koepsell, T.D.; Checkoway, H.; Morgan, M.S.; van Belle, G. Occupational exposures and amyotrophic lateral sclerosis. A population-based case-control study. Am. J. Epidemiol. 1997, 145, 1076–1088. [Google Scholar] [CrossRef] [PubMed]
  39. Noonan, C.W.; Reif, J.S.; Yost, M.; Touchstone, J. Occupational exposure to magnetic fields in case-referent studies of neurodegenerative diseases. Scand. J. Work. Environ. Health 2002, 28, 42–48. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Park, R.M.; Schulte, P.A.; Bowman, J.D.; Walker, J.T.; Bondy, S.C.; Yost, M.G.; Touchstone, J.A.; Dosemeci, M. Potential occupational risks for neurodegenerative diseases. Am. J. Ind. Med. 2005, 48, 63–77. [Google Scholar] [CrossRef] [PubMed]
  41. Parlett, L.E.; Bowman, J.D.; van Wijngaarden, E. Evaluation of occupational exposure to magnetic fields and motor neuron disease mortality in a population-based cohort. J. Occup. Environ. Med. 2011, 53, 1447–1451. [Google Scholar] [CrossRef] [PubMed]
  42. Peters, T.L.; Kamel, F.; Lundholm, C.; Feychting, M.; Weibull, C.E.; Sandler, D.P.; Wiebert, P.; Sparen, P.; Ye, W.; Fang, F. Occupational exposures and the risk of amyotrophic lateral sclerosis. Occup. Environ. Med. 2017, 74, 87–92. [Google Scholar] [CrossRef] [PubMed]
  43. Ruder, A.M.; Hein, M.J.; Hopf, N.B.; Waters, M.A. Mortality among 24,865 workers exposed to polychlorinated biphenyls (PCBs) in three electrical capacitor manufacturing plants: A ten-year update. Int. J. Hyg. Environ. Health 2014, 217, 176–187. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Savitz, D.A.; Checkoway, H.; Loomis, D.P. Magnetic field exposure and neurodegenerative disease mortality among electric utility workers. Epidemiology 1998, 9, 398–404. [Google Scholar] [CrossRef] [PubMed]
  45. Savitz, D.A.; Loomis, D.P.; Tse, C.K. Electrical occupations and neurodegenerative disease: Analysis of U.S. mortality data. Arch. Environ. Health 1998, 53, 71–74. [Google Scholar] [CrossRef] [PubMed]
  46. Sorahan, T.; Kheifets, L. Mortality from Alzheimer’s, motor neurone and Parkinson’s disease in relation to magnetic field exposure: Findings from the study of UK electricity generation and transmission workers, 1973–2004. Occup. Environ. Med. 2007. [Google Scholar] [CrossRef] [PubMed]
  47. Sorahan, T.; Mohammed, N. Neurodegenerative disease and magnetic field exposure in UK electricity supply workers. Occup. Med. 2014, 64, 454–460. [Google Scholar] [CrossRef] [PubMed]
  48. Stampfer, M.J. Welding occupations and mortality from Parkinson’s disease and other neurodegenerative diseases among United States men, 1985-1999. J. Occup. Environ. Hyg. 2009, 6, 267–272. [Google Scholar] [CrossRef] [PubMed]
  49. Steenland, K.; Hein, M.J.; Cassinelli, R.T., 2nd; Prince, M.M.; Nilsen, N.B.; Whelan, E.A.; Waters, M.A.; Ruder, A.M.; Schnorr, T.M. Polychlorinated biphenyls and neurodegenerative disease mortality in an occupational cohort. Epidemiology 2006, 17, 8–13. [Google Scholar] [CrossRef] [PubMed]
  50. Vanacore, N.; Cocco, P.; Fadda, D.; Dosemeci, M. Job strain, hypoxia and risk of amyotrophic lateral sclerosis: Results from a death certificate study. Amyotroph. Lateral Scler. 2010, 11, 430–434. [Google Scholar] [CrossRef] [PubMed]
  51. Weisskopf, M.G.; McCullough, M.L.; Morozova, N.; Calle, E.E.; Thun, M.J.; Ascherio, A. Prospective study of occupation and amyotrophic lateral sclerosis mortality. Am. J. Epidemiol. 2005, 162, 1146–1152. [Google Scholar] [CrossRef] [PubMed]
  52. Weisskopf, M.G.; Morozova, N.; O’Reilly, E.J.; McCullough, M.L.; Calle, E.E.; Thun, M.J.; Ascherio, A. Prospective study of chemical exposures and amyotrophic lateral sclerosis. J. Neurol. Neurosurg. Psychiatry 2009, 80, 558–561. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Veldink, J.H.; Kalmijn, S.; Groeneveld, G.J.; Titulaer, M.J.; Wokke, J.H.; van den Berg, L.H. Physical activity and the association with sporadic ALS. Neurology 2005, 64, 241–245. [Google Scholar] [CrossRef] [PubMed]
  54. Vergara, X.; Mezei, G.; Kheifets, L. Case-control study of occupational exposure to electric shocks and magnetic fields and mortality from amyotrophic lateral sclerosis in the US, 1991–1999. J. Expo. Sci. Environ. Epidemiol. 2015, 25, 65–71. [Google Scholar] [CrossRef] [PubMed]
  55. Andrew, A.S.; Caller, T.A.; Tandan, R.; Duell, E.J.; Henegan, P.L.; Field, N.C.; Bradley, W.G.; Stommel, E.W. Environmental and Occupational Exposures and Amyotrophic Lateral Sclerosis in New England. Neurodegener. Dis. 2017, 17, 110–116. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  56. Armon, C.; Kurland, L.T.; Daube, J.R.; O’Brien, P.C. Epidemiologic correlates of sporadic amyotrophic lateral sclerosis. Neurology 1991, 41, 1077–1084. [Google Scholar] [CrossRef] [PubMed]
  57. Belli, S.; Vanacore, N. Proportionate mortality of Italian soccer players: Is amyotrophic lateral sclerosis an occupational disease? Eur. J. Epidemiol. 2005, 20, 237–242. [Google Scholar] [CrossRef] [PubMed]
  58. Bertke, S.J.; Lehman, E.J.; Wurzelbacher, S.J.; Hein, M.J. Mortality of lead smelter workers: A follow-up study with exposure assessment. Am. J. Ind. Med. 2016, 59, 979–986. [Google Scholar] [CrossRef] [PubMed]
  59. Binazzi, A.; Belli, S.; Uccelli, R.; Desiato, M.T.; Talamanca, I.F.; Antonini, G.; Corsi, F.M.; Scoppetta, C.; Inghilleri, M.; Pontieri, F.E.; et al. An exploratory case-control study on spinal and bulbar forms of amyotrophic lateral sclerosis in the province of Rome. Amyotroph. Lateral Scler. 2009, 10, 361–369. [Google Scholar] [CrossRef] [PubMed]
  60. Buckley, J.; Warlow, C.; Smith, P.; Hilton-Jones, D.; Irvine, S.; Tew, J.R. Motor neuron disease in England and Wales, 1959–1979. J. Neurol. Neurosurg. Psychiatry 1983, 46, 197–205. [Google Scholar] [CrossRef] [PubMed]
  61. Burns, C.J.; Beard, K.K.; Cartmill, J.B. Mortality in chemical workers potentially exposed to 2,4-dichlorophenoxyacetic acid (2,4-D) 1945-94: An update. Occup. Environ. Med. 2001, 58, 24–30. [Google Scholar] [CrossRef] [PubMed]
  62. Chancellor, A.M.; Slattery, J.M.; Fraser, H.; Warlow, C.P. Risk factors for motor neuron disease: A case-control study based on patients from the Scottish Motor Neuron Disease Register. J. Neurol. Neurosurg. Psychiatry 1993, 56, 1200–1206. [Google Scholar] [CrossRef] [PubMed]
  63. Chio, A.; Meineri, P.; Tribolo, A.; Schiffer, D. Risk factors in motor neuron disease: A case-control study. Neuroepidemiology 1991, 10, 174–184. [Google Scholar] [CrossRef] [PubMed]
  64. Das, K.; Ghosh, M.; Nag, C.; Nandy, S.P.; Banerjee, M.; Datta, M.; Devi, G.; Chaterjee, G. Role of familial, environmental and occupational factors in the development of Parkinson’s disease. Neurodegener. Dis. 2011, 8, 345–351. [Google Scholar] [CrossRef] [PubMed]
  65. Davanipour, Z.; Sobel, E.; Bowman, J.D.; Qian, Z.; Will, A.D. Amyotrophic lateral sclerosis and occupational exposure to electromagnetic fields. Bioelectromagnetics 1997, 18, 28–35. [Google Scholar] [CrossRef]
  66. Deapen, D.M.; Henderson, B.E. A case-control study of amyotrophic lateral sclerosis. Am. J. Epidemiol. 1986, 123, 790–799. [Google Scholar] [CrossRef] [PubMed]
  67. Fang, F.; Quinlan, P.; Ye, W.; Barber, M.K.; Umbach, D.M.; Sandler, D.P.; Kamel, F. Workplace exposures and the risk of amyotrophic lateral sclerosis. Environ. Health Perspect. 2009, 117, 1387–1392. [Google Scholar] [CrossRef] [PubMed]
  68. Furby, A.; Beauvais, K.; Kolev, I.; Rivain, J.G.; Sebille, V. Rural environment and risk factors of amyotrophic lateral sclerosis: A case-control study. J. Neurol. 2010, 257, 792–798. [Google Scholar] [CrossRef] [PubMed]
  69. Gallagher, J.P.; Sanders, M. Trauma and amyotrophic lateral sclerosis: A report of 78 patients. Acta Neurol. Scand. 1987, 75, 145–150. [Google Scholar] [CrossRef] [PubMed]
  70. Govoni, V.; Granieri, E.; Fallica, E.; Casetta, I. Amyotrophic lateral sclerosis, rural environment and agricultural work in the Local Health District of Ferrara, Italy, in the years 1964–1998. J. Neurol. 2005, 252, 1322–1327. [Google Scholar] [CrossRef] [PubMed]
  71. Graham, A.J.; Macdonald, A.M.; Hawkes, C.H. British motor neuron disease twin study. J. Neurol. Neurosurg. Psychiatry 1997, 62, 562–569. [Google Scholar] [CrossRef] [PubMed]
  72. Granieri, E.; Carreras, M.; Tola, R.; Paolino, E.; Tralli, G.; Eleopra, R.; Serra, G. Motor neuron disease in the province of Ferrara, Italy, in 1964–1982. Neurology 1988, 38, 1604–1608. [Google Scholar] [CrossRef] [PubMed]
  73. Gresham, L.S.; Molgaard, C.A.; Golbeck, A.L.; Smith, R. Amyotrophic lateral sclerosis and occupational heavy metal exposure: A case-control study. Neuroepidemiology 1986, 5, 29–38. [Google Scholar] [CrossRef] [PubMed]
  74. Gunnarsson, L.G.; Lygner, P.E.; Veiga-Cabo, J.; de Pedro-Cuesta, J. An epidemic-like cluster of motor neuron disease in a Swedish county during the period 1973–1984. Neuroepidemiology 1996, 15, 142–152. [Google Scholar] [CrossRef] [PubMed]
  75. Kalfakis, N.; Vassilopoulos, D.; Voumvourakis, C.; Ndjeveleka, M.; Papageorgiou, C. Amyotrophic lateral sclerosis in southern Greece: An epidemiologic study. Neuroepidemiology 1991, 10, 170–173. [Google Scholar] [CrossRef] [PubMed]
  76. Kondo, K.; Tsubaki, T. Case-control studies of motor neuron disease: Association with mechanical injuries. Arch. Neurol. 1981, 38, 220–226. [Google Scholar] [CrossRef] [PubMed]
  77. Kurtzke, J.F.; Beebe, G.W. Epidemiology of amyotrophic lateral sclerosis: 1. A case-control comparison based on ALS deaths. Neurology 1980, 30, 453–462. [Google Scholar] [CrossRef] [PubMed]
  78. Lewis, R.J.; Schnatter, A.R.; Katz, A.M.; Thompson, F.S.; Murray, N.; Jorgensen, G.; Theriault, G. Updated mortality among diverse operating segments of a petroleum company. Occup. Environ. Med. 2000, 57, 595–604. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  79. Malek, A.M.; Barchowsky, A.; Bowser, R.; Heiman-Patterson, T.; Lacomis, D.; Rana, S.; Youk, A.; Stickler, D.; Lackland, D.T.; Talbott, E.O. Environmental and occupational risk factors for amyotrophic lateral sclerosis: A case-control study. Neurodegener. Dis. 2014, 14, 31–38. [Google Scholar] [CrossRef] [PubMed]
  80. Mitchell, J.D.; Davies, R.B.; al-Hamad, A.; Gatrell, A.C.; Batterby, G. MND risk factors: An epidemiological study in the north west of England. J. Neurol. Sci. 1995, 129, 61–64. [Google Scholar] [CrossRef]
  81. Morahan, J.M.; Pamphlett, R. Amyotrophic lateral sclerosis and exposure to environmental toxins: An Australian case-control study. Neuroepidemiology 2006, 27, 130–135. [Google Scholar] [CrossRef] [PubMed]
  82. Pamphlett, R. Exposure to environmental toxins and the risk of sporadic motor neuron disease: An expanded Australian case-control study. Eur. J. Neurol. 2012, 19, 1343–1348. [Google Scholar] [CrossRef] [PubMed]
  83. Pamphlett, R.; Rikard-Bell, A. Different occupations associated with amyotrophic lateral sclerosis: Is diesel exhaust the link? PLoS ONE 2013, 8, e80993. [Google Scholar] [CrossRef] [PubMed]
  84. Pinkerton, L.E.; Hein, M.J.; Meyers, A.; Kamel, F. Assessment of ALS mortality in a cohort of formaldehyde-exposed garment workers. Amyotroph. Lateral Scler. Front. Degener. 2013, 14, 353–355. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  85. Pinkerton, L.E.; Hein, M.J.; Grajewski, B.; Kamel, F. Mortality from neurodegenerative diseases in a cohort of US flight attendants. Am. J. Ind. Med. 2016, 59, 532–537. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  86. Provinciali, L.; Giovagnoli, A.R. Antecedent events in amyotrophic lateral sclerosis: Do they influence clinical onset and progression? Neuroepidemiology 1990, 9, 255–262. [Google Scholar] [CrossRef] [PubMed]
  87. Pupillo, E.; Messina, P.; Giussani, G.; Logroscino, G.; Zoccolella, S.; Chio, A.; Calvo, A.; Corbo, M.; Lunetta, C.; Marin, B.; et al. Physical activity and amyotrophic lateral sclerosis: A European population-based case-control study. Ann. Neurol. 2014, 75, 708–716. [Google Scholar] [CrossRef] [PubMed]
  88. Roberts, A.L.; Johnson, N.J.; Cudkowicz, M.E.; Eum, K.D.; Weisskopf, M.G. Job-related formaldehyde exposure and ALS mortality in the USA. J. Neurol. Neurosurg. Psychiatry 2015. [Google Scholar] [CrossRef] [PubMed]
  89. Roelofs-Iverson, R.A.; Mulder, D.W.; Elveback, L.R.; Kurland, L.T.; Molgaard, C.A. ALS and heavy metals: A pilot case-control study. Neurology 1984, 34, 393–395. [Google Scholar] [CrossRef] [PubMed]
  90. Roosli, M.; Lortscher, M.; Egger, M.; Pfluger, D.; Schreier, N.; Lortscher, E.; Locher, P.; Spoerri, A.; Minder, C. Mortality from neurodegenerative disease and exposure to extremely low-frequency magnetic fields: 31 years of observations on Swiss railway employees. Neuroepidemiology 2007, 28, 197–206. [Google Scholar] [CrossRef] [PubMed]
  91. Savettieri, G.; Salemi, G.; Arcara, A.; Cassata, M.; Castiglione, M.G.; Fierro, B. A case-control study of amyotrophic lateral sclerosis. Neuroepidemiology 1991, 10, 242–245. [Google Scholar] [CrossRef] [PubMed]
  92. Scarmeas, N.; Levy, G.; Tang, M.X.; Manly, J.; Stern, Y. Influence of leisure activity on the incidence of Alzheimer’s disease. Neurology 2001, 57, 2236–2242. [Google Scholar] [CrossRef] [PubMed]
  93. Schulte, P.A.; Burnett, C.A.; Boeniger, M.F.; Johnson, J. Neurodegenerative diseases: Occupational occurrence and potential risk factors, 1982 through 1991. Am. J. Public Health 1996, 86, 1281–1288. [Google Scholar] [CrossRef] [PubMed]
  94. Strickland, D.; Smith, S.A.; Dolliff, G.; Goldman, L.; Roelofs, R.I. Amyotrophic lateral sclerosis and occupational history. A pilot case-control study. Arch. Neurol. 1996, 53, 730–733. [Google Scholar] [CrossRef] [PubMed]
  95. Strickland, D.; Smith, S.A.; Dolliff, G.; Goldman, L.; Roelofs, R.I. Physical activity, trauma, and ALS: A case-control study. Acta Neurol. Scand. 1996, 94, 45–50. [Google Scholar] [CrossRef] [PubMed]
  96. Sutedja, N.A.; Veldink, J.H.; Fischer, K.; Kromhout, H.; Wokke, J.H.; Huisman, M.H.; Heederik, D.J.; Van den Berg, L.H. Lifetime occupation, education, smoking, and risk of ALS. Neurology 2007, 69, 1508–1514. [Google Scholar] [CrossRef] [PubMed]
  97. Valenti, M.; Pontieri, F.E.; Conti, F.; Altobelli, E.; Manzoni, T.; Frati, L. Amyotrophic lateral sclerosis and sports: A case-control study. Eur. J. Neurol. 2005, 12, 223–225. [Google Scholar] [CrossRef] [PubMed]
  98. Yu, Y.; Su, F.C.; Callaghan, B.C.; Goutman, S.A.; Batterman, S.A.; Feldman, E.L. Environmental risk factors and amyotrophic lateral sclerosis (ALS): A case-control study of ALS in Michigan. PLoS ONE 2014, 9, e101186. [Google Scholar] [CrossRef] [PubMed]
  99. Hill, A.B. The environment and disease: Association or causation? Proc. R. Soc. Med. 1965, 58, 295–300. [Google Scholar] [CrossRef] [PubMed]
  100. Begg, C.B.; Mazumdar, M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994, 50, 1088–1101. [Google Scholar] [CrossRef] [PubMed]
  101. Palmer, T.M.; Sterne, J.A.C.; Newton, H.J.; Cox, N.J. Meta-Analysis in Stata: An Updated Collection from the Stata Journal, 2nd ed.; Stata Press Publication: College Station, TX, USA, 2016. [Google Scholar]
  102. Horner, R.D.; Feussner, J.R.; Kasarskis, E.J. Prospective study of military service and mortality from ALS. Neurology 2005, 65, 180–181. [Google Scholar] [CrossRef] [PubMed]
  103. Ioannides, Z.A.; Ngo, S.T.; Henderson, R.D.; McCombe, P.A.; Steyn, F.J. Altered Metabolic Homeostasis in Amyotrophic Lateral Sclerosis: Mechanisms of Energy Imbalance and Contribution to Disease Progression. Neurodegener. Dis. 2016, 16, 382–397. [Google Scholar] [CrossRef] [PubMed]
  104. Garzillo, E.M.; Lamberti, M.; Genovese, G.; Pedata, P.; Feola, D.; Sannolo, N.; Daniele, L.; Trojsi, F.; Monsurro, M.R.; Miraglia, N. Blood lead, manganese, and aluminum levels in a regional Italian cohort of ALS patients: Does aluminum have an influence? J. Occup. Environ. Med. 2014, 56, 1062–1066. [Google Scholar] [CrossRef] [PubMed]
Ijerph 15 02371 g001 550
Figure 1. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to heavy physical work. Results for men only are indicated by M; otherwise the results concern both sexes. Random effect models were used, with stratification of activities. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity. RR = relative risk.
Figure 1. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to heavy physical work. Results for men only are indicated by M; otherwise the results concern both sexes. Random effect models were used, with stratification of activities. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity. RR = relative risk.
Ijerph 15 02371 g001
Ijerph 15 02371 g002 550
Figure 2. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to chemicals. Results for men only are indicated by M, results for women only are indicated by W; otherwise the results concern both sexes. Random effect models were used, with stratification of exposure related to chemical exposure related to agricultural work or other chemical exposures. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Figure 2. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to chemicals. Results for men only are indicated by M, results for women only are indicated by W; otherwise the results concern both sexes. Random effect models were used, with stratification of exposure related to chemical exposure related to agricultural work or other chemical exposures. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Ijerph 15 02371 g002
Ijerph 15 02371 g003 550
Figure 3. Funnel plot for the 16 RR (relative risk) estimates of the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to chemicals in Figure 2.
Figure 3. Funnel plot for the 16 RR (relative risk) estimates of the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to chemicals in Figure 2.
Ijerph 15 02371 g003
Ijerph 15 02371 g004 550
Figure 4. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to metals. Results for men only are indicated by M; otherwise the results concern both sexes. Random effect models were used, with stratification of exposure to lead or other metals. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Figure 4. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to metals. Results for men only are indicated by M; otherwise the results concern both sexes. Random effect models were used, with stratification of exposure to lead or other metals. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Ijerph 15 02371 g004
Ijerph 15 02371 g005 550
Figure 5. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to welding. Results for men only are indicated by M; otherwise the results concern both sexes. Random effect models were used. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Figure 5. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to welding. Results for men only are indicated by M; otherwise the results concern both sexes. Random effect models were used. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Ijerph 15 02371 g005
Ijerph 15 02371 g006 550
Figure 6. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to work with electricity and electromagnetic fields (EMF). Results for men only are indicated by M, results for women only are indicated by W; otherwise the results concern both sexes. Random effect models were used, with stratification of exposure to work with electricity or EMF. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Figure 6. Forest plot for studies assessing the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to work with electricity and electromagnetic fields (EMF). Results for men only are indicated by M, results for women only are indicated by W; otherwise the results concern both sexes. Random effect models were used, with stratification of exposure to work with electricity or EMF. Heterogeneity was tested by the I2 statistic, with p < 0.05 indicating rejection of homogeneity.
Ijerph 15 02371 g006
Ijerph 15 02371 g007 550
Figure 7. Cumulative meta-analyses on the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to EMF or work with electricity with the pooled estimate (Cum RR) updated for every new study, year by year.
Figure 7. Cumulative meta-analyses on the association between ALS (amyotrophic lateral sclerosis) and occupational exposure to EMF or work with electricity with the pooled estimate (Cum RR) updated for every new study, year by year.
Ijerph 15 02371 g007
Table
Table 1. Publications fulfilling good scientific standards (Armon class II or III) [15] according to the algorithm presented in the Appendix A.
Table 1. Publications fulfilling good scientific standards (Armon class II or III) [15] according to the algorithm presented in the Appendix A.
PublicationYearDiagnosisExposureStudy Group: Selection, Controls, Missing DataMethods AnalysisArmon Global Class [15]Funding:Exposures
AssessmentGradeNumber of Exposed CasesGrade
Beghi [18]20101Q2312–32–3IIIPUHard physical work, physical trauma
Bonvicini [19]20101Q21322IIPAChemicals (pesticides)
Chio [20]20092–3EC2822–3IIIPA, PUProfessional sports
Fang [21]20101EC215111IIPA, PULead
Feychting [22]20033JEM2–3234 *11IIIPUOccupation (work with electricity, EMF, welding)
Fischer [23]20152–3JEM2766 *11IIIPU, IOccupation (EMF, welding)
Gait [24]20033JEM213 *2–32III?Metals and solvents
Gallo [25]20163Q2–31722–3III?Hard physical work
Gunnarsson [26]19913JEM232 *22IIIPUOccupation (electricians, chemicals, nurse)
Gunnarsson [27]19921Q210 *21IIPUMetals, chemicals, welding
Harwood [28]20162Q217522IIIPUWork related physical activity
Horner [29]20081EC24822IIPUGulf War veterans
Huisman [30]20131Q110321IIPUHard physical work
Huss [31]20153JEM246 *21IIIPUOccupation (work with electricity, EMF)
Håkansson [32]20033JEM2–31321IIIIOccupation (EMF)
Johansen [33]19983JEM2–31422IIIPU, IOccupation (EMF)
Johansen [34]20002JEM21522IIPU, IOccupation (work with electricity)
Kamel [35]20021Q11121IIPULead
Lehman [36]20123EC1722IIIPUProfessional sports
Longstreth [37]19981Q2592–32IIIPAHard physical work
McGuire [38]19971Q2712–32IIIPUChemicals, metals
Noonan [39]20023JEM2–319 *21–2III?Occupation (work with electricity, EMF)
Park [40]20053JEM2–35965 *22IIIPUOccupation (EMF, pesticides and other chemicals, welding, nurse)
Parlett [41]20113JEM1102–32–3IIInoneOccupation (EMF)
Peters [42]20172–3JEM2–3611 *11IIIPA, PUOccupation (chemicals, lead, nurse)
Ruder [43]20143EC12022IIIPUChemicals (PCB)
Savitz [44]1998a3JEM2–3922IIIIOccupation (EMF)
Savitz [45]1998b3JEM2–311422III?Occupation (work with electricity)
Sorahan [46]20073JEM26222IIIIPower station workers (work with electricity)
Sorahan [47]20143JEM21122IIIIPower station workers (EMF)
Stampfer [48]20093JEM2–37122IIIIOccupation (welding)
Steenland [49]20063JEM11122III?Chemicals (PCB)
Vanacore [50]20103JEM2–362–32–3III?Hard physical work, professional sports
Weisskopf [51]20053Q2–330 *22III?Occupation (work with electricity, nurse)
Weisskopf [52]20093Q2–314222IIIPUChemicals
Veldink [53]20051Q2252–32–3IIIPUStrenuous physical work
Vergara [54]20153JEM2–3503 *2–31–2IIIPUOccupation (EMF, work with electricity, welding)
Diagnosis was graded with a score from 1–3 (diagnosis from a neurologist, in-patient care or general practitioner (GP)/mortality register). The other categories were graded with either a single score of 2 or were included in the interval 2–3 (1 = good, 2 = sufficient, 3 = uncertain/insufficient). Assessment of exposure: JEM = Job Exposure Matrix, Q = questionnaires/interviews, EC = exposed cohort. EMF = electromagnetic fields, PCB = polychlorinated biphenyls. Funding: I = industry, PA = patient association, PU = public, ? = funding is not possible to classify based on information in source text. * In case of multiple exposures, the highest number of exposed cases is registered.
Table
Table 2. Publications not fulfilling good scientific standards [3,9,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98] (Armon class IV or V) [15] according to the algorithm presented in the Appendix A.
Table 2. Publications not fulfilling good scientific standards [3,9,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98] (Armon class IV or V) [15] according to the algorithm presented in the Appendix A.
PublicationYearDiagnosisExposureStudy Group:
Selection Controls Missing Data		Methods AnalysisArmon Global Class [15]FundingExposures
Andrew [55]201733–442–3VPU, PAChemicals, metals, water-related activities
Armon [56]19911232–3IVPUPhysical work, trauma, metals
Belli [57]20053223IV?Professional soccer player
Bertke [58]20163323–4IVPUMetals
Binazzi [59]20091232–3IV?All possible
Breland [3]19671–232–33IV?Physical work
Buckley [60]19833–432–33IVPUOccupation
Burns [61]20013123IVI2,4-dichlorophenoxyacetic acid
Chancellor [62]19931232–3IVPAChemicals, manual work
Chiò [63]1991133–43IVIOccupation
Das [64]20121333IVPUPesticides
Davanipour [65]19971232–3IV?EMF
Deapen [66]19862–3233IVPAPhysical trauma (electric shock)
Fang [67]20091232–3IVPUChemicals
Furby [68]20101232–3IVIAgricultural work and chemicals
Gallagher [69]19871244V?Physical trauma
Govoni [70]20051433VPULiving in an agricultural area
Graham [71]199732–332–3IVPUChemicals
Granieri [72]19881232–3IVPUOccupation (Physical work, farmer, forest work)
Gresham [73]198612–333IV?Metals
Gunnarsson [74]19961422–3VPULiving in an agricultural area
Kalfakis [75]19911344V?Occupation
Kamel [9]20121232–3IVPUPesticides
Kondo [76]198132–333IVPUPhysical trauma
Kurtzke [77]19803322–3IVPUAll possible
Lewis [78]200033–422IVIWork at a petroleum company
Malek [79]201412–33–42–3IVPAMetals, pesticides
Mitchell [80]19951233IVPUAll possible
Morahan [81]200612–33–43IVPASolvents, pesticides
Pamphlett [82]201212–33–43IVPASolvents, pesticides (extension of Morahan 2006)
Pamhplett [83]20132343VPADiesel
Pinkerton [84]201333–423IVPUChemicals (formaldehyde)
Pinkerton [85]201633–423–4IVPUFlight attendant
Provinciali [86]1990133–43IV?Metals, trauma, heavy manual labour
Pupillo [87]20141231–2IVPU, IPhysical work
Roberts [88]2015332–33IVPAFormaldehyde
Roelofs-Iverson [89]198412–343VPUHeavy metals
Röösli [90]20073123IVPUOccupation (EMF)
Savettieri [91]199132–342VPUAll possible
Scarmeas [92]20021232–3IV?Athletic activity
Schulte [93]19963323IV?Occupation (farmers)
Strickland [94]199612–332–3IVPUMetals
Strickland [95]199612–332–3IVPUPhysical work
Sutedja [96]200712–332IVPUOccupation (education)
Valenti [97]20051333IV?Athletic activity
Yu [98]201411–23–43IVPU, IPhysical activity, metals, pesticides
These publications have either Diagnosis graded as 4 or any other category (Exposure, Study Group, Methods) graded with a single score of 3 or 4 (3 = uncertain/insufficient, 4 = unacceptable). I = industry, PA = patient association, PU = public, ? = funding is not possible to classify based on information in source text, EMF = electromagnetic fields.

Share and Cite

MDPI and ACS Style

Gunnarsson, L.-G.; Bodin, L. Amyotrophic Lateral Sclerosis and Occupational Exposures: A Systematic Literature Review and Meta-Analyses. Int. J. Environ. Res. Public Health 2018, 15, 2371. https://doi.org/10.3390/ijerph15112371

AMA Style

Gunnarsson L-G, Bodin L. Amyotrophic Lateral Sclerosis and Occupational Exposures: A Systematic Literature Review and Meta-Analyses. International Journal of Environmental Research and Public Health. 2018; 15(11):2371. https://doi.org/10.3390/ijerph15112371

Chicago/Turabian Style

Gunnarsson, Lars-Gunnar, and Lennart Bodin. 2018. "Amyotrophic Lateral Sclerosis and Occupational Exposures: A Systematic Literature Review and Meta-Analyses" International Journal of Environmental Research and Public Health 15, no. 11: 2371. https://doi.org/10.3390/ijerph15112371

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop