Next Article in Journal
Polycyclic Aromatic Hydrocarbons in Sediments/Soils of the Rapidly Urbanized Lower Reaches of the River Chaohu, China
Previous Article in Journal
Programming Effects of Prenatal Stress on Neurodevelopment—The Pitfall of Introducing a Self-Fulfilling Prophecy
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Concentrations of Lead, Mercury, Arsenic, Cadmium, Manganese, and Aluminum in Blood of Romanian Children Suspected of Having Autism Spectrum Disorder

by
Manouchehr Hessabi
1,†,
Mohammad H. Rahbar
1,2,3,†,
Iuliana Dobrescu
4,
MacKinsey A. Bach
1,2,
Liana Kobylinska
4,
Jan Bressler
2,5,
Megan L. Grove
2,5,
Katherine A. Loveland
6,
Ilinca Mihailescu
4,
Maria Cristina Nedelcu
4,
Mihaela Georgeta Moisescu
7,
Bogdan Mircea Matei
7,
Christien Oktaviani Matei
7 and
Florina Rad
4,*
1
Biostatistics/Epidemiology/Research Design (BERD) core, Center for Clinical and Translational Sciences (CCTS), The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
2
Department of Epidemiology, Human Genetics, and Environmental Sciences, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
3
Division of Clinical and Translational Sciences, Department of Internal Medicine, McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
4
Department of Child and Adolescent Psychiatry, University of Medicine and Pharmacy of Carol Davila, Psychiatry Clinical Hospital Alexandru Obregia, Bucharest, sector 4, 041914, Romania
5
Human Genetics Center, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
6
Department of Psychiatry and Behavioral Sciences, McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX 77054, USA
7
Department of Biophysics and Cellular Biotechnology, University of Medicine and Pharmacy of Carol Davila, Bucharest, sector 5, 050747, Romania
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Int. J. Environ. Res. Public Health 2019, 16(13), 2303; https://doi.org/10.3390/ijerph16132303
Submission received: 29 April 2019 / Revised: 24 June 2019 / Accepted: 25 June 2019 / Published: 28 June 2019
(This article belongs to the Section Environmental Health)

Abstract

:
Environmental exposure to lead (Pb), mercury (Hg), arsenic (As), cadmium (Cd), manganese (Mn), and aluminum (Al) has been associated with neurodevelopmental disorders including autism spectrum disorder (ASD). We conducted a pilot study during May 2015–May 2107 to estimate blood concentrations of six metals (Pb, Hg, As, Cd, Mn, and Al) and identify their associated factors for children with ASD or suspected of having ASD in Romania. Sixty children, age 2–8 years, were administered versions of ADOS or ADI-R translated from English to Romanian. After assessment, 2–3 mL of blood was obtained and analyzed for the concentrations of the six metals. The mean age of children was 51.9 months and about 90% were male. More than half (65%) of the children were born in Bucharest. Over 90% of concentrations of As and Cd were below limits of detection. Geometric mean concentrations of Pb, Mn, Al, and Hg were 1.14 μg/dL, 10.84 μg/L, 14.44 μg/L, and 0.35 μg/L, respectively. Multivariable linear regression analysis revealed that children who were female, had less educated parents, exhibited pica, and ate cold breakfast (e.g., cereal), watermelon, and lamb had significantly higher concentrations of Pb compared to their respective referent categories (all p < 0.05 except for eating lamb, which was marginally significant, p = 0.053). Although this is the first study that provides data on concentrations of the six metals for Romanian children with ASD, the findings from this study could be useful for designing future epidemiologic studies for investigating the role of these six metals in ASD in Romanian children.

1. Introduction

Environmental exposure to several metals, particularly some heavy metals, has been associated with neurodevelopmental disorders [1,2,3,4]. For example, several studies have reported that perinatal or postnatal exposure to lead (Pb) is associated with poor neurodevelopmental outcomes in children, including lower intelligence quotient (IQ) and cognitive ability [5] and autism spectrum disorder (ASD) [6,7]. Children with neurodevelopmental disorders have also been found to have higher blood or urine concentrations of metals and metalloids, such as mercury (Hg) [1], arsenic (As) [8], cadmium (Cd) [9], manganese (Mn) [10,11], and aluminum (Al) [12].
ASD is a neurodevelopmental disorder [13,14] that may be associated with the exposure to metals [5,6,15,16], either alone or through gene-environment interactions [17] and epigenetic changes [18]. Some studies have reported that heavy metals, such as Pb, Hg, Al, As, and Cd are associated with ASD [6,15,16,19,20,21,22,23,24,25,26], while other studies found no such associations [27,28,29,30,31,32,33]. However, some of these studies did not have the necessary information to control for potential confounders [6,16]. For example, it is well-known that children with ASD have a higher incidence of gastrointestinal problems and, as a result, their diets are very different from those of typically developing (TD) children [34]. Furthermore, some studies did not explore potential interactions while assessing associations between environmental exposure to these metals and ASD. For example, Rahbar et al. [35] did not find an association between Mn and ASD based on additive models. However, they found a significant interaction between Mn and glutathione S-transferase pi 1 (GSTP1) in relation to ASD after controlling for a mixture of Pb, Hg, As, and Cd as well as several other covariates [36].
Limited data are available regarding ASD in Romania [37]. Although a 2012 report from the Romanian Ministry of Health indicated 7,179 diagnoses of ASD [38], experts estimate that there are at least 15,000 ASD diagnoses in Romania annually [38]. This discrepancy is because the frequency of ASD cases reported by the Ministry of Health do not account for ASD cases diagnosed by general practitioners and other physicians in the private sector [38]. Most regions of Romania have a poor service infrastructure, an education system ill-equipped to integrate ASD children, and limited availability of services for adults with ASD [38]. However, people in Romania may be at increased risk of neurodevelopmental disorders (e.g., ASD) and other health outcomes since they are exposed to a variety of environmental pollutants, including heavy metals [39,40,41,42,43,44,45]. Romania has many reported sources of pollution, including water, soil, and air pollution [39,40,44,45]. Mining activity in Romania has resulted in elevated levels of heavy metals in soil [46,47], which may have found their way into the food chain, particularly in locally sourced vegetables [48]. Romania has industrial areas that used heavy metals for extraction and processing of non-ferrous ores, resulting in long-term heavy metals pollution. Although the Romanian government has put in place some environmental protection measures (e.g., allowable limits) for heavy metals in soil, the soil in certain areas continues to contain 2–3 times the allowable limits for some of these metals, which can potentially contaminate fruits and vegetables. For example, leafy vegetables (e.g., lettuce) have been found to contain concentrations of Cd and Pb that are seven and 17 times greater than the maximum allowable limits, respectively, in Romania [49]. Another study, assessed levels of minerals and heavy metals in commercial fruit juices from Romanian markets (apple, peach, apricot, orange, kiwi, pear, pineapple, and multi-fruit) and reported that while concentrations of As and Cd in fruit juices were below the allowable limits, the Pb content in fruit juices (apple, peach, and orange) exceeded the allowable limits for drinking water [50] recommended by the World Health Organization (WHO) (2014) [51]. An assessment of soil contamination with heavy metals revealed that recommended safety threshold values are exceeded for As, Al, Pb, Mn, and Hg in 34 Romanian counties [40]. In addition, water contaminated with heavy metals is a major concern in the Tisa (Tisza) and Danube Rivers, and soil in areas near the Danube River Delta that is contaminated with heavy metals [44,45]. Pb was heavily exploited by mining industries in northwestern Romania, causing elevated lead levels in soil near mining and ore processing sites [47]. Moreover, in the central Romania Copsa Mica area, children are at risk of being exposed to Pb [46,52]. One study has reported adverse human health effects including saturnine encephalopathy (lead encephalopathy), radial nerve paralysis, and lead colic due to such exposures expressing concerns that Pb- or Cd-contaminated soils and vegetables may have significantly contributed to decreasing life expectancy of people living in the affected areas near Baia Mare and Copsa Mica in the last two decades [46].
To our knowledge, few papers have been published regarding concentrations of heavy metals in blood of Romanian children [53,54,55]. We planned a pilot case-control study to assess the feasibility of conducting a large-scale epidemiologic study in Romania for investigating the role of environmental exposure to six metals (Pb, Hg, As, Mn, Cd, and Al) and their interactions with select genes that are involved in contaminant metabolism in relation to ASD. Although the pilot project revealed difficulty in enrolling suitable TD children as controls, in this paper we use data from the 60 ASD cases to report concentration of the aforementioned six metals and identify factors associated with blood Pb levels in Romanian children with or suspected of having ASD.

2. Materials and Methods

2.1. General Description

In collaboration between faculty at University of Medicine and Pharmacy of Carol Davila (UMF Carol Davila) in Romania and the University of Texas Health Science Center at Houston (UTHealth), we conducted a pilot study to assess the feasibility of conducting long-term epidemiologic research focused on gene-environment interactions in relation to ASD in Romania. Currently, our study team in the department of Child and Adolescent Psychiatry at UMF Carol Davila diagnoses at least 1400 children per year who meet the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) [56] and International Classification of Diseases, 10th Revision (ICD-10) [57] criteria for ASD. From May 2015 to May 2017, after obtaining consents from the parents, sixty (60) children between ages 2–8 years who met the DSM-IV and ICD-10 criteria for ASD, were reevaluated based on translated versions of the Autism Disorder Observation Schedule (ADOS) [58] and Autism Diagnostic Interview—Revised (ADI-R) [59] in Romanian [60,61]. Those who exceeded the threshold of ADI-R domains (e.g., A) qualitative abnormalities in reciprocal social interaction; B) qualitative abnormalities in communication; C) restricted, repetitive, and stereotyped patterns of behavior; and D) abnormality of development evident at or before 36 months) [59] and ADOS domains (e.g., communication and social interaction domains (social affect score)) were classified as children with confirmed ASD. In our pilot study, 67% of children had confirmed diagnosis of ASD as they exceeded both ADOS and ADI-R thresholds, but the remaining 33% exceeded only one of these two thresholds (herein labeled as being “suspected of having ASD”).
We also administered a socioeconomic status (SES) questionnaire to assess demographic characteristics, socioeconomic position, education level of the parents, and occupation of parents with a particular focus on potential exposure to metals, and family history of psychiatric disorders. A detailed food frequency questionnaire was administered to capture possible dietary exposure to metals and data were obtained about the frequency of the child’s weekly consumption of local and imported seafood, fruits and vegetables, meat/organ meat (e.g., liver), dairy products, eggs, grain/starches, peas, beans, nuts, juices/soft drinks, and sources of drinking and cooking water. In the food frequency questionnaire we asked the parents, how often their child ate certain foods and parents could select one of the following responses: I) Never or almost never, II) 1–3 times per month, III) Once per week, IV) 2–4 times per week, or V) 5 or more times per week. At the end of the interview, 2–3 mL of whole blood was collected from each child for assessment of exposure to the metals. The blood samples were stored at −20 °C at UMF Carol Davila, and then shipped to the Michigan Department of Health and Human Services (MDHHS) in Lansing, MI, USA, for analysis of the six metals. Institutional Review Boards (IRBs) of UTHealth (Project identification code: HSC-GEN-15-0844) and UMF Carol Davila (Project identification code: PO-35-F-03) approved this study.

2.2. Methods for Assessment of the Metals Exposure

For this study, assays for trace metals (Pb, Hg, As, Cd, Mn, and Al) were performed at MDHHS Trace Metals Laboratory, a facility certified by the Centers for Disease Control and Prevention (CDC). Venous whole blood samples were diluted and analyzed for trace metals using a PerkinElmer Elan DRCII inductively-coupled plasma mass spectrometer (PerkinElmer, Waltham, MA, USA). The methods for analysis of the six metal concentrations in blood by MDHHS have been described previously [62,63].
The limits of detection (LoD) for Pb, Mn, Al, Hg, As, and Cd were 0.25 μg/dL, 2.5 μg/L, 5.0 μg/L, 0.25 μg/L, 1.3 μg/L, and 0.13 μg/L, respectively. All Pb and Mn concentrations were above the LoD. The percentage of concentrations below LoD for Al, Hg, As, and Cd were 30%, 61.7%, 90%, and 95%, respectively.

2.3. Statistical Analysis

We conducted descriptive analyses to examine the distributions of various characteristics of the study population including demographics and SES. Since the distributions of blood concentrations of Pb, Hg, Mn, and Al were skewed, we transformed these data using the natural logarithm (ln) in order to produce distributions that better approximated a normal distribution. The means of the ln-transformed blood concentrations of the four metals were transformed to their original scale (i.e., μg/dL for Pb and μg/L for the other metals) by applying an exponential function, herein called geometric mean. We also report the arithmetic (i.e., untransformed) means for comparative purposes. For Al and Hg concentrations below the LoD were replaced by the LoD for that metal divided by the square root of two (i.e., 2−1/2LoD) [64]. Since over 90% of concentrations of As and Cd were below LoD, we did not report arithmetic or geometric mean concentrations for these metals.
For Pb, Mn, and Al, which had more than 70% concentrations above LoD, we used univariable and multivariable General Linear Models (GLMs) to identify factors associated with concentrations of these metals. For GLM analysis, we categorized all responses in the food frequency questionnaire into binary variables “never consumed” = “no” or “consumed” = “yes”. Potential factors considered included gender, education of parents, distance of residence from a major road, and binary consumption of various kinds of seafood or fish, leafy vegetables (e.g., lettuce), fruits (e.g., watermelon), meat (e.g., lamb), grains (e.g., cereal for breakfast), and nuts. All variables that were significantly (i.e., p < 0.05) or marginally (i.e., 0.05 ≤ p <0.10) associated with metals in univariable analysis were entered in multivariable GLMs. As a result, we could not report significantly associated factors from multivariable GLMs for Mn and Al. In addition, for Hg, As, and Cd, which had more than 65% of concentrations below LoD, we did not identify factors associated with these metal concentrations. All statistical tests were performed at 5% level of significance. All statistical analyses were conducted using SAS 9.4. [65].

3. Results

3.1. Characteristics of Study Sample

The mean age of children was 51.9 months and about 90% were male. More than half (65%) of the children were born in Bucharest, and 45.5% of fathers and 71.7% of mothers had education beyond high school. Nearly 48% of children had family history of different psychiatric or neurodevelopment disorders as shown in Table 1. Demographic and SES characteristics are displayed in Table 1.

3.2. Arithmetic and Geometric Means and Median Blood Metal Concentration of ASD Children

Geometric mean (GM) concentrations of Pb, Mn, Al, and Hg were 1.14 μg/dL, 10.84 μg/L, 14.44 μg/L, and 0.35 μg/L, respectively. Arithmetic mean concentrations of these metals are also reported in Table 2. Percentage of children with elevated levels of the metals was 5% for Pb; 17% for Mn; 60% for Al, 3.3% for Hg; 0% for As; and 1.7% for Cd, based on cut-off values recommended by the Agency for Toxic Substances and Disease Registry (ATSDR) [66,67,68,69,70], Centers for Disease Control and Prevention (CDC) [71], and Laboratory Corporation of America (LabCorp) [72], reported in Table 2.

3.3. Factors Associated with Blood Lead Concentrations Based on Univariable and Multivariable General Linear Models

Univariable linear regression analysis revealed that, compared to boys, girls had a significantly higher concentration of Pb (GM of 2.48 μg/dL vs. 1.05 μg/dL, p < 0.01). Children with less educated parents had a significantly higher mean concentration of Pb than those whose parents had higher education (GM of 1.78 μg/dL vs. 0.96 μg/dL, p < 0.01). Those who exhibited pica (i.e., eating mud) had a significantly higher mean concentration of Pb than those who did not (GM of 3.44 μg/dL vs. 1.01 μg/dL, p < 0.01). Residential proximity to high traffic roads was also associated with blood Pb concentrations; children living near the roads (i.e., within a quarter of a mile) had significantly higher concentration of Pb compared to children living further away from the roads (GM of 1.47 μg/dL vs. 0.98 μg/dL, p = 0.04). Compared to children who did not eat the following foods, respectively, children who ate these foods had a significantly higher blood concentrations of Pb; lettuce (GM of 1.50 μg/dL vs. 0.99 μg/dL, p = 0.04); watermelon (GM of 1.40 μg/dL vs. 0.87 μg/dL, p = 0.02); lamb (GM of 1.84 μg/dL vs. 1.05 μg/dL, p = 0.04); cold breakfast such as cereal (GM of 1.32 μg/dL vs. 0.84 μg/dL, p = 0.03); peanuts (GM of 1.49 μg/dL vs. 0.89 μg/dL, p < 0.01); tree nuts (GM of 1.53 μg/dL vs. 0.92 μg/dL, p < 0.01); and oil from animal fat (e.g., used for cooking) (GM of 2.02 μg/dL vs. 1.07 μg/dL, p < 0.05). Other univariable results are reported in Table 3.
In the final multivariable model that included all variables significantly associated with Pb in univariable analysis as described earlier, the following variables were identified as independently associated with having higher levels of Pb in the presence of other variables in the model: child’s sex (GM of 2.58 μg/dL for girls vs. 1.52 μg/dL for boys, p = 0.03), children of parents with lower education (GM of 2.48 μg/dL vs. 1.58 μg/dL, p < 0.01), children who exhibited pica (GM of 2.68 μg/dL vs. 1.46 μg/dL, p = 0.01), children who ate cold breakfast such as cereal (GM of 2.67 μg/dL vs. 1.46 μg/dL, p < 0.01); and children who ate watermelon (GM of 2.38 μg/dL vs. 1.64 μg/dL, p < 0.01). Consumption of lamb became marginally significant (p = 0.05) in the final multivariable model as shown in Table 3.

3.4. Factors Associated with Blood Manganese Concentrations Based on Univariable General Linear Models

Univariable linear regression analysis for Mn revealed that children who ate the following foods had a significantly higher geometric mean concentration of Mn compared to those who did not eat these foods: fresh shellfish (lobster, crab, crawfish) (GM of 20.00 μg/L vs. 10.72 μg/L, p = 0.05); banana (GM of 11.13 μg/L vs. 9.10 μg/L, p = 0.09); and polenta (GM of 11.48 μg/L vs. 9.88 μg/L, p = 0.07), though two of these associations were marginally significant as shown in Table 4. When all of these variables were entered in a multivariable linear regression model, none of these factors were significantly associated with Mn concentrations at 5% level of significance.

3.5. Factors Associated with Blood Aluminum Concentrations Based on Univariable General Linear Models

Similarly, in univariable analysis for Al, children who ate the following foods had significantly higher Al concentrations compared to those who did not eat these foods: cantaloupe or honeydew (GM of 20.47 μg/L vs. 10.86 μg/L, p = 0.04); shrimp (GM of 44.95 μg/L vs. 13.02 μg/L; p = 0.03); and lamb (GM of 32.06 μg/L vs. 12.54 μg/L, p = 0.03) as shown in Table 5. Again, when these variables were entered in a multivariable linear regression model, none of these factors were significantly associated with Al.

4. Discussion

A limited number of studies have been published regarding concentrations of the heavy metals in blood and teeth of Romanian children [53,54,55]. Nicolescu et al. investigated associations between environmental exposure of Romanian children to neurotoxic metals (Pb, Hg, and Al) and attention-deficit hyperactivity disorder (ADHD) [73]. To our knowledge, our study is the first study that reports concentrations of the six heavy metals in blood of children with ASD or suspected of having ASD in Romania. It is important to note throughout the discussion section that our results may not be generalizable to the general population of Romanian children due to the presence of a neurodevelopmental disorder. Additionally, we can reasonably expect that children with ASD may have different dietary exposures than children in the general population due to the high rate (23%–70%) of comorbid gastrointestinal disorders among children with ASD [74,75]. As a majority of the relevant literature is focused on exposures to Pb, in the following sections we will discuss Pb separately from the other five metals.

4.1. Blood Lead Concentrations and Their Associated Factors

In the 1980s, the Environment and Health Information System (ENHIS) study, a collaborative study between the World Health Organization and the European Commission, reported that geometric mean concentrations of Pb in children’s blood ranged from 18.2–18.9 μg/dL in Bulgaria, Hungary, and Romania, which were much higher than the concentrations of blood Pb in children from other countries in Europe such as Italy (11.0 μg/dL) and Germany (7.4 μg/dL) [53]. In 1998, the World Bank reported that many Central and Eastern European countries had gradually reduced the maximum allowed concentration of lead in gasoline to the current European Union standard (0.15 g/L), however Romania still allowed lead concentrations of up to 0.6 g/L [76]. After phasing out the use of lead in gasoline, the Pb blood concentrations in European countries dropped. For example, Romania phased out the use of lead in gasoline in 2005, which may partially explain why Nicolescu et al. reported lower arithmetic blood Pb concentrations of children with ADHD in 2006 in Bucharest (3.2 μg/dL) and Pantelimon (5.1 μg/dL), (a city near a metal-processing plant) [73] compared with the levels reported by WHO for Romania in the mid-1980s [53]. In the present study, our results from the univariable models revealed that children who have been living within quarter of a mile of a high traffic road had significantly (p = 0.04) higher mean blood Pb concentrations than those lived further away. On the other hand, we found relatively lower geometric mean concentrations of Pb in blood of children with ASD in Romania (1.14 μg/dL) compared to those reported by Nicolescu et al. [73]. Furthermore, the percentage of children with elevated levels of Pb (i.e., >5 μg/dL) in our study was 5%, lower than 25% reported by Nicolescu et al. [73]. The difference between the mean blood Pb concentrations and percentages of elevated levels of Pb in our study and that of Nicolescu et al., could be due to the areas where the sampled children live. Specifically, Nicolescu et al. stated that their study samples were collected from children in Bucharest (44.6%) and from a more Pb contaminated industrial area of Pantelimon, about 2 km south-east of Bucharest (55.4%), whereas in our study a majority (65%) of children were from Bucharest. Overall, our study and Nicolescu et al. [73] provide important information indicating that lowering lead in gasoline may have had a significant effect in reducing blood Pb concentrations among children in Romania. However, additional interventions may be needed to eliminate sources of Pb exposure through air pollution.
Significantly higher concentrations of Pb in children from the present study who ate lettuce (GM of 1.50 μg/dL vs. 0.99 μg/dL, p = 0.04) is consistent with a report that showed high levels of Pb in leafy vegetables (lettuce, parsley, dill, and orach) produced in Romania [49]. Industrial areas located near the south-east region of Bucharest, which has higher Pb concentrations in soil, could affect blood Pb concentrations of children living nearby [77]. This point was also emphasized by a study from Romania (Deliea et al.) that assessed Pb concentrations in children’s teeth and reported that the levels were 48% higher for children living in industrial areas, (Slatina city with high level of pollution) when compared to the reference region (Pătârlagele, a small village in a mountainous region with lower level of pollution) [55]. A more recent study from Romania in 2015 by Brad et al. also reported higher median concentration of Pb in blood (3.3 μg/dL (ranging from 1.3–13.0 μg/dL)) in contaminated areas that have mining activities, in north Romania [54]. As a result, the Romanian government could raise awareness of people that are living in these areas.
In this study, we found that girls had a significantly higher concentrations of Pb compared to boys (GM of 2.48 μg/dL vs. 1.05 μg/dL). While there are currently no comparable studies from Romania, there are studies from neighboring countries including Poland and Italy that investigated the association of sex and blood Pb concentrations and reported findings different from our own. For example, a study, conducted in 2009 in Poland, reported no significant differences in blood Pb concentrations between boys and girls [78], while another study, conducted in 2012 in Poland, reported a slightly higher geometric mean concentration of blood Pb for boys than girls (GM of 2.54 μg/dL vs. 2.39 μg/dL), though this difference was only marginally significant (p = 0.08) [79]. A study from Italy carried out during 1993–1994 also reported no significant difference between blood lead concentrations in boys and girls [80]. One possible reason for the differences between our results and the Polish and Italian studies is that, our study included children with ASD, which resulted in having a much higher ratio of boys to girls (9:1) than would be expected in the general population. We also had a very limited sample size (only six girls), which warrants further replication of our finding.
Parental education was another factor associated with blood lead concentrations in the present study; we found that children who had less educated parents, had significantly higher concentrations of Pb compared to children whose parents had higher education (GM of 1.78 μg/dL vs. 0.96 μg/dL). In the previously mentioned Polish study, Barrett reported no significant difference in mean blood Pb concentrations between children based on the education levels of their parents [78].
Our finding that children who exhibited pica had a significantly higher blood concentration of Pb than those who did not exhibit pica is consistent with two other studies in India and Pakistan. [81,82]. These findings are potentially important for Romanian children since, as discussed earlier, lead levels in soil are elevated above the maximum allowable limits in many areas across Romania.
In our univariable analysis, we found that children who ate lamb had a higher blood Pb concentration, compare to those who did not eat lamb. However, in the multivariable model this association became marginally significant. Based on other studies, lead poisoning affects livestock of all ages. Another possible source of exposure to Pb in farm livestock could be from their habit of chewing disposed batteries or eating items contaminated with used oil from machinery [83]. In addition, we found that children who ate watermelon had a significantly higher blood lead concentrations. Since Romania is ranked twenty-sixth in terms of production of watermelon in the world (https://www.tridge.com/intelligences/watermelon/RO/production) and contributes 0.4% share of world imports, our finding has revealed a specific source of exposure to lead in Romanian children. Moreover, in the multivariable model we found children who ate cold breakfast such as cereal had a significantly higher geometric mean concentration of lead in blood (GM of 2.67 μg/dL vs. 1.46 μg/dL, p < 0.01), compared to those who did not eat such breakfast. Although we did not find any other studies that reported a significant association between eating cereals and higher Pb concentrations in blood of children, a recent study from Australia, reported that levels of Pb found in breakfast cereals, rice products and diets of a selected group of children in Brisbane, ranged from <0.01 to 0.25 mg/kg. The estimated daily intake of Pb in children varied widely and ranged from 0.90 to 11.7 (5.6 ± 3.5 μg/day) [84]. Since lead is found in soil of some parts of Romania [46,47], we suggest further investigation of the Pb in cereal marketed in Romania.

4.2. Blood Concentrations of Other Metals

The literature on blood concentrations of Mn, Al, Hg, Cd, and As in children from Romania and surrounding countries is very limited. In the present study, we found that the geometric mean blood concentrations of Mn in Romanian children with ASD is 10.84 μg/L. This is much higher than the concentrations reported in a study on adults from Italy (1.35 µg/L) [85].
We reported that geometric mean concentrations of Al in blood of Romanian children with ASD is 14.44 μg/L, which is lower than the previously mentioned study on Romanian children with ADHD (36.0 μg/L to 49.0 μg/L) [73] but much higher than a study on TD Italian children (1.49 µg/L in serum) [85]. The reasons for such a large difference may need to be investigated in future studies. On the other hand, based on recommended cut-off value of 9.0 μg/L for elevated blood Al concentration reported in Table 2, our result shows that the percentage of children with elevated levels of Al was 60%, much lower than that of a study on Romanian children with ADHD that was nearly 99% [73].
In this study, we found that the geometric mean concentration of Hg in blood of Romanian children with ASD was 0.35 μg/L and 3.3% of children had elevated Hg concentrations (i.e., >5 μg/L). Nicolescu et al. reported lower arithmetic blood Hg concentrations of children with ADHD in Romania (0.1 μg/L), and none of the children had elevated concentrations [73]. National European Surveys and other studies reported geometric mean blood mercury concentrations for many countries in Europe [86]. Our results from Romania are comparable to results from TD children (8–10 years old) in the Czech Republic in 2008 and German TD children (3–14 years old) during 2003–2006, who have geometric mean blood Hg concentrations of 0.45 µg/L and 0.23 µg/L, respectively. In contrast, our results were about half of that reported in Italian TD children (13–15 years old) during 2008–2010 (0.84 µg/L) [87,88].
In our study, since 95% of concentrations for Cd were below LoD, we did not report the geometric and arithmetic means. However, percentage of elevated levels for Cd was 1.7%. Our finding suggests lower concentrations of Cd than levels reported in a 2012 Polish study on preschool children living near environmental hazards (0.25 μg/L for boys and 0.24 μg/L for girls) [79]. Additionally, a 1998 study on people living in mining areas of Seville, Spain, reported a geometric mean of blood Cd concentration of 0.19 µg/L in people living close to the spill of toxic material from mining, which was higher than that of the unaffected population (0.14 µg/L) [89].
Similarly, we did not report the geometric and arithmetic means for As due to 90% of concentrations being lower than LoD. As a result, none of children in our study had elevated levels for As. On the other hand, we did not find blood As concentrations for children in other European countries to compare our results with. However, in our Jamaican autism study, we found that geometric mean blood arsenic concentration for Jamaican children with ASD was 2.48 μg/L, much higher than that of Romanian children with ASD.

5. Limitations

Despite interesting findings reported from this pilot study, we acknowledge that this study has several limitations. First, this study was a pilot study with limited funding. Although our Romanian partners could have enrolled a larger number of children with ASD, shipment of the specimens to the US and assessment of the six metals would have not been possible for more than 60 children. Second, the sample size of 60 children in this study may not provide sufficient statistical power for investigating factors associated with concentrations of metals, especially those that had a sizable proportion below LoD. Third, blood may not be the most suitable biomarker for assessing exposures to all six of these metals. For example, urine is considered as a better biomarker for assessment of exposure to As [67] and Cd [90]. Hair is usually used for assessment of Hg [91] and Mn [92]. However, blood is considered as a good biomarker for assessment of Pb [93] and Al [94]. Fourth, due to 50% missing data regarding parents’ specific occupation, we did not report information regarding occupation of parents as a potential source of exposure to the metals. Fifth, the sample of 60 children with ASD or suspected of having ASD may not be representative of children from Romania due to the presence of a neurodevelopmental disorder. Additionally, the children in our study were enrolled primarily in and around Bucharest. It is also important to keep in mind that the type I error of 5% is for testing individual factors associated with Pb and does not account for multiple testing performed in this study.

6. Conclusions

This is the first study from Romania reporting concentrations of the six heavy metals in blood of children with ASD or suspected of having ASD. Considering that Romania has higher levels of some of these metals in soil, this study provides important information regarding concentrations of these metals in blood of potentially exposed Romanian children. We have also identified factors associated with concentrations of Pb in blood of children with ASD or suspected of having ASD, including dietary factors. These factors include child’s sex, parental education, exhibiting pica, and eating watermelon, lamb, and cold breakfast such as cereal. The findings from this study may help with designing relevant interventions to reduce concentrations of Pb in blood of Romanian children. These findings also provide important information for designing future epidemiologic studies, particularly those focused on assessing the role of these six metals in ASD among children in Romania.

Author Contributions

Conceptualization, M.H.R., I.D. and F.R.; methodology, M.H.R., M.H., I.D. and F.R.; software, M.H.; formal analysis, M.H.R., M.H.; investigation, F.R., M.C.N., L.K., and I.M.; resources, M.H.R. and I.D.; data curation, M.H., F.R., M.L.G., L.K., I.M., M.C.N., M.G.M., B.M.M., and C.O.M.; writing—original draft preparation, M.H.R., M.H., M.A.B.; writing—review and editing, M.H.R., M.H., M.A.B., J.B., K.A.L., and F.R.; visualization, M.H.R. and M.H.; supervision, M.H.R., I.D., and F.R.; project administration, M.H.R., M.A.B.; funding acquisition, M.H.R.”.

Funding

This study was partly funded by NIH/NCATS grant (UL1 TR000371) awarded to University of Texas Health Science Center at Houston. For data management we used REDCap, which is developed by Vanderbilt University and with partial grant support from NIH/NCATS (UL1 TR000445), and it is shared with other institutions, including University of Texas Health Science Center at Houston.

Acknowledgments

We acknowledge the support provided by the Biostatistics/Epidemiology/Research Design (BERD) component of the Center for Clinical and Translational Sciences (CCTS) for this project. CCTS is mainly funded by a grant from the National Center for Advancing Translational Sciences (NCATS), (UL1 TR000371), awarded to the University of Texas Health Science Center at Houston. Furthermore, we acknowledge that the data management were done using REDCap [95]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NCATS. Moreover, we acknowledge contributions by colleagues in the Analytical Chemistry Lab at MDHHS for analyzing and storing the whole blood samples for the assessments of heavy metal concentrations, under a service contract. Finally, we acknowledge contributions of members of the UMF Carol Davila Team in Romania (Gianina Anghel, Alexandra Buica, Alina Stan, Ana Ciubotariu, Giorgiana Murariu, Andrei Emanuela, Mihailescu Georgiana, Irimie-Ana Alecsandra, Mirela Hatis, and Anca Rusu), who provided assistance with collecting data, or any other tasks that for their contributions could not be offered co-authorship on this manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

LeadPb
MercuryHg
CadmiumCd
ArsenicAs
ManganeseMn
AluminumAl
Autism Spectrum DisorderASD
Typically developingTD
glutathione S-transferase pi 1GSTP1
World Health OrganizationWHO
Intelligence quotientIQ
University of Medicine and Pharmacy of Carol DavilaUMF Carol Davila
The University of Texas Health Science Center at HoustonUTHealth
Diagnostic and Statistical Manual of Mental Disorders, Fourth EditionDSM-IV
International Classification of Diseases, 10th RevisionICD-10
Autism Disorder Observation ScheduleADOS
Autism Diagnostic Interview—RevisedADI-R
Socioeconomic statusSES
Michigan Department of Health and Human ServicesMDHHS
Agency for Toxic Substances and Disease RegistryATSDR
Centers for Disease Control and PreventionCDC
Laboratory Corporation of AmericaLabCorp
Natural Logarithmln
Limits of DetectionLoD
General Linear ModelsGLMs
Geometric meanGM
Attention-deficit hyperactivity disorderADHD
Environment and Health Information SystemENHIS

References

  1. Rossignol, D.A.; Genuis, S.J.; Frye, R.E. Environmental toxicants and autism spectrum disorders: A systematic review. Transl. Psychiatry 2014, 4, e360. [Google Scholar] [CrossRef] [PubMed]
  2. Chen, P.; Miah, M.R.; Aschner, M. Metals and Neurodegeneration. F1000Research 2016, 5. [Google Scholar] [CrossRef] [PubMed]
  3. Ha, M.; Kwon, H.J.; Lim, M.H.; Jee, Y.K.; Hong, Y.C.; Leem, J.H.; Sakong, J.; Bae, J.M.; Hong, S.J.; Roh, Y.M.; et al. Low blood levels of lead and mercury and symptoms of attention deficit hyperactivity in children: A report of the children’s health and environment research (CHEER). Neurotoxicology 2009, 30, 31–36. [Google Scholar] [CrossRef] [PubMed]
  4. Claus, H.B.; Coull, B.A.; Wright, R.O. Chemical mixtures and children’s health. Curr. Opin. Pediatr. 2014, 26, 223–229. [Google Scholar] [CrossRef] [PubMed]
  5. Bellinger, D.C. Very low lead exposures and children’s neurodevelopment. Curr. Opin. Pediatr. 2008, 20, 172–177. [Google Scholar] [CrossRef] [PubMed]
  6. Fido, A.; Al-Saad, S. Toxic trace elements in the hair of children with autism. Autism 2005, 9, 290–298. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Blaurock-Busch, E.; Amin, O.R.; Rabah, T. Heavy metals and trace elements in hair and urine of a sample of arab children with autistic spectrum disorder. Maedica 2011, 6, 247–257. [Google Scholar] [PubMed]
  8. Tsuji, J.S.; Garry, M.R.; Perez, V.; Chang, E.T. Low-level arsenic exposure and developmental neurotoxicity in children: A systematic review and risk assessment. Toxicology 2015, 337, 91–107. [Google Scholar] [CrossRef]
  9. Jiang, H.M.; Han, G.A.; He, Z.L. Clinical significance of hair cadmium content in the diagnosis of mental retardation of children. Chin. Med. J. 1990, 103, 331–334. [Google Scholar]
  10. Bouchard, M.; Laforest, F.; Vandelac, L.; Bellinger, D.; Mergler, D. Hair manganese and hyperactive behaviors: Pilot study of school-age children exposed through tap water. Environ. Health Perspect. 2007, 115, 122–127. [Google Scholar] [CrossRef]
  11. Khan, K.; Factor-Litvak, P.; Wasserman, G.A.; Liu, X.; Ahmed, E.; Parvez, F.; Slavkovich, V.; Levy, D.; Mey, J.; van Geen, A.; et al. Manganese exposure from drinking water and children’s classroom behavior in Bangladesh. Environ. Health Perspect. 2011, 119, 1501–1506. [Google Scholar] [CrossRef] [PubMed]
  12. Prakash, D.; Gopinath, K.; Sudhandiran, G. Fisetin enhances behavioral performances and attenuates reactive gliosis and inflammation during aluminum chloride-induced neurotoxicity. Neuromol. Med. 2013, 15, 192–208. [Google Scholar] [CrossRef] [PubMed]
  13. Genuis, S.J. Is autism reversible? Acta Paediatr. 2009, 98, 1575–1578. [Google Scholar] [CrossRef] [PubMed]
  14. Volkmar, F.R.; Chawarska, K. Autism in infants: An update. World Psychiatry 2008, 7, 19–21. [Google Scholar] [CrossRef] [PubMed]
  15. Blaurock-Busch, E.; Amin, O.R.; Dessoki, H.H.; Rabah, T. Toxic Metals and Essential Elements in Hair and Severity of Symptoms among Children with Autism. Maedica 2012, 7, 38–48. [Google Scholar] [PubMed]
  16. Kern, J.K.; Grannemann, B.D.; Trivedi, M.H.; Adams, J.B. Sulfhydryl-reactive metals in autism. J. Toxicol. Environ. Health Part A 2007, 70, 715–721. [Google Scholar] [CrossRef]
  17. Chaste, P.; Leboyer, M. Autism risk factors: Genes, environment, and gene-environment interactions. Dialogues Clin. Neurosci. 2012, 14, 281–292. [Google Scholar]
  18. Rangasamy, S.; D’Mello, S.R.; Narayanan, V. Epigenetics, autism spectrum, and neurodevelopmental disorders. Neurotherapeutics 2013, 10, 742–756. [Google Scholar] [CrossRef]
  19. Seneff, S.; Davidson, R.M.; Liu, J. Empirical Data Confirm Autism Symptoms Related to Aluminum and Acetaminophen Exposure. Entropy 2012, 14, 2227–2253. [Google Scholar] [CrossRef] [Green Version]
  20. Yorbik, O.; Kurt, I.; Hasimi, A.; Ozturk, O. Chromium, cadmium, and lead levels in urine of children with autism and typically developing controls. Biol. Trace Elem. Res. 2010, 135, 10–15. [Google Scholar] [CrossRef]
  21. Soden, S.E.; Lowry, J.A.; Garrison, C.B.; Wasserman, G.S. 24-hour provoked urine excretion test for heavy metals in children with autism and typically developing controls, a pilot study. Clin. Toxicol. 2007, 45, 476–481. [Google Scholar] [CrossRef] [PubMed]
  22. Clark, B.; Vandermeer, B.; Simonetti, A.; Buka, I. Is lead a concern in Canadian autistic children? Paediatr. Child Health 2010, 15, 17–22. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. DeSoto, M.C.; Hitlan, R.T. Concerning Blood Mercury Levels and Autism: A Need to Clarify. J. Child Neurol. 2008, 23, 463–465. [Google Scholar] [CrossRef]
  24. Lakshmi Priya, M.; Geetha, A. Level of Trace Elements (Copper, Zinc, Magnesium and Selenium) and Toxic Elements (Lead and Mercury) in the Hair and Nail of Children with Autism. Biol. Trace Elem. Res. 2010, 142, 148–158. [Google Scholar] [CrossRef] [PubMed]
  25. Blaylock, R.L. Aluminum induced immunoexcitotoxicity in neurodevelopmental and neurodegenerative disorders. Curr. Inorg. Chem. 2012, 2, 46–53. [Google Scholar] [CrossRef]
  26. Choi, A.L.; Levy, J.I.; Dockery, D.W.; Ryan, L.M.; Tolbert, P.E.; Altshul, L.M.; Korrick, S.A. Does living near a Superfund site contribute to higher polychlorinated biphenyl (PCB) exposure? Environ. Health Perspect. 2006, 114, 1092–1098. [Google Scholar] [CrossRef] [PubMed]
  27. Tian, Y.; Green, P.G.; Stamova, B.; Hertz-Picciotto, I.; Pessah, I.N.; Hansen, R.; Yang, X.; Gregg, J.P.; Ashwood, P.; Jickling, G.; et al. Correlations of gene expression with blood lead levels in children with autism compared to typically developing controls. Neurotox. Res. 2011, 19, 1–13. [Google Scholar] [CrossRef]
  28. Hertz-Picciotto, I.; Green, P.G.; Delwiche, L.; Hansen, R.; Walker, C.; Pessah, I.N. Blood mercury concentrations in CHARGE Study children with and without autism. Environ. Health Perspect. 2010, 118, 161–166. [Google Scholar] [CrossRef]
  29. Rahbar, M.H.; Samms-Vaughan, M.; Ardjomand-Hessabi, M.; Loveland, K.A.; Dickerson, A.S.; Chen, Z.; Bressler, J.; Shakespeare-Pellington, S.; Grove, M.L.; Bloom, K.; et al. The role of drinking water sources, consumption of vegetables and seafood in relation to blood arsenic concentrations of Jamaican children with and without Autism Spectrum Disorders. Sci. Total Environ. 2012, 433, 362–370. [Google Scholar] [CrossRef] [Green Version]
  30. Rahbar, M.H.; Samms-Vaughan, M.; Loveland, K.A.; Ardjomand-Hessabi, M.; Chen, Z.; Bressler, J.; Shakespeare-Pellington, S.; Grove, M.L.; Bloom, K.; Pearson, D.A.; et al. Seafood Consumption and Blood Mercury Concentrations in Jamaican Children With and Without Autism Spectrum Disorders. Neurotox. Res. 2013, 23, 22–38. [Google Scholar] [CrossRef]
  31. Rahbar, M.H.; Samms-Vaughan, M.; Dickerson, A.S.; Loveland, K.A.; Ardjomand-Hessabi, M.; Bressler, J.; Lee, M.; Shakespeare-Pellington, S.; Grove, M.L.; Pearson, D.A.; et al. Role of fruits, grains, and seafood consumption in blood cadmium concentrations of Jamaican children with and without Autism Spectrum Disorder. Res. Autism Spectr. Disord. 2014, 8, 1134–1145. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Rahbar, M.H.; Samms-Vaughan, M.; Ma, J.; Bressler, J.; Loveland, K.A.; Ardjomand-Hessabi, M.; Dickerson, A.S.; Grove, M.L.; Shakespeare-Pellington, S.; Beecher, C.; et al. Role of Metabolic Genes in Blood Arsenic Concentrations of Jamaican Children with and without Autism Spectrum Disorder. Int. J. Environ. Res. Public Health 2014, 11, 7874–7895. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Rahbar, M.H.; Samms-Vaughan, M.; Dickerson, A.S.; Loveland, K.A.; Ardjomand-Hessabi, M.; Bressler, J.; Shakespeare-Pellington, S.; Grove, M.L.; Pearson, D.A.; Boerwinkle, E. Blood Manganese Concentrations in Jamaican Children with and without Autism Spectrum Disorder. Environ. Health 2014, 13, 69. [Google Scholar] [CrossRef] [PubMed]
  34. Cermak, S.A.; Curtin, C.; Bandini, L.G. Food selectivity and sensory sensitivity in children with autism spectrum disorders. J. Am. Diet. Assoc. 2010, 110, 238–246. [Google Scholar] [CrossRef] [PubMed]
  35. Rahbar, M.H.; Samms-Vaughan, M.; Ma, J.; Bressler, J.; Loveland, K.A.; Hessabi, M.; Dickerson, A.S.; Grove, M.L.; Shakespeare-Pellington, S.; Beecher, C.; et al. Interaction between GSTT1 and GSTP1 allele variants as a risk modulating-factor for autism spectrum disorders. Res. Autism Spectr. Disord. 2015, 12, 1–9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Rahbar, M.H.; Samms-Vaughan, M.; Lee, M.; Christian, M.A.; Bressler, J.; Hessabi, M.; Grove, M.L.; Shakespeare-Pellington, S.; Coore Desai, C.; Reece, J.A.; et al. Interaction between manganese and GSTP1 in relation to autism spectrum disorder while controlling for exposure to mixture of lead, mercury, arsenic, and cadmium. Res. Autism Spectr. Disord. 2018, 55, 50–63. [Google Scholar] [CrossRef] [PubMed]
  37. David, D.; Dobrean, A.; Mogoase, C.; Dobrescu, I. Screening for Autism in Romanian Population: An Initial Study Investigating the Clinical Utility of the Screening Questionnaire for Autism Spectrum Disorders (Chestionarul de Screening pentru Tulbur-âri de Spectru Autist-CS-TSA). Transylv. J. Psychol. 2013, 14, 127–148. [Google Scholar]
  38. Autismul—Cifre, Statistici la Nivel Mondial. Situatia din Romania. Available online: http://sfatulparintilor.ro/prescolari/sanatate-prescolari/autismul-cifre-statistici-la-nivel-mondial-situatia-din-romania/ (accessed on 28 June 2019).
  39. World Health Organization. Health Risks of Heavy Metals from Long-Range Transboundary Air Pollution; World Health Organization Regional Office Europe: Copenhagen, Denmark, 2007. [Google Scholar]
  40. Moldoveanu, A.M. Assessment of Soil Pollution with Heavy Metals in Romania. In Environmental Risk Assessment of Soil Contamination; Hernandez-Soriano, M.C., Ed.; IntechOpen: London, UK, 2014; pp. 285–305. [Google Scholar]
  41. Dinescu, L.C.; Duliu, O.G. Heavy metal pollution of some Danube Delta lacustrine sediments studied by neutron activation analysis. Appl. Radiat. Isot. 2001, 54, 853–859. [Google Scholar] [CrossRef]
  42. Bird, G.; Brewer, P.A.; Macklin, M.G.; Serban, M.; Balteanu, D.; Driga, B. Heavy metal contamination in the Aries river catchment, western Romania: Implications for development of the Rosia Montana gold deposit. J. Geochem. Explor. 2005, 86, 26–48. [Google Scholar] [CrossRef]
  43. Sakan, S.; Grzetic, I.; Dordevic, D. Distribution and fractionation of heavy metals in the Tisa (Tisza) river sediments. Environ. Sci. Pollut. Res. Int. 2007, 14, 229–236. [Google Scholar] [CrossRef]
  44. Matache, M.L.; Marin, C.; Rozylowicz, L.; Tudorache, A. Plants accumulating heavy metals in the Danube River wetlands. J. Environ. Health Sci. Eng. 2013, 11, 39. [Google Scholar] [CrossRef] [PubMed]
  45. Subotic, S.; Visnjic, J.Z.; Spasic, S.; Hegedis, A.; Krpo-Cetkovic, J.; Lenhardt, M. Distribution and accumulation of elements (As, Cu, Fe, Hg, Mn, and Zn) in tissues of fish species from different trophic levels in the Danube River at the confluence with the Sava River (Serbia). Environ. Sci. Pollut. Res. Int. 2013, 20, 5309–5317. [Google Scholar] [CrossRef] [PubMed]
  46. Lacatusu, R.; Rauta, C.; Carstea, S.; Ghelase, I. Soil-plant-man relationships in heavy metal polluted areas in Romania. Appl. Geochem. 1996, 11, 105–107. [Google Scholar]
  47. Senila, M. Real and simulated bioavailability of lead in contaminated and uncontaminated soils. J. Environ. Health Sci. Eng. 2014, 12, 108. [Google Scholar] [CrossRef] [PubMed]
  48. Soceanu, A.; Dobrinas, S.; Stanciu, G.; Popescu, V. Polycyclic aromatic hydrocarbons in vegetables grown in urban and rural areas. Environ. Eng. Manag. J. 2014, 13, 2311–2315. [Google Scholar]
  49. Lacatusu, R.; Lacatusu, A.R. Vegetable and fruits quality within heavy metals polluted areas in Romania. Carpathian J. Earth Environ. Sci. 2008, 3, 115–129. [Google Scholar]
  50. Dehelean, A.; Magdas, D.A. Analysis of mineral and heavy metal content of some commercial fruit juices by inductively coupled plasma mass spectrometry. Sci. World J. 2013, 2013, 215423. [Google Scholar] [CrossRef]
  51. World Health Organization. Guidelines for Drinking Water Quality; World Health Organization: Geneva, Switzerland, 2014. [Google Scholar]
  52. Iurian, S.I.; Neamtu, M.L.; Neamtu, B.M. The Biological Effects of Heavy Metals Exposure in Children. Clinical Study. Pediatr. Res. 2010, 68, 698. [Google Scholar]
  53. Rudnai, P. Levels of Lead in Children’s Blood; World Health Organization-Europe, European Environmental and Health Information System (ENIHS); National Institute of Environmental Health: Budapest, Hungary, 2009.
  54. Brad, A.; Blaga, A.B.; Pop, C. Children’s Exposure to Lead and Cadmium in Baia Mare area, Romania. Studia Universitatis Babes-Bolyai Ambientum 2015, 60, 31–38. [Google Scholar]
  55. Dilea, M.; Prelipcean, D.D.; Ionita, D. About oral health of romanian children from various polluted area due to heavy metals. UPB Bull. Stiintific Ser. B Chem. Mater. Sci. 2012, 74, 171–182. [Google Scholar]
  56. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV); American Psychiatric Publishing, Inc.: Washington, DC, USA, 1994. [Google Scholar]
  57. World Health Organization. International Classification of Diseases, 10th Revision (ICD-10), 2nd ed.; World Health Organization: Geneva, Switzerland, 2004. [Google Scholar]
  58. Lord, C.; Rutter, M.; Couteur, A. Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J. Autism Dev. Disord. 1994, 24, 659–685. [Google Scholar] [CrossRef] [PubMed]
  59. Rutter, M.; Le, C.A.; Lord, C. Autism Diagnostic Interview-Revised (ADI-R); Western Psychological Services: Los Angeles, CA, USA, 2003. [Google Scholar]
  60. David, D. Autism Diagnostic Interview-Revised (ADI-R)-Translated to Romanian Language; Testcentral: Bucharest, Romania, 2010. [Google Scholar]
  61. David, D. Autism Disorder Observation Schedule (ADOS)-Translated to Romanian Language; Testcentral: Bucharest, Romania, 2009. [Google Scholar]
  62. Rahbar, M.H.; Samms-Vaughan, M.; Ma, J.; Bressler, J.; Dickerson, A.S.; Hessabi, M.; Loveland, K.A.; Grove, M.L.; Shakespeare-Pellington, S.; Beecher, C.; et al. Synergic effect of GSTP1 and blood manganese concentrations in Autism Spectrum Disorder. Res. Autism Spectr. Disord. 2015, 18, 73–82. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  63. Rahbar, M.H.; Samms-Vaughan, M.; Dickerson, A.S.; Loveland, K.A.; Ardjomand-Hessabi, M.; Bressler, J.; Shakespeare-Pellington, S.; Grove, M.L.; Pearson, D.A.; Boerwinkle, E. Blood Lead Concentrations in Jamaican Children with and without Autism Spectrum Disorder. Int. J. Environ. Res. Public Health 2014, 12, 83–105. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  64. Meeker, J.D.; Sathyanarayana, S.; Swan, S.H. Phthalates and other additives in plastics: Human exposure and associated health outcomes. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2009, 364, 2097–2113. [Google Scholar] [CrossRef] [PubMed]
  65. SAS Institute Inc. SAS® 9.4. 2013; SAS Institute Inc.: Cary, NC, USA, 2013. [Google Scholar]
  66. Agency for Toxic Substances and Disease Registry. Toxicological Profile for Chlorinated Dibenzo-p-Dioxins; Agency for Toxic Substances and Disease Registry (ATSDR): Atlanta, GA, USA, 1998.
  67. Agency for Toxic Substances and Disease Registry. Toxicological Profile for Arsenic; Agency for Toxic Substances and Disease Registry (ATSDR): Atlanta, GA, USA, 2007.
  68. Agency for Toxic Substances and Disease Registry. Toxicological Profile for Aluminum; Agency for Toxic Substances and Disease Registry (ATSDR): Atlanta, GA, USA, 2008.
  69. Agency for Toxic Substances and Disease Registry; Centers for Disease Control and Prevention. Children’s Exposure to Elemental Mercury: A National Review of Exposure Events; Agency for Toxic Substances and Disease Registry (ATSDR): Atlanta, GA, USA, 2009.
  70. Agency for Toxic Substances and Disease Registry. Toxicological Profile for Cadmium; Agency for Toxic Substances and Disease Registry (ATSDR): Atlanta, GA, USA, 2008.
  71. Centers for Disease Control and Prevention (CDC). Blood Lead Levels in Children; Centers for Disease Control and Prevention (CDC): Atlanta, GA, USA, 2012.
  72. Laboratory Corporation of America. Aluminum, Serum, Plasma, or Whole Blood; Laboratory Corporation of America: Burlington, NC, USA, 2019. [Google Scholar]
  73. Nicolescu, R.; Petcu, C.; Cordeanu, A.; Fabritius, K.; Schlumpf, M.; Krebs, R.; Kramer, U.; Winneke, G. Environmental exposure to lead, but not other neurotoxic metals, relates to core elements of ADHD in Romanian children: Performance and questionnaire data. Environ. Res. 2010, 110, 476–483. [Google Scholar] [CrossRef] [PubMed]
  74. Chaidez, V.; Hansen, R.L.; Hertz-Picciotto, I. Gastrointestinal problems in children with autism, developmental delays or typical development. J. Autism Dev. Disord. 2014, 44, 1117–1127. [Google Scholar] [CrossRef] [PubMed]
  75. Gorrindo, P.; Williams, K.C.; Lee, E.B.; Walker, L.S.; McGrew, S.G.; Levitt, P. Gastrointestinal dysfunction in autism: Parental report, clinical evaluation, and associated factors. Autism Res. 2012, 5, 101–108. [Google Scholar] [CrossRef] [PubMed]
  76. Lovei, M. Phasing Out Lead From Gasoline; World Bank: Washington, DC, USA, 1998. [Google Scholar]
  77. Velea, T.; Gherghe, L.; Predica, V.; Krebs, R. Heavy metal contamination in the vicinity of an industrial area near Bucharest. Environ. Sci. Pollut. Res. Int. 2009, 16 (Suppl. 1), S27–S32. [Google Scholar] [CrossRef]
  78. Barrett, J.R. CHILDREN’S HEALTH: Sex-Specific Cognitive Effects of Lead. Environ. Health Perspect. 2009, 117. [Google Scholar] [CrossRef]
  79. Kowalska, M.; Kulka, E.; Jarosz, W.; Kowalski, M. The determinants of lead and cadmium blood levels for preschool children from industrially contaminated sites in Poland. Int. J. Occup. Med. Environ. Health 2018, 31, 351–359. [Google Scholar] [CrossRef]
  80. Amodio-Cocchieri, R.; Arnese, A.; Prospero, E.; Roncioni, A.; Baruffo, L.; Ullucci, R.; Romano, V. Lead in human blood from children living in Campania, Italy. J. Toxicol. Environ. Health 1996, 47, 311–320. [Google Scholar] [CrossRef] [PubMed]
  81. Raju, G.; Praveen Kumar, N. Study on blood lead levels in children. Int. Arch. Integr. Med. (IAIM) 2017, 4, 84–93. [Google Scholar]
  82. Rahbar, M.H.; White, F.; Agboatwalla, M.; Hozhabri, S.; Luby, S. Factors associated with elevated blood lead concentrations in children in Karachi, Pakistan. Bull. World Health Organ. 2002, 80, 769–775. [Google Scholar] [PubMed]
  83. Siddiqui, M.F.M.F.; Rajurkar, G.R. Lead-An Emerging threat to livestock. Vet. World 2008, 1, 213. [Google Scholar]
  84. Tinggi, U.; Schoendorfer, N. Analysis of lead and cadmium in cereal products and duplicate diets of a small group of selected Brisbane children for estimation of daily metal exposure. J. Trace Elem. Med. Biol. 2018, 50, 671–675. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  85. Garzillo, E.M.; Lamberti, M.; Genovese, G.; Pedata, P.; Feola, D.; Sannolo, N.; Daniele, L.; Trojsi, F.; Monsurro, M.R.; Miraglia, N. Blood lead, manganese, and aluminum levels in a regional Italian cohort of ALS patients: Does aluminum have an influence? J. Occup. Environ. Med. 2014, 56, 1062–1066. [Google Scholar] [CrossRef] [PubMed]
  86. Ruggieri, F.; Majorani, C.; Domanico, F.; Alimonti, A. Mercury in Children: Current State on Exposure through Human Biomonitoring Studies. Int. J. Environ. Res. Public Health 2017, 14, 519. [Google Scholar] [CrossRef] [PubMed]
  87. Puklová, V.; Krsková, A.; Cerná, M.; Cejchanová, M.; Rehûrková, I.; Ruprich, J.; Kratzer, K.; Kubínová, R.; Zimová, M. The mercury burden of the Czech population: An integrated approach. Int. J. Hyg. Environ. Health 2010, 213, 243–251. [Google Scholar] [CrossRef]
  88. Pino, A.; Amato, A.; Alimonti, A.; Mattei, D.; Bocca, B. Human biomonitoring for metals in Italian urban adolescents: Data from Latium Region. Int. J. Hyg. Environ. Health 2012, 215, 185–190. [Google Scholar] [CrossRef]
  89. European Commission. Science for Environment Policy In-Depth Report: Soil Contamination: Impacts on Human Health; European Commission DG Environment: Brussels, Belgium, 2013. [Google Scholar]
  90. Jung, K.; Pergande, M.; Graubaum, H.J.; Fels, L.M.; Endl, U.; Stolte, H. Urinary proteins and enzymes as early indicators of renal dysfunction in chronic exposure to cadmium. Clin. Chem. 1993, 39, 757–765. [Google Scholar]
  91. Berglund, M.; Lind, B.; Bjornberg, K.A.; Palm, B.; Einarsson, O.; Vahter, M. Inter-individual variations of human mercury exposure biomarkers: A cross-sectional assessment. Environ. Health 2005, 4, 20. [Google Scholar] [CrossRef] [PubMed]
  92. Santamaria, A.B.; Sulsky, S.I. Risk assessment of an essential element: Manganese. J. Toxicol. Environ. Health A 2010, 73, 128–155. [Google Scholar] [CrossRef] [PubMed]
  93. Barbosa, F., Jr.; Tanus-Santos, J.E.; Gerlach, R.F.; Parsons, P.J. A critical review of biomarkers used for monitoring human exposure to lead: Advantages, limitations, and future needs. Environ. Health Perspect. 2005, 113, 1669–1674. [Google Scholar] [CrossRef] [PubMed]
  94. Exley, C. Human exposure to aluminium. Environ. Sci. Process. Impacts 2013, 15, 1807–1816. [Google Scholar] [CrossRef] [Green Version]
  95. Harris, P.A.; Taylor, R.; Thielke, R.; Payne, J.; Gonzalez, N.; Conde, J.G. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 2009, 42, 377–381. [Google Scholar] [CrossRef]
Table 1. Characteristics of children with ASD (or suspected of ASD) and their parents (N = 60).
Table 1. Characteristics of children with ASD (or suspected of ASD) and their parents (N = 60).
VariablesCategoriesn (%)
Child’s sexMale54 (90.0)
Child’s age (months)Age < 4832 (53.3)
48 ≤ Age < 7218 (30.0)
Age ≥ 7210 (16.7)
Child’s race Caucasian (white)59 (98.3)
Other1 (1.7)
Child’s ethnicity Romanian59 (98.3)
Roma1 (1.7)
Maternal age (at child’s birth) a< 35 years50 (87.7)
≥ 35 years7 (12.3)
Paternal age (at child’s birth) b< 35 years42 (71.2)
≥ 35 years17 (28.8)
Maternal race cCaucasian (white)59 (98.3)
Paternal raceCaucasian (white)59 (98.3)
Other1 (1.7)
Maternal education (at child’s birth) dUp to high school *15 (28.3)
Beyond high school **38 (71.7)
Paternal education (at child’s birth) eUp to high school *30 (54.5)
Beyond high school **25 (45.5)
Socioeconomic status (SES)Car ownership11 (18.3)
Home ownership41 (68.3)
Family history of neurodevelopment disorders fAutism spectrum disorder (ASD)4 (6.7)
Attention-deficit/hyperactivity disorder (ADHD)7 (11.7)
Learning problems8 (13.3)
Speech problems19 (31.7)
Behavior or disruptive problems at school5 (8.3)
Difficulties interacting with other people3 (5.0)
Intellectual disability2 (3.3)
Seizure5 (8.3)
* Up to high school education means attended Primary/Jr. Secondary, and Secondary/High/Technical schools; ** Beyond high school education means attended a Vocational, Tertiary College, or University. a Maternal age was missing for three mothers. b Paternal age was missing for one father. c Maternal race was missing for one mother. d Maternal education was missing for seven mothers. e Paternal education was missing for five fathers. f Family history includes; mother, father, siblings, grandparents, uncles, aunts, cousins, nieces or nephews.
Table 2. Arithmetic and geometric mean blood metal concentration of ASD children (N = 60).
Table 2. Arithmetic and geometric mean blood metal concentration of ASD children (N = 60).
VariablesLoD% Below LoDArithmetic MeanMedianGeometric Mean aRecommended Cut-Off LevelPercentage above the Cut-Off Levels
Lead (μg/dL)0.2501.551.101.1455
Manganese (μg/L)2.5011.4110.5010.841517
Aluminum (μg/L)5.03029.7714.0014.44960
Mercury (μg/L)0.2561.70.820.250.3553.3
Arsenic (μg/L)1.390NRNRNR120
Cadmium (μg/L)0.1395NRNRNR0.3151.7
a Mean blood metal concentration indicates the geometric mean = Exp. [Mean (ln metal concentration)]. NR: Geometric and arithmetic means as well as medians are not reported, because over 90% of As and Cd concentrations in blood were below LoD.
Table 3. Factors associated with blood lead concentrations based on univariable and multivariable General Linear Models (N = 60).
Table 3. Factors associated with blood lead concentrations based on univariable and multivariable General Linear Models (N = 60).
Exposure VariablesCategoryUnivariableMultivariable ***
YesNop-ValueYesNop-Value
Geometric Mean Pb (μg/dL) *NGeometric Mean Pb (μg/dL) *NAdjusted Geometric Mean Pb (μg/dL) **Adjusted Geometric Mean Pb (μg/dL) **
Sex of childMale1.05542.4860.0061.522.580.03
Maternal age a (at child’s birth)More than 35 years0.9771.17500.53
Paternal age b (at child’s birth)More than 35 years1.45170.99420.06---
Parental education levels c (at child’s birth)At least one of the parents had education beyond high school 0.96361.78140.0051.582.480.004
Socioeconomic status (SES)High SES (own a car)1.17111.14490.92
Source of drinking water dBottled water1.07510.9330.74
Source of cooking water ePiped water0.56471.1550.03
Pica (habitually put items in mouth)Mud3.4461.0154<0.00012.681.460.01
Living near a high traffic road f1.47240.98350.04---
Home environmentLiving with adults whose jobs involve construction1.16101.14500.94
Types of toys the child plays withPlastic1.13592.8010.23
Electronic 1.03441.53160.07
Battery operated1.07511.6690.11
Stuffed1.20151.12450.78
Types of pots, pans, and dishes used at homeCast iron g1.46181.02410.10---
Teflon 1.16550.9650.59
Aluminum h1.22291.06290.49
Fruits and vegetables consumptionLeafy vegetablesLettuce1.50210.99390.04---
Spinach, kale0.94291.34310.07
FruitsBlackberry0.77131.27470.03
Banana1.11521.4180.40
Watermelon1.40340.87260.022.381.640.008
Other melons (cantaloupe, honeydew)1.22271.08330.54
Meat consumptionBeef0.99421.59180.02
Lamb1.8491.05510.042.371.650.053
Goat2.2341.09560.6---
Pork1.22440.95160.26
Minced meat (beef/pork/chicken)1.27470.78130.04---
Grain/Starches consumptionDark bread0.88211.31390.05
Oatmeal0.94101.19500.38
Cold breakfast (cereal)1.32410.84190.032.671.460.0001
Pasta, macaroni, noodles 1.06501.67100.08
Biscuits1.17590.2810.06---
Polenta1.22371.04230.42
Peas, beans, and nutsRed beans1.0481.16520.71
Peanuts, cashews1.49290.89310.008---
Tree nuts (pecans, walnuts)1.53260.92340.007---
Do you use oil from animal fat for cooking?2.0261.0754<0.05---
* Mean Pb indicate the geometric mean = Exp. [Mean (lnPb)]; ** Adjusted for sex, parental education levels, pica (habitually put items in mouth), ate watermelon, consumed lamb, and ate cold breakfast (cereal); *** Levels of parental education was missing for 10 parents. As a result the sample size for multivariable analysis was based on N = 50. The Yes column includes children who met the category specified in front of each exposure variable or the food consumed. The No column includes children who did not meet the category specified in front of each exposure variable or never consumed the food. “-“ Indicates exposure variables that had p-value < 0.10 in univariable analysis but were not significantly associated blood metal concentrations in multivariable models at 5% level. a Maternal age was missing for three mothers. b Paternal age was missing for one father. c Parental education levels were missing for 10 parents. d Source of drinking water was missing for six households. e Source of cooking water was missing for eight households. f Living near a high traffic road was missing for one household. g Cast iron was missing for one household. h Aluminum pots, pans, and dishes used at home was missing for 2 households.
Table 4. Factors associated with blood manganese concentrations based on univariable General Linear Models (N = 60).
Table 4. Factors associated with blood manganese concentrations based on univariable General Linear Models (N = 60).
Exposure VariablesCategoryUnivariable
YesNop-Value
Geometric Mean Mn (μg/L) *NGeometric Mean Mn (μg/L) *N
Sex of childMale11.905410.7260.44
Maternal age a (at child’s birth)More than 35 years12.75710.83500.19
Paternal age b (at child’s birth)More than 35 years11.241710.45420.38
Parental education levels c (at child’s birth)At least one of the parents had education beyond high school10.853610.77140.94
Socioeconomic status (SES)High SES (own a car)9.681111.12490.19
Source of drinking water dBottled water10.615111.4430.66
Source of cooking water ePiped water10.374710.5750.88
Fruits and vegetables consumptionLeafy vegetablesLettuce10.462111.05390.52
Spinach, kale10.042911.53310.09
FruitsBlackberry9.821311.14470.20
Banana11.13529.1080.09
Watermelon10.713411.00260.75
Other melons (cantaloupe, honeydew)10.822710.85330.99
Seafood consumptionImported seafoodAte shrimp11.05510.82550.89
Fresh seafoodAte channel catfish11.35310.81570.80
Ate lake/pond fish (catfish, crappie)10.49610.88540.79
Ate bay fish (speckled Trout, redfish, flounder)10.641110.88490.83
Ate river fish (bass, trout)10.872210.82380.96
Ate offshore fish (tuna, snapper, whiting)10.96510.83550.93
Ate shellfish (lobster, crab, crawfish)20.00110.72590.05
Grain/Starches consumptionDark bread10.482111.03390.55
Oatmeal11.861010.64500.32
Cold breakfast (cereal)10.774110.98190.82
Pasta, macaroni, noodles 10.605012.10100.22
Biscuits10.87598.9010.53
Polenta11.48379.88230.07
* Mean Mn indicate the geometric mean = Exp. [Mean (lnMn)]; The Yes column includes children who met the category specified in front of each exposure variable or the food consumed. The No column includes children who did not meet the category specified in front of each exposure variable or never consumed the food. a Maternal age was missing for 3 mothers. b Paternal age was missing for 1 father. c Parental education levels were missing for 10 parents d Source of drinking water was missing for 6 households. e Source of cooking water was missing for 8 households.
Table 5. Factors associated with blood aluminum concentrations based on univariable General Linear Models (N = 60).
Table 5. Factors associated with blood aluminum concentrations based on univariable General Linear Models (N = 60).
Exposure VariablesCategoryUnivariable
YesNop-Value
Geometric Mean Al (μg/L) *NGeometric Mean Al (μg/L) *N
Sex of childMale14.535413.6760.91
Maternal age a (at child’s birth)More than 35 years13.83714.39500.94
Paternal age b (at child’s birth)More than 35 years15.511713.20420.64
Parental education levels c (at child’s birth)At least one of the parents had education beyond high school 15.983615.17140.89
Socioeconomic status (SES)High SES (own a car)20.731113.31490.28
Source of drinking water dBottled water15.80519.8030.50
Source of cooking water ePiped water6.514717.1950.08
Types of toys the child plays withPlastic 14.005988.0010.14
Electronic 14.584414.06160.92
Battery operated15.41519.9990.33
Stuffed19.181513.14450.30
Types of pots, pans, and dishes used at homeCast iron f18.761813.32410.32
Teflon 14.685512.0950.74
Aluminum g13.112915.32290.64
Fruits and vegetables consumptionLeafy vegetablesLettuce13.342115.07390.72
Spinach, kale18.312911.89310.18
FruitsBlackberry13.601314.68470.84
Banana15.235210.2180.39
Watermelon17.573411.17260.16
Other melons (cantaloupe, honeydew)20.472710.86330.04
Seafood consumptionImported seafoodAte shrimp44.95513.02550.03
Fresh seafoodAte channel catfish32.08313.85570.25
Ate lake/pond fish (catfish, crappie)9.92615.06540.43
Ate bay fish (speckled trout, redfish, flounder)17.091113.90490.62
Ate river fish (bass, trout)11.252216.69380.23
Ate offshore fish (tuna, snapper, whiting)21.08513.95550.47
Ate shellfish (lobster, crab, crawfish)27.00114.29590.61
Meat consumptionBeef16.064211.27180.31
Lamb32.06912.54510.03
Goat37.90413.48560.10
Pork14.064415.55160.78
Minced meat (beef/pork/chicken)14.154715.52130.81
Juices/ Soft DrinkJuices (e.g., orange, tomato, etc.)12.224523.84150.07
Flavored beverages15.492413.78360.72
Soft drinks (soda, Coke/Pepsi)29.94713.11530.09
Hot tea (e.g., black, Earl Grey, green)13.952714.86330.84
Iced tea19.05913.75510.46
* Mean Al indicate the geometric mean = Exp. [Mean (lnAl)]. The Yes column includes children who met the category specified in front of each exposure variable or the food consumed. The No column includes children who did not meet the category specified in front of each exposure variable or never consumed the food. a Maternal age was missing for three mothers. b Paternal age was missing for one father. c Parental education levels were missing for 10 parents d Source of drinking water was missing for six households. e Source of cooking water was missing for eight households. f Cast iron was missing for one household. g Aluminum pots, pans, and dishes used at home was missing for two households.

Share and Cite

MDPI and ACS Style

Hessabi, M.; Rahbar, M.H.; Dobrescu, I.; Bach, M.A.; Kobylinska, L.; Bressler, J.; Grove, M.L.; Loveland, K.A.; Mihailescu, I.; Nedelcu, M.C.; et al. Concentrations of Lead, Mercury, Arsenic, Cadmium, Manganese, and Aluminum in Blood of Romanian Children Suspected of Having Autism Spectrum Disorder. Int. J. Environ. Res. Public Health 2019, 16, 2303. https://doi.org/10.3390/ijerph16132303

AMA Style

Hessabi M, Rahbar MH, Dobrescu I, Bach MA, Kobylinska L, Bressler J, Grove ML, Loveland KA, Mihailescu I, Nedelcu MC, et al. Concentrations of Lead, Mercury, Arsenic, Cadmium, Manganese, and Aluminum in Blood of Romanian Children Suspected of Having Autism Spectrum Disorder. International Journal of Environmental Research and Public Health. 2019; 16(13):2303. https://doi.org/10.3390/ijerph16132303

Chicago/Turabian Style

Hessabi, Manouchehr, Mohammad H. Rahbar, Iuliana Dobrescu, MacKinsey A. Bach, Liana Kobylinska, Jan Bressler, Megan L. Grove, Katherine A. Loveland, Ilinca Mihailescu, Maria Cristina Nedelcu, and et al. 2019. "Concentrations of Lead, Mercury, Arsenic, Cadmium, Manganese, and Aluminum in Blood of Romanian Children Suspected of Having Autism Spectrum Disorder" International Journal of Environmental Research and Public Health 16, no. 13: 2303. https://doi.org/10.3390/ijerph16132303

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop