Next Article in Journal
Associations between Pharmacotherapy for Cardiovascular Diseases and Periodontitis
Next Article in Special Issue
Increased Alopecia Areata Risk in Children with Attention-Deficit/Hyperactivity Disorder and the Impact of Methylphenidate Use: A Nationwide Population-Based Cohort Study
Previous Article in Journal
Microplastic Contamination in Snow from Western Italian Alps
Previous Article in Special Issue
Objective Analysis of Movement in Subjects with ADHD. Multidisciplinary Control Tool for Students in the Classroom
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Stress/Immune Biomarkers in Saliva among Children with ADHD Status

by
Anna Krahel
1,
Elzbieta Paszynska
1,*,†,
Agnieszka Slopien
2,†,
Maria Gawriolek
1,
Justyna Otulakowska-Skrzynska
1,
Szymon Rzatowski
1,
Amadeusz Hernik
1,
Tomasz Hanć
3,
Ewa Bryl
3,
Paula Szczesniewska
3,
Karolina Bilska
4,
Joanna Duda
4,
Marta Tyszkiewicz-Nwafor
2,‡ and
Monika Dmitrzak-Weglarz
4,‡
1
Department of Integrated Dentistry, Poznan University of Medical Sciences, 60-812 Poznan, Poland
2
Department of Child and Adolescent Psychiatry, Poznan University of Medical Sciences, 60-572 Poznan, Poland
3
Institute of Human Biology and Evolution, Faculty of Biology, Adam Mickiewicz University, 61-614 Poznan, Poland
4
Psychiatric Genetics Unit, Department of Psychiatry, Poznan University of Medical Sciences, 60-806 Poznan, Poland
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work and shared the second authorship.
These authors contributed equally to this work and shared the last authorship.
Int. J. Environ. Res. Public Health 2021, 18(2), 769; https://doi.org/10.3390/ijerph18020769
Submission received: 12 December 2020 / Revised: 6 January 2021 / Accepted: 9 January 2021 / Published: 18 January 2021
(This article belongs to the Special Issue Diagnosis and Treatment of ADHD in Adolescents)

Abstract

:
Background. This cross-sectional study aimed to evaluate stress and immune biomarkers in saliva samples of attention-deficit/hyperactivity disorder ADHD compared to healthy non-ADHD children. Material and methods. A total of 132 children under 11 years old (8.5 ± 1.1) enrolled in a cross-sectional study: with confirmed ADHD (n = 60) and healthy controls (n = 72). The clinical evaluation included physical measurements (height, waist, hip circumference, body weight, body mass index BMI, BMI z-score) and unstimulated saliva collection and measurements of free cortisol, salivary alpha-amylase (sAA), and secreted immunoglobulins (sIgA, IgG, and IgM) with quantitative assay (ELISA) analysis. Unpaired t-test, Welch test, or Mann–Whitney U test were applied for group comparisons when appropriate, and the correlation between variables was analyzed with Spearman’s rank coefficient. Results were considered significant at p < 0.05. Results. In the ADHD group, body weight (p ≤ 0.01), BMI (p ≤ 0.009), and hip circumference (p ≤ 0.001) significantly differed, while waist size and BMI z-score did not (p > 0.05). Significant elevation of the salivary sAA (p = 0.03), sIgA (p = 0.02), and IgM (p ≤ 0.001) biomarkers were detected, without differences in the morning cortisol (p > 0.05). Significant correlations between cortisol and BMI, hip size, and IgA, as well as between IgG and sAA and IgA were obtained. Conclusions. Saliva can be used to monitor ADHD status with regard to biomarkers indicating the hypothalamus–pituitary–adrenal axis, as HPA axis, and sympathetic activity. The results indicate that morning collection of saliva in contrast to unchanged salivary cortisol, may evaluate mentioned above system dysregulations by measurements of sAA and immunoglobulins among ADHD children.

1. Introduction

A dental office is a frequently visited health-care facility by children and adolescents under 18 years of age. This allows easy contact with children and may also help in monitoring mental stress during the developmental period of life among healthy or unhealthy subjects.
Diagnosing and managing attention-deficit/hyperactivity disorder (ADHD) in childhood are exceptional achievements for medical clinicians [1]. Research has shown that, for children with ADHD, there is not an established, adequate diagnostic related to clinical symptoms, and they may suffer from significant dysfunction [2,3]. Some of the ADHD symptoms, such as extreme hyperactivity, may appear or disappear over time. Additionally, obtaining information from the family could be limited [1]. The abovementioned difficulties are only examples of scientific gaps in research that may affect the ability to screen, diagnose, and treat ADHD. A critical level of stress and inflammatory biomarkers in saliva among young ADHD individuals could be a source of potential knowledge to determine the diagnosis and course of ADHD disease empirically [4].
The salivary glands work together with the neuroendocrine system, which makes saliva sensitive to stress or mood behaviors [5]. Valuable stress indicators such as salivary cortisol, salivary alpha-amylase (sAA), secretory immunoglobulin A (sIgA), G (IgG), and M (IgM) can commonly be found in saliva as well as in other body fluids. Still, as a response to mental stress, the largest variability in these biomarkers’ concentrations, may be recorded in saliva [6,7].
Many studies have described a significant effect of stress on the activation of the hypothalamic–pituitary–adrenal (HPA) axis, resulting in increased cortisol production [6,7]. Cortisol is released according to a person’s circadian rhythm, with a peak in the morning after waking up. Its level decreases during the day to reach the lowest value just before bedtime [8]. Exposure to a stress factor is associated with an increase in cortisol production and a rapid return to its resting level [8]. Chronically sustained elevated or lowered cortisol levels may lead to mood disorders [9,10], social isolation [8], eating disorders [11], and neurodevelopmental disorders [12].
Currently, it is proposed that the HPA axis and the activity of the sympathetic nervous system (SNS) have to be monitored in parallel during mental stress [13,14,15]. Saliva can be a source of information on SNS activity through salivary alpha-amylase concentration [16]. Complementary, stress-induced alterations in saliva were detected in other groups of patients with mental disturbances [17,18].
Just as salivary cortisol and amylase have been studied quite frequently, the salivary immunoglobulins of the immune system, such as IgA, IgG, and IgM, have been unexplored in ADHD [19]. The oral mucosal immune system is one of the defense effectors against human pathogens and also reveals a subclinical immune reaction to hyperactive behavior in ADHD [20]. The measurement of salivary immunoglobulins might be indicated as a non-invasive method to determine the immune status of ADHD children in terms of general and oral infections. Investigations focusing on stress/immune biomarkers and ADHD are few. An examination of the abovementioned biomarkers may at the same time provide a comprehensive knowledge about all aspects of stress regulation [21,22,23,24]. Therefore, this study aimed to evaluate salivary cortisol, sAA, IgA, IgG, and IgM among children with clinically confirmed ADHD against healthy, non-ADHD children.

2. Material and Methods

2.1. Sample of the ADHD Group

The intention was to select children with ADHD from one part of the area, the western sector of the country (Supplementary Figure S1). The recruitment of children with suspected ADHD was announced in GP clinics (n = 100) and mental health clinics for children and adolescents (n = 20). In total, 69 families having a child with a suspected ADHD diagnosis applied for recruitment and were willing to include their child in the investigation. Ultimately, in the 69 children, a diagnosis of ADHD was determined by two specialists, a psychiatrist and a clinical psychologist, according to the diagnostic criteria of DSM-V [25,26]. Most of the examined children had symptoms from all criteria areas of the disorder, which allowed for the diagnosis of a mixed form of ADHD. Among the abovementioned children who came for the study, the diagnosis was confirmed in 64 of them. Two children were excluded due to a lack of cooperation during a dental examination in the office, and the parents of two children withdrew their consent. Ultimately, the study group consisted of 60 children aged 6 to 11 years (8.6 ± 1.2) with a confirmed diagnosis of ADHD: 50 boys (82.1% of all participants in the study group) and 10 girls (17.9%) (Supplementary Figure S1).

Sample of the Control Group

In one agglomeration, four primary schools were randomly selected from opposite geographic regions––eastern, western, northern, and southern. In the schools, the project was proposed to parents of children aged 6 to 11. The researchers offered a free child-development analysis and dental examination at the dental office. Four hundred willing parents were enrolled from four schools to have their children examined. Regarding the inclusion criteria (Table 1), 150 children were pre-qualified for further evaluation. After the general interview, 50 children did not meet inclusion age criterion because they were over 12 years old. In addition, 18 children did not cooperate during the dental examination or did not attend school for more than 4 weeks. Finally, 72 children were allocated to the control group, with ages ranging from 6 to 11 years (8.3 ± 1.1); 45 boys (62.5% of the control group) and 27 girls (37.5%) were recruited among primary schoolchildren (Supplementary Figure S1).

2.2. Ethical Approval

The cross-sectional project involved non-invasive analysis methods such as salivary sampling and questionnaires to confirm or deny a diagnosis of ADHD. The study plan was correct according to the standards of Good Clinical Practice (GCP) and had a positive opinion of the local Ethics Committee of the Medical University of Poznań (Resolution No. 26/18) [27]. All parents and children were informed about the aims and methods of the research, and their legal representatives gave their written consent.

2.3. Assessment of Child Development

The anthropometric assessment of a child’s development consisted of body measurements, such as height, waist, hip circumference, and body weight. The standing body height was measured with a SECA 216 wall-mounted stadiometer with an accuracy of 0.1 cm. The body weight of children was assessed in light clothing on a digital scale with an accuracy of 0.1 kg. The waist circumference was determined at the midpoint between the lower edge of the costal arch and the upper iliac crest using a metric tape [28]. The hip circumference was measured parallel to the floor and taken as the greatest circumference of the buttocks using a metric tape [28]. All subjects were assessed according to Body Mass Index (BMI), age, weight, and height. The BMI was transformed into z- scores based on WHO growth charts [29].

2.4. ELISA Salivary Samples

All participants in both groups underwent the same procedures in the dental office to avoid any bias that could affect salivary flow during the examination. The patients were examined in the same season of the calendar year (fall/winter) and at the same time in the morning, i.e., between 9:00 and 10:00. The same examiner took the saliva samples. All patients were examined 60 min after breakfast and oral hygiene procedures. Unstimulated total saliva was spat into sterile containers for 15 min. Children were instructed to move their body to forward position and focus on the spitting activity. The first portion of the saliva was collected in a separate container and then allowed to accumulate. It was then spit out into a sterile disposable tube [30]. Immediately after the collection, the container was sealed and subjected to centrifugation, supernatant separation, and two-stage freezing at −20 and then −80 °C. The tests for salivary biomarkers were performed as follows.

2.4.1. Salivary Cortisol

The concentration of free cortisol in the saliva was quantitatively determined by the ELISA method using the ELISA Kit DES6611 for the In Vitro Diagnostic (IVD) (Demeditec Diagnostics GmbH, Kiel, Germany). The standard curve ranged from 0 to 30 ng/mL, the intra- and inter-assay variability coefficient was assessed to be below 6%, and the standard curve was statistically significant (r2 = 0.998, p < 0.001) [31,32].

2.4.2. Salivary Alpha-Amylase (sAA)

The sAA concentrations were quantitatively determined by enzyme-linked immunosorbent assay (ELISA) using the ELISA Kit DEEQ6231 for the In Vitro Diagnostic (IVD) (Demeditec Diagnostics GmbH, Kiel, Germany). The standard curve ranged from 0 to 500 U/mL, the intra- and inter-assay variability coefficients were assessed to be below 5%, and the standard curve was statistically significant (r2 = 0.982, p < 0.001) [33].

2.4.3. Salivary Secretory IgA (sIgA)

Salivary sIgA concentrations were quantitatively determined by the ELISA method using the ELISA Kit DEXK276 for the In Vitro Diagnostic (IVD) (Demeditec Diagnostics GmbH, Kiel, Germany). The standard curve ranged from 0 to 400 µg/mL, the intra- and inter-assay variability coefficient were assessed to be below 5%, and the standard curve was statistically significant (r2 = 0.995, p < 0.001) [34,35].

2.4.4. Salivary IgG

Salivary IgG concentrations were quantitatively determined by the ELISA method using the ELISA Kit E0544h for research use only (RUO) (EIAab, Wuhan, China). The standard curve ranged from 0 to 5000 pg/mL, the intra-assay variability coefficient was <4.4%, and the inter-assay variability was <7.8%, respectively (r2 = 0.999, p < 0.001) [36,37].

2.4.5. Salivary IgM

Salivary IgM concentrations were quantitatively determined by the ELISA method using the ELISA Kit E0543h for research use only (RUO) (EIAab, Wuhan, China). The standard curve ranged from 0 to 500 ng/mL, the intra-assay variability coefficient was <6.2%, and the inter-assay variability was <9.1%, respectively (r2 = 0.986, p < 0.001) [38].
All ELISA tests were performed according to the manufacturer’s instructions without any modifications. All samples and standards were run in duplicate, and the mean value of the two assays was used for statistical evaluation. Optical density was read with a spectrophotometric plate reader (Asys UVM 340 Microplate Reader from Biochrom Ltd., Cambridge, UK) for a wavelength of 450 nm ± 10 nm. A four-parameter algorithm (four-parameter logistic) was used to assay the concentration in the tested samples.

2.5. Statistical Data Analysis

The analyzed data were expressed as mean ± standard deviation, median, and minimum and maximum values as appropriate. The normality of the distribution was tested using the Shapiro–Wilk test, and the equality of variances was checked using Levene’s test. A comparison of two unpaired groups was performed using the Mann–Whitney U-test. The relationship between variables was analyzed with Spearman’s rank correlation coefficient. All results were considered significant at p < 0.05. Statistical analyses were performed with STATISTICA v13 software (StatSoft Inc., Tulsa, U.S.).

3. Results

3.1. Sample

The final sample consisted of 132 participants: (60 ADHD and 72 control children) (Table 1). The mean age of the ADHD children was 8.6 ± 1.1 and that of the controls was 8.3 ± 1.2 years with no statistically significant difference for age and height (p > 0.05). The ADHD participants weighed 10.5% more than those in the control group (p ≤ 0.01), as body weight (p ≤ 0.01), BMI (p ≤ 0.009), and hip circumference (p ≤ 0.0002) were statistically different between patients and controls, but not the BMI z-score or waist size (p > 0.05) (Table 2).

3.1.1. Stress and Immune Biomarker Levels

An analysis of the salivary samples revealed a significant elevation of the sAA (p = 0.03), sIgA (p = 0.02), and IgM (p = 0.0000001) levels in the ADHD group (Table 2). However, no differences in the free-cortisol level were detected between the ADHD and healthy children of the control group (p = 0.739) (Table 2).

3.1.2. Correlations

In the ADHD group, the Spearman analysis showed a significant correlation between cortisol and BMI (rs = −0.35; p = 0.01), hip size (rs = −0.28; p = 0.04), and IgA (rs = 0.35; p = 0.01). IgG was correlated to alpha-amylase (rs = −0.38; p = 0.005) and IgA (rs = −0.47; p = 0.0005) (Table 3).
In the control group, significant correlations were found only between IgA and IgM (rs = −0.308; p = 0.01) (Table 3).

4. Discussion

The selected salivary biomarkers were chosen by evaluating previous studies on salivary stress and immune biomarkers and ADHD. However, there are limited results in the scientific literature, mostly connected with cortisol and sAA but not IgA, IgG, or IgM [19]. Saliva proved to be a diagnostic fluid to obtain information about the state of the children’s health, but not without significance is the non-invasive nature of the examination [39]. Our results showed similar cortisol concentrations in the groups of children with and without ADHD. Previously published investigations, based on 24-h cortisol monitoring in children with ADHD suggested a different regulation of the hypothalamic–pituitary–adrenal axis with a decrease concentration, especially in the morning [12,40]. In subsequent studies, children with ADHD revealed lower cortisol levels in the morning and at bedtime than during the day [41,42]. Therefore, coordination and control of cortisol levels with abnormal fluctuations throughout the day were considered. Unbalanced peaks and decreases in cortisol secretion and its irregular continuity towards increased concentrations may have affected the reversal results [40,41,42,43,44,45,46,47]. The hypothesis that bad life experiences, permanent stress, and adversities in early childhood may affect the HPA axis, leading to a decrease in cortisol levels in ADHD, has not been confirmed, except for a positive association between childhood adversity and a morning cortisol rise [42]. Therefore, the lack of significance in the cortisol level between the groups is justified because cortisol is characterized by a relatively high variability in secretion after the stress factor applied within a few minutes. It then quickly changes its level and returns to the daily profile. Even if children show increased cortisol concentration during the day, its liberation is accelerated at night [48,49]. The abovementioned mechanisms of cortisol variability and further analyses of its level in children with ADHD would require assessments over a 24-h period.
Nevertheless, determinations in saliva open many paths due to the non-invasiveness and repeatability of the measurements [50,51].
The results in this study of an increased sAA concentration compared to cortisol found raised some questions and notes. In the analyses of other authors, the sAA was collected during acute and chronic stress stimuli, and higher concentrations were obtained [52,53].
In one ADHD study, a lower evening and diurnal sAA secretions in response to stress were noted [54]. It should be remembered that the salivary glands are innervated by both the sympathetic and parasympathetic parts of the autonomic nervous system (ANS). Therefore, its innervation suggests changes in response to stress [7]. It should also be underlined that children with neurodevelopmental disorders may show atypical stress responses and have alterations in their diurnal profile of stress biomarkers. In our study, the significantly demonstrated level of sAA suggests that hyperactivity of the HPA axis in subjects with ADHD may be accompanied by disturbances in the regulation of the ANS [44].
In the present study, all immune biomarkers, IgA, IgG, and IgM, were significantly increased in the salivary samples of ADHD children. The level of secretory IgA was markedly higher among ADHD children than in those in the control group and correlated to free cortisol and IgG levels in the saliva. This finding shows that sIgA levels may be linked to the stress response and may also confirm a link to HPA axis hyperactivity. According to the current literature, an increase in IgA in oral saliva may also indicate a greater hypersensitivity of the oral mucosa and susceptibility to infectious diseases [55]. This increase in IgA levels can be attributed to high oral responsiveness to injuries, gingivitis, and an increased concentration of yeast and Lactobacillus [56]. Additionally, overall our results may help to interpret other research findings relating to ADHD and immune responses between children and adolescents [20]. It could also indicate pathological oral conditions, especially those that are infection related [57].
Our research suggests many directions in which salivary enzymes and immunoglobulins may be analyzed. Their concentrations in saliva are essential also for stimulating dental caries development in the oral cavity. Our previous study showed high dental caries in primary and permanent dentition among ADHD children [58]. Additionally, tooth decay was related to sweetened drink intake [58]. Therefore, alpha-amylase and immune biomarkers may both be at an enhanced level, and further research is necessary to link oral homeostasis and ADHD.
Our results showed a significant increase in body mass, hip circumference, and BMI with correlation to cortisol. Comparative research also indicates excessive body mass or BMI compared to pediatric norms or control groups [59,60,61,62]. The ADHD–obesity relationship can be explained by genes [63], neurobiological features [64], deficits in executive functions [65], fetal programming [62], sleep disorders (circadian rhythm), incorrect eating habits [66], high-fat diet [67], reduced physical activity [61,68], and stress [69], as well as a sedentary lifestyle. It can be assumed that poor stress resistance affects physical health in overweight children with ADHD [70].

5. Conclusions

This cross-sectional study aimed to evaluate stress and immune biomarkers in salivary samples of children with attention-deficit/hyperactivity disorder (ADHD) compared with those of healthy, non-ADHD children.
In our opinion, saliva may be a promising material for non-invasive ADHD monitoring. The results indicate that morning collection of saliva in contrast to unchanged salivary cortisol, may evaluate HPA and SNS dysregulations by measurements of sAA and immunoglobulins among ADHD children.

Supplementary Materials

The following are available online at https://www.mdpi.com/1660-4601/18/2/769/s1, Figure S1. Flow chart of the study.

Author Contributions

Conceptualization, E.P. and M.D.-W.; data curation, A.K. and M.G.; formal analysis, M.D.-W.; funding acquisition, M.D.-W.; investigation, A.K., A.H., K.B., J.D., J.O.-S., S.R., E.B., P.S., and M.T.-N.; methodology, A.S.; project administration, T.H.; resources, A.S.; software, M.D.-W.; supervision, E.P.; writing original draft, A.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Science Center (grant no. 2016/21/B/NZ5/00492).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of Poznan University of Medical Sciences (Resolution No. 26/18).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Acknowledgments

The authors acknowledge the parents, patients, school managers, teachers, and schoolchildren that agreed to participate in this study. A particular thanks go to all examiners who performed clinical and biochemical examinations. We would like to thank Tomasz Maksymiuk for his assistance with the English language editing.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the study’s design; in the collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Abbreviations

ADHD—Attention Deficit Hyperactivity Disorder, BMI—Body Mass Index, DSM-V—Diagnostic and Statistical Manual of Mental Disorders (5th ed.), ECC—Early Childhood Caries, GCP—Good Clinical Practice, HPA—hypothalamus–pituitary–adrenal axis, ICD-10—International Statistical Classification of Diseases and Related Health Problems (10th edition), sAA—salivary alpha-amylase, sIgA—secreted immunoglobulin A, SNS—sympathetic nervous system, WHO—World Health Organization.

References

  1. Hanć, T.; Cortese, S. Attention deficit/hyperactivity-disorder and obesity: A review and model of current hypotheses explaining their comorbidity. Neurosci. Biobehav. Rev. 2018, 92, 16–28. [Google Scholar] [CrossRef] [PubMed]
  2. Polanczyk, G.; de Lima, M.S.; Horta, B.L.; Biederman, J.; Rohde, L.A. The worldwide prevalence of ADHD: A systematic review and metaregression analysis. Am. J. Psychiatry 2007, 164, 942–948. [Google Scholar] [CrossRef] [PubMed]
  3. Polanczyk, G.; Willcutt, E.G.; Salum, G.A.; Kieling, C.; Rohde, L.A. ADHD prevalence estimates across three decades: An updated systematic review and meta-regression analysis. Int. J. Epidemiol. 2014, 43, 434–442. [Google Scholar] [CrossRef] [PubMed]
  4. Corominas, M.; Ramos-Quiroga, J.M.; Ferrer, M.; Sáez-Francàs, N.; Palomar, G.; Bosch, R.; Casas, M. Cortisol responses in children and adults with attention deficit hyperactivity disorder (ADHD): A possible marker of inhibition deficits. ADHD Atten. Deficit Hyperact. Disord. 2012, 4, 63–75. [Google Scholar] [CrossRef]
  5. Jasim, H.; Carlsson, A.; Hedenberg-Magnusson, B.; Ghafouri, B.; Ernberg, M. Saliva as a medium to detect and measure biomarkers related to pain. Sci. Rep. 2018, 19, 3220. [Google Scholar] [CrossRef] [Green Version]
  6. Lawrence, H.P. Salivary markers of systemic disease: Non-invasive diagnosis of disease and monitoring of general health. J. Can. Dent. Assoc. 2002, 68, 170–174. [Google Scholar]
  7. Obayashi, K. Salivary mental stress proteins. Clin. Chim. Acta 2013, 21, 196–201. [Google Scholar] [CrossRef]
  8. Sapolsky, R.M.; Alberts, S.C.; Altmann, J. Hypercortisolism associated with social subordinance or social isolation among wild baboons. Arch. Gen. Psychiatry 1997, 54, 1137–1143. [Google Scholar] [CrossRef]
  9. Parker, K.J.; Schatzberg, A.F.; Lyons, D.M. Neuroendocrine aspects of hypercortisolism in major depression. Horm. Behav. 2003, 43, 60–66. [Google Scholar] [CrossRef]
  10. Gold, P.W.; Loriaux, D.L.; Roy, A.; Kling, M.A.; Calabrese, J.R.; Kellner, C.H.; Nieman, L.K.; Post, R.M.; Pickar, D.; Gallucci, W. Responses to corticotrophin-releasing hormone in the hypercortisolism of depression and Cushing’s disease. Pathophysiologic and diagnostic implications. N. Engl. J. Med. 1986, 314, 1329–1335. [Google Scholar] [CrossRef]
  11. Paszynska, E.; Dmitrzak-Weglarz, M.; Tyszkiewicz-Nwafor, M.; Slopien, A. Salivary alpha-amylase, secretory IgA and free cortisol as neurobiological components of the stress response in the acute phase of anorexia nervosa. World J. Biol. Psychiatry 2016, 17, 266–273. [Google Scholar] [CrossRef] [PubMed]
  12. King, J.A.; Barkley, R.A.; Barrett, S. Attention-deficit hyperactivity disorder and the stress response. Biol. Psychiatry 1998, 44, 72–74. [Google Scholar] [CrossRef]
  13. Nater, U.M.; Rohleder, N. Salivary alpha-amylase as a non-invasive biomarker for the sympathetic nervous system: Current state of research. Psychoneuroendocrinology 2009, 34, 486–496. [Google Scholar] [CrossRef] [PubMed]
  14. Almela, M.; Hidalgo, V.; Villada, C.; van der Meij, L.; Espín, L.; Gómez-Amor, J.; Salvador, A. Salivary alpha-amylase response to acute psychosocial stress: The impact of age. Biol. Psychol. 2011, 87, 421–429. [Google Scholar] [CrossRef] [PubMed]
  15. Schumacher, S.; Kirschbaum, K.; Fydrich, T.; Strohle, A. Is salivary alpha-amylase an indicator of autonomic nervous system dysregulations in mental disorders?—A review of preliminary findings and the interactions with cortisol. Psychoneuroendocrinology 2013, 38, 729–743. [Google Scholar] [CrossRef] [PubMed]
  16. Nater, U.M.; La Marca, R.; Florin, L.; Moses, A.; Langhans, W.; Koller, M.M.; Ehlert, M.M. Stress–induced changes in human salivary alpha-amylase activity–associations with adrenergic activity. Psychoneuroendocrinology 2006, 31, 49–58. [Google Scholar] [CrossRef] [PubMed]
  17. Scheutzel, P.; Gerlach, U. Alpha-amylase isoenzymes in serum and saliva of patients with anorexia and bulimia nervosa. Gastroenterology 1991, 29, 339–345. [Google Scholar]
  18. Schlueter, N.; Ganss, C.; Pötschke, S.; Klimek, J.; Hannig, C. Enzyme activities in the oral fluids of patients suffering from bulimia: A control clinical trial. Caries Res. 2012, 46, 130–139. [Google Scholar] [CrossRef] [Green Version]
  19. Ertugrul, C.C.; Kirzioglu, Z.; Aktepe, E.; Savas, H.B. The effects of psychostimulants on oral health and saliva in children with attention deficit hyperactivity disorder: A case-control study. Niger. J. Clin. Pract. 2018, 21, 1213–1220. [Google Scholar] [CrossRef]
  20. Bekdas, M.; Tufan, A.E.; Hakyemez, I.N.; Tas, T.; Altunhan, H.; Demircioglu, F.; Kısmet, E. Abant Izzet Baysal University Faculty of Medicine. Subclinical immune reactions to viral infections may correlate with child and adolescent diagnosis of attention-deficit/hyperactivity disorder: A preliminary study from Turkey. Afr. Health Sci. 2014, 14, 439–445. [Google Scholar] [CrossRef]
  21. Bosch, J.A.; Ring, C.; de Geus, E.J.C.; Veerman, E.C.I.; Amerongen, A.V.N. Stress and secretory immunity. Int. Rev. Neurobiol. 2002, 52, 213–253. [Google Scholar]
  22. Nater, U.M.; Rohleder, N.; Schlotz, W.; Ehlert, U.; Kirschbaum, C. Determinants of the diurnal course of salivary alpha-amylase. Psychoneuroendocrinology 2007, 32, 392–401. [Google Scholar] [CrossRef]
  23. Wolf, J.M.; Nicholls, E.; Chen, E. Chronic stress, salivary cortisol, and alpha-amylase in children with asthma and healthy children. Biol. Psychol. 2008, 78, 20–28. [Google Scholar] [CrossRef]
  24. Ali, N.; Pruessner, J.C. The salivary alpha amylase over cortisol ratio as a marker to assess dysregulations of the stress systems. Physiol. Behav. 2012, 106, 65–72. [Google Scholar] [CrossRef]
  25. Pinninti, N.R.; Madison, H.; Musser, E.; Rissmiller, D. MINI International Neuropsychiatric Schedule: Clinical utility and patient acceptance. Eur. Psychiatry 2003, 18, 361–364. [Google Scholar] [CrossRef]
  26. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 5th ed.; American Psychiatric Publishing: Arlington, VA, USA, 2013. [Google Scholar]
  27. Guideline for Good Clinical Practice (e6). International Conference on Harmonization ICH. 1996. Available online: http://www.ich.org/LOB/media/MEDIA482.pdf (accessed on 26 May 2018).
  28. World Health Organization. Waist Circumference and Waist–Hip Ratio: Report of a WHO Expert Consultation; WHO: Geneva, Switzerland, 2008; pp. 1–47. [Google Scholar]
  29. Sedgwick, P. Standardising outcome measures using z scores. BMJ 2014, 1, 349. [Google Scholar] [CrossRef]
  30. Edgar, W.M.; Dawes, C.; O’Mullane, D. Saliva and Oral Health; London British Dental Association: London, UK, 2004; pp. 14–50. [Google Scholar]
  31. Jessop, D.S.; Turner-Cobb, J.M. Measurement and meaning of salivary cortisol: A focus on health and disease in children. Stress 2007, 11, 1–14. [Google Scholar] [CrossRef]
  32. Dorn, L.D.; Lucke, J.F.; Loucks, T.L.; Berga, S.L. Salivary cortisol reflects serum cortisol: Analysis of circadian profiles. Ann. Clin. Biochem. 2007, 44, 281–284. [Google Scholar] [CrossRef]
  33. Wang, D.; Zhuo, J.Q.; Zhao, M.P. A simple and rapid competitive enzyme-linked immunosorbent assay (cELISA) for high through-put measurement of secretory immunoglobulin A (sIgA) in saliva. Talanta 2010, 82, 432–436. [Google Scholar] [CrossRef]
  34. Valdimarsdottir, H.B.; Stone, A.A. Psychosocial factors and secretory immunoglobulin A. Crit. Rev. Oral Biol. Med. 1997, 8, 461–474. [Google Scholar] [CrossRef]
  35. Latiff, A.H.A.; Kerr, M.A. The clinical significance of immunoglobulin A deficiency. Ann. Clin. Biochem. 2007, 44, 131–139. [Google Scholar] [CrossRef] [Green Version]
  36. Hettegger, P.; Huber, J.; Paßecker, K.; Soldo, R.; Kegler, U.; Nöhammer, C.; Weinhäusel, A. High similarity of IgG antibody profiles in blood and saliva opens opportunities for saliva based serology. PLoS ONE 2019, 20, e0218456. [Google Scholar] [CrossRef]
  37. Błochowiak, K.; Sokalski, J.; Golusińska, E.; Trzybulska, D.; Witmanowski, H.; Bodnar, M.; Marszałek, A. Salivary levels and immunohistochemical expression of selected angiogenic factors in benign and malignant parotid gland tumours. Clin. Oral Investig. 2019, 23, 995–1006. [Google Scholar] [CrossRef] [Green Version]
  38. Norhagen, G.; Engström, P.E.; Hammarström, L.; Söder, P.O.; Smith, C.I. Immunoglobulin levels in saliva in individuals with selective IgA deficiency: Compensatory IgM secretion and its correlation with HLA and susceptibility to infections. J. Clin. Immunol. 1989, 9, 279–286. [Google Scholar] [CrossRef]
  39. Castagnola, M.; Scarano, E.; Passali, G.C.; Messana, I.; Cabras, T.; Iavarone, F.; Di Cintio, G.; Fiorita, A.; De Corso, E.; Paludetti, G. Salivary biomarkers and proteomics: Future diagnostic and clinical utilities. Acta Otorhinolaryngol. Ital. 2017, 37, 94–101. [Google Scholar] [CrossRef]
  40. Blomqvist, M.; Holmberg, K.; Lindblad, F.; Fernell, E.; Ek, U.; Dahllof, G. Salivary cortisol levels and dental anxiety in children with attention deficit hyperactivity disorder. Eur. J. Oral Sci. 2007, 115, 1–6. [Google Scholar] [CrossRef]
  41. Isaksson, J.; Nilsson, K.W.; Nyberg, F.; Hogmark, A.; Lindblad, F. Cortisol levels in children with attention-deficit/hyperactivity disorder. J. Psychiatr. Res. 2012, 46, 1398–1405. [Google Scholar] [CrossRef]
  42. Isaksson, J.; Nilsson, K.W.; Lindblad, F. Early psychosocial adversity and cortisol levels in children with attention-deficit/hyperactivity disorder. Eur. Child Adolesc. Psychiatry 2013, 22, 425–432. [Google Scholar] [CrossRef]
  43. Angeli, A.; Frajria, R.; Dogliotti, L.; Crosazzo, C.; Rigoli, F.; Ceresa, F. Differences between temporal patterns of plasma cortisol and corticosteroid-binding globulin binding capacity throughtout the twenty-four hour day and the menstrual cycle. J. Endocrinol. Investig. 1978, 1, 31–38. [Google Scholar] [CrossRef]
  44. Angeli, E.; Korpa, T.; Johnson, E.O.; Apostolakou, F.; Papassotiriou, I.; Chrousos, G.P.; Pervanidou, P. Salivary cortisol and alpha-amylase diurnal profiles and stress reactivity in children with Attention Deficit Hyperactivity Disorder. Psychoneuroendocrinology 2018, 90, 174–181. [Google Scholar] [CrossRef]
  45. Hucklebridge, F.; Hussain, T.; Evans, P.; Clow, A. The diurnal patterns of the adrenal steroids cortisol and dehydroepiandrosterone (DHEA) in relation to awakening. Psychoneuroendocrinology 2005, 30, 51–57. [Google Scholar] [CrossRef]
  46. Northover, C.; Thapar, A.; Langley, K.; Fairchild, G.; van Goozen, S.H.M. Cortisol levels at baseline and under stress in adolescent males with attention-deficit hyperactivity disorder, with or without comorbid conduct disorder. Psychiatry Res. 2016, 30, 130–136. [Google Scholar] [CrossRef] [Green Version]
  47. Corominas-Roso, M.; Palomar, G.; Ferrer, R.; Real, A.; Nogueira, M.; Corrales, M.; Casas, M.; Ramos-Quiroga, J.A. Cortisol response to stress in adults with attention deficit hyperactivity disorder. Int. J. Neuropsychopharmacol. 2015, 18, pyv027. [Google Scholar] [CrossRef] [Green Version]
  48. Peeters, F.; Berkhof, J.; Delespaul, P.; Rottenberg, J.; Nicolson, N.A. Diurnal mood variation in major depressive disorder. Emotion 2006, 6, 383–391. [Google Scholar] [CrossRef] [Green Version]
  49. Stetler, C.; Dickerson, S.S.; Miller, G.E. Uncoupling of social zeitgebers and diurnal cortisol secretion in clinical depression. Psychoneuroendocrinology 2004, 29, 1250–1259. [Google Scholar] [CrossRef]
  50. Chen, E.; Cohen, S.; Miller, G.E. How low socioeconomic status affects 2-year hormonal trajectories in children. Psychol. Sci. 2010, 21, 31–37. [Google Scholar] [CrossRef]
  51. Fernald, L.C.; Gunnar, M.R. Poverty-alleviation program participation and salivary cortisol in very low-income children. Soc. Sci. Med. 2009, 68, 2180–2189. [Google Scholar] [CrossRef] [Green Version]
  52. Chatterton, R.T., Jr.; Vogelsong, K.M.; Lu, Y.C.; Ellman, A.B.; Hudgens, G.A. Salivary alpha-amylase as a measure of endogenous adrenergic activity. Clin. Physiol. 1996, 16, 433–448. [Google Scholar] [CrossRef]
  53. Behringer, V.; Deschner, T.; Möstl, E.; Selzer, D.; Hohmann, G. Stress affects salivary alpha-Amylase activity in bonobos. Physiol. Behav. 2012, 105, 476–482. [Google Scholar] [CrossRef]
  54. Anesiadou, S.; Makris, G.; Michou, M.; Bali, P.; Papassotiriou, I.; Apostolakou, F.; Korkoliakou, P.; Papageorgiou, C.; Chrousos, G.; Pervanidou, P. Salivary cortisol and alpha-amylase daily profiles and stress responses to an academic performance test and a moral cognition task in children with neurodevelopmental disorders. Stress Health 2020. [Google Scholar] [CrossRef]
  55. Brandtzaeg, P. Do salivary antibodies reliably reflect both mucosal and systemic immunity? Ann. N. Y. Acad. Sci. 2007, 1098, 288–311. [Google Scholar] [CrossRef]
  56. Sikorska, M.H.J.; Mielnik-Blaszczak, M.; Kapec, E. The relationship between the levels of sIgA, lactoferrin and α1proteinaze inhibitor in saliva and permanent dentition caries in 15-year-old. Oral Microbiol. Immunol. 2002, 17, 272–276. [Google Scholar] [CrossRef]
  57. Challacombe, S.J.; Percival, R.S.; Marsh, P.D. Age-related changes in immunoglobulin isotypes in whole and parotid saliva and serum in healthy individuals. Oral Microbiol. Immunol. 1995, 10, 202–207. [Google Scholar] [CrossRef]
  58. Paszynska, E.; Dmitrzak-Węglarz, M.; Perczak, A.; Gawriolek, M.; Hanć, T.; Bryl, E.; Mamrot, P.; Dutkiewicz, A.; Roszak, M.; Tyszkiewicz-Nwafor, M.; et al. Excessive Weight Gain and Dental Caries Experience among Children Affected by ADHD. Int. J. Environ. Res. Public Health 2020, 17, 5870. [Google Scholar] [CrossRef]
  59. Anderson, S.E.; Cohen, P.; Naumova, E.N.; Must, A. Relationship of childhood behaviour disorders to weight gain from childhood into adulthood. Ambul. Pediatr. 2006, 6, 297–301. [Google Scholar]
  60. Hanć, T.; Dmitrzak-Węglarz, M.; Borkowska, A.; Wolańczyk, T.; Pytlińska, N.; Rybakowski, F.; Słopień, R.; Słopień, A. Overweight in boys with is related to candidate genes and not to deficits in cognitive functions. J. Atten. Disord. 2018, 22, 1158–1172. [Google Scholar] [CrossRef]
  61. Cook, B.G.; Li, D.; Heinrich, K.M. Obesity, physical activity, and sedentary behaviour of youth with learning disabilities and ADHD. J. Learn. Disabil. 2015, 48, 563–576. [Google Scholar] [CrossRef]
  62. Hanć, T.; Słopień, A.; Wolańczyk, T.; Szwed, A.; Czapla, Z.; Durda, M.; Dmitrzak-Węglarz, M.; Ratajczak, J. Attention-Deficit/Hyperactivity Disorder is Related to Decreased Weight in the Preschool Period and to Increased Rate of Overweight in School-Age Boys. J. Child Adolesc. Psychopharmacol. 2015, 25, 691–700. [Google Scholar] [CrossRef]
  63. Albayrak, Ö.; Pütter, C.; Volckmar, A.-L.; Cichon, S.; Hoffmann, P.; Nöthen, M.M.; Jöckel, K.-H.; Schreiber, S.; Wichmann, H.-E.; Faraone, S.V.; et al. Psychiatric GWAS Consortium: ADHD Subgroup: Common Obesity Risk Alleles in Childhood Attention-Deficit/Hyperactivity Disorder. Am. J. Med. Genet. B Neuropsychiatr. Genet. 2013, 162, 295–305. [Google Scholar]
  64. Liu, L.L.; Lin, B.M.; Wang, Y.W. Does dopaminergic reward system contribute to explaining comorbidity obesity and ADHD? Med. Hypotheses 2008, 70, 1118–1120. [Google Scholar] [CrossRef]
  65. Choundhry, Z.; Sengupta, S.M.; Grizenko, N.; Harvey, W.J.; Fortier, M.E.; Schmitz, N.; Joober, R. Body weight and ADHD: Examining the role of self-regulation. PLoS ONE 2013, 8, e55351. [Google Scholar]
  66. Voegel, S.W.; Bijlenga, D.; Tanke, M.; Bron, T.I.; Heijden, K.B.; Swaab, H.; Beekman, A.T.F.; Kooij, S.J.J. Circadian rhythm disruption as a link between Attention-Deficit/Hyperactivity Disorder and obesity? J. Psychosom. Res. 2015, 79, 443–450. [Google Scholar] [CrossRef]
  67. Ptacek, R.; Kuzelova, H.; Stefano, G.B.; Raboch, J.; Sadkova, T.; Goetz, M.; Kream, R.M. Disruptive patterns of eating behaviors and association lifestyles in males with ADHD. Med. Sci. Monit. 2014, 20, 608–613. [Google Scholar]
  68. do Nascimento, E.M.F.; Contreira, A.R.; da Silva, E.V.A.; de Souza, L.P.; Beltrame, T.S. Motor performance and nutritional status with attention deficit hyperactivity disorder. J. Hum. Growth Dev. 2013, 23, 357–363. [Google Scholar] [CrossRef] [Green Version]
  69. Pauli-Pott, U.; Reinhardt, A.; Bagus, E. Psychosocial risk factors underlie the link between attention deficit hyperactivity symptoms and overweight at school entry. Eur. Child Adolesc. Psychiatry 2017, 26, 67–73. [Google Scholar] [CrossRef]
  70. Lissak, G. Adverse physiological and psychological effects of screen time on children and adolescents: Literature review and case study. Environ. Res. 2018, 164, 149–157. [Google Scholar] [CrossRef]
Table 1. Inclusion and exclusion criteria for both groups.
Table 1. Inclusion and exclusion criteria for both groups.
Inclusion Criteria for the Study GroupInclusion Criteria for the Control GroupExclusion Criteria from the
Study and Control Groups
Children of both sexes aged 6–11Children of both sexes aged 6–11Children with disorders of the central nervous system (e.g., epilepsy, serious injuries, and CNS infections)
Children with diagnosed ADHD in accordance with DSM-V diagnostic criteria
(diagnosis confirmed separately by psychiatrist and clinical psychologist after a standardized and structured interview)
Lack of mental
disorders—assessment with
the use of a MINI-Kid
questionnaire [25]
Co-existing:
schizophrenia,
bipolar affective disorder, or
any serious somatic disorders
Clinically significant ADHD symptoms lasting over six monthsA parent’s or legal guardian’s approvalChronic somatic diseases
Children without hereditary mental disorders (first-degree relatives) Persistent pharmacotherapy, hormonotherapy
A parent’s or legal guardian’s consent Lack of acceptance from parents or legal guardians
ADHD—Attention Deficit Hyperactivity Disorder; DSM-V—Diagnostic and Statistical Manual of Mental Disorders (5th ed.), MINI-Kid–MINI International Neuropsychiatric Interview for Kids; CNS—Central Nervous System.
Table 2. Anthropometric data and salivary concentrations of cortisol, alpha-amylase (sAA), and IgA, IgG, and IgM in the ADHD and control groups.
Table 2. Anthropometric data and salivary concentrations of cortisol, alpha-amylase (sAA), and IgA, IgG, and IgM in the ADHD and control groups.
GroupADHD (n = 60)
Mean ± SD
Median (min–max)
Control (n = 72)
Mean ± SD
Median (min–max)
p-Value
Parameter
Age (years)8.6 ± 1.1
8 (6–11)
8.3 ± 1.2
8 (6–11)
ns
BMI z-score0.6 ± 1.50.5 ± 1.2ns
0.4 (4.1 − (−2.6))0.4 (3.5 − (−1.9))
Waist size (cm)63.9 ± 11
62 (49–101)
60.2 ± 6.6
60 (38–80)
ns
Hip size (cm)76.1 ± 8.5
75 (57–104)
70.7 ± 6.3
70 (50–88)
0.001
Cortisol (ng/mL)5.5 ± 4
4.3 (1.5–17.8)
5.5 ± 6.6
4.1 (0.2–54.2)
ns
sAA (U/mL)112.6 ± 71.2
104.6 (11.7–296)
82.9 ± 49.8
76.1 (1.3–217.2)
0.030
IgA (ug/mL)143.3 ± 72.7
160.8 (9.8–298.2)
115.5 ± 57.1
98.8 (22.3–235.4)
0.022
IgG (pg/mL)4866.1 ± 1194.9
5352.5 (2032.3–6110)
5804 ± 2794.5
6013.1 (38.1–11,830.4)
0.017
IgM (ng/mL)468.7 ± 115.3
469.6 (0.04–738.4)
414.4 ± 40.4
410.7 (318.6–556.8)
0.001
Statistical significance is given according to p-value with the statistical difference (p ≤ 0.05, p ≤ 0.01, p ≤ 0.001) vs. non-significant values in statistical analysis (ns). Abbreviations: n, number of patients; ADHD, patients diagnosed with attention-deficit/hyperactivity disorder; control group, healthy children. Statistical tests used were the Mann–Whitney U test. Abbreviations: Results are expressed as mean ± standard deviation, median (min–max ranges); n, number of examined children.
Table 3. Spearman’s analysis of correlations (cortisol, alpha-amylase, IgA, IgG, IgM, BMI, and waist and hip sizes) between the ADHD and control groups.
Table 3. Spearman’s analysis of correlations (cortisol, alpha-amylase, IgA, IgG, IgM, BMI, and waist and hip sizes) between the ADHD and control groups.
Control Group
IgA (ug/mL)sAA
(U/mL)
Cortisol
(ng/mL)
IgG
(pg/mL)
IgM
(ng/mL)
BMI
(kg/m2)
Waist Size (cm)Hip Size (cm)
IgA (ug/mL) 0.1780.095−0.117−0.3080.043−0.0470.065
sAA (U/mL)0.178 0.161−0.1770.001−0.017−0.0120.025
Cortisol (ng/mL)0.0950.161 −0.203−0.047−0.1050.004−0.046
IgG (pg/mL)−0.117−0.177−0.203 * 0.0900.077−0.012−0.060
IgM (ng/mL)−0.308 *0.001−0.0470.090 0.0550.008−0.019
BMI (kg/m2)0.043−0.017−0.1050.0770.055 0.7520.611
Waist size (cm)−0.047−0.0120.004−0.0120.0080.752 * 0.717
Hip size (cm)0.0650.025−0.046−0.060−0.0190.611 *0.717 *
ADHD Group
IgA (ug/mL)sAA
(U/mL)
Cortisol
(ng/mL)
IgG
(pg/mL)
IgM
(ng/mL)
BMI (kg/m2)Waist Size (cm)Hip Size (cm)
IgA (ug/mL) 0.2290.348−0.4670.175−0.142−0.049−0.169
sAA (U/mL)0.229 * −0.119−0.384−0.118−0.130−0.189−0.104
Cortisol (ng/mL)0.348 *−0.119 −0.031−0.055−0.350−0.182−0.282
IgG (pg/mL)−0.467 *−0.384 *−0.031 −0.0940.1310.1130.225
IgM (ng/mL)0.175−0.118−0.055−0.094 0.1340.1590.112
BMI (kg/m2)−0.142−0.130−0.350 *0.1310.134 0.7850.834
Waist size (cm)−0.049−0.189−0.1820.1130.1590.785 * 0.807
Hip size (cm)−0.169−0.104−0.282 *0.225 *0.1120.834 *0.807 *
A force of correlation:0.00–0.200.21–0.390.40–0.690.70–1.00
p-value in bold *, >0.05.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Krahel, A.; Paszynska, E.; Slopien, A.; Gawriolek, M.; Otulakowska-Skrzynska, J.; Rzatowski, S.; Hernik, A.; Hanć, T.; Bryl, E.; Szczesniewska, P.; et al. Stress/Immune Biomarkers in Saliva among Children with ADHD Status. Int. J. Environ. Res. Public Health 2021, 18, 769. https://doi.org/10.3390/ijerph18020769

AMA Style

Krahel A, Paszynska E, Slopien A, Gawriolek M, Otulakowska-Skrzynska J, Rzatowski S, Hernik A, Hanć T, Bryl E, Szczesniewska P, et al. Stress/Immune Biomarkers in Saliva among Children with ADHD Status. International Journal of Environmental Research and Public Health. 2021; 18(2):769. https://doi.org/10.3390/ijerph18020769

Chicago/Turabian Style

Krahel, Anna, Elzbieta Paszynska, Agnieszka Slopien, Maria Gawriolek, Justyna Otulakowska-Skrzynska, Szymon Rzatowski, Amadeusz Hernik, Tomasz Hanć, Ewa Bryl, Paula Szczesniewska, and et al. 2021. "Stress/Immune Biomarkers in Saliva among Children with ADHD Status" International Journal of Environmental Research and Public Health 18, no. 2: 769. https://doi.org/10.3390/ijerph18020769

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop