Next Article in Journal
Sensitivity Analysis for Transient Thermal Problems Using the Complex-Variable Finite Element Method
Next Article in Special Issue
TTCNN: A Breast Cancer Detection and Classification towards Computer-Aided Diagnosis Using Digital Mammography in Early Stages
Previous Article in Journal
Discriminative Frequencies and Temporal EEG Segmentation in the Motor Imagery Classification Approach
Previous Article in Special Issue
MFDGCN: Multi-Stage Spatio-Temporal Fusion Diffusion Graph Convolutional Network for Traffic Prediction
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predicting Children with ADHD Using Behavioral Activity: A Machine Learning Analysis

School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Fukushima 965-8580, Japan
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(5), 2737; https://doi.org/10.3390/app12052737
Submission received: 14 January 2022 / Revised: 4 February 2022 / Accepted: 3 March 2022 / Published: 7 March 2022
(This article belongs to the Special Issue Biomedical Signal Processing, Data Mining and Artificial Intelligence)

Abstract

:
Attention deficit hyperactivity disorder (ADHD) is one of childhood’s most frequent neurobehavioral disorders. The purpose of this study is to: (i) extract the most prominent risk factors for children with ADHD; and (ii) propose a machine learning (ML)-based approach to classify children as either having ADHD or healthy. We extracted the data of 45,779 children aged 3–17 years from the 2018–2019 National Survey of Children’s Health (NSCH, 2018–2019). About 5218 (11.4%) of children were ADHD, and the rest of the children were healthy. Since the class label is highly imbalanced, we adopted a combination of oversampling and undersampling approaches to make a balanced class label. We adopted logistic regression (LR) to extract the significant factors for children with ADHD based on p-values (<0.05). Eight ML-based classifiers such as random forest (RF), Naïve Bayes (NB), decision tree (DT), XGBoost, k-nearest neighborhood (KNN), multilayer perceptron (MLP), support vector machine (SVM), and 1-dimensional convolution neural network (1D CNN) were adopted for the prediction of children with ADHD. The average age of the children with ADHD was 12.4 ± 3.4 years. Our findings showed that RF-based classifier provided the highest classification accuracy of 85.5%, sensitivity of 84.4%, specificity of 86.4%, and an AUC of 0.94. This study illustrated that LR with RF-based system could provide excellent accuracy for classifying and predicting children with ADHD. This system will be helpful for early detection and diagnosis of ADHD.

1. Introduction

Attention deficit hyperactivity disorder (ADHD) is one of the most frequent neurodevelopmental behavioral disorders in childhood [1]. Children with ADHD have the following symptoms: hyperactivity, inattention, and impulsivity [1]. According to the Centers for Disease Control (CDC) and prevention, the number of children in the USA who have been diagnosed with ADHD has fluctuated over time as follows: about 4.4 million children between the ages of 2 and 17 years were diagnosed with ADHD in 2003, 5.4 million children in 2007, 6.4 million children in 2011, and 6.1 million children in 2016 [2]. About 12.9% of male children and 5.6% of females were diagnosed with ADHD [2,3]. Globally, the prevalence of adults with ADHD was 2.8% in 2016 [4] and 0.96% in 2019; and 7.8% of children were diagnosed with ADHD in 2003, 9.5% in 2007, and 11% in 2007 [5]. There were 62% of children who had taken medication for ADHD, and 46.7% of those children had also received behavioral treatment [2]. It is noted that the number of children with ADHD has been increasing day by day. Therefore, it is necessary to propose a model for the identification of the risk factors for ADHD.
Researchers are trying to determine the risk factors to reduce the number of children with ADHD. A study showed that genetic factors played a significant role and were linked with ADHD [6]. Genetic factors are responsible for almost 75% of the risk of ADHD in younger children [7]. Besides the genetic factors, there were several risk factors for ADHD such as brain injury, alcohol/tobacco use during pregnancy, and premature delivery [6]. Previous studies also showed that age, sex, asthma, race, anxiety, depression, obesity, cigarette smoking, and socio-economic status were also associated with children with ADHD [5,8,9,10,11,12,13,14,15]. These studies were conducted only to identify the risk factors for children with ADHD. It is necessary to propose a prediction model. In this regard, in comparison with classical approaches, machine learning (ML)-based models may be used for prediction. ML-based models have been also used for the identification and prediction in the field of medical imaging [16,17,18], healthcare [19,20,21], and mental health [22,23].
Several ML-based classifiers were applied to predict children with ADHD [24,25,26,27,28,29]. Uluyagmur-Ozturk et al. [30] conducted a study on the emotional status of children and classified them as ASD, ADHD, and control based on their diagnosis in Turkey. They extracted the data of 61 children from Maramara University Medical Hospital. There were 18 children with ASD, 30 children with ADHD, and 13 healthy children. The average ages of the children with respective groups were 10.50, 9.46, and 9.22 years. They utilized ReliefF to determine the most significant features of ASD and ADHD. They also utilized five ML-based algorithms like decision tree (DT), random forest (RF), support vector machine (SVM), k-nearest neighbor (KNN), and AdaBoost (AB) to classify children as ASD, ADHD, and healthy. They showed that AB provided an 80% accuracy rate in differentiating children as having ASD, ADHD, and healthy.
Slobodin et al. [31] also diagnosed children with ADHD based on a continuous performance test (CPT). They selected 458 children aged 6–12 years. The selected children had an average age of 8.7 ± 1.8 years and 59.0% of the children were boys, with 46.51% of the children having ADHD problems. They found that there was no significant age difference between ADHD and non-ADHD (p-value = 0.94). They partitioned the dataset into the training set and holdout. They applied several ML-based classifiers like RF, MOXO, and neural network (NN) for the prediction of ADHD. ML-based classifiers were trained on 60% of the dataset, and 40% of the dataset was used as test set for the evaluation of ML-based classifiers. They showed that their proposed ML-based classifiers (MOXO) provided the highest accuracy of 87.0%, the sensitivity of 89.0%, and the specificity of 84.0%.
Morrow et al. [32] also conducted a study on children who received treatment for ADHD. They extracted the data of 6630 children with ADHD (age: 3–17 years) from the National Survey of Children’s Health (NSCH), 2016–2017. The average age of the children with ADHD was 12.4 years. Four ML-based classifiers like classification and regression tree (CART), logistic regression (LR), ensemble decision forest (EDR), and deep multi-layer neural network (DeepNet) were employed to determine the associated factors with children who received treatment for ADHD. They showed that the DeepNet-based classifier gave the highest AUC of 0.72 compared to CART, EDR, and LR.
Despite the rapid development of ML-based classifiers, their application to ADHD diagnosis remains a difficult task. Yet, various ML-based classifiers have been utilized to predict children with ADHD in different countries using different ADHD datasets. However, the models’ performance has to be improved. The current study had the following objectives: (i) to extract the risk factors of children with ADHD; and (ii) to propose an ML-based classifier to classify and predict children as either having ADHD or healthy.
The overall layout of this study is as follows: Section 2 presents the materials and methods; we present descriptions of dataset, predictor and outcome variables, statistical analysis, imbalance management methods, feature section method, machine learning techniques, and performance evaluation criteria. Results are presented in Section 3. Section 4 presents a detailed discussion, and finally, the conclusion is presented in Section 5.

2. Materials and Methods

2.1. Dataset

The data utilized for this study was extracted from the 2018–2019 NSCH [33], which is a nationally representative survey based on child health and well-being. Participants were 59,963 youths aged 0 to 17 years from the NSCH, 2018–2019. We enrolled 56,006 participants aged 3–17 years for our study purpose. The dataset contained some missing and unusual observations. Excluding these, about 45,779 participants were considered for our final analysis. Among them, 5218 children with ADHD and the rest of the children were healthy.

2.2. Predictor Variable

Based on an extensive literature review about ADHD, the predictor variables were used in this paper as: child’s age [34,35,36,37,38,39,40], sex of the child [34,35,36,37,38,40,41,42], mother’s age [35,41,42], allergies, arthritis, asthma, brain injury, headaches, anxiety [34,38,39,43], depression [34,36,38,43], health insurance [34,35], alcohol [36,41], race [28,34,35,37,38,40], family structure [34,35], mother’s education [34,35,38,40,42], very low birth weight (LBW) [38], LBW [35,38], premature [35,36,38,42], and poverty [34,35]. The variable names, question types, along with their categories are described in Table 1.

2.3. Outcome Variable

In this study, we considered the outcome variables by asking the following question to their parents: “Has a doctor or health professional ever told you that the selected child (S.C.) has attention deficit disorder or attention deficit hyperactive disorder, that is, ADD or ADHD?” [34,44]. We categorized this outcome variable as “1” if the response was “Yes” and “0” if the response was “No”.

2.4. Statistical Analysis

We used Stata version 14.10 for descriptive analysis and Python version 3.9, and Scikit-learn version 1.0.2 for ML-based analysis. First, data is presented as mean ± standard deviation (SD) for continuous variables and frequency (%) for categorical variables. Second, an independent t-test for continuous variables and Chi-square tests for categorical variables were used to compare the differences in variables between ADHD and healthy children. Third, all tests were two-tailed and the factors were statistically significant whose p-values are less than 0.05.

2.5. Imbalanced Management Method

A dataset is called imbalanced when one class label is larger than the other class label. To classify imbalanced data, an ML-based algorithm will be biased to the majority class. To solve this problem, we adopted two types of data sampling methods as follows: (i) oversampling and (ii) undersampling. Oversampling is a sampling technique that randomly selects the samples with replacement from the minority class and adds them to the training dataset. As a result, the performance of ML-based classifiers will be improved [45,46]. Undersampling is also a sampling technique to randomly select samples without replacement from the majority class until the balance of the class label is reached [47].

2.6. Feature Selection Method

Feature selection (FS) is also known as the variable selection in statistics and machine learning (ML). FS is a process for selecting the most informative features to improve the performance of ML-based algorithms. FS is needed for the following reasons: (i) to simplify models to make them easy to interpret by readers [48]; (ii) to reduce overfitting and the complexity of problems the model [49]; (iii) to reduce the training time and cost [50]; (iv) to avoid the curse of dimensionality [51]; and (v) to improve the accuracy of ML-based models [52]. In this study, we used LR as an FS method [53,54] to extract the most significant risk factors of the children with ADHD. LR is used as supervised learning in the community of ML. In statistics, LR is also used to extract the most informative features [36,38,41,54,55]. The LR-based feature extraction procedure is described as follows:
LR is used when the output variable is binary (1/0) and the input variables may be discrete or continuous. LR evaluates the connection between the output and one or more input variables by estimating the probability of the logit function. The logit function is the linear combination of input variables (X) and output variable (Y) (here, ADHD), which can be represented as follows:
logit ( P j ) = log e P j 1 P j = i = 0 r B i X i
where, P j is the probability of children who have ADHD and takes a value, Y = 1, and 1 P j is the probability of healthy children and takes a value, Y = 0. B i (i = 0, 1, …, r), are the unknown parameters, known as regression coefficients that need to be estimated, where, r represents the total number of the input variables. The steps of LR-based FS method are as follows: (i) Write down the likelihood function; (ii) Estimate the regression coefficients by maximum likelihood estimator (MLE) and one can get easily odds ratio (ORs) by taking the exponent of the regression coefficients (ORs = exp(B)); and (iii) Test the regression coefficients using a normal/z-test and calculate the p-values. We select the features that correspond to regression coefficients with p-values less than 0.05 [53,54,55].

2.7. Machine Learning Techniques

This study aimed to predict children with ADHD using eight ML-based classifiers. We select the best classifier who performed the better performance scores. We divided the dataset into two sets: training set and test set. We took 90% of the dataset as training set and the rest of the dataset was treated as the test set. We fitted each of eight ML-based classifiers: random forest (RF) [56], Naïve Bayes (NB) [57], decision tree (DT) [58], XGBoost [59], k-nearest neighbor (KNN) [60], multilayer perceptron (MLP) [61], support vector machine (SVM) [62], and 1-dimensional convolution network (1D CNN) [63] for the training set. The five ML-based classifiers (RF, DT, KNN, MLP, and SVM) out of eight classifiers had additional parameters, called hyperparameters. We optimized the hyperparameters based on the grid search function. The grid search function takes as input arrays of all possible hyperparameters values for each classifier and uses a cross-validation (CV) protocol on the training set to extract the optimal values of the hyperparameters. In this study, we used 10-fold CV and selected the sets of hyperparameter values with the highest classification accuracy. Then, we fit the ML-based classifiers after choosing the optimal values of the hyperparameters. The hyperparameters of different classifiers are presented in Table 2. We used the sigmoid function and the Adam optimizer for 1D CNN. After choosing the optimum value of hyperparameters, we have now predicted the children with ADHD for the test set and computed the performance scores of each ML-based classifier.

2.8. Performance Evaluation Criteria

Accuracy, sensitivity (SE), and specificity (SP) are used to evaluate the performance of all ML-based classifiers, which are computed based on true positive (TP), true negative (TN), false positive (FP), and false-negative (FN) and defined as follows:
  • Accuracy
Accuracy is the ratio between the total number of correctly classified classes and the total number of populations and mathematically defined as:
A c c u r a c y ( % ) = T P + T N T P + F N + F P + T N × 100
  • Sensitivity
Sensitivity (SE) is the ratio between the total number of correctly classified positive classes and the total number of positive classes and mathematically defined as:
S E ( % ) = T P T P + F N × 100
  • Specificity
Specificity (SP) is the ratio between the total number of correctly classified negative classes and the total number of negative classes and mathematically defined as:
S P ( % ) = F P F P + T N × 100

3. Results

In this study, we adopted a feature selection method and eight ML-based classifiers for the prediction. We performed three experiments, such as (i) Baseline and demographic characteristics of children with ADHD; (ii) balanced dataset formation; (iii) selecting the prominent significant risk factors of children with ADHD using LR; and (iv) comparison of performance of ML-based classifiers for the prediction of children with ADHD. The results of these three experiments were discussed in Section 3.1, Section 3.2, Section 3.3 and Section 3.4, respectively.

3.1. Baseline and Demographic Characteristics of Children with ADHD

The baseline and demographic characteristics of children with ADHD aged 3–17 years are shown in Table 3. Before balancing the class label, the overall prevalence of ADHD was 11.4%. The age range included in our analysis was from 3–17 years, with the average age of the children being 10.6 ± 4.4 years, with an ADHD disease age of 12.4 ± 3.4 years. In this study, 52.2% were male; 79.2% were white, 6.29% were black, and 14.6% were of other race. About 15.1% of male children had ADHD. Our results showed that 37.6% and 41.9% of children with ADHD suffered from anxiety and depression problems, respectively. It was observed that all factors were statistically significantly associated with ADHD (p < 0.05).

3.2. Balanced Dataset Formation

The main aim of this section is to balance the class label (ADHD vs. healthy) using a combination of oversampling and undersampling methods. The database utilized in this study was comprised of 5218 (11.4%) children with ADHD, and 40,561 (88.6%) children were healthy. Here, the ratio between children with ADHD and healthy children was 1:9. In order to reduce the difference in the number of samples per class, we take 3 times of the positive class (ADHD) ( 3 × 5218 ) = 15,654 children with ADHD using oversampling and also take 15,654 healthy children from 40,561 using undersampling.

3.3. Prominent Risk Factors of Children with ADHD Using LR

One of the objectives of this study was to select the high-risk factors for children with ADHD. After balancing the class label, LR was adopted for feature selection. We need to check the associations between different factors and children with ADHD before applying LR. We chose only the factors for LR whose factors were statistically significantly associated with children who had ADHD. Table 4 summarizes identifying the risk factors for children with ADHD using LR. The odds ratios (ORs) with their 95% confidence intervals (CIs), standard error (SE), and p-values are also summarized in Table 4. The following factors were associated with a higher likelihood of being diagnosed with ADHD: child’s age (OR: 1.103, 95% CI: 1.096–1.110); a male child (OR: 2.727; 95% CI: 2.586–2.877), had allergies (OR: 1.161; 95% CI: 1.098–1.228); had asthma (OR: 1.225; 95% CI: 1.140–1.316); had anxiety (OR: 5.217; 95% CI: 4.848–5.613); had depression (OR: 1.807; 95% CI: 1.628–2.005); drinking alcohol (OR: 1.383; 95% CI: 1.202–1.591); had health insurance (OR: 1.440, 95% CI: 1.330–1.558); was white (OR: 1.431; 95% CI: 1.323–1.548), black (OR: 1.636; 95% CI: 1.449–1.848), had very LBW (OR: 1.353; 95% CI: 1.083–1.691), premature child (OR: 1.474; 95% CI: 1.346–1.615), belonged to ≤200% poverty level (OR: 1.093; 95% CI: 1.012–1.178). A child had a significantly lower chance of being diagnosed with ADHD if she/he lived in a two-parent family (OR:0.833; 95% CI: 0.781–0.887), and mother’s age (OR: 0.971 95% CI: (0.967–0.975) was a low risk factor. At 5% level of significance, it was discovered that child’s age, child’s sex, mother’s age, allergies, asthma, anxiety, depression, alcohol, insurance, race, family structure, very LBW, premature child, and poverty were statistically significant risk factors of ADHD (see Table 4).

3.4. Comparisons of Performances of Machine Learning Techniques

The main objective of this section was to predict children with ADHD using eight ML-based classifiers. The comparison of the performances of ML-based classifiers for the prediction of children with ADHD is shown in Table 5. It was noted that RF-based classifier gave the highest classification accuracy of 85.5%, sensitivity of 84.4%, and specificity of 86.4%, whereas NB provided the lowest classification accuracy of 69.8%, sensitivity of 77.3%, and specificity of 65.3%. It was also noted that DT provided 84.6% accuracy, 83.4% sensitivity, and 86.0% specificity, whereas KNN provided 84.0% accuracy, 82.6% sensitivity, and 85.6% specificity. It was also observed that RF-based classifier achieved the highest AUC of 0.94 compared to other classifiers. The corresponding ROC curve of eight ML-based classifiers is depicted in Figure 1. Therefore, the RF-based classifier performed better performance scores for the prediction of children with ADHD.

4. Discussion

Our current study was conducted based on the latest nationally representative survey of NSCH, 2018–2019, with children aged 3–17 years. The study aim was as follows: (i) to investigate the risk factors of the children with ADHD; and (ii) to predict the children with ADHD. The current diagnostic process for ADHD is time-consuming and complicated by behavioral symptom overlaps. Since the incidence rate of ADHD is high, it is necessary to provide a tool that can swiftly and correctly predict the risk of ADHD. There were some ML-based works in previous studies to correctly detect and predict ADHD [28,29,64,65,66] and children with ADHD who received treatment [32]. Our current study expands these previous works by implementing an LR-based model for the risk factor extraction method and eight ML-based classifiers for the prediction of the children with ADHD. LR results illustrated that several factors (child’s age, child’s sex, mother’s age, allergies, asthma, anxiety, depression, alcohol, insurance, race, family structure, very LBW, premature child, and poverty) were identified as the high-risk predictors of the children who had ADHD. This present study also adopted eight ML-based classifiers for prediction. Eight ML-based classifiers for the prediction of children with ADHD gave an accuracy range of 69.8% to 85.5% and an AUC of 0.78 to 0.94. RF-based classifiers correctly predicted the children with ADHD with an excellent accuracy of 85.5% and also an excellent AUC of 0.94.

5. Conclusions

This study presented a comprehensive investigation into the risk factors of the children with ADHD. This study illustrated that LR with RF-based classifier could provide excellent accuracy in correctly classifying and predicting children with ADHD. This study will assist physicians in detecting and treating children with ADHD at an early stage.

Author Contributions

Conceptualization, J.S., M.A.M.H. and M.M.; methodology, M.A.M.H. and M.M.; software, M.M.; validation, M.A.M.H. and M.M.; formal analysis, M.M.; investigation, M.A.M.H. and M.M.; resources, J.S. and M.A.M.H.; data curation and collection, M.M.; writing—original draft preparation, M.M.; writing—review and editing, J.S. and M.A.M.H.; visualization, J.S. and M.M; supervision, J.S. and M.A.M.H.; project administration, J.S.; funding acquisition, J.S. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research (KAKENHI), Japan (Grant Numbers JP20K11892).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

This study was based on an analysis of existing public domain survey datasets that are freely available online with all identifier information removed. One can use the dataset from the following link as [https://www.census.gov/programs-surveys/nsch/data/datasets.html].

Acknowledgments

The authors are grateful to the National Survey of Children’s Health (NSCH) for providing the datasets. The authors also want to express their gratitude to the editors and reviewers for their valuable comments and suggestions to improve the manuscript.

Conflicts of Interest

The authors declared no conflict of interest for this research.

References

  1. Edition, F. Diagnostic and statistical manual of mental disorders. Am. Psychiatric Assoc. 2013, 21, 591–643. [Google Scholar]
  2. Danielson, M.L.; Bitsko, R.H.; Ghandour, R.M.; Holbrook, J.R.; Kogan, M.D.; Blumberg, S.J. Prevalence of parent-reported ADHD diagnosis and associated treatment among US children and adolescents, 2016. J. Clin. Child. Adolesc. Psychol. 2018, 47, 199–212. [Google Scholar] [CrossRef] [PubMed]
  3. Mowlem, F.D.; Rosenqvist, M.A.; Martin, J.; Lichtenstein, P.; Asherson, P.; Larsson, H. Sex differences in predicting ADHD clinical diagnosis and pharmacological treatment. Eur. Child Adolesc. Psychiatry 2019, 28, 481–489. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Fayyad, J.; Sampson, N.A.; Hwang, I.; Adamowski, T.; Aguilar-Gaxiola, S.; Al-Hamzawi, A.; Andrade, L.H.; Borges, G.; de Girolamo, G.; Florescu, S.; et al. The descriptive epidemiology of DSM-IV adult ADHD in the world health organization world mental health surveys. Atten. Defic. Hyperact. Disord. 2017, 9, 47–65. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Visser, S.N.; Lesesne, C.A.; Perou, R. National estimates and factors associated with medication treatment for childhood attention-deficit/hyperactivity disorder. Pediatrics 2007, 119, S99–S106. [Google Scholar] [CrossRef] [Green Version]
  6. Faraone, S.V.; Banaschewski, T.; Coghill, D.; Zheng, Y.; Biederman, J.; Bellgrove, M.A.; Newcorn, J.H.; Gignac, M.; Al Saud, N.M.; Manor, I.; et al. The world federation of ADHD international consensus statement: 208 evidence-based conclusions about the disorder. Neurosci. Biobehav. Rev. 2021, 128, 789–818. [Google Scholar] [CrossRef]
  7. Brikell, I.; Kuja-Halkola, R.; Larsson, H. Heritability of attention-deficit hyperactivity disorder in adults. Am. J. Med. Genet. B Neuropsychiatr. Genet. 2015, 168, 406–413. [Google Scholar] [CrossRef]
  8. Freeman-Fobbs, P. Feeding our children to death: The tragedy of childhood obesity in America. J. Natl. Med. Assoc. 2003, 95, 119. [Google Scholar]
  9. Stevens, J.; Harman, J.S.; Kelleher, K.J. Race/ethnicity and insurance status as factors associated with ADHD treatment patterns. J. Child Adolesc. Psychopharmacol. 2005, 15, 88–96. [Google Scholar] [CrossRef]
  10. Bazar, K.A.; Yun, A.J.; Lee, P.Y.; Daniel, S.M.; Doux, J.D. Obesity and ADHD may represent different manifestations of a common environmental oversampling syndrome: A model for revealing mechanistic overlap among cognitive, metabolic, and inflammatory disorders. Med. Hypotheses 2006, 66, 263–269. [Google Scholar] [CrossRef]
  11. Agranat-Meged, A.N.; Deitcher, C.; Goldzweig, G.; Leibenson, L.; Stein, M.; Galili-Weisstub, E. Childhood obesity and attention deficit/hyperactivity disorder: A newly described comorbidity in obese hospitalized children. Int. J. Eat. Disord. 2005, 37, 357–359. [Google Scholar] [CrossRef] [PubMed]
  12. Cortese, S.; Angriman, M.; Maffeis, C.; Isnard, P.; Konofal, E.; Lecendreux, M.; Purper-Ouakil, D.; Vincenzi, B.; Bernardina, B.D.; Mouren, M.C. Attention-deficit/hyperactivity disorder (ADHD) and obesity: A systematic review of the literature. Crit. Rev. Food Sci. Nutr. 2008, 48, 524–537. [Google Scholar] [CrossRef] [PubMed]
  13. Waring, M.E.; Lapane, K.L. Overweight in children and adolescents in relation to attention-deficit/hyperactivity disorder: Results from a national sample. Pediatrics 2008, 122, e1–e6. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Bramlett, M.D.; Blumberg, S.J. Family structure and children’s physical and mental health. Health Aff. 2007, 26, 549–558. [Google Scholar] [CrossRef] [PubMed]
  15. Kollins, S.H.; McClernon, F.J.; Fuemmeler, B.F. Association between smoking and attention-deficit/hyperactivity disorder symptoms in a population-based sample of young adults. Arch. Gen. Psychiatry 2005, 62, 1142–1147. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Choy, G.; Khalilzadeh, O.; Michalski, M.; Do, S.; Samir, A.E.; Pianykh, O.S.; Geis, J.R.; Pandharipande, P.V.; Brink, J.A.; Dreyer, K.J. Current applications and future impact of machine learning in radiology. Radiology 2018, 288, 318–328. [Google Scholar] [CrossRef]
  17. Zhou, L.Q.; Wang, J.Y.; Yu, S.Y.; Wu, G.G.; Wei, Q.; Deng, Y.B.; Wu, X.L.; Cui, X.W.; Dietrich, C.F. Artificial intelligence in medical imaging of the liver. World J. Gastroenterol. 2019, 25, 672. [Google Scholar] [CrossRef]
  18. Ghaderzadeh, M.; Asadi, F.; Hosseini, A.; Bashash, D.; Abolghasemi, H.; Roshanpour, A. Machine learning in detection and classification of leukemia using smear blood images: A systematic review. Scient. Program. 2021, 2021, 1–14. [Google Scholar] [CrossRef]
  19. Alanazi, H.O.; Abdullah, A.H.; Qureshi, K.N. A critical review for developing accurate and dynamic predictive models using machine learning methods in medicine and health care. J. Med. Syst. 2017, 41, 1–10. [Google Scholar] [CrossRef]
  20. Zea-Vera, R.; Ryan, C.T.; Havelka, J.; Corr, S.J.; Nguyen, T.C.; Chatterjee, S.; Wall, M.J., Jr.; Coselli, J.S.; Rosengart, T.K.; Ghanta, R.K. Machine Learning to Predict Outcomes and Cost by Phase of Care after Coronary Artery Bypass Grafting. Ann. Thorac. Surg. 2021, 112, S0003–4975. [Google Scholar] [CrossRef]
  21. Battineni, G.; Sagaro, G.G.; Chinatalapudi, N.; Amenta, F. Applications of machine learning predictive models in the chronic disease diagnosis. J. Pers. Med. 2020, 10, 21. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Kessler, R.C.; Bernecker, S.L.; Bossarte, R.M.; Luedtke, A.R.; McCarthy, J.F.; Nock, M.K.; Pigeon, W.R.; Petukhova, M.V.; Sadikova, E.; VanderWeele, T.J.; et al. The role of big data analytics in predicting suicide. In Person. Psychiatry-Big Data Analytics in Mental Health; Springer Nature: New York, NY, USA, 2019. [Google Scholar]
  23. Burke, T.A.; Ammerman, B.A.; Jacobucci, R. The use of machine learning in the study of suicidal and non-suicidal self-injurious thoughts and behaviors: A systematic review. J. Affect. Disord. 2019, 245, 869–884. [Google Scholar] [CrossRef] [PubMed]
  24. Kim, J.W.; Sharma, V.; Ryan, N.D. Predicting methylphenidate response in ADHD using machine learning approaches. Int. J. Neuropsychopharmacol. 2015, 18, 1–7. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Kim, S.; Lee, H.; Lee, K. Can the MMPI Predict Adult ADHD? An Approach Using Machine Learning Methods. Diagnostics 2021, 11, 976. [Google Scholar] [CrossRef]
  26. Zhang-James, Y.; Helminen, E.C.; Liu, J.; Franke, B.; Hoogman, M.; Faraone, S.V. Evidence for similar structural brain anomalies in youth and adult attention-deficit/hyperactivity disorder: A machine learning analysis. Transl. Psychiatry 2021, 11, 1–9. [Google Scholar] [CrossRef]
  27. Yasumura, A.; Omori, M.; Fukuda, A.; Takahashi, J.; Yasumura, Y.; Nakagawa, E.; Koike, T.; Yamashita, Y.; Miyajima, T.; Koeda, T.; et al. Applied machine learning method to predict children with ADHD using prefrontal cortex activity: A multicenter study in Japan. J. Atten. Disord. 2020, 24, 2012–2020. [Google Scholar] [CrossRef]
  28. Duda, M.; Ma, R.; Haber, N.; Wall, D. Use of machine learning for behavioral distinction of autism and ADHD. Transl. Psychiatry 2016, 6, e732. [Google Scholar] [CrossRef] [Green Version]
  29. Duda, M.; Haber, N.; Daniels, J.; Wall, D. Crowdsourced validation of a machine-learning classification system for autism and ADHD. Transl. Psychiatry. 2017, 7, e1133. [Google Scholar] [CrossRef]
  30. Uluyagmur-Ozturk, M.; Arman, A.R.; Yilmaz, S.S.; Findik, O.T.P.; Genc, H.A.; Carkaxhiu-Bulut, G.; Yazgan, M.Y.; Teker, U.; Cataltepe, Z. ADHD and ASD classification based on emotion recognition data. In Proceedings of the 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), Anaheim, CA, USA, 18–20 December 2016; pp. 810–813. [Google Scholar]
  31. Slobodin, O.; Yahav, I.; Berger, I. A Machine-Based Prediction Model of ADHD Using CPT Data. Front. Hum. Neurosci. 2020, 14, 383. [Google Scholar] [CrossRef]
  32. Morrow, A.S.; Campos Vega, A.D.; Zhao, X.; Liriano, M.M. Leveraging machine learning to identify predictors of receiving psychosocial treatment for Attention Deficit/Hyperactivity Disorder. Adm. Policy Ment. Health 2020, 47, 680–692. [Google Scholar] [CrossRef]
  33. Child and Adolescent Health Measurement Initiative. 2018–2019 National Survey of Children’s Health (2 Years Combined), [(SAS/SPSS/Stata)] Indicator Data Set. In Data Resource Center for Child and Adolescent Health supported by Cooperative Agreement from the U.S.; Department of Health and Human Services, Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB): Washington, DC, USA, 2013. [Google Scholar]
  34. Lingineni, R.K.; Biswas, S.; Ahmad, N.; Jackson, B.E.; Bae, S.; Singh, K.P. Factors associated with attention deficit/hyperactivity disorder among US children: Results from a national survey. BMC Pediatr. 2012, 12, 1–10. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. DeCarlo, D.K.; Swanson, M.; McGwin, G.; Visscher, K.; Owsley, C. ADHD and vision problems in the National Survey of Children’s Health. Optom. Vis. Sci. 2016, 93, 459. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Wüstner, A.; Otto, C.; Schlack, R.; Hölling, H.; Klasen, F.; Ravens-Sieberer, U. Risk and protective factors for the development of ADHD symptoms in children and adolescents: Results of the longitudinal BELLA study. PLoS ONE 2019, 14, e0214412. [Google Scholar] [CrossRef] [Green Version]
  37. DuPaul, G.J.; Chronis-Tuscano, A.; Danielson, M.L.; Visser, S.N. Predictors of receipt of school services in a national sample of youth with ADHD. J. Atten. Disord. 2019, 23, 1303–1319. [Google Scholar] [CrossRef]
  38. Zarei, K.; Xu, G.; Zimmerman, B.; Giannotti, M.; Strathearn, L. Adverse childhood experiences predict common neurodevelopmental and behavioral health conditions among US children. Children 2021, 8, 761. [Google Scholar] [CrossRef]
  39. Ren, Y.; Fang, X.; Fang, H.; Pang, G.; Cai, J.; Wang, S.; Ke, X. Predicting the adult clinical and academic outcomes in boys with ADHD: A 7-to 10-year follow-up study in China. Front. Pediatr. 2021, 9, 751. [Google Scholar] [CrossRef] [PubMed]
  40. DuPaul, G.J.; Evans, S.W.; Owens, J.S.; Cleminshaw, C.L.; Kipperman, K.; Fu, Q.; Benson, K. School-based intervention for adolescents with attention-deficit/hyperactivity disorder: Effects on academic functioning. J. Sch. Psychol. 2021, 87, 48–63. [Google Scholar] [CrossRef] [PubMed]
  41. Hoang, H.H.; Tran, A.T.N.; Van Hung Nguyen, T.T.B.; Nguyen, T.A.P.N.; Le, D.D.; Jatho, A.; Onchonga, D.; Van Duong, T.; Nguyen, M.T.; Tran, B.T. Attention Deficit Hyperactivity Disorder (ADHD) and Associated Factors Among First-Year Elementary School Students. J. Multidiscip. Healthc. 2021, 14, 997. [Google Scholar] [CrossRef]
  42. Rahman, M.S.; Takahashi, N.; Iwabuchi, T.; Nishimura, T.; Harada, T.; Okumura, A.; Takei, N.; Nomura, Y.; Tsuchiya, K.J. Elevated risk of attention deficit hyperactivity disorder (ADHD) in Japanese children with higher genetic susceptibility to ADHD with a birth weight under 2000 g. BMC Med. 2021, 19, 1–13. [Google Scholar] [CrossRef]
  43. AlZaben, F.N.; Sehlo, M.G.; Alghamdi, W.A.; Tayeb, H.O.; Khalifa, D.A.; Mira, A.T.; Alshuaibi, A.M.; Alguthmi, M.A.; Derham, A.A.; Koenig, H.G. Prevalence of attention deficit hyperactivity disorder and comorbid psychiatric and behavioral problems among primary school students in western Saudi Arabia. Saudi Med. J. 2018, 39, 52. [Google Scholar] [CrossRef]
  44. Wang, C.; Preisser, J.; Chung, Y.; Li, K. Complementary and alternative medicine use among children with mental health issues: Results from the National Health Interview Survey. BMC Complement. Altern. Med. 2018, 18, 1–17. [Google Scholar] [CrossRef] [PubMed]
  45. Schubach, M.; Re, M.; Robinson, P.N.; Valentini, G. Imbalance-aware machine learning for predicting rare and common disease-associated non-coding variants. Sci. Rep. 2017, 7, 1–12. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Matsuoka, D. Classification of imbalanced cloud image data using deep neural networks: Performance improvement through a data science competition. Prog. Earth Planet. Sci. 2021, 8, 1–11. [Google Scholar] [CrossRef]
  47. Bunkhumpornpat, C.; Sinapiromsaran, K.; Lursinsap, C. MUTE: Majority under-sampling technique. In Proceedings of the 2011 8th International Conference on Information, Communications & Signal Processing, Singapore, 13–16 December 2011; IEEE: Hoboken, NJ, USA, 2011; pp. 1–4. [Google Scholar]
  48. James, G.; Witten, D.; Hastie, T.; Tibshirani, R. An Introduction to Statistical Learning; Springer: New York, NY, USA, 2013; Volume 112. [Google Scholar]
  49. Fogelman-Soulié, F. Mining Massive Data Sets for Security: Advances in Data Mining, Search, Social Networks and Text Mining, and Their Applications to Security; IOS Press: Amsterdam, The Netherlands, 2008; Volume 19, pp. 1–366. [Google Scholar]
  50. Liu, H.; Motoda, H. Feature Selection for Knowledge Discovery and Data Mining; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2012; Volume 454. [Google Scholar]
  51. Kramer, M.A. Nonlinear principal component analysis using autoassociative neural networks. AIChE J. 1991, 37, 233–243. [Google Scholar] [CrossRef]
  52. Kratsios, A.; Hyndman, C. Neu: A meta-algorithm for universal uap-invariant feature representation. J. Mach. Lear. Res. 2021, 22, 1–51. [Google Scholar]
  53. Cuadrado-Godia, E.; Maniruzzaman, M.; Araki, T.; Puvvula, A.; Rahman, M.J.; Saba, L.; Suri, H.S.; Gupta, A.; Banchhor, S.K.; Teji, J.S.; et al. Morphologic TPA (mTPA) and composite risk score for moderate carotid atherosclerotic plaque is strongly associated with HbA1c in diabetes cohort. Comput. Biol. Med. 2018, 101, 128–145. [Google Scholar] [CrossRef]
  54. Maniruzzaman, M.; Rahman, M.J.; Ahammed, B.; Abedin, M.M. Classification and prediction of diabetes disease using machine learning paradigm. Health Inf. Sci. Syst. 2020, 8, 1–14. [Google Scholar] [CrossRef]
  55. Maniruzzaman, M.; Rahman, M.J.; Al-MehediHasan, M.; Suri, H.S.; Abedin, M.M.; El-Baz, A.; Suri, J.S. Accurate diabetes risk stratification using machine learning: Role of missing value and outliers. J. Med. Syst. 2018, 42, 1–17. [Google Scholar] [CrossRef] [Green Version]
  56. Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
  57. Rish, I. An empirical study of the naive Bayes classifier. In IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence; ResearchGate: Berlin, Germany, 2001; Volume 3, pp. 41–46. [Google Scholar]
  58. Quinlan, J.R. Simplifying decision trees. Int. J. Man. Mach. Stud. 1987, 27, 221–234. [Google Scholar] [CrossRef] [Green Version]
  59. Chen, T.; Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
  60. Peterson, L.E. K-nearest neighbor. Scholarpedia J. 2009, 4, 1883. [Google Scholar] [CrossRef]
  61. Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning; Springer Series in Statistics; Springer: Berlin/Heidelberg, Germany, 2001. [Google Scholar]
  62. Cortes, C.; Vapnik, V. Support-vector networks. Mach. learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
  63. Zhang, X.; Wu, F.; Li, Z. Application of convolutional neural network to traditional data. Expert Syst. Appl. 2021, 168, 114185. [Google Scholar] [CrossRef]
  64. Tenev, A.; Markovska-Simoska, S.; Kocarev, L.; Pop-Jordanov, J.; Müller, A.; Candrian, G. Machine learning approach for classification of ADHD adults. Int. J. Psychophysiol. 2014, 93, 162–166. [Google Scholar] [CrossRef] [PubMed]
  65. Chu, K.C.; Huang, H.J.; Huang, Y.S. Machine learning approach for distinction of ADHD and OSA. In Proceedings of the 2016 IEEE/ACM international Conference on Advances in Social Networks Analysis and Mining (ASONAM), San Francisco, CA, USA, 18–21 August 2016; pp. 1044–1049. [Google Scholar]
  66. Christiansen, H.; Chavanon, M.L.; Hirsch, O.; Schmidt, M.H.; Meyer, C.; Müller, A.; Rumpf, H.J.; Grigorev, I.; Hoffmann, A. Use of machine learning to classify adult ADHD and other conditions based on the Conners’ Adult ADHD Rating Scales. Sci. Rep. 2020, 10, 1–10. [Google Scholar] [CrossRef] [PubMed]
Figure 1. ROC curves of eight ML-based classifiers for children with ADHD.
Figure 1. ROC curves of eight ML-based classifiers for children with ADHD.
Applsci 12 02737 g001
Table 1. Description of variable names, question types, and their categories.
Table 1. Description of variable names, question types, and their categories.
Variable NamesQuestion TypesCategories
Child’s ageChild age in yearsContinuous
SexSex of the childMale and Female
Mother’s ageMother’s age in yearsContinuous
AllergiesHas a doctor ever told you that the selected child (S.C.) has allergies?Yes and No
ArthritisHas a doctor ever told you that S.C. has arthritis?Yes and No
AsthmaHas a doctor ever told you that S.C. has asthma?Yes and No
Brain injuryHas a doctor ever told you that S.C. has a brain injuryYes and No
HeadachesHas a doctor ever told you that S.C. has frequent or severe headaches or migraine?Yes and No
AnxietyHas a doctor ever told you that S.C. had anxiety problems?Yes and No
DepressionHas a doctor ever told you that S.C. had depression problems?Yes and No
InsuranceIs S.C. currently covered by any kind of health insurance plan?Yes and No
AlcoholTo the best of your knowledge, has S.C. ever experienced lived with anyone who had a problem with alcohol or drugsYes and No
RaceWhat is this child’s race?White, Black, and Other
Family structureFamily structureTwo-parent-biological/step/adopted and Other-single mother/ father/other
Mother’s educationHighest level of education<High school, High school, and > High school
Very LBWIs child-birth weight <1.5 kg?Yes and No
LBWIs child-birth weight <2.5 kg?Yes and No
PrematurePremature birth (>3 weeks before due date)Yes and No
PovertyIncome-based on federal poverty level status<200% and >=200%
Table 2. Optimized hyperparameters of different classifiers using the grid search method.
Table 2. Optimized hyperparameters of different classifiers using the grid search method.
Classifier TypesHyper-ParametersOptimized Values
RFmax_depth = (2, 3, 5), n_estimators = (25, 50, 100, 200, 300, 600, 1200), min_samples_split = (2, 3, 10), min_samples_leaf = (1, 3, 10), criterion = (gini, entropy)max_depth = 3, n_estimators = 200, min_samples_split= 10, min_samples_leaf = 10, criterion= entropy
DTmin_samples_leaf = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)min_samples_leaf = 9
KNNk = (5, 6, 7,8, 9, 10), leaf_size = (1, 2, 3, 5)k = 8, leaf_size = 1
MLPhidden_layer = [(50, 50, 50), (50, 160, 50), (100, 1)], activation = (relu, tanh, logistic), alpha = (0.0001, 0.05), learning_rate = (constant, adaptive)hidden_layer = (50, 160, 50), activation = logistic, alpha = 0.05, learning_rate = adaptive
SVMC = (1, 10, 100, 1000), γ = (0.001, 0.0001)C = 1000, and γ = 0.001
Table 3. Baseline and demographic characteristics of children with ADHD, 3–17 years.
Table 3. Baseline and demographic characteristics of children with ADHD, 3–17 years.
VariablesOverall, n (%)Healthy, n (%)ADHD, n (%)p-Value 1
Total45,77940,561 (88.6)5218 (11.4)
Child’s age10.6 ± 4.410.4 ± 4.512.4 ± 3.4<0.001
Sex, Male23,901 (52.2)20,304 (84.9)3597 (15.1)<0.001
Mother’s age30.0 ± 5.830.2 ± 5.728.7 ± 6.3<0.001
Allergies, Yes13,930(30.4)11,889 (85.4)2041 (14.6)<0.001
Arthritis, Yes182 (0.4)151 (83.0)31 (17.0)0.039
Asthma, Yes6293 (13.8) 5180 (82.3)1113 (17.7)<0.001
Brain injury, Yes2482 (5.4)2048 (82.5)434 (17.5)<0.001
Headache, Yes2457 (5.4)1931 (78.6)526 (21.4)<0.001
Anxiety, Yes5850 (12.8)3652 (62.4)2198 (37.6)<0.001
Depression, Yes2744 (6.0)1595 (58.1)1149 (41.9)<0.001
Insurance, Yes44,057 (96.2)39,008 (88.5)5049 (11.46)0.035
Alcohol, Yes4728 (10.3)3712 (78.5)1016 (21.5)<0.001
Race, White36,235 (79.2)31,935 (88.1)4300 (11.9)<0.001
Family structure, two parent-biological/step/adopted35,551 (77.7)31,940 (89.8)3611 (10.2)<0.001
Mother’s education, High school39,048 (85.3)34,725 (88.9)4323 (11.1)<0.001
Very LBW, Yes558 (1.2)451 (80.8)107 (19.2)<0.001
LBW, Yes3837 (8.4)3289 (85.7)548 (14.3)<0.001
Premature, Yes5090 (11.1)4287 (84.2)803 (15.8)<0.001
Poverty, <200%12,079 (26.4)10,426 (86.3)1653 (13.7)<0.001
Table 4. Identifying the risk factors for ADHD using LR.
Table 4. Identifying the risk factors for ADHD using LR.
VariablesOR (95% CI)SEp-Value 1
Child’s age1.103 (1.096–1.110)0.004<0.001
Sex   
Male2.727 (2.586–2.877)0.074<0.001
Female®1.000  
Mother’s age0.971 (0.967–0.975)0.002<0.001
Allergies   
Yes1.161 (1.098–1.228)0.033<0.001
No®1.000  
Arthritis   
Yes0.688 (0.442–1.026)0.1530.088
No®1.000  
Asthma   
Yes1.225 (1.140–1.316)0.045<0.001
No®1.000  
Brain injury   
Yes0.933 (0.837–1.039)0.0510.260
No®1.000  
Headache   
Yes0.979 (0.879-1.090)0.0540.702
No®1.000  
Anxiety   
Yes5.217 (4.848–5.613)0.195<0.001
No®1.000  
Depression   
Yes1.807 (1.628–2.005)0.096<0.001
No®1.000  
Alcohol   
Yes1.383 (1.202–1.591)0.099<0.001
No®1.000  
Insurance   
Yes1.440 (1.330–1.558)0.058<0.001
No®1.000  
Alcohol   
Yes1.393 (1.274–1.523)0.0640.001
No®   
Race   
White1.431 (1.323–1.548)0.057<0.001
Black1.636 (1.449–1.848)0.102<0.001
Others®1.000  
Family structure   
Two parent biological/step/adopted0.833 (0.781–0.887)0.027<0.001
Other-single mother/father/other®1.000  
Mother’s education   
<High school0.876 (0.733–1.046)0.0790.142
High school1.062 (0.983–1.149)0.0420.129
>High school®1.000  
Very LBW   
Yes1.353 (1.083–1.691)0.1950.003
No®1.000  
LBW   
Yes1.015 (0.910–1.132)0.0570.791
No®1.000  
Premature   
Yes1.474 (1.346–1.615)0.069<0.001
No®1.000  
Poverty   
<200%1.093 (1.012–1.178)0.0420.002
>=200%®1.000  
Table 5. Performances of ML-based classifiers for predicting children with ADHD.
Table 5. Performances of ML-based classifiers for predicting children with ADHD.
Classifier TypesAccuracy (%)SE (%)SP (%)AUC
RF85.584.486.40.94
NB69.877.365.30.78
DT84.683.486.00.90
XGBoost77.076.877.30.86
KNN84.082.685.60.92
MLP79.980.179.60.88
SVM-RBF71.972.970.90.80
1D CNN72.180.184.40.80
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Maniruzzaman, M.; Shin, J.; Hasan, M.A.M. Predicting Children with ADHD Using Behavioral Activity: A Machine Learning Analysis. Appl. Sci. 2022, 12, 2737. https://doi.org/10.3390/app12052737

AMA Style

Maniruzzaman M, Shin J, Hasan MAM. Predicting Children with ADHD Using Behavioral Activity: A Machine Learning Analysis. Applied Sciences. 2022; 12(5):2737. https://doi.org/10.3390/app12052737

Chicago/Turabian Style

Maniruzzaman, Md., Jungpil Shin, and Md. Al Mehedi Hasan. 2022. "Predicting Children with ADHD Using Behavioral Activity: A Machine Learning Analysis" Applied Sciences 12, no. 5: 2737. https://doi.org/10.3390/app12052737

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop