Next Article in Journal
Physiological and Neural Changes with Rehabilitation Training in a 53-Year Amputee: A Case Study
Next Article in Special Issue
Association between Individual Norepinephrine Transporter (NET) Availability and Response to Pharmacological Therapy in Adults with Attention-Deficit/Hyperactivity Disorder (ADHD)
Previous Article in Journal
The Neurobiological Basis of Love: A Meta-Analysis of Human Functional Neuroimaging Studies of Maternal and Passionate Love
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Accurate Identification of ADHD among Adults Using Real-Time Activity Data

by
Amandeep Kaur
1,* and
Karanjeet Singh Kahlon
2
1
Department of Computer Engineering and Technology, Guru Nanak Dev University, Amritsar 143005, Punjab, India
2
Department of Computer Science, Guru Nanak Dev University, Amritsar 143005, Punjab, India
*
Author to whom correspondence should be addressed.
Brain Sci. 2022, 12(7), 831; https://doi.org/10.3390/brainsci12070831
Submission received: 4 June 2022 / Revised: 20 June 2022 / Accepted: 23 June 2022 / Published: 26 June 2022
(This article belongs to the Special Issue Current Research in ADHD among Adults)

Abstract

:
Attention Deficit Hyperactivity Disorder (ADHD) is a neurodevelopment disorder that affects millions of children and typically persists into adulthood. It must be diagnosed efficiently and consistently to receive adequate treatment, otherwise, it can have a detrimental impact on the patient’s professional performance, mental health, and relationships. In this work, motor activity data of adults suffering from ADHD and clinical controls has been preprocessed to obtain 788 activity-related statistical features. Afterwards, principal component analysis has been carried out to obtain significant features for accurate classification. These features are then fed into six different machine learning algorithms for classification, which include C4.5, kNN, Random Forest, LogitBoost, SVM, and Naive Bayes. The detailed evaluation of the results through 10-fold cross-validation reveals that SVM outperforms other classifiers with an accuracy of 98.43%, F-measure of 98.42%, sensitivity of 98.33%, specificity of 98.56% and AUC of 0.983. Thus, a PCA-based SVM approach appears to be an effective choice for accurate identification of ADHD patients among other clinical controls using real-time analysis of activity data.

1. Introduction

ADHD (Attention Deficit Hyperactivity Disorder) is a prevalent childhood disorder characterized by inattention, hyperactivity, and impulsivity [1]. Children with ADHD have poor behavioral regulation and control, causing them to react inappropriately to a variety of stimuli. The incidence rate for school-age children globally is about 5%; approximately 80–85 percent of ADHD tweens tend to face difficulties into adolescence, and 60% encounter symptoms in adults [2]. ADHD can have a significant impact on academic attainment and social interactions, as it makes it difficult for the patients to focus their attention and manage their behavior [3,4]. Adult ADHD frequently has a more diverse clinical presentation that extends beyond the basic motor symptoms seen in children and includes a broader range of emotional dysregulation and functional impairment [5]. Adult ADHD tends to be characterized by inattention, which manifests itself as disorganization and distraction-, task- and focus-related difficulties, as well as a proclivity for daydreaming [6]. Adult ADHD is usually linked to other mental conditions such as depression, mood, anxiety and sleeplessness [7,8]. Moreover, the symptoms of ADHD frequently coincide with those of other psychiatric illnesses, such as affective and internalizing disorders, autism spectrum disorder, and autistic-like features [9]. There may be external factors that raise the chance of ADHD, such as exposure to chemicals and maternal lifestyle throughout pregnancy, but these characteristics are not prevalent in all human beings [10]. Furthermore, the disorder’s occurrence cannot be linked to a particular biomarker, such as a single faulty gene [11].
While males are more likely to be diagnosed with ADHD and the percentage of them receiving treatment is high, the existing research trends show that prevalence has been rising amongst females and people of various races over the last century [12]. Although females have lesser incidence rates than males, they have severe impairments associated with their diagnoses that necessitate early detection to minimize the chances of psychological illnesses, self-harm and suicidality [13]. The lack of clear ADHD diagnosis is due to a lack of understanding of the brain mechanisms that cause the disorder. The severity of ADHD symptoms, IQ, co-morbidities, family conflict or environmental discomfort, earlier care for ADHD, and the incidence of concurrent diseases are all significant factors that influence the trajectory of ADHD [6,14]. In the diagnosis of this disorder, stimulants are undoubtedly regarded as the most promising treatment, however, they carry a high risk of misuse, addiction, and pose different side effects such as loss of appetite, high blood pressure, heart problems, mood disorders, etc., [1]. However, there exist strong associations between adult ADHD with a higher risk of substance abuse [15]. The current way of diagnosing ADHD is a subjective evaluation and thorough clinical observation which are mostly imprecise and erroneous, so there is a need for more objective methods. Since the diagnosis is exclusively based on observed behavior and reported symptoms, there is a possibility of over-and-under-diagnosis and also there are no valid objective tests to identify ADHD [16].
Nowadays, sensory data gathered from patients and evaluated using machine learning approaches has attracted a lot of attention as a way to supplement existing subjective diagnostic practices in mental health, particularly ADHD [17]. Recent technological advancements, notably in the field of wearable devices, have enabled unparalleled access to physiological data. Moreover, wearable devices are employed in a variety of applications such as analyzing physical health and mental health. The studies using actigraph data from both adolescents and adults have found that motor activity has a significant role to play in the ADHD diagnosis. Moreover, HRV has been researched concerning ADHD, although the results are relatively less promising [1]. Generally, Electrocardiogram (ECG) [18], electroencephalogram (EEG) [19,20], magnetic resonance imaging (MRI) [21,22], game simulators [23], accelerometers [24] and many other methods are available for collecting sensory data. In practice, research is also carried out to analyze the EEG signal in the field of concentration of attention in mothers and children [25]. However, the activity data collected from actigraphic devices are considered as a gold standard for detecting different patterns such as sleep and awake states. The results derived from activity data are more precise, accurate, and reliable [26].
In light of these considerations, the present study presents a Support Vector Machine (SVM)-based approach for accurately identifying patients with ADHD among other clinical controls using activity data analysis. This approach makes use of statistical features related to the motor activity of patients, which are appropriately selected using Principal Component Analysis (PCA). The comparative analysis of the given SVM classifier with other state-of-the-art classifiers viz; Random Forest, C4.5 Decision tree, k-Nearest Neighbor, Naïve Bayes and LogitBoost, evidently reveals that the proposed PCA- and SVM-based activity features approach is an effective method for precise detection of ADHD patients among other patients suffering from different disorders.
This paper has been organized into various sections: Section 2 introduces the key contributions in this field. The proposed framework and study approaches are elaborated in Section 3. Section 4 addresses the classification models to be employed along with their hyper-parameter tuning. The results drawn are presented in Section 5. While Section 6 discusses the research findings and outlines the current work’s constraints and potential future directions. Finally, the brief conclusions reached from the work’s primary outcomes are explained in Section 7.

2. Literature Survey

The goal of this systematic literature review is to evaluate the most relevant research on ADHD Diagnosis utilizing activity-related data and its analysis using machine learning techniques. A variety of search engines and databases were used, including Google Scholar, PubMed, IEEE Explore, Science Direct and others. The keywords were input as a Boolean search string that included (‘ADHD’ or ‘attention deficit hyperactivity disorder’), (‘activity data’ or ‘actigraphy’ or ‘accelerometer’) and(‘machine learning’ or ‘deep learning’ or ‘artificial intelligence). The step-by-step search method and filtering of the article stage-wise are depicted in detail in Figure 1.
In the first step of the search, 389 items were discovered, whereas 202 of these articles were removed due to duplicate entries. A further modification was done to match the titles and abstracts of the retrieved publications. As a result, the set was reduced to 98 items at this stage by discarding 89 articles that did not fulfill the targeted objectives. However, this collection also included publications that employed a variety of approaches to diagnose ADHD, including PPG, Multimodal, MRI, MEG, and others. However, the use of activity data for identification is a topic of concern. As a result, the other methods were abandoned, and ten publications based on activity data were scanned. Indeed, upon further analysis, five were deemed to be suitable and were chosen as the final group for literature analysis. Table 1 shows a detailed comparison of these articles based on different parameters.
Indeed, the literature review has certain inadequacies that lead to the current investigation through this paper. Some of the few drawbacks that are of concern are:
  • Only one study out of five has utilized an adult dataset to diagnose ADHD.
  • The majority of the studies relied on private datasets, which necessitated a significant amount of effort and time for data gathering and processing.
  • On the activity datasets, only five studies have used machine learning approaches.
  • The machine learning models applied to the activity dataset even did not provide many reliable and precise results.
As a result of these limitations, this study aims to improve the precision with which activity data can be utilized to distinguish ADHD patients from other clinical controls. In addition, there are relatively few publicly available datasets in this domain, and via this, a new publicly available dataset has been examined.

3. Methods and Materials

This section provides an overview of the proposed framework for the identification of ADHD among clinical controls. Figure 2 illustrates the proposed framework, and based on the dataset, this paradigm is useful for classifying patients with ADHD or Non-ADHD. In the first stage, data is gathered from varied sources, which consist of details of patients containing information regarding their background and medical history. Moreover, it takes as input the results of computerized tests and the patient’s activity log using wearable devices. Then, the relevant data pertinent to the goals are chosen. Following the selection of necessary data, it is processed through statistical feature extraction and selection, using principal component analysis. Afterwards, the analysis is performed using several classification algorithms to precisely choose an optimal algorithm based on various performance metrics for accurate identification of ADHD patients.

3.1. Data Acquisition

The present work makes use of a publicly available ‘Hyperaktiv’ dataset [1] for accurate classification of ADHD and non-ADHD patients. It comprises health, activity, and heart rate data from adult ADHD patients and other clinical controls. It is accessible at https://osf.io/3agwr (accessed on 2 April 2022) and is licensed under Creative Commons Attribution-Noncommercial 4.0 International (CC BY-NC 4.0) [1]. For acquisition of this dataset,103 patients are enrolled out of which 51 patients have been diagnosed with ADHD and 52 with other disorders termed as clinical controls. The information gathered includes recordings of motor activity, heart rate, results of a computerized test as well as multiple diagnostic and clinical assessments. The activity log of one of the patients diagnosed with ADHD is depicted in Figure 3, which has been analyzed over a span of 24 h. The movement activity is measured in the Inertial Measurement Unit (IMU) using a wrist-worn actigraph device (Actiwatch, Cambridge Neurotechnology Ltd., England, model AW4). It consists of a piezoelectric accelerometer configured to record the integration of movement strength, amount, and duration in the x, y, and z-axes movements greater than 0.05 g at a sampling frequency of 32 Hz [1,30]. It is clear from the below figure that patients diagnosed with ADHD are more hyperactive during early mornings and in the evenings. The activity variation has a pivotal role in the diagnosis of ADHD because of the time-dependent nature of ADHD hyperactivities [31].
In this dataset, all the patients are diagnosed by two experienced and qualified psychiatrists using the Mini-International Neuropsychiatric Interview (MINI Plus, version 5.0.0) [32]. The clinical assessments are done by an automated and digital neuropsychological response test commonly known as CPT-II (Conner’s Continuous Performance Test) [33], which is used to access problems with attention. Apart from activity data, the present dataset also contains ECG-based heart rate data, which have been recorded with a compact battery-powered chest-worn device that allows for mobility and extensive recording times. This type of heart rate data can be used to determine the Heart Rate Variability (HRV) [34], which is a representation of the time difference between successive heartbeats. In the present data, a total of 80 patients provided heart rate recordings, with 38 ADHD sufferers logging their heart rates for an average of 20.5 ± 3.9 h, and 42 clinical controls logging their heart rates for an average of 21 ± 4 h [1].
Furthermore, the feature’s file consists of pre-extracted attributes that are then used for training and testing of given machine learning models being employed in this work. The file consisting of patient information contains 32 different attributes for each of 103 patients. For visual representation and analysis, the software utilized is open source cloud computing environment Google Colaboratory utilizing Python 3.7.13 [35]. Figure 4a represents the count of the number of males and females in the study. Figure 4b represents the different age patterns of the patients involved in the study, where 1 = 17–29 yrs, 2 = 30–39 yrs, 3 = 40–49 yrs, and 4 = 50–67 yrs. Figure 4c,d represent the number of patients diagnosed with ADHD and other disorders, including attention deficit disorder (ADD). Figure 4e picturizes the output for the CPT-II test results for the Montgomery and Asberg Depression Rating Scale (MADRS) [36], which consists of 21 items that access the intensity of depression. Figure 4f represents the Wender Utah rating scale (WURS) [37], which evaluates the prevalence and severity of childhood ADHD symptoms.

3.2. Feature Extraction

In the present work, 788 statistical features have been employed which are extracted from the activity data of the patients. These features are listed in the features file of the Hyperaktiv dataset [1]. The features are extracted using the open-source python package tsfresh [38], which aids in the feature engineering of the time-series data of the activity dataset [39]. These features are associated with standard deviation, variance, skewness, kurtosis, root mean square value, entropy and other related metrics.
Since the number of the features present in the dataset file are cumbersome, so to get a brief overview some of these features are shown graphically using different types of plots. Figure 5a presents a violin plot of the absolute energy of the activity data. The standard deviation ranges are shown using a histogram in Figure 5b. While Figure 5c,d analyze the kernel density estimator for kurtosis and skewness. The values of autocorrelation and ricker’s continuous wavelet transform are represented in Figure 5e,f respectively. Lastly, Figure 5g,h represent the fast Fourier transform coefficient values and permutation entropy values using kernel density estimator and histogram plots, respectively.

3.3. Feature Processing

This step entails the processing of the relevant features related to the activity recordings. Due to the small dataset size, training and testing would not accurately reflect a machine learning model’s generalization capabilities and would unnecessarily diminish its predictive power. Therefore, Principal Component Analysis (PCA) has been employed in this work in order to find significant features having strong patterns for accurate identification of patients with ADHD. PCA is an unsupervised dimensionality reduction approach. In this approach, by lowering the variance, the strong patterns in the provided dataset can be extracted. It is a statistical technique that turns observations of correlated features into a set of linearly uncorrelated data through orthogonal transformation. In this way, the newly altered features that are discovered are called the principal components. The reduced principal component features are either less than or equal to the features initially passed as an input. The purpose of PCA is to keep as much information as feasible while reducing the number of variables in a data set [40]. The various steps involved in PCA are standardizing the range of variables, calculating the covariance matrix, and finally determining the eigen vectors and eigen values from the matrix [41]. Dimensionality reduction is accomplished in this dataset by selecting eigenvectors to account for a specific proportion of variance in the original data. The default variance is 0.95. The data is centered rather than being standardized. PCA is calculated using the covariance matrix instead of the correlation matrix. A maximum of five attributes are selected in the formation of modified attributes. Moreover, all the attributes are considered to be equally important and are retained in the converted ranked attributes.
To investigate the accuracy further, the dataset size is resampled. It creates a random subset of the dataset, with or without replacement. The entire sample needs to be fitted into the memory perfectly. It allows the calculation of standard errors and confidence intervals easily. The confidence intervals become more reliable as the sample size grows. However, it increases the risk of overfitting noise in the data. The consequences of this problem can be reduced by combining the resampling method with the cross-validation procedure, which has been employed in the current work.

4. Model Selection

4.1. Machine Learning Techniques and Hyperparameter Tuning

To categorize patients with ADHD and other clinical controls, six different machine learning algorithms are employed which include Naive Bayes (NB), k-Nearest Neighbor (kNN), Support Vector Machine (SVM), C 4.5 Decision Tree, Random Forest (RF), and LogitBoost. The eventual aim is to create algorithms with the highest classification accuracy while using the fewest possible features possible. To boost the performance of these models, essential hyperparameters must be identified and tuned to exactly fit these machine learning models using given data for creating reliable and accurate models. Moreover, these parameters assist in finding the right balance of bias and variance and hence preventing the model from overfitting and underfitting. The machine learning models and hyperparameters tuning are implemented using WEKA 3.8.6. The hyperparameter tuning for varied chosen algorithms is as follows:

4.1.1. Support Vector Machine (SVM)

SVM is a kernel-based machine learning model for classification and regression challenges. Due to its strong theoretical foundation and capacity to generalize, SVMs have become one of the most extensively used classification methods in recent years. SVM’s main goal is to separate classes in a training set by using a surface that maximizes the margin between them. It maximizes the generalization capabilities of the model [42]. This algorithm is typically used for linearly separable data, but it can also be used for non-linearly separable data using the kernel approach.
In this work, the type of SVM used is C-SVC, where C stands for the penalty parameter of the error term, and the cost value is taken as 1. It strikes a balance between the need for a smooth decision border and the accuracy with which the training points are identified. The batch size is indeed 100, and the cache is 40 MB. The termination criteria tolerance (eps) is 0.001. The type of kernel employed in this model is the radial basis function, and the degree of the kernel is set to three. The seed value is taken as 1 and the shrinking heuristics are configured to true.

4.1.2. Naive Bayes (NB)

The Naive Bayes classifier is a probabilistic classifier based on Bayes’ theorem, which states that each feature contributes equally to the target class. The NB classifier assumes that each feature is independent of the others and does not interact with them, implying that each feature contributes equally to the probability of a sample belonging to a specific class [43]. It is simple to use and compute and performs well on large datasets with many dimensions. This algorithm works best when associated features are removed from the model since they are voted twice in the model. It is noise-robust and ideal for real-time applications [43].
For this classifier, the preferred number of instances that are processed in one batch is 100. The ‘use kernel estimator’ is set to false, so normal distribution is used for numeric attributes. The ‘use supervised discretization’ value and ‘do not check capabilities’ are both initialized to false.

4.1.3. C 4.5 Decision Tree

C4.5 is based on the information gain ratio being evaluated by entropy. The test features are picked using the information gain ratio measure at each node in the tree. These measures are known as feature or attribute selection measures. The test feature for the current node is the attribute with the highest information gain ratio. The proportion of observations to total observations is known as the Information Gain Ratio (IGR) [44]. When a data set is large, the decision tree is pruned, which implies that extraneous branches that are insignificant in the overall computation are deleted.
In this classifier, the batch size and seed parameters are both set to 100 and 1, respectively. When the ‘collapse tree’ property is set to True, the sections of the tree that do not minimize training error are deleted. The value of the number of folds is taken as three, which defines the amount of data used for reduced error pruning where two folds are used for tree growth and one fold is utilized for pruning. While the confidence factor is set to 0.25, the ‘subtree raising’ is considered true when pruning. The minimum number of the object variable is set to two. The MDL correction is utilized when discovering splits on numeric attributes.

4.1.4. Random Forest (RF)

Random Forest is based on ensemble learning, which is a way to integrate multiple classifiers to solve complicated problems and enhance the effectiveness of the algorithm. It is a classifier that combines several decision trees on different subsets of a dataset and averages their results to increase the dataset’s predictive accuracy. The random forest collects forecasts from each tree and predicts the ultimate output based on the majority votes of projections, rather than depending on a single decision tree. The more the number of trees in the forest, the higher the accuracy and lower the risk of overfitting. It predicts output with good accuracy and runs quickly even with a huge dataset even if data values are missing [45].
In RF classifier, the bag size percentage, number of iterations, and batch size are all set to 100 by default. The maximum depth and number of iterations of the tree are both set to 0. To create the ensemble, there is a need for one execution slot. The “Store out of Bag Predictions” is tuned to false and the seed value is fixed to one.

4.1.5. k-Nearest Neighbor (kNN)

kNN is a case-based classification algorithm that keeps all of the training data. It is a simple but efficient classification approach that has been proved to be one of the most powerful techniques in text categorization. A model is created for kNN to improve its efficiency while keeping its classification accuracy. To classify a data record t, the k closest neighbors are obtained, forming a t neighborhood [46]. However, in order to apply kNN, the proper value for k is chosen, as the performance of classification is strongly dependent on this number. There are a few ways to figure out the k value, but one simple option is to run the algorithm numerous times with different inputs.
In kNN, the number of neighbors (k) is chosen to one. The batch size is one hundred. In the nearest neighbor search algorithm, the linear NN Search algorithm is employed in the analysis. When conducting cross-validation, the mean absolute error is utilized. The window size parameter is set to 0, implying that there is no limit on the number of training instances. The distance weighting and cross-validate are set to false.

4.1.6. Logit Boost (LBoost)

LogitBoost is a popular boosting type for categorizing binary and multi-class data. It is an additive tree regression method that reduces the logistic loss. The boosting algorithm for LogitBoost is deduced from the statistical approach in which the loss, the function model and the optimization strategy are all three essential components. This algorithm uses a binomial log-likelihood to modify the loss function linearly. In this, outliers and noise are less noticeable [47].
For this classifier, the Zmax response threshold is set to 3.0. All of the seed, shrinkage, pool size and number of threads variables are set to one. The chosen base classifier is the Decision Stump. The number of iterations is set to ten. The likelihood threshold is set equal to −1.79769313, while the weight threshold and batch size are adjusted at 100.

4.2. Performance Evaluation

The performance of an algorithm for a specific task is evaluated using different metrics. For this purpose, the confusion matrix is widely used, which is a four-quadrant table with rows and columns for presenting the classifier’s classification results. It aids in the compilation of performance evaluation measures. The four quadrants in the matrix consist of True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN). Various performance metrics derived from the confusion matrix, which are employed in the present work, include accuracy [48], sensitivity [48], specificity [48], F-measure [49] and AUC [50]. These metrics are illustrated as per the following details:
Accuracy = T P + T N T P + T N + F P + F N
F-measure =   2 Precision Recall Precision   +   Recall
Sensitivity = T P T P + F N
Specificity = T N T N + F P
AUC is a cumulative measure of performance that takes into account all possible classification thresholds. AUC is a two-dimensional region beneath the complete AUC curve from (0, 0) to (1, 1). The AUC value varies from 0 to 1. The AUC of a model whose predictions are 100 percent incorrect is 0.0, whereas the AUC of a model whose predictions are 100 percent correct is 1.0 [50].

5. Results

This section demonstrates the results of multiple classification algorithms to ensure the relevancy of the proposed methodological approach for the detection of ADHD. The different classification algorithms chosen for this work are C4.5, kNN, LBoost, NB, SVM, and RF. For all the algorithms, ten-fold cross-validation is used to analyze the result. The performance of the employed classification algorithms is presented in Table 2 in terms of accuracy, sensitivity, specificity, F-measure and AUC.
Furthermore, the comparison is drawn graphically for accuracy values among given ML algorithms in Figure 6, which also shows the superiority of SVM among other algorithms for accurate identification of ADHD patients, whereas Figure 7, Figure 8 and Figure 9 display the performance of given ML algorithms in terms of sensitivity, specificity and F-measure values of ADHD and non-ADHD classes. Figure 10 presents AUC curves for SVM, RF, C4.5, kNN, LB and NB classifiers.

6. Discussion

6.1. This Work

It is evident from Table 2 that the SVM classifier outperforms with an accuracy of 98.43% and the Naïve Bayes algorithm exhibits the lowest accuracy of 80.39% among other classification algorithms. The other performance metrics values such as F-Measure, sensitivity, specificity, and AUC for SVM are 98.42%, 98.33%, 98.56%, and 0.983, respectively. However, the evaluation results of RF and kNN are quite close to the performance of SVM. They both have showcased accuracies of nearly 97.25% and 97.65%, respectively. While the AUC for RF is the maximum among all the algorithms, which is 0.999. According to the findings of the evaluation, the NB and LBoost do not appear to be appropriate classifiers for identifying ADHD as they exhibit much lower values of performance metrics in comparison to the other models.
Figure 7, Figure 8 and Figure 9 pictorially represent how well these metrics performed on various models to differentiate between the two classes of classifying the ADHD patients from clinical controls. As expected, SVM classifier has performed well for this task in terms of sensitivity, specificity and F-measure having class ADHD values of 100%, 100% and 98.54%, respectively.
As per Figure 10 and Table 2, SVM exhibits a significantly high value of AUC along with Random Forest having values of 0.983 and 0.999, respectively. On the other hand, NB classifier shows a poor AUC of 0.889 only. Therefore, the thorough analysis of given ML algorithms reveals that SVM algorithm performs better amongst other algorithms with maximum values of accuracy, sensitivity, specificity, F-measure and an optimal value of AUC, which makes it a suitable choice for accurate identification of ADHD patients among other clinical controls.

6.2. Contributions and Limitations

This research work utilizing activity data provides an efficient machine learning model, which can efficiently differentiate between ADHD and clinical controls with an accuracy of 98.43% and AUC of 0.983. However, this study also poses some limitations. The key drawback is the scarcity of data samples due to clinical data collection difficulties. Another issue is that the subtype of ADHD was not taken into account due to the diversity of the ADHD population [11]. Additionally, new strategies for detecting ADHD, such as transfer learning, can be applied [51]. Deep learning algorithms can also be used to distinguish ADHD patients from clinical controls using EEG data [52]. Moreover, indicators such as pupil size can be used to determine whether or not a patient has ADHD [16]. In connection to ADHD, the gender and age difference factors could also be used [12]. Furthermore, the relationship between heart rates and ADHD could also be investigated, and the validity of computerized tests used to diagnose ADHD could be tested.

6.3. Future Scope

Future research plans include looking at ADHD in context with the patients’ heart rates. The research should be expanded to look into the causes and effects of additional conditions such as anxiety, bipolar disorder, unipolar disorder, and depression on ADHD. Additionally, the validity of the computerized tests that are currently used to assess the prevalence of ADHD must be examined.

7. Conclusions

In this study, the optimal autonomic model for the diagnosis of ADHD is determined using six machine learning algorithms. The phenomenal performance of classifiers in this work is attributed to the computed feature selection approach and most importantly, the identification and combination of complementary features using PCA. The SVM model proposed is the most accurate classifier, classifying at 98.43% accuracy with high sensitivity, specificity and F-measure. The overall analysis of the proposed work clearly reveals that PCA based SVM approach is a highly suited method for accurate identification of ADHD patients. The overall findings provide new information regarding accurate machine learning algorithms, which can reduce misinterpretation and can be used to assess treatment effectiveness.

Author Contributions

Conceptualization, A.K.; methodology, A.K.; software, A.K; validation, A.K.; formal analysis, A.K.; investigation, A.K.; resources, A.K.; data curation, A.K; writing—original draft preparation, A.K.; writing—review and editing, A.K and K.S.K.; visualization, A.K.; supervision, K.S.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data is available at https://osf.io/3agwr (accessed on 2 April 2022).

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

ADHDAttention Deficit Hyperactivity Disorder
ADDAttention Deficit Disorder
AUCArea Under the ROC Curve
HRVHeart Rate Variability
IGRInformation Gain Ratio
IMUInertial Measurement Unit
kNNk-Nearest Neighbor
LBoostLogit Boost
MLMachine Learning
NBNaïve Bayes
PCAPrincipal Component Analysis
RFRandom Forest
SVMSupport Vector Machine
WURSWender Utah Rating Scale
MADRSMontgomery and Asberg Depression Rating Scale

References

  1. Hicks, S.A.; Stautland, A.; Fasmer, O.B.; Forland, W.; Hammer, H.L.; Halvorsen, P.; Mjeldheim, K.; Oedegaard, K.J.; Osnes, B.; Syrstad, V.E.G.; et al. Hyperaktiv: An Activity Dataset from Adult Patients with Attention-Deficit/Hyperactivity Disorder (ADHD). In Proceedings of the 12th ACM Multimedia Systems Conference (MMSys ’21), Istanbul, Turkey, 28 September–1 October 2021; Volume 1, pp. 314–319. [Google Scholar]
  2. Brahmbhatt, K.; Hity, D.M.; Hah, M.; Han, J.; Angkustsiri, K.; Schweitzer, J. Diagnosis and Treatment of Adhd During Adolescence in the Primary Care Setting: Review and Future Directions. J. Adolesc. Health 2016, 59, 135–143. [Google Scholar] [CrossRef] [Green Version]
  3. Seidman, L.J.; Valera, E.M.; Makris, N. Structural brain imaging of attention-deficit/hyperactivity disorder. Biol. Psychiatry 2005, 57, 1263–1272. [Google Scholar] [CrossRef] [PubMed]
  4. Bassett, D.S.; Meyer-Lindenberg, A.; Achard, S.; Duke, T.; Bullmore, E. Adaptive reconfiguration of fractal small-world human brain functional networks. Proc. Natl. Acad. Sci. USA 2006, 103, 19518–19523. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Shaw, P.; Stringaris, A.; Nigg, J.; Leibenluft, E. Emotion dysregulation in attention deficit hyperactivity disorder. American J. Psychiatry 2014, 171, 276–293. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Ellison, A.T.; Johnson, B.B.; Harrell, M.N. Attention-Deficit/Hyperactivity Disorders (ADHD); Elsevier: Amsterdam, The Netherlands, 2016. [Google Scholar]
  7. Sayal, K.; Prasad, V.; Daley, D.; Ford, T.; Coghill, D. ADHD in children and young people: Prevalence, care pathways, and service provision. Lancet Psychiatry 2018, 5, 175–186. [Google Scholar] [CrossRef]
  8. Syrstad, V.E.G.; Oedagaard, K.J.; Fasmer, O.B.; Halmay, A.; Haavik, J.; Dilsaver, S.; Gjestad, R. Cyclothymic temperament: Associations with ADHD, other psychopathology, and medical morbidity in the general population. J. Affect. Disord. 2019, 260, 440–447. [Google Scholar] [CrossRef]
  9. Tistarelli, N.; Fagnani, C.; Troianiello, M.; Stazi, M.A.; Adriani, W. The nature and nurture of ADHD and its comorbidities: A narrative review on twin studies. Neurosci. Biobehav. Rev. 2020, 109, 63–77. [Google Scholar] [CrossRef]
  10. Barkley, A.R. Attention Deficit Hyperactivity Disorder: A Handbook for Diagnosis and Treatment; Guilford Press: New York, NY, USA, 2015; Volume 20. [Google Scholar]
  11. Swanson, J.M.; Kinsbourne, M.; Nigg, J.; Lamphear, B.; Stefanatos, G.A.; Volkow, N.; Taylor, E.; Casey, B.J.; Castellanos, F.X.; Wadhwa, P.D. Etiologic Subtypes of Attention-Deficit/Hyperactivity Disorder: Brain Imaging, Molecular Genetic and Environmental Factors and the Dopamine Hypothesis. Neuropsychol. Rev. 2007, 17, 39–59. [Google Scholar] [CrossRef]
  12. Rantekkar, U.P.; Reirsen, A.M.; Todoron, A.A.; Todd, R.D. Sex and age differences in Attention-Deficit/Hyperactivity Disorder symptoms and diagnoses: Implications for DSM-V and ICD-11. J. Am. Acad. Child Adolesc. Psychiatry 2011, 49, 217–228. [Google Scholar]
  13. Hinshaw, S.P.; Owens, E.B.; Zalecki, C.; Huggins, S.P.; Nevado, A.J.M.; Schrodek, E.; Swanson, E.N. Prospective follow-up of girls with attention-deficit/ hyperactivity disorder into early adulthood: Continuing impairment includes elevated risk for suicide attempts and self-injury. J. Consult. Clin. Psychol. 2012, 80, 1041–1051. [Google Scholar] [CrossRef] [Green Version]
  14. Hinshaw, S.P.; Arnold, L.E. ADHD, Multimodal Treatment, and Longitudinal Outcome: Evidence, Paradox, and Challenge. Wiley Interdiscip. Rev. Cogn. Sci. 2015, 6, 39–52. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Volkow, N.; Swanson, J. Adult attention deficit–hyperactivity disorder. New Engl. J. Med. 2013, 369, 1935–1944. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Das, W.; Khanna, S. A Robust Machine Learning Based Framework for the Automated Detection of ADHD Using Pupillometric Biomarkers and Time Series Analysis. Sci. Rep. 2021, 11, 16370. [Google Scholar] [CrossRef] [PubMed]
  17. Garcia-Ceja, E.; Riegler, M.; Nordgreen, T.; Jakobsen, P.; Oedgaard, K.J.; Tørresen, J. Mental health monitoring with multimodal sensing and machine learning: A survey. Pervasive Mob. Comput. 2018, 51, 1–26. [Google Scholar] [CrossRef]
  18. Koh, J.E.W.; Ooi, C.P.; Vicnesh, J.; Oh, S.L. Automated classification of attention deficit hyperactivity disorder and conduct disorder using entropy features with ECG signals. Comput. Biol. Med. 2022, 140, 105–120. [Google Scholar] [CrossRef]
  19. Ahmadi, A.; Kashefi, M.; Shahrokhi, S.; Nazari, M.A. Computer aided diagnosis system using deep convolutional neural networks for ADHD subtypes. Biomed. Signal Processing Control 2021, 63, 102–227. [Google Scholar] [CrossRef]
  20. Öztoprak, H.; Toucan, M.; Alp, Y.K.; Arıkan, O.; Doğutepe, E.; Karakaş, S. Machine-based classification of ADHD and non-ADHD participants using time/frequency features of event-related neuroelectric activity. Clin. Neurophysiol. 2017, 128, 2400–2410. [Google Scholar] [CrossRef]
  21. Bohland, J.W.; Saperstein, S.; Pereira, F.; Rapin, J.; Grady, L. Network, anatomical, and non-imaging measures for the prediction of ADHD diagnosis in individual subjects. Front. Syst. Neurosci. 2012, 6, 1–36. [Google Scholar] [CrossRef] [Green Version]
  22. Tang, Y.; Sun, J.; Wang, C.; Zhong, Y.; Jiang, A.; Liu, G.; Liu, X. ADHD classification using auto-encoding neural network and binary hypothesis testing. Artif. Intell. Med. 2022, 123, 102–109. [Google Scholar] [CrossRef]
  23. Heller, M.D.; Roots, K.; Srivastava, S.; Schumann, J.; Srivastava, J.; Hale, T.S. A Machine Learning-Based Analysis of Game Data for Attention Deficit Hyperactivity Disorder Assessment. Games Health J. 2013, 2, 291–298. [Google Scholar] [CrossRef]
  24. O’Mahony, N.; Florentino-Liano, B.; Carballo, J.J.; Baca-García, E.; Rodríguez, A.A. Objective diagnosis of ADHD using IMUs. Med. Eng. Phys. 2014, 36, 922–926. [Google Scholar] [CrossRef] [PubMed]
  25. Zolubak, M.; Paszkiel, S. EEG Analysis and Neurofeedback Therapy Concentration Problems in Mother and Child. In Advances in Intelligent Systems and Computing (AISC); Springer: Charm, Switzerland, 2021; Volume 362, pp. 79–93. [Google Scholar]
  26. Martin, J.L.; Hakim, A.D. Wrist actigraphy. Chest 2011, 139, 1514–1527. [Google Scholar] [CrossRef] [PubMed]
  27. Muñoz-Organero, M.; Powell, L.; Heller, B.; Harpin, V.; Parker, J. Automatic extraction and detection of characteristic movement patterns in children with ADHD based on a convolutional neural network (CNN) and acceleration images. Sensors 2018, 18, 3924. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Faedda, G.L.; Ohashi, K.; Hernandez, N.; McGreenery, C.E.; Grant, M.C.; Baroni, A.; Polcari, A.; Teicher, M.H. Actigraph measures discriminate pediatric bipolar disorder from attention-deficit/hyperactivity disorder and typically developing controls. J. Child Psychol. Psychiatry Allied Discip. 2016, 57, 706–716. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Amado-Caballero, P.; Casaseca-da-la-Higuera, P.; Alberola-Lopez, S.; Andres-de-Llano, J.M.; Villalobos, J.A.L.; Garmendia-Leiza, J.R.; Alberola-Lopez, C. Objective ADHD Diagnosis Using Convolutional Neural Networks over Daily-Life Activity Records. IEEE J. Biomed. Health Inform. 2020, 24, 2690–2700. [Google Scholar] [CrossRef] [Green Version]
  30. Fasmer, O.B.; Fasmer, E.E.; Mjeldheim, K.; Forland, W.; Syrstad, V.E.G.; Jakobsen, P.; Berle, J.Ø.; Henriksen, T.E.G.; Sepasdar, Z.; Hauge, E.R.; et al. Diurnal variation of motor activity in adult ADHD patients analyzed with methods from graph theory. PLoS ONE 2020, 15, e0241991. [Google Scholar] [CrossRef]
  31. Usami, M.; Okada, T.; Sasayama, D.; Iwadare, Y.; Watanable, K.; Ushijima, H.; Kodaira, M.; Sugiyama, N.; Sawa, T.; Saito, K. What time periods of the day are concerning for parents of children with attention deficit hyperactivity disorder? PLoS ONE 2013, 8, e79806. [Google Scholar] [CrossRef] [Green Version]
  32. Álvarez, R.F.P.; Batra, C.; Carpentier, P.J.; Carrithers, S.; Crunelle, C.L.; Demetrovics, Z.; Dom, G.; Faraone, S.V.; Franck, J.; Johnson, B.; et al. Validity of the ADHD module of the Mini International Neuropsychiatric Interview PLUS for screening of adult ADHD in treatment seeking substance use disorder patients: ADHD screening with MINI-Plus. Rev. Psiquiatr. Y Salud Ment. 2020; in press. [Google Scholar] [CrossRef]
  33. Conners, C.K.; Sitarenios, G. Conners’ Continuous Performance Test (CPT), Encyclopedia of Clinical Neuropsychology; Springer: New York, NY, USA, 2011. [Google Scholar]
  34. Brage, S.; Brage, N.; Franks, P.W.; Ekelund, U.; Wareham, N.J. Reliability and validity of the combined heart rate and movement sensor Actiheart. Eur. J. Clin. Nutr. 2005, 59, 561–570. [Google Scholar] [CrossRef] [Green Version]
  35. Google Colaboratory, Welcome to Colaboratory. Available online: https://colab.research.google.com (accessed on 20 May 2022).
  36. Hawley, C.J.; Gale, T.M.; Sivakumaran, T. Defining remission by cut-off score on the MADRS: Selecting the optimal value. J. Affect. Disord. 2002, 72, 177–184. [Google Scholar] [CrossRef]
  37. Brevik, E.J.; Lundervold, A.J.; Haavik, J.; Posserud, M.B. Validity and accuracy of the Adult Attention-Deficit/Hyperactivity Disorder (ADHD) Self-Report Scale (ASRS) and the Wender Utah Rating Scale (WURS) symptom checklists in discriminating between adults with and without ADHD. Brain Behav. 2020, 10, e01605. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Christ, M.; Braun, N.; Neuffer, J.; Kempa-Liehr, A.W. Time Series Feature Extraction on basis of Scalable Hypothesis tests (tsfresh—A Python package). Neurocomputing 2018, 307, 72–77. [Google Scholar] [CrossRef]
  39. Heaton, J. An empirical analysis of feature engineering for predictive modeling. In Proceedings of the SoutheastCon2016, Norfolk, VA, USA, 30 March–3 April 2016; pp. 1–6. [Google Scholar]
  40. Jolliffe, I.T.; Cadima, J. Principal component analysis: A review and recent developments. Philos. Trans. R. Soc. A 2016, 374, 20150202. [Google Scholar] [CrossRef] [PubMed]
  41. Zhang, Z.; Castelló, A. Principal components analysis in clinical studies. Ann. Transl. Med. 2017, 5, 3–9. [Google Scholar] [CrossRef] [Green Version]
  42. Cervantes, J.; Garcia-Lamont, F.; Rodriguez-Mazahua, L.; Lopez, A. A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing 2020, 408, 189–215. [Google Scholar] [CrossRef]
  43. Kaviani, P.; Dhotre, S. Short Survey on Naïve Bayes Algorithm. Int. J. Adv. Res. Comput. Sci. Manag. 2017, 4, 607–611. [Google Scholar]
  44. Patel, H.; Prajapati, P. Study and Analysis of Decision Tree based classification algorithms. Int. J. Comput. Sci. Eng. 2018, 6, 74–78. [Google Scholar] [CrossRef]
  45. Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
  46. Gou, G.; Wang, H.; Bell, D.A.; Bi, Y.; Greer, K. kNN Model Based Approach in Classification. In On the Move to Meaningful Internet Systems, 2003: CoopIS, DOA and ODBASE.OTM 2003. Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2003; Volume 2888. [Google Scholar]
  47. Sun, P.; Reid, M.; Zhou, J. An improved multiclass LogitBoost using adaptive one-vs-one. Mach. Learn. 2014, 97, 295–326. [Google Scholar] [CrossRef]
  48. Baratloo, A.; Hosseini, M.; Negida, A.; Ashal, G.A.I. Part 1: Simple Definition and Calculation of Accuracy, Sensitivity and Specificity. 2015. Available online: https://www.jemerg.com (accessed on 21 April 2022).
  49. Sasaki, Y. The truth of the F-measure. Teach Tutor Mater 2007, 1, 1–5. [Google Scholar]
  50. Hajian-Tilaki, K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Casp. J. Intern. Med. 2013, 4, 627–635. [Google Scholar]
  51. Lin, J.; Chang, Y.; Chang, C.; Ko, L. Applications of Transfer Learning in Field verification for children in Attention Deficit Hyperactivity Disorder. In Proceedings of the International Conference in Fuzzy Theory and its Applications, Taitung, Taiwan, 5–8 October 2021. [Google Scholar]
  52. Vahid, A.; Bluschke, A.; Roessner, V.; Stober, S.; Bete, C. Deep Learning based on Event-Related EEG differentiates children with ADHD from Healthy Controls. J. Clin. Med. 2021, 8, 1055. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Flow diagram for filtering of articles.
Figure 1. Flow diagram for filtering of articles.
Brainsci 12 00831 g001
Figure 2. The proposed framework.
Figure 2. The proposed framework.
Brainsci 12 00831 g002
Figure 3. Activity variation in ADHD patient with respect to time.
Figure 3. Activity variation in ADHD patient with respect to time.
Brainsci 12 00831 g003
Figure 4. Patient information in terms of different parameters: (a) males vs. females; (b) age group of patients; (c) ADD vs. non-ADD; (d) ADHD vs. non-ADHD; (e) MADRS values; and (f) WURS values.
Figure 4. Patient information in terms of different parameters: (a) males vs. females; (b) age group of patients; (c) ADD vs. non-ADD; (d) ADHD vs. non-ADHD; (e) MADRS values; and (f) WURS values.
Brainsci 12 00831 g004
Figure 5. Visual representation of various activity related features: (a) absolute energy; (b) standard deviation; (c) kurtosis; (d) skewness; (e) autocorrelation values; (f) continuous wavelet transform (g) fast Fourier transform; and (h) permutation entropy.
Figure 5. Visual representation of various activity related features: (a) absolute energy; (b) standard deviation; (c) kurtosis; (d) skewness; (e) autocorrelation values; (f) continuous wavelet transform (g) fast Fourier transform; and (h) permutation entropy.
Brainsci 12 00831 g005
Figure 6. Performance comparison of ML algorithms in terms of accuracy.
Figure 6. Performance comparison of ML algorithms in terms of accuracy.
Brainsci 12 00831 g006
Figure 7. Performance of ML algorithms in terms of sensitivity of ADHD and non-ADHD classes.
Figure 7. Performance of ML algorithms in terms of sensitivity of ADHD and non-ADHD classes.
Brainsci 12 00831 g007
Figure 8. Performance of ML algorithms in terms of sensitivity of ADHD and non-ADHD classes.
Figure 8. Performance of ML algorithms in terms of sensitivity of ADHD and non-ADHD classes.
Brainsci 12 00831 g008
Figure 9. Performance of ML algorithms in terms of sensitivity of ADHD and non-ADHD classes.
Figure 9. Performance of ML algorithms in terms of sensitivity of ADHD and non-ADHD classes.
Brainsci 12 00831 g009
Figure 10. Area under the ROC curve (AUC) for different models.
Figure 10. Area under the ROC curve (AUC) for different models.
Brainsci 12 00831 g010
Table 1. Description of studies that used actigraphy and accelerometer data.
Table 1. Description of studies that used actigraphy and accelerometer data.
S.NoReferenceYearDatasetAge GroupPublic/PrivateMethodFeaturesModelValidation ApproachHighest Accuracy
1Munoz-Organero et al. [27]201822 school children with ADHD = 11,
Paired Controls = 11
6–15 yearsPrivateTwo trial axial accelerometers: one on the wrist of the dominant arm and the other on the axle of the dominant leg2D acceleration imagesDeep learning4-fold cross-validationCNN 87.5% with wrist sensor and 93.8% with axle sensor
2Faedda et al. [28]2016155 children with
ADHD = 44
ADHD + Depression = 21
Bipolar = 48
Controls = 42
5–18 yearsPrivateBelt worn actigraphs28 metricsMachine Learning4-Fold cross validationSVM 83.1%
3Amado-Caballeroat et al. [29]2020148 children with
ADHD = 73
Normal = 75
6–15 yearsPrivateWrist Worn ActiGraph GT3xEnd-to-EndDeep Learning10 fold cross validationCNN 98.6%
4O’Mahony et al. [24]201443 children with
ADHD = 24
Normal = 19
6–11 yearsPrivateTwo IMU one at the waist and the other at the ankle of the dominant legInertial measurement UnitsMachine LearningLeave one out cross-validationSVM 95.1%
5Hicks et al. [1]2021103 patients
with ADHD = 51
Normal = 52
17–67 yearsPublicWrist-worn Actigraph deviceFeature extraction with tsfreshMachine Learning10 fold cross-validationRandom Forest gives 72%
Table 2. Performance evaluation results of different classification algorithms.
Table 2. Performance evaluation results of different classification algorithms.
S.NoModelAccuracy
(%)
Sensitivity
(%)
Specificity
(%)
F-Measure
(%)
AUC
1C4.595.2995.2895.2895.280.973
2kNN97.6597.6497.6497.640.975
3LBoost89.0289.0388.9688.990.941
4NB80.3979.8681.2180.020.889
5SVM98.4398.3398.5698.420.983
6RF97.2597.2797.2397.250.999
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kaur, A.; Kahlon, K.S. Accurate Identification of ADHD among Adults Using Real-Time Activity Data. Brain Sci. 2022, 12, 831. https://doi.org/10.3390/brainsci12070831

AMA Style

Kaur A, Kahlon KS. Accurate Identification of ADHD among Adults Using Real-Time Activity Data. Brain Sciences. 2022; 12(7):831. https://doi.org/10.3390/brainsci12070831

Chicago/Turabian Style

Kaur, Amandeep, and Karanjeet Singh Kahlon. 2022. "Accurate Identification of ADHD among Adults Using Real-Time Activity Data" Brain Sciences 12, no. 7: 831. https://doi.org/10.3390/brainsci12070831

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop