Internet Gaming Disorder Clustering Based on Personality Traits in Adolescents, and Its Relation with Comorbid Psychological Symptoms
Atención e Investigación en Socioadicciones (AIS), Mental Health and Addictions Network, Generalitat de Catalunya (XHUB), 08014 Barcelona, Spain
Department of Psychiatry & Forensic Medicine, Institute of Neurosciences, Universitat Autònoma de Barcelona, Bellaterra, 08193 Barcelona, Spain
Research and Development Unit, Parc Sanitari Sant Joan de Déu, Fundació Sant Joan de Déu, Sant Boi de Llobregat, 08830 Barcelona, Spain
Department of Statistics and Operations Research, Polytechnic University of Catalonia-BarcelonaTech, 08028 Barcelona, Spain
Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Instituto de Salud Carlos III, 28029 Madrid, Spain
Pathological Gambling Unit, Department of Psychiatry, Bellvitge University Hospital-IDIBELL, 08907 Barcelona, Spain
Ciber Fisiopatología Obesidad y Nutrición (CIBERObn), Instituto de Salud Carlos III, 28029 Madrid, Spain
Department of Clinical Sciences, School of Medicine and Health Sciences, University of Barcelona, 08907 Barcelona, Spain
Nursing Department of Mental Health, Public Health, Maternal and Child Health, Nursing School of the University of Barcelona, 08907 Barcelona, Spain
Authors to whom correspondence should be addressed.
These authors contributed equally to this work and share first authorship.
Int. J. Environ. Res. Public Health 2020, 17(5), 1516; https://doi.org/10.3390/ijerph17051516
Received: 20 December 2019 / Revised: 20 February 2020 / Accepted: 23 February 2020 / Published: 26 February 2020
(This article belongs to the Special Issue Internet and Smartphone Use-Related Addiction Health Problems: Treatment, Education and Research)
In recent years, the evidence regarding Internet Gaming Disorder (IGD) suggests that some personality traits are important risk factors for developing this problem. The heterogeneity involved in problematic online gaming and differences found in the literature regarding the comorbid psychopathology associated with the problem could be explained through different types of gamers. Clustering analysis can allow organization of a collection of personality traits into clusters based on similarity. The objectives of this study were: (1) to obtain an empirical classification of IGD patients according to personality variables and (2) to describe the resultant groups in terms of clinical and sociodemographic variables. The sample included 66 IGD adolescent patients who were consecutive referrals at a mental health center in Barcelona, Spain. A Gaussian mixture model cluster analysis was used in order to classify the subjects based on their personality. Two clusters based on personality traits were detected: type I “higher comorbid symptoms” (n = 24), and type II “lower comorbid symptoms” (n = 42). The type I included higher scores in introversive, inhibited, doleful, unruly, forceful, oppositional, self-demeaning and borderline tendency traits, and lower scores in histrionic, egotistic and conforming traits. The type I obtained higher scores on all the Symptom Check List-90 items-Revised, all the State-Trait Anxiety Index scales, and on the DSM-5 IGD criteria. Differences in personality can be useful in determining clusters with different types of dysfunctionality.
According to the ICD-11 (International Classification of Diseases) , Internet Gaming Disorder (IGD) is defined as “a pattern of persistent or recurrent gaming behavior (‘digital gaming’ or ‘video-gaming’), which may be online (i.e., over the Internet) or offline, manifested by: (1) impaired control over gaming (e.g., onset, frequency, intensity, duration, termination, context); (2) increasing priority given to gaming to the extent that gaming takes precedence over other life interests and daily activities; and (3) continuation or escalation of gaming despite the occurrence of negative consequences. The behavior pattern is of sufficient severity to result in significant impairment in personal, family, social, educational, occupational or other important areas of functioning. The pattern of gaming behavior may be either continuous or, on the other hand, episodic and recurrent. The gaming behavior and other features are normally evident over a period of at least 12 months for a diagnosis to be assigned, although the required duration may be shortened if all diagnostic requirements are met and symptoms are severe”.
Nowadays, behavioral addictions, including IGD, are increasingly being documented worldwide . The current versions of the official diagnostic classification manuals have included addictions without substances in the behavioral addictions category. At the moment, only Gambling Disorder has been included in this category, and although IGD seems to share many factors with this disorder such as the negative reinforcement as a maintaining variable in the long-term maintenance of the behavior, or the use of positive reinforcement as a developing mechanism at the beginning of the problem [3,4], the DSM-5 work group decided to include IGD in Section III of the diagnostic manual DSM-5  as a condition that requires further study.
In recent years, numerous factors involved in the etiology of problematic online gaming and of IGD have been identified. Although more research is needed, the evidence suggests that some personality traits are important risk factors for developing this problem [6,7]. Personality traits are different across individuals, influence their behavior, reflect people’s characteristic patterns of thoughts, attitudes, emotions, and behaviors , and they have relatively high stability over a lifetime, even though some authors have found changes after cognitive behavioral therapies . Specifically, researchers have linked patients with IGD to personality traits common in other addictive disorders.
Regarding the Big Five Model of personality, some of the factors of the model seem to have relevance in the addiction process . Among them, high neuroticism is the most commonly present in IGD, and has been interpreted as a way to use online gaming to overcome negative life effects  or to modify negative emotions [12,13]. The results regarding the other traits of the model and male gamers are various, with low agreeableness , low conscientiousness and low extraversion  being the most consistent findings. These domains reflect an impaired offline social cooperation with their peers, low self-discipline, and low motivation to maintain positive interpersonal relations.
Taking other personality models as a reference, some authors have found that low sociability , low openness to experience , and the combination of low self-directedness and cooperativeness  were correlated with problematic gaming and IGD. Finally, low self-directness  has been found as a predictor of IGD. This trait reflects the lack of regulation and adaptation of one’s own behavior in order to achieve personally chosen goals and values, and the tendency to be laid back.
All these traits combined can predispose people to avoid social interactions and new activities. This diminished social participation in the “real world” and the need to establish some kind of social interactions with others could lead gamers to find online relationships, which are commonly more distant and superficial, with people with similar interests. Thus, they expand their social network  and find themselves in a safe environment when they are in the online world.
Clustering can be defined as the statistical methods that allow us to organize a collection of data points into clusters based on similarity . These methods are part of what we call unsupervised learning and they are used in a wide range of research fields such as psychology, biology and market research [21,22,23]. Clustering methods have been used to classify individuals based upon behavioral patterns , personality traits  and severity of mental disorder .
Several studies have focused on classifying the personality traits of patients with behavioral addictions using clusters, and their relationships with psychiatric comorbidity and sociodemographic characteristics. The study of personality profiles for gambling disorder [25,27,28] and compulsive buying  shows that there is a heterogeneity in the personality patterns of the affected people, and these different profiles are associated with differences in the number and severity of other psychological comorbid symptoms. These results suggest that the experience with problematic behavior varies between patients, and that the processing of the negative consequences derived from it may affect them in different ways.
Focusing on the use of videogames, Billieux et al.  classified a group of 1057 general population online gamers, playing a massive multiplayer online roleplaying-game, into five reliable clusters (three problematic and two non-problematic clusters) according to several psychological risk factors (impulsivity, motives to play, self-esteem) and potential consequences of playing (addiction symptoms, positive and negative affect). Members of the two non-problematic clusters were defined by low impulsivity and high and low levels of self-esteem, respectively. The motivations to play of these two clusters vary between non-fundamental, to motivations related to social exchange and roleplaying.
With respect to the three problematic clusters, the first one was composed of gamers with poor self-esteem and high impulsivity (but low sensation seeking), and by low achievement and high escapism motives. The second problematic cluster included high self-esteem, high impulsivity, and motivations regarding achievements in the game. The last cluster comprise gamers with high self-esteem, high impulsivity, and motives related to roleplaying, achievements and escapism. This study demonstrated the existence of distinct subtypes of problematic online gamers, emphasizing the high heterogeneity and the wide range of psychological factors involved in the problem.
Therefore, it could be possible that, similar to what happens in other behavioral addictions, the different personality types of the gamers have a role not only in the development or perpetuation of the problem, but also in the comorbid psychopathology associated to the disorder. In that sense, several authors have analyzed the different personality and psychopathological features among IGD patients [31,32], finding fewer functional personality traits and higher psychopathological scores compared with a normative population.
Nevertheless, the relationship between personality and psychopathology in IGD remains unclear, and the associations between comorbidity and IGD in adolescents and young adults have shown inconsistent results with depression, anxiety, ADHD or hyperactivity, social phobia/anxiety, and obsessive-compulsive disorder, finding among different authors full, partial and no associations with these symptoms . It is possible that these inconsistencies could be related to the analysis as a whole of different personality types among video game players.
In other words, cluster analyses can help in the conceptualization of patients consulting for IGD, and these profiles of similarities and differences among individuals can contribute to clarifying some of the results found in previous research regarding clinical profiles, and can help to improve the clinical treatments. Considering this, the objectives of this study are as follows: (1) to obtain an empirical classification of IGD patients according to personality variables and (2) to describe the resultant groups in terms of clinical and sociodemographic variables.
2. Materials and Methods
The sample included 66 IGD patients who were consecutive referrals for assessment and outpatient treatment at the Behavioral Addiction Unit in the mental health center AIS-PRO JUVENTUD (Care and Research in Behavioral Addiction) (AIS), located in Barcelona, Spain.
The required sample size was calculated based on the standard deviations of the questionnaire Millon Adolescent Personality Inventory (MACI). Thus, by setting an alpha risk of 0.05 and a beta risk of 0.20 in a two-sided test with a 10% estimated dropout rate, we required a sample size of 59 individuals to detect a minimum expected difference between groups of 6 units. We therefore decided to recruit 66 patients.
The exclusion criteria for being included in the analyses were: (1) had neurological disorders or primary psychiatric conditions that could affect cognitive function (assessed through semi-structured, face-to-face, clinical interview, (2) had a head injury with loss of consciousness for more than 2 min or a learning disorder, (3) used psychostimulants or drugs that could interfere with the assessment, (4) were older than 21 years or younger than 12. No potential participants were excluded based on exclusion criteria 1, 2, or 3.
This study was carried out according to the latest version of the Declaration of Helsinki. The Ethics Committee of CEIC Fundació Unió Catalana d’Hospitals (CEIC14/71) approved the study, and informed consent (signed document) was obtained from parents of adolescents under the age of 18 years and adolescents over the age of 18 years (and assent in adolescents under the age of 18 years).
The characteristics of the sample were as follows: all patients were Caucasian and male. All the adolescents included have as their main and problematic videogame an online videogame, with a mean age of 15.80 (SD 2.18) years. Most patients had elementary education (92.4%). Regarding their main problem, all the patients included played online videogames, and the mean duration of the problem was 2.2 (SD 1.7) years. There was no consumption of alcohol or drugs and only a 3.0% of the sample were smokers.
2.2.1. Millon Adolescent Personality Inventory (MACI)
This personality test  has 160 items and is self-administered. It measures thirty-one scales: twelve Personality Patterns scales (Axis II), eight Expressed Concerns Scales, seven Clinical Syndrome Scales, three Modifying Indices (particular response styles), and a Validity scale. The instrument has been translated and validated into a Spanish population with a good internal consistency of 0.82 (mean Cronbach’s alpha) . The MACI is one of the most widely used personality assessment tests for adolescents [36,37,38,39,40]. The MACI is constructed using an underlying theory of personality and psychopathology, and can identify and assess a wide range of psychological difficulties in adolescents. Studies have examined the potential utility of the MACI for assessing substance use disorders , reporting support for MACI as a screening instrument. We used this instrument in order to facilitate current and future comparisons in other studies regarding IGD or other psychological problems.
2.2.2. Symptom CheckList-90 Items-Revised (SCL-90-R)
This questionnaire  evaluates psychological problems and psychopathological symptoms. It contains ninety items and measures nine primary symptom dimensions: somatization, obsession-compulsion, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. It also includes three global indices, such as a global severity index (GSI), that measures overall psychological distress, a positive symptom distress index (PSDI), to measure the intensity of symptoms; and a positive symptom total (PST). This scale has been translated to Spanish and validated in a Spanish population , and presents a good internal consistency (mean Cronbach’s alpha = 0.75).
2.2.3. State-Trait Anxiety Index (STAI)
This questionnaire  includes forty items on a 4-point rating scale and is self-reported, measuring state anxiety (twenty items) and trait anxiety (twenty items). The minimum score is 20 and the maximum is 80 points. The state anxiety uses items that measure subjective feelings of apprehension, tension, nervousness, worry, and activation/arousal of the autonomic nervous system and evaluates the current state of anxiety. The trait anxiety scale includes general states of calmness, confidence, and security and evaluates relatively stable aspects of “anxiety proneness”. The STAI has been translated to Spanish and validated in the Spanish population with a mean Cronbach’s alpha coefficient of 0.92 .
2.2.4. DSM-5 IGD Criteria
This instrument is a questionnaire evaluating the criteria for IGD proposed in the DSM-5 . The diagnostic criteria of IGD are composed of 9 items: preoccupation, withdrawal, tolerance, unsuccessful attempts to control, loss of other interests, continued excessive use despite psychosocial problems, deceiving regarding online gaming, escape, and functional impairment. Five or more DSM-5 criteria of IGD indicates Internet gaming problems. The criteria were asked in a questionnaire form using a “yes” or “no” response.
2.2.5. Sociodemographical Variables
Additional demographic, clinical, and social/family variables related to internet gaming were measured using a semi-structured face-to-face clinical interview, including age, sex, duration of the problem, and education level.
First, at intake, a face-to-face specific clinical interview and a functional analysis of IGD was carried by experienced psychologists (more than 5 years of clinical experience in behavioral addictions) using the semi-structured clinical interview SCID-I . The questions included in this interview were about tolerance, preoccupation, withdrawal, loss of control, playing for long periods, escaping from adverse mental states, risking or losing relationships or opportunities because of the behavior, deception/covering up, giving up other activities, persistence of the behavior despite problems, and functional impairment (e.g., functional impairment in familial relationships, other social relationships, and academic achievement), and questions regarding demographic data.
During a second session (with an average duration of 90 min) before starting the treatment, were administered the above-mentioned questionnaires.
With regard to meeting the diagnostic criteria for IGD, the results obtained through the DSM-5 diagnostic criteria questionnaire were compared post hoc with the results obtained through the face-to-face clinical interview, and only patients who met the DSM-5 criteria for IGD were included in our analysis.
2.4. Statistical Analysis
Statistical analysis was carried out using the statistical software R version 3.5.3 (R Core Team, Vienna, Austria) and, in particular, its R packages mclust  and factoextra .
We obtained clusters based on the scores of the 12 sub-scales of the personality patterns scale of the MACI (Introversive, Inhibited, Doleful, Submissive, Histrionic, Egotistic, Unruly, Forceful, Conforming, Oppositional, Self-Demeaning, and Borderline Tendency) conducting a Gaussian Mixture Model (GMM) cluster analysis. Model-fitting is performed using the expectation-maximization (EM) algorithm [49,50], which later is used to initialize a hierarchical model-based agglomerative clustering [51,52]. The optimal number of clusters was selected based on the Bayesian Information Criterion (BIC), where the lowest values indicate better fit. Each individual was allocated to one class only according to their highest probability of membership.
Chi-square tests (χ2) for categorical variables and t-test (2 groups) for quantitative measures were computed to assess differences between clusters. When the normality assumption was not accomplished according to Shapiro–Wilk test, the equivalent non-parametric Mann–Whitney U test was performed. A two-sided p-value < 0.05 was considered statistically significant. Cohen’s d was used to measure the effect size for power analysis purposes. The effect size was classified as high (d = 0.8), medium (d = 0.5) or low (d = 0.2) according to Cohen .
3.1. Cluster Composition: Description for the Cluster Indicators
In the MACI scale (n = 66 men, age: 15.80 ± 2.18) there are 12 sub-scales: Introversive, Inhibited, Doleful, Submissive, Histrionic, Egotistic, Unruly, Forceful, Conforming, Oppositional, Self-Demeaning, and Borderline Tendency. The descriptive statistics of the sample are presented in Table 1.
We conducted a Gaussian finite mixture model cluster analysis. The results, for the personality patterns scale, showed that a 2 cluster (VEE: ellipsoidal, equal shape and orientation) was the optimal solution. The cluster classification for each subject is shown in Figure 1. The first cluster has 24 subjects and the second cluster has 42 subjects.
The results of the t-test and the Mann–Whitney U test comparing the two clusters created by the model-based clustering algorithm show that there are significant differences in all the sub-scales, except in the Submissive. The first cluster has higher values in the following sub-scales: introversive, inhibited, doleful, unruly, forceful, oppositional, self-demeaning and borderline tendency, while in the histrionic, egotistic and conforming sub-scales the first cluster has lower values (see Table 2 and Table 3). The results of the Cohen’s d show that the effect size was high in all scales apart from the Submissive and Unruly scales (d-values < 0.8; Table 2 and Table 3).
3.2. Comparison between the Clusters in Sociodemographic and Clinical Variables
After that, we compared the two clusters regarding a series of sociodemographic variables, i.e., age, age of onset of the disorder, disorder duration, and education level. The results of the t-tests showed that there were no significant differences between clusters regarding age, age of onset, and years with IGD. Besides, the Fisher exact test, to elucidate whether the education level was related to the cluster classification, was also non-significant.
In order to investigate if there were other significant psychopathological differences between clusters, we conducted Student’s t-tests and Mann–Whitney U-tests to compare the clusters in the Symptom Check List-90 items-Revised scores, in the DSM 5 diagnostic criteria scores and the STAI. The SCL-R has nine dimensions: somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation and psychoticism. The results of the analyses showed that the first cluster had significantly increased scores in all dimensions of the SCL-R, except for the somatization. Regarding the diagnostic criteria, the analysis comparing the cluster regarding the DSM scores yielded a significant result (U = 684, p < 0.05) indicating that the subjects in the first cluster had higher scores. Regarding Cohen’s d values, we observed high effect sizes except for the three dimensions of the SCL-90-R (Somatization, Obsessive-Compulsive and Phobia), and for the DSM-5 scores, the effect size was moderate.
In a similar manner, we conducted the same analyses for the anxiety-trait and anxiety-state of the STAI and all the comparisons were statistically significant (all t = 5.54; p < 0.001 for anxiety trait and t = 3.58; p < 0.001 for anxiety state), indicating that the first cluster of each scale had higher anxiety levels than the members of the second cluster.
This work explores the existence of empirical clusters for IGD starting from the patients’ personality trait scores, using a person-centered methodology.
Two clusters were detected: type I “higher comorbid symptoms” and type II “lower comorbid symptoms”. The main differences in the clustering variables between both clusters were the mean scores in the personality traits introversive, inhibited, doleful, unruly, forceful, oppositional, self-demeaning and borderline tendency (higher scores for the type I), histrionic, egotistic and conforming (lower scores for the type I). It should be noted the similarity between this cluster I and the IGD dimensional personality profile found in other studies.
In line with our results, literature about personality traits and IGD suggests that high scores in neuroticism , impulsivity  and introversion [15,18], and lower levels in agreeableness [14,15], cooperativeness  and self-regulation , are risk factors for this disorder. Furthermore, some authors have found that low levels of self- directedness uniquely predicted video game abuse in an adult population with Gambling Disorder , and others have identified lower responsibility as a risk factor associated with this IGD . Summarizing the conclusions of this previous research, this combination of personality traits could lead the affected people to use online gaming as a maladaptive mood modifier and/or a strategy to overcome negative life events, to having higher tendency toward competition, lower life satisfaction, lower expectations of self-efficacy, less face-to-face social support, and increased feelings of anger.
Analyzing the comorbidity level of both clusters, the type I has the higher comorbidity, obtaining higher scores on all SCL-90R scales (i.e., somatization, obsessive-compulsive, sensitivity, depression, anxiety, hostility, phobia, paranoid ideation and psychoticism) all STAI scales (anxiety trait and state), and on the IGD diagnostic criteria, being the obsessive-compulsive, sensitivity, depression, anxiety, hostility, paranoid ideation, psychoticism and anxiety trait above general population scoring. Despite the literature regarding comorbid psychopathology in adolescents and young adults with IGD having found full associations with anxiety , depression , ADHD or hyperactivity symptoms , and social phobia/anxiety , other authors have found partial associations [61,62] and no associations  with these same symptoms. These contradictory results and a possible publication bias  makes it difficult to detect the directionality and the relationship of these associations and shows the existing complexity of the relationship between IGD, personality and psychopathology.
Some authors have found relationships between personality factors and psychopathology in adolescents. According to the results found by Castellanos-Ryan , general psychopathology was related to high disinhibition/impulsivity, low agreeableness, high neuroticism and hopelessness, high delay-discounting and poor response inhibition. Of all of this, high neuroticism is the Big Five trait most strongly associated with several psychopathological symptoms, especially with anxiety and mood disorders and their comorbidity . It has been hypothesized that the negative emotionality related with this trait could mediate this association . Other personality trait predictors of psychopathology in adolescents are low conscientiousness, low extraversion , hopelessness and high impulsivity . Regarding the Egotistic personality scale, where cluster 2 scored higher, adolescents who score high in the Egotistic Personality subscale have a passive-independent pattern, are perceived as conceited, have strong self-esteem, may take others for granted, and may fantasize about future success and power . Some authors showed that certain narcissistic features are adaptive when paired with high levels of self-esteem . Taking these results into account, the differences between the Type I and Type II personality traits could suggest that the higher negative affect, the lower positive affect, the difficulties in establishing social support, and the poor general behavioral control involved in the Type I personality profiles may be characterizing the chances of developing comorbid psychopathology, and could partially explain the inconsistent results regarding IGD and psychopathology.
When interpreting the results of this study, the effects of maturity and personality development in adolescence must be taken into account. Recent research has shown a spontaneous recovery of video game addiction [71,72] and it must be noted that personality traits may change during individual development . Therefore, it is possible that the dysfunctional personality traits found will change or disappear in the future, and with them the influence they are having on the development or maintenance of IGD and the associated comorbidity.
In addition, our results show that there is a group of patients with IGD whose comorbid mental health symptoms are non-existent, or similar to the non-psychiatric population. Since there are no differences in the age at which the disorder begins or in the duration of the disorder between both cluster groups, it seems that in at least some individuals, IGD presents itself as a solitary condition or as an originating condition.
Regardless of the debate about the diagnosis and comorbidity , across the years the prevalence of the disordered gaming and the incidence of patients seeking treatment for IGD has remained stable . This study is focused on the heterogeneity of the disorder and in the existence of different subgroups of IGD patients, based on personality, with differences in the seriousness of their psychological comorbidity. Our results suggest that these variables appear to be useful in determining clusters which represent different clinical subtypes with different degrees of severity. Such differences among online gamers imply that the experience of playing may vary in patients and the general population. Therefore, in order to succeed in developing instruments, planning efficient prevention programs for general population and targeting at-risk gamers, these dependent effects should be considered. Furthermore, the data regarding standard therapy for IGD are limited, with cognitive-behavioral therapy, family therapy and pharmacological intervention showing some significant results [76,77]. In order to develop more specific and accurate treatment interventions, the existence of IGD clusters should be considered.
Concerning limitations, the results of this study are based on a small sample, but considering that the sample is formed by a clinical sample, evaluated in a controlled setting, there may be a reason to anticipate that our findings may be of value for similar profile patients. Second, the measures used are based on self-administered questionnaires, although a trained psychologist supervised the entire process. Third, the cross-sectional design does not allow us to determine causality of the variables assessed. Fourth, the severity of IGD has not been reflected in this paper, as the objective of this work was to describe the different types of IGD-diagnosed adolescents with respect to personality traits and to analyze the comorbidity of each cluster groups.
Future research could complement these results using longitudinal designs addressing the potential mediating role of personality in the etiological factors and clinical course of IGD, in order to try to develop a predictive model of problematic video games use and its comorbidity.
Problematic use of online videogames is a growing clinical issue in developed countries. There is ongoing debate about the suitability of the current proposed definition and diagnostic criteria and about its related dysfunctional consequences and clinical correlates. The current literature shows that IGD is a complex problem where neurobiological systems, personality, and socioeconomic and environmental variables are involved. The objective of this study was to explore the heterogeneity of the affected people and the existence of differentiated subgroups based on different personality traits.
The results suggest that differences in personality can be useful in determining clusters with different levels of comorbidity. These differences imply that the experience of playing online videogames may affect different players, and clinicians and researchers should consider them in order to develop treatment options or assessment instruments. Future research should include more heterogeneous samples in relation to age and sex, and consider treatment outcome as a variable to analyze.
D.F., V.G.-B. and J.J.S. conceived and planned this article. V.G.-B. and J.J.S. carried out the search and revision of the literature. D.F. and I.O. analyzed the data. V.G.-B. and J.J.S. drafted the study. All authors (V.G.-B., J.J.S., D.F., I.O., E.M., M.B., S.J.-M., A.d.P.-G. and J.R.) revised the article critically for important intellectual content. All authors (V.G.-B., J.J.S., D.F., I.O., E.M., M.B., S.J.-M., A.d.P.-G. and J.R.) commented on and approved the final manuscript and are accountable for all aspects of the work. All authors have read and agreed to the published version of the manuscript.
This work was funded by an AIS (Atención e Investigación en Socioaddiciones) intramural research program. This research is partially supported by the Marsden grant E2987-3648 (Royal Society of New Zealand), by grant RTI2018-100927-J-I00 Retos Investigación from Ministerio de Ciencia, Innovación y Universidades (Spain), and by grant 2017 SGR 622 (GRBIO) from the Departament d’Economia i Coneixement de la Generalitat de Catalunya (Spain). These partial funders had no role in the study design, data collection, and analysis, decision to publish, or preparation of the manuscript.
Conflicts of Interest
The authors declare no conflict of interest.
- World Health Organization. International Classification of Diseases for Mortality and Morbidity Statistics 11th Revision; World Health Organization: Geneva, Switzerland, 2018. [Google Scholar]
- Poli, R. Internet addiction update: Diagnostic criteria, assessment and prevalence. Neuropsychiatry 2017, 07, 4–8. [Google Scholar] [CrossRef]
- Fauth-Bühler, M.; Mann, K. Neurobiological correlates of internet gaming disorder: Similarities to pathological gambling. Addict. Behav. 2017, 64, 349–356. [Google Scholar] [CrossRef] [PubMed]
- Kaptsis, D.; King, D.; Delfabbro, P.; Gradisar, M. Withdrawal symptoms in internet gaming disorder: A systematic review. Clin. Psychol. Rev. 2016, 43, 58–66. [Google Scholar] [CrossRef] [PubMed]
- APA. Diagnostic and Statistical Manual of Mental Disorders, 5th ed.; (DSM-5); American Psychiatric Association, Ed.; APA: Washington, DC, USA, 2013. [Google Scholar]
- Gervasi, A.M.; La Marca, L.; Costanzo, A.; Pace, U.; Guglielmucci, F.; Schimmenti, A. Personality and Internet Gaming Disorder: A Systematic Review of Recent Literature. Curr. Addict. Rep. 2017, 4, 293–307. [Google Scholar] [CrossRef]
- Şalvarlı, Ş.İ.; Griffiths, M.D. Internet Gaming Disorder and Its Associated Personality Traits: A Systematic Review Using PRISMA Guidelines. Int. J. Ment. Health Addict. 2019. [Google Scholar] [CrossRef]
- Matthews, G.; Deary, I.J.; Whiteman, M.C. Personality Traits, 3rd ed.; Cambridge University Press: Cambridge, UK, 2009; ISBN 9780511812743. [Google Scholar]
- Agüera, Z.; Krug, I.; Sánchez, I.; Granero, R.; Penelo, E.; Peñas-Lledó, E.; Jiménez-Murcia, S.; Menchón, J.M.; Fernández-Aranda, F. Personality Changes in Bulimia Nervosa after a Cognitive Behaviour Therapy. Eur. Eat. Disord. Rev. 2012, 20, 379–385. [Google Scholar] [CrossRef]
- Kim, E.J.; Namkoong, K.; Ku, T.; Kim, S.J. The relationship between online game addiction and aggression, self-control and narcissistic personality traits. Eur. Psychiatry 2008, 23, 212–218. [Google Scholar] [CrossRef]
- Kardefelt-Winther, D. The moderating role of psychosocial well-being on the relationship between escapism and excessive online gaming. Comput. Hum. Behav. 2014, 38, 68–74. [Google Scholar] [CrossRef]
- Mehroof, M.; Griffiths, M.D. Online gaming addiction: The role of sensation seeking, self-control, neuroticism, aggression, state anxiety, and trait anxiety. Cyberpsychol. Behav. Soc. Netw. 2010, 13, 313–316. [Google Scholar] [CrossRef]
- Khazaal, Y.; Chatton, A.; Rothen, S.; Achab, S.; Thorens, G.; Zullino, D.; Gmel, G. Psychometric properties of the 7-item game addiction scale among french and German speaking adults. BMC Psychiatry 2016, 16, 132. [Google Scholar] [CrossRef]
- Jiménez-Murcia, S.; Fernández-Aranda, F.; Granero, R.; Chóliz, M.; La Verde, M.; Aguglia, E.; Signorelli, M.S.; Sá, G.M.; Aymamí, N.; Gómez-Peña, M.; et al. Video game addiction in gambling disorder: Clinical, psychopathological, and personality correlates. BioMed Res. Int. 2014, 7, 105–110. [Google Scholar] [CrossRef]
- Müller, K.W.; Beutel, M.E.; Egloff, B.; Wölfling, K. Investigating risk factors for Internet gaming disorder: A comparison of patients with addictive gaming, pathological gamblers and healthy controls regarding the big five personality traits. Eur. Addict. Res. 2014, 20, 129–136. [Google Scholar] [CrossRef]
- Festl, R.; Scharkow, M.; Quandt, T. Problematic computer game use among adolescents, younger and older adults. Addiction 2013, 108, 592–599. [Google Scholar] [CrossRef] [PubMed]
- Wang, C.W.; Ho, R.T.H.; Chan, C.L.W.; Tse, S. Exploring personality characteristics of Chinese adolescents with internet-related addictive behaviors: Trait differences for gaming addiction and social networking addiction. Addict. Behav. 2015, 42, 32–35. [Google Scholar] [CrossRef] [PubMed]
- Montag, C.; Flierl, M.; Markett, S.; Walter, N.; Jurkiewicz, M.; Reuter, M. Internet addiction and personality in first-person-shooter video gamers. J. Media Psychol. 2011, 23, 163–173. [Google Scholar] [CrossRef]
- Goby, V.P. Personality and Online/Offline Choices: MBTI Profiles and Favored Communication Modes in a Singapore Study. CyberPsychol. Behav. 2006, 9, 5–13. [Google Scholar] [CrossRef] [PubMed]
- Jain, A.K.; Jain, A.K.; Murty, M.N.; Flynn, P.J. Data Clustering: A Review. ACM Comput. Surv. 1999, 31, 264–323. [Google Scholar] [CrossRef]
- Noori, H.R.; Helinski, S.; Spanagel, R. Cluster and meta-analyses on factors influencing stress-induced alcohol drinking and relapse in rodents. Addict. Biol. 2014, 19, 225–232. [Google Scholar] [CrossRef]
- Alikhanian, H.; Crawford, J.D.; DeSouza, J.F.X.; Cheyne, D.O.; Blohm, G. Adaptive cluster analysis approach for functional localization using magnetoencephalography. Front. Neurosci. 2013. [Google Scholar] [CrossRef]
- Henry, D.; Dymnicki, A.B.; Mohatt, N.; Allen, J.; Kelly, J.G. Clustering Methods with Qualitative Data: A Mixed-Methods Approach for Prevention Research with Small Samples. Prev. Sci. 2015, 16, 1007–1016. [Google Scholar] [CrossRef]
- Gamito, P.S.; Morais, D.G.; Oliveira, J.G.; Brito, R.; Rosa, P.J.; Gaspar De Matos, M. Frequency is not enough: Patterns of use associated with risk of Internet addiction in Portuguese adolescents. Comput. Hum. Behav. 2016, 58, 471–478. [Google Scholar] [CrossRef]
- Álvarez-Moya, E.M.; Jiménez-Murcia, S.; Aymamí, M.N.; Gómez-Peña, M.; Granero, R.; Santamaría, J.; Menchón, J.M.; Fernández-Aranda, F. Subtyping study of a pathological gamblers sample. Can. J. Psychiatry 2010, 55, 498–506. [Google Scholar] [CrossRef] [PubMed]
- de la Fuente-Tomas, L.; Arranz, B.; Safont, G.; Sierra, P.; Sanchez-Autet, M.; Garcia-Blanco, A.; Garcia-Portilla, M.P. Classification of patients with bipolar disorder using k-means clustering. PLoS ONE 2019, 14. [Google Scholar] [CrossRef] [PubMed]
- Vachon, D.D.; Bagby, R.M. Pathological Gambling Subtypes. Psychol. Assess. 2009, 21, 608–615. [Google Scholar] [CrossRef]
- Suomi, A.; Dowling, N.A.; Jackson, A.C. Problem gambling subtypes based on psychological distress, alcohol abuse and impulsivity. Addict. Behav. 2014, 39, 1741–1745. [Google Scholar] [CrossRef]
- Granero, R.; Fernandez-Aranda, F.; Mestre-Bach, G.; Steward, T.; Bano, M.; Agvera, Z.; Mallorqui-Bague, N.; Aymami, N.; Gomez-Pena, M.; Sancho, M.; et al. Cognitive behavioral therapy for compulsive buying behavior: Predictors of treatment outcome. Eur. Psychiatry 2017, 39. [Google Scholar] [CrossRef]
- Billieux, J.; Thorens, G.; Khazaal, Y.; Zullino, D.; Achab, S.; Van der Linden, M. Problematic involvement in online games: A cluster analytic approach. Comput. Hum. Behav. 2015, 43, 242–250. [Google Scholar] [CrossRef]
- Torres-Rodríguez, A.; Griffiths, M.D.; Carbonell, X.; Oberst, U. Internet gaming disorder in adolescence: Psychological characteristics of a clinical sample. J. Behav. Addict. 2018, 7, 707–718. [Google Scholar] [CrossRef]
- Mallorquí-Bagué, N.; Fernández-Aranda, F.; Lozano-Madrid, M.; Granero, R.; Mestre-Bach, G.; Baño, M.; Pino-Gutiérrez, A.D.; Gómez-Peña, M.; Aymamí, N.; Menchón, J.M.; et al. Internet gaming disorder and online gambling disorder: Clinical and personality correlates. J. Behav. Addict. 2017, 6, 669–677. [Google Scholar] [CrossRef]
- González-Bueso, V.; Santamaría, J.; Fernández, D.; Merino, L.; Montero, E.; Ribas, J. Association between Internet Gaming Disorder or Pathological Video-Game Use and Comorbid Psychopathology: A Comprehensive Review. Int. J. Environ. Res. Public Health. 2018, 15, 668. [Google Scholar] [CrossRef]
- Millon, T. Manual of Millon Adolescent Clinical Inventory; National Computer Systems: Minneapolis, MN, USA, 1993. [Google Scholar]
- Aguirre, G. Adaptación Español del MACI; TEA: Madrid, Spain, 2004. [Google Scholar]
- Murrie, D.C.; Cornell, D.G. Psychopathy screening of incarcerated juveniles: A comparison of measures. Psychol. Assess. 2002, 14, 390–396. [Google Scholar] [CrossRef] [PubMed]
- Salekin, R.T.; Larrea, M.A.; Ziegler, T. Relationships between the MACI and the BASC in the assessment of child and adolescent offenders. J. Forensic Psychol. Pract. 2002, 2, 35–50. [Google Scholar] [CrossRef]
- Velting, D.M.; Rathus, J.H.; Miller, A.L. MACI personality scale profiles of depressed adolescent suicide attempters: A pilot study. J. Clin. Psychol. 2000, 56, 1381–1385. [Google Scholar] [CrossRef]
- Burton, D.L.; Duty, K.J.; Leibowitz, G.S. Differences between sexually victimized and nonsexually victimized male adolescent sexual abusers: Developmental antecedents and behavioral comparisons. J. Child Sex. Abus. 2011, 20, 77–93. [Google Scholar] [CrossRef] [PubMed]
- Ferrer, L.; Kirchner, T. Suicidal tendency among adolescents with adjustment disorder: Risk and protective personality factors. Crisis 2015, 36, 202–210. [Google Scholar] [CrossRef] [PubMed]
- Grilo, C.M.; Fehon, D.C.; Walker, M.; Martino, S. A comparison of adolescent inpatients with and without wubstance abuse using the Millon Adolescent Clinical Inventory. J. Youth Adolesc. 1996, 25, 379–388. [Google Scholar] [CrossRef]
- Derogatis, L.R. SCL-90-R. Administration, Scoring and Procedures Manual; Clinical Psychometric: Baltimore, MD, USA, 1990. [Google Scholar]
- Derogatis, L.R. SCL-90-R. Cuestionario de 90 Síntomas-Manual. [Questionnaire of the 90 Symptoms-Manual]; TEA Ediciones: Madrid, Spain, 2002. [Google Scholar]
- Spielberger, C.; Gorsuch, R.L.; Lushene, R.E. Manual for the State/Trait Anxiety Inventory; Consulting Psychologists Press: Palo Alto, CA, USA, 1970. [Google Scholar]
- Guillén-Riquelme, A.; Buela-Casal, G. Psychometric revision and differential item functioning in the State Trait Anxiety Inventory (STAI). Psicothema 2011, 23, 510–515. [Google Scholar]
- First, M.; Gibbon, M.; Spitzer, R.; Williams, J. Users Guide for the Structured Clinical Interview for DSM IV Axis I Disorders—Research Version (SCID-I, Version 2.0); New York State Psychiatric Institute: New York, NY, USA, 1996. [Google Scholar]
- Scrucca, L.; Raftery, A.E. Clustvarsel: A package implementing variable selection for Gaussian model-based clustering in R. J. Stat. Softw. 2018, 84. [Google Scholar] [CrossRef]
- Kassambara, A.; Mundt, F. Factoextra: Extract and Visualize the Results of Multivariate Data Analyses. R Package Version 2017, 1, 1–74. [Google Scholar]
- Dempster, A.P.; Laird, N.M.; Rubin, D.B. Maximum Likelihood from Incomplete Data via the EM Algorithm. J. R. Stat. Soc. B 1977, 39, 1–22. [Google Scholar]
- McLachlan, G.J.; Krishnan, T. (Thriyambakam) The EM Algorithm and Extensions; Wiley-Interscience: New York, NY, USA, 2008; ISBN 9780471201700. [Google Scholar]
- Fraley, C.; Raftery, A.E. Model-based methods of classification: Using the mclust software in chemometrics. J. Stat. Softw. 2007, 18, 1–13. [Google Scholar] [CrossRef]
- Mclachlan, G.J.; Lee, S.X.; Rathnayake, S.I. Annual Review of Statistics and Its Application Finite Mixture Models. Annu. Rev. Stat. 2019. [Google Scholar] [CrossRef]
- Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; L. Erlbaum Associates: Hillsdale, NJ, USA, 1988; ISBN 9780805802832. [Google Scholar]
- Lehenbauer-Baum, M.; Klaps, A.; Kovacovsky, Z.; Witzmann, K.; Zahlbruckner, R.; Stetina, B.U. Addiction and Engagement: An Explorative Study Toward Classification Criteria for Internet Gaming Disorder. Cyberpsychology. Behav. Soc. Netw. 2015, 18, 343–349. [Google Scholar] [CrossRef] [PubMed]
- Gentile, D.A.; Choo, H.; Liau, A.; Sim, T.; Li, D.; Fung, D.; Khoo, A. Pathological video game use among youths: A two-year longitudinal study. Pediatrics 2011, 127, e319–e329. [Google Scholar] [CrossRef] [PubMed]
- Seay, A.F.; Kraut, R.E. Project Massive: Self-Regulation and Problematic Use of Online Gaming. In Proceedings of the Conference on Human Factors in Computing Systems, San Jose, CA, USA, 24–28 April 2007; pp. 829–838. [Google Scholar]
- Müller, K.W.; Koch, A.; Dickenhorst, U.; Beutel, M.E.; Duven, E.; Wölfling, K. Addressing the Question of Disorder-Specific Risk Factors of Internet Addiction: A Comparison of Personality Traits in Patients with Addictive Behaviors and Comorbid Internet Addiction. Biomed Res. Int. 2013. [Google Scholar] [CrossRef]
- Van Rooij, A.J.; Schoenmakers, T.M.; Vermulst, A.A.; Van Den Eijnden, R.J.J.M.; Van De Mheen, D. Online video game addiction: Identification of addicted adolescent gamers. Addiction 2011, 106, 205–212. [Google Scholar] [CrossRef]
- Baer, S.; Bogusz, E.; Green, D.A. Stuck on screens: Patterns of computer and gaming station use in youth seen in a psychiatric clinic. J. Can. Acad. Child Adolesc. Psychiatry 2011, 20, 86–94. [Google Scholar]
- Cole, S.H.; Hooley, J.M. Clinical and Personality Correlates of MMO Gaming. Soc. Sci. Comput. Rev. 2013, 31, 424–436. [Google Scholar] [CrossRef]
- Starcevic, V.; Berle, D.; Porter, G.; Fenech, P. Problem Video Game Use and Dimensions of Psychopathology. Int. J. Ment. Health Addict. 2011, 9, 248–256. [Google Scholar] [CrossRef]
- Stetina, B.U.; Kothgassner, O.D.; Lehenbauer, M.; Kryspin-Exner, I. Beyond the fascination of online-games: Probing addictive behavior and depression in the world of online-gaming. Comput. Hum. Behav. 2011, 27, 473–479. [Google Scholar] [CrossRef]
- King, D.L.; Delfabbro, P.H.; Zwaans, T.; Kaptsis, D. Clinical features and axis I comorbidity of Australian adolescent pathological Internet and video game users. Aust. N. Zeal. J. Psychiatry 2013, 47, 1058–1067. [Google Scholar] [CrossRef] [PubMed]
- Castellanos-Ryan, N.; Briere, F.N.; O’Leary-Barrett, M.; Banaschewski, T.; Bokde, A.; Bromberg, U.; Biichel, C.; Flor, H.; Frouin, V.; Gallinat, J.; et al. The IMAGEN Consortium The structure of psychopathology in adolescence and its common personality and cognitive correlates. J. Abnorm. Psychol. 2016, 125, 1039–1052. [Google Scholar] [CrossRef] [PubMed]
- Kotov, R.; Gamez, W.; Schmidt, F.; Watson, D. Linking “Big” personality traits to anxiety, depressive, and substance use disorders: A meta-analysis. Psychol. Bull. 2010, 136, 768–821. [Google Scholar] [CrossRef] [PubMed]
- Tackett, J.L.; Kushner, S.C.; De Fruyt, F.; Mervielde, I. Delineating Personality Traits in Childhood and Adolescence: Associations Across Measures, Temperament, and Behavioral Problems. Assessment 2013, 20, 738–751. [Google Scholar] [CrossRef] [PubMed]
- Malouff, J.M.; Thorsteinsson, E.B.; Schutte, N.S. The relationship between the five-factor model of personality and symptoms of clinical disorders: A meta-analysis. J. Psychopathol. Behav. Assess. 2005, 27, 101–114. [Google Scholar] [CrossRef]
- Carragher, N.; Teesson, M.; Sunderland, M.; Newton, N.C.; Krueger, R.F.; Conrod, P.J.; Barrett, E.L.; Champion, K.E.; Nair, N.K.; Slade, T. The structure of adolescent psychopathology: A symptom-level analysis. Psychol. Med. 2016, 46, 981–994. [Google Scholar] [CrossRef]
- McCann, J.T. The MACI: Composition and Clinical Applications. In The Millon Inventories: Clinical and Personality Assessment; Millon, T., Ed.; The Guilford Press: New York, NY, USA, 1997; pp. 363–388. [Google Scholar]
- Barry, C.T.; Frick, P.J.; Killian, A.L. The relation of narcissism and self-esteem to conduct problems in children: A preliminary investigation. J. Clin. Child Adolesc. Psychol. 2003, 32, 139–152. [Google Scholar] [CrossRef]
- Przybylski, A.K.; Weinstein, N.; Murayama, K. Internet gaming disorder: Investigating the clinical relevance of a new phenomenon. Am. J. Psychiatry 2017, 174, 230–235. [Google Scholar] [CrossRef]
- Rothmund, T.; Klimmt, C.; Gollwitzer, M. Low Temporal Stability of Excessive Video Game Use in German Adolescents. J. Media Psychol. 2018, 30, 53–65. [Google Scholar] [CrossRef]
- Soto, C.J.; John, O.P.; Gosling, S.D.; Potter, J. Age Differences in Personality Traits From 10 to 65: Big Five Domains and Facets in a Large Cross-Sectional Sample. J. Pers. Soc. Psychol. 2011, 100, 330–348. [Google Scholar] [CrossRef]
- Bean, A.M.; Nielsen, R.K.L.; van Rooij, A.J.; Ferguson, C.J. Video game addiction: The push to pathologize video games. Prof. Psychol. Res. Pract. 2017, 48, 378–389. [Google Scholar] [CrossRef]
- Feng, W.; Ramo, D.E.; Steven, R.C.; Bourgeois, J.A. Internet gaming disorder: Trends in prevalence 1998–2016. Addict. Behav. 2017, 75, 17–24. [Google Scholar] [CrossRef] [PubMed]
- Han, D.H.; Kim, S.M.; Lee, Y.S.; Renshaw, P.F. The effect of family therapy on the changes in the severity of on-line game play and brain activity in adolescents with on-line game addiction. Psychiatry Res. Neuroimaging 2012, 202, 126–131. [Google Scholar] [CrossRef] [PubMed]
- Kim, S.M.; Han, D.H.; Lee, Y.S.; Renshaw, P.F. Combined cognitive behavioral therapy and bupropion for the treatment of problematic on-line game play in adolescents with major depressive disorder. Comput. Hum. Behav. 2012, 28, 1954–1959. [Google Scholar] [CrossRef]
Figure 1. Classification of subjects for each cluster.
Table 1. Descriptive statistics of the personality traits of the Millon Adolescent Personality Inventory (MACI) scale (n = 66).
SD = Standard Deviation, IQR = Interquartile range.
Table 2. T-test comparison between clusters of the personality traits of the MACI and the State-Trait Anxiety Index (STAI) scales (cluster 1 n = 24, cluster 2 n = 42).
|Independent Samples Test|
|CLUSTER 1||CLUSTER 2|
|Mean (SD)||Mean (SD)||t||df||Sig. (2 tailed)||Cohen’s d|
|Submissive||46.48 (11.86)||42.24 (8.42)||1.69||64||0.126||0.43|
|Egotistic||27.80 (3.03)||36.93 (7.33)||−3.16||33.15||0.003||−0.92|
|Unruly||34.12 (9.70)||27.85 (8.45)||2.76||64||0.010||0.70|
|Conforming||39.44 (8.19)||49.04 (7.49)||−4.86||64||0.000||−1.23|
|Oppositional||28.76 (8.58)||15.14 (6.46)||7.32||64||0.000||1.86|
|Anxiety State||18.72 (8.35)||11.38 (7.52)||3.67||63||0.000||0.94|
|Anxiety Trait||23.48 (8.11)||12.64 (7.00)||5.75||64||0.000||1.46|
SD = Standard Deviation; df = degrees of freedom (Cohen’s d > 0.80 in bold).
Table 3. Mann–Whitney U-test for the MACI, Symptom CheckList-90 items-Revised (SCL-90-R), and DSM 5 criteria mean scores (cluster 1 n = 24, cluster 2 n = 42).
|Mann–Whitney U Test|
|CLUSTER 1||CLUSTER 2|
|Mean (SD)||Mean (SD)||U||Sig. (2 tailed)||Cohen’s d|
|Introversive||35.76 (9.27)||18.76 (7.44)||948.50||0.000||2.08|
|Inhibited||29.88 (11.04)||15.37 (5.94)||883||0.000||1.76|
|Doleful||20.44 (9.42)||6.00 (5.07)||941||0.000||2.05|
|Histrionic||29.88 (10.95)||40.61 (6.70)||223.50||0.000||−1.26|
|Forceful||14.56 (7.49)||8.17 (5.07)||781||0.000||1.05|
|Self-Demeaning||32.96 (11.51)||11.17 (6.97)||974||0.000||2.45|
|Borderline Tend.||19.44 (5.34)||6.97 (4.39)||998.5||0.000||2.61|
|Somatization||0.55 (0.54)||0.26 (0.27)||718||0.006||0.75|
|Obsessive-comp||1.03 (0.60)||0.61 (0.51)||719||0.006||0.76|
|Interp. sens.||1.00 (0.81)||0.38 (0.39)||781||0.000||1.04|
|Depression||0.93 (0.72)||0.28 (0.30)||840.50||0.000||1.28|
|Anxiety||0.69 (0.81)||0.21 (0.22)||700||0.012||0.90|
|Hostility||1.10 (0.91)||0.56 (0.45)||708||0.009||0.81|
|Phobia||0.40 (0.57)||0.10 (0.19)||782.50||0.000||0.77|
|Paranoid ideation||1.03 (0.82)||0.38 (0.43)||773||0.000||1.06|
|Psychoticism||0.56 (0.50)||0.14 (0.20)||845||0.000||1.18|
|DSM 5 criteria||5.84 (1.82)||4.95 (1.55)||684||0.019||0.54|
SD = Standard Deviation; df = degrees of freedom (Cohen’s d > 0.80 in bold).
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).