Next Article in Journal
From Tutoring Gross Anatomy to Pancreatic Surgery Innovation
Next Article in Special Issue
Contribution of Leisure Context, Motivation and Experience to the Frequency of Participation in Structured Leisure Activities among Adolescents
Previous Article in Journal
Assessment of Erosion Characteristics in Purple and Yellow Soils Using Simulated Rainfall Experiments
Previous Article in Special Issue
Child-Friendly Legal Aid and Individual Assessment of Children in Conflict with the Law: Building the Basis for Effective Participation
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 19
Issue 1
10.3390/ijerph19010358
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Difficulties in Establishing “Truth” Conditions in the Assessment of Addictive Smartphone Use in Young Adults

by
Javier García-Manglano
1,*,
Claudia López-Madrigal
2,
Charo Sádaba-Chalezquer
3,
Cecilia Serrano
4 and
Olatz Lopez-Fernandez
5,*
1
Institute of Culture and Society, University of Navarra, 31008 Pamplona, Spain
2
School of Education and Psychology, University of Navarra, 31008 Pamplona, Spain
3
School of Communication, University of Navarra, 31008 Pamplona, Spain
4
Department of Sociology, Catholic University of the Sacred Heart, 20123 Milan, Italy
5
Health Research Institute, Hospital Universitario Fundación Jiménez Díaz, 28040 Madrid, Spain
*
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(1), 358; https://doi.org/10.3390/ijerph19010358
Submission received: 29 October 2021 / Revised: 9 December 2021 / Accepted: 16 December 2021 / Published: 30 December 2021
(This article belongs to the Special Issue A Further Look at Risk Behaviors in Adolescents: An Update on Assessment Instruments, Explanation Models, and Interventions Focused on Individual and Environmental Risk/Protective Factors)
Versions Notes

Abstract

:
The smartphone revolution has placed powerful, multipurpose devices in the hands of youth across the globe, prompting worries about the potential negative consequences of these technologies on mental health. Many assessment tools have been created, seeking to classify individuals into problematic and non-problematic smartphone users. These are identified using a cutoff value: a threshold, within the scale range, at which higher scores are expected to be associated with negative outcomes. Lacking a clinical assessment of individuals, the establishment of this threshold is challenging. We illustrate this difficulty by calculating cutoff values for the Short Version of the Smartphone Addiction Scale (SAS-SV) in 13 Spanish-speaking samples in 11 countries, using common procedures (i.e., reliability, validity, ROC methodology). After showing that results can be very heterogeneous (i.e., they lead to diverse cutoff points and rates of addiction) depending on the decisions made by the researchers, we call for caution in the use of these classifications, particularly when researchers lack a clinical definition of true addiction—as is the case with most available scales in the field of behavioral addictions—which can cause an unnecessary public health alert.

1. Introduction

Over the past decade, as handheld digital devices with constant internet access became ubiquitous, worries about the consequences of problematic smartphone use on youth development have multiplied. Even though touchscreen handheld phones were introduced in the early 2000s, it was the release of the iPhone and the Android OS in 2007 that initiated the smartphone revolution; the number of smartphones in active use in the United States doubled from 50 to 100 million between 2009 and 2011, and doubled again from 100 to 200 million between 2011 and 2014. These numbers were estimated to be near 300 million in 2019 [1] in a population of around 330 million. Globally, the number of active smartphones (mobile subscriptions) grew from 724 million in 2011 to 6401 million in 2021 [2].
The smartphone changed communications in important ways. By providing full portability, it opened the possibility of constant connectivity, anytime and anywhere. By creating an ecosystem for mobile applications it expanded functionality from voice calls to all sorts of activities—from calendar to videogames, from entertainment to note-taking. With time, dropping smartphone prices broke income barriers, bringing constant connectivity to places where desktops and laptops had not yet arrived, e.g., low-income areas and groups. All of this made it increasingly common to see powerful devices in the hand of adolescents and teens. By 2018, 95% of US teens had a smartphone, with little class variability, and the percentage was 93% among teens in households with an income lower than $30 K, and 97% among teens in households with an income higher than $75 K [3].
It soon became apparent that mobile phones could have implications for well-being and mental health, and the 2010s witnessed the development of a plethora of instruments (inventories, scales) aimed at classifying individuals by their problematic mobile phone use, in different languages and cultures [4,5].
These classifying tools are varied in their perspective and goals; some take a more behavioral approach, seeking to assess ‘problematic’ use, while others take a more clinical approach and claim to be able to identify signs of ‘addictive’ use. In this paper, we do not aim to determine whether the behavioral or the clinical approach is more convenient. Instead, we explore, using one of the most popular assessment tools available to the scientific community, the complexities of establishing clear classifications of individuals in this area of research, sounding a call for caution in the use of these assessment tools.

1.1. One Construct, Many Tools

One of the first literature reviews on mobile phone abuse or addiction was undertaken by Pedrero and colleagues [6]. This study identified some important scientific problems, including the conceptual vagueness of the concepts used to refer to the excessive use of mobile phones, the disparity in the adoption of diagnostic criteria, and the numerous psychometric instruments for the assessment of these concepts. They recorded estimated prevalence ranges between 0 and 38%, depending on the scale used (out of the 15 scales tested between 2005 and 2011, almost all without robust psychometric tests and replication) and the characteristics of the population (usually adolescents and young adults from Asia).
In a recent systematic review of available problematic phone use scales, Harris et al. [7] analyzed the properties of 78 such instruments—all of them created between 2004 and 2019. These scales were of three types, most of them attempting to measure problematic smartphone use (70 scales), some exploring smartphone use motivations and attitudes (six scales), and a few assessing smartphone use frequency (three scales, one of which was also included in the previous group). The authors identified a series of problems in the development of smartphone use scales, such as the exclusive use of self-reported data (even when seeking to assess criterion-related validity), or the frequent absence of test-retest reliability coefficients (available in only 10 out of the 78 scales) to assess the temporal stability of the instruments. Thus, the study of problematic mobile phone use is still rife with potential measurement problems.

1.2. The Smartphone Addiction Scale

The Smartphone Addiction Scale (SAS) is an instrument created with a clinical perspective, which is gaining higher recognition over the MPPUS, possibly because it is more adapted to the current smartphones rather than mobile phones. Both have shown similar psychometric properties, but the SAS is the most representative scale to measure present problematic smartphone use for two reasons according to a recent review [8]: (i) it is highly cited across numerous research fields showing it remains influential; (ii) among the ten scales used in the empirical study, this was the one which captured the largest variance from the construct shared across all measures. Our choice of instrument, however, does not imply that we endorse clinical approaches in general, or this scale in particular—instead, we use the SAS because of its popularity and evidence-based support: it has been cited over a thousand times in less than 10 years, and it has been translated into multiple languages covering the immense majority of the world population, from the most populous languages, like Chinese, Spanish, English, Arabic, French, German, Italian or Portuguese, to lesser used languages, like Malay or Persian. See Table 1.
The SAS was originally developed in 2013 by Min Kwon and colleagues at the Catholic University of Korea, in Seoul. The development of the scale began with a 48-item questionnaire assessing seven dimensions of addiction (i.e., daily-life disturbance, disturbance of reality, positive anticipation, withdrawal, cyberspace-oriented relationship, overuse, and tolerance), and a 7-item visual analogue scale in which respondents subjectively rated their own addiction to smartphones [17]. The same authors developed a short version of the SAS (SAS-SV) with 10 items in the Korean language, which was particularly suited for adolescents [9].
Kwon and colleagues went on to calculate a cutoff point dividing adolescents between those exhibiting symptoms of ‘addiction’ in the use of their smartphones and those making a moderate use. Notably, this process included the clinical assessment of 150 adolescents (90 boys, 60 girls) by professional psychologists; in this assessment, 16.6% of the boys and 26.6% of the girls exhibited symptoms of addiction, tolerance, and withdrawal. These clinical assessments were used as true conditions for a ROC analysis [18], determining cutoff points of 31 for men and 33 for women [9]. We used the SAS-SV as the basis for our analysis, in its Spanish adaptation [10], which in turn suggested a unique cutoff value of 32 for both boys and girls.

1.3. The SAS-SV and Concurrent Outcomes

Over the years, researchers have explored the association between problematic smartphone use and a variety of variables, from socioeconomic characteristics to mental health issues. We review some of those associations, with particular attention to studies using the SAS-SV and the four outcomes we will use in our analysis—see below.

1.3.1. Psychological Distress: Anxiety, Depression, Stress

Due to their prevalence and interference in daily activities, anxiety, depression, and stress have become the gold standard in the assessment of psychological distress. Multiple studies have found significant associations between addictive or problematic smartphone use as measured by the SAS-SV, and anxiety [19,20,21,22,23,24,25], depression [21,25,26,27,28,29], and stress [24,30,31,32].
Similar results were found in studies that assessed problematic smartphone use with different scales, such as the Bergen Social Networking Addiction Scale, BSNAS [28,33], the Problematic Mobile Phone Use Scale, PMPUS [34,35], the Technology Use Questionnaire [36], the Problematic Use of Mobile Phone, PUMP [37], and the Smartphone Addiction Inventory, SPAI [38].
A systematic review of 23 peer-reviewed papers [39] explains that people with mental health issues try to compensate for their emotional difficulties with an overuse of the smartphone to alleviate their negative emotions; a technique which turns out to be ineffective. Because psychopathological symptoms are related to difficulties in regulating emotions and in enacting adaptive strategies, problematic use of technology can relate to the presence of psychopathology [27]. However, more longitudinal studies are needed to establish directionality between depression, anxiety, stress, and smartphone use.

1.3.2. Screen Time and Social Media

Studies have also established a link between higher scores of problematic smartphone use and time spent on screens and social media [20,25,27,40,41,42,43,44,45]. According to displacement theory [46], time spent on social media can displace important activities, such as face-to-face interactions or sleep [47]. Similar research shows that time spent on screens might be detrimental to one’s own well-being by altering or displacing positive lifestyle habits, such as sports and hobbies [29,48,49,50,51].
However, small or non-significant relations have also been reported between smartphone use and both the SAS-SV scores [52] and other problematic smartphone use scales [36,37,53,54,55]. These studies contend that time spent on social media is not necessarily a predictor of poor mental health when we account for other processes, such as use of motivations and socio-contextual variables that impact both mental health and time spent on devices [56]. Additionally, youth use multiple devices for multiple purposes (e.g., gaming, texting, entertainment, work, study), some of which are positive, so the amount of screen time spent alone might be a poor proxy for problematic smartphone use.
In sum, the aim of this paper is to warn about a common practice in the field of smartphone addiction research: the proposal and use of cutoff points to classify users as addicted or non-addicted, while lacking a clinical assessment of addiction to use as ‘true’ condition against which to compare the statistical instrument (e.g., scale, assessment tool). We sound our warning by replicating common practice, using one of the most widely used smartphone addiction scales (SAS-SV) across 13 Spanish-speaking samples. Even though we calculate cutoff points, our goal is not to select the best one among them, but to show that picking one among many possible cutoff points is a difficult task that should be carried out in a more careful, transparent way than what we find in most studies to date.

2. Materials and Methods

2.1. Participants and Procedure

Data for this study were collected mostly between March and September of 2020, except for one sample which was collected in March of 2021. The data collection effort began in the context of a project devised in 2019. Ethical approval was obtained from the Ethics Committee of the University of Navarra (2020.087) in early 2020. Researchers went into the field in late February 2020, before the global COVID-19 pandemic. Data collection was interrupted by the health crisis in March 2020, when 624 telephone surveys had been completed. In this extraordinary situation, the team of researchers decided that the original data collection effort in Spain would be interrupted until the lockdown was lifted. Starting in May 2020, as limitations waned, researchers went back into the field to collect the remaining 576 telephone surveys (completing the sample Spain 1 on 18 June 2020). This Spanish sample of 1200 youth aged 18 to 22 (Spain 1) was recontacted a year later (March 2021) for a follow-up telephone interview (Spain 2—retention rate was 87% of the original sample, and the remaining 13% was replaced with youth with similar sociodemographic characteristics). These two telephone interviews are the only nationally representative samples in our study. Even though these are panel data, for the purposes of this study we use the two samples separately.
In March 2020, when the pandemic started, the team decided to expand the project internationally, inviting academic peers in 10 Spanish-speaking countries to contribute with online surveys. This resulted in the collection of 11 cross-sectional convenience samples (using the same questionnaire) in Argentina, Chile, Colombia, Ecuador, El Salvador, Guatemala, México, Perú, Spain (Spain 3), Uruguay, and Venezuela.
Table 2 provides sample sizes and some demographic data on each sample. To increase comparability across samples, we selected sub-samples of individuals ages 18 to 29 years old in each country. The sample sizes vary between 212 in Argentina and 1200 in Spain. Online surveys produced convenience samples, which resulted in an over-representation of women which was particularly acute in Colombia (80.1% women) and Argentia (77.8% women).
Average ages ranged from 20 years old in Spain 1 to almost 24 years old in Ecuador. Respondents with higher levels of education were over-represented in all non-nationally representative samples, particularly in Argentina and Chile, where about half of the sample had reached higher-education levels.

2.2. Instruments

The online survey used for the present study comprised three sections: (a) sociodemographics, (b) the Spanish SAS-SV version previously validated by Lopez-Fernandez (2017), and (c) scales with related constructs. The socio-demographic variables included were gender, age, and country.
The SAS-SV included 10 items rated on from 1 (‘strongly disagree’) to 6 (‘strongly agree’), giving a score between 10 and 60 in which the highest score shows the maximum presence of ‘smartphone addiction’. It has originally shown content and concurrent validity and internal consistency (Cronbach’s Alpha: 0.91). Items include situations, such as ‘missing planned work due to smartphone use’, ‘being told by people around me that I use my smartphone too much’ or ‘feeling impatient and fretful when not holding my smartphone’ (for a complete list of the items, in English and Spanish, see Appendix A). The scales of related constructs were the following:
  • Psychological distress. Anxiety, depression, and stress through the DASS-21 scale [57]; following the authors’ manual [58]. It is a self-reporting scale with 21 items, seven per category: depression (e.g., loss of self-esteem and low mood), anxiety (e.g., fear anticipation of negative outcomes), and stress (e.g., state of hyper-arousal and low-frustration tolerance). Each item is measured with a 4-point severity/frequency rating scale from 0 (‘did not apply to me at all’) to 3 (‘applied to me very much, or most of the time’), with higher scores representing more severe levels of psychological distress in the last week. The original Cronbach Alphas are 0.91, 0.84 and 0.90 for the subscales of depression, anxiety, and stress, respectively. We flagged respondents with severe or extremely severe scores in each subscale: anxiety (a value of 8 or more), depression (11 or more), and stress (13 or more).
  • Time in social media. Following common practice [16,59,60], we flagged individuals with more than 4 h of self-reported daily use of social media (net of the time spent in direct communication, which was collected as a separate question).

2.3. Analysis Plan

Adapting and validating scales involves a series of steps, each of which includes, in turn, several small but potentially relevant decisions. The goal of this analysis is to unpack those decisions and to illustrate their relevance, particularly regarding the classification of participants into categories below and above some theoretically relevant cutoff point. We focus on decisions concerning the estimation of a threshold to classify participants into ‘addicted’ (note that when we classify individuals as ‘addicted’ we are not making a clinical assessment, but simply following the SAS-SV (Smartphone Addiction Scale) rationale. We are of the belief that the addiction framework is not appropriate for this sort of scale, and will refer to ‘problematic use’ in all cases, except when we explicitly refer to the output of the SAS-SV. In any case, we will always use quotation marks to indicate that ‘addicted’ is a scale-derived construct, not necessarily a diagnosis or an accurate description of reality) or ‘non-addicted’ individuals. We use all 13 samples to illustrate the difficulties inherent in this process; the Spanish version of the SAS-SV was applied in all samples. We used Stata 16 for our analyses.
Following the original SAS-SV paper [17], we took three evaluating steps (omitting those directly related to item selection):
  • Reliability: evaluates internal consistency, i.e., the degree to which the items that make up a scale behave consistently with respect to the construct being evaluated. This is evaluated using the overall Cronbach Alpha of SAS-SV in each sample.
  • Concurrent validity (criterion-related): refers to the extent to which a scale correlates with indicators (criteria) that are theoretically expected to be associated with it. We tested Pearson’s correlations between the SAS-SV and four external criteria: anxiety, depression, stress, and excessive social media.
  • Cutoff point calculation: a process that involves the following sub-steps:
    • Determine a ‘true condition’, a criterion capable of identifying ‘addicted’ smartphone users. The gold standard would be to carry out a clinical evaluation of each individual. However, this is expensive and seldom done (in the case of the SAS-SV and the nine language versions presented in Table 1, only the original scale by Kwon and colleagues used clinical evaluations to determine true smartphone ‘addiction’). Lacking a clinical evaluation, researchers use external criteria which are expected to be associated with the scale, to distinguish between truly ‘addicted’ individuals (what we will call ‘problematic profiles’ below). These external criteria include, for example, addiction symptoms from the DSM-IV [61,62], lack of self-control [63], subjective feeling of smartphone addiction [32] symptoms of depression and anxiety [21,64], or correlation with other internet addiction scales [65].
    • Compare SAS-SV scores with the true condition above, calculating true positives (TP: ‘addicted’ individuals who were correctly classified as such), false positives (FP: people classified ‘addicted’ by the SAS-SV who are not truly ‘addicts’), true negatives (TN: subjects who are not ‘addicted’ and were correctly classified as such), and false negatives (FN: ‘addicts’ who are classified as not having any problem with their smartphone use). From these values, one calculates the sensitivity [TP/(TP + FN)] and the specificity [TN/(TN + FP)] of the SAS-SV scale.
    • Calculate a cutoff point: using receiver operator characteristics (ROC) curve methodology, look for the cutoff value that maximizes the area under this (ROC) curve (AUC). This is given by the highest Youden Index among all possible combinations of true conditions and cutoff points.
This is a complex process that involves a series of decisions that might sway the results one way or another. Particularly problematic is, we argue, the definition of true conditions when researchers lack expert clinical evaluations of the outcome being assessed (in our case, smartphone ‘addiction’). In the remainder of this paper, we use our 13 Spanish-speaking samples to show that, depending on the decisions taken along the way, researchers can obtain very heterogeneous results. Here we present only results from the general sample of individuals under the age of 30.

3. Results

3.1. Reliability and Concurrent Validity

The SAS-SV exhibits a strong degree of reliability across samples, with Cronbach Alphas above 0.8, with Argentina being (α = 0.81) the lowest and Ecuador (α = 0.87) the highest. See Table 3.
With respect to concurrent validity (also called criterion-related validity), we observe significant correlations between SAS-SV scores and four of the criteria that the literature generally associates with problematic smartphone use: anxiety, depression, stress, and daily time in social media. Correlations are as high as 0.51 between SAS-SV scores and severe stress in Venezuela; the lowest scores (still statistically significant) are with social media use, ranging from 0.15 in Colombia to 0.34 in the Spain 1 sample.
Generally, this implies that the SAS-SV is a reliable estimate, one that covaries significantly with measures of psychological distress and with social media screen time. Most adaptations of the scale (and most uses and adaptations of similar tools designed to measure problematic smartphone use) stop here, assuming that this is enough to guarantee a proper use of the scale. Below, we argue that here is where things might get messier; a series of decisions, and differences across samples, can still lead to very highly heterogeneous results.

3.2. Difficulties in Establishing a Cutoff Point Points for Each Sample

After assessing reliability and concurrent validity, we build 15 potential problematic profiles, by combining (individually, in pairs, and in groups of three and four) four variables commonly associated with problematic smartphone use: anxiety, depression, stress, and time in social media. We flag individuals with severe symptoms in the problematic smartphone use related constructs, such as psychological distress and time on social media.
Table 4 presents five of the fifteen potential problematic profiles. It shows the proportion of individuals in our samples who exhibit severe rates of anxiety, depression, or stress, as well as those using social media for more than 4 h every day. We note that the prevalence of these conditions, taken one by one, among youth aged 18 to 29, is relatively high; most proportions are between 20% and 40%.
The last column in Table 4 shows the percentage of youth scoring high in all four of the criteria; a very strict condition, met by less than 10% of youth in all countries. To make our case as conservative as possible, we will use this most stringent criterion to determine whether a participant is truly problematic in his use of the smartphone (‘addicted’, in SAS-SV terminology). We considered adding other criteria, such as subjective sense of control in the use of the smartphone, or youth’s attachment styles; however, these were discarded from our final models due to weak correlation with other indicators and with SAS-SV (results available upon request).
The next step, after determining the true condition of the individuals in our sample (i.e., participants who show problematic symptomatology potentially associated with detrimental smartphone use), was to calculated sensitivity (the ability to correctly identify a person with smartphone addiction, i.e., to minimize false negatives) and specificity (the ability to correctly identify a person with a healthy level of smartphone use, i.e., to minimize false positives). These calculations were repeated for each sample, under each of the 15 potential problematic profiles described above, providing 195 cutoff points for the determination of ‘smartphone addiction’ using the SAS-SV (data available upon request).
These cutoff points are extremely diverse, ranging from a low of 21 to a high of 45; it is important to notice that these cutoff points would all be statistically possible; theoretically, they would only differ in the way they define the true condition, lacking a clinical assessment of individuals. To illustrate this disparity, the low and high cutoff points correspond to the following scenarios:
  • In Ecuador, if we determined the true condition with ‘social media’ as the only criterion (individuals using it for longer than 4 h per day), the statistically ideal cutoff point would be 21 (out of 60). This would lead to the conclusion that 69% of Ecuadorians are ‘addicted’ to their smartphone.
  • In Venezuela, if we determined the true condition as those with severe symptoms of anxiety and depression, who also use social media more than 4 h a day, the statistically ideal cutoff point would be 45. We would conclude that only 7.8% of Venezuelans are ‘addicted’ to the smartphone.
In between these two extremes, we find all possible combinations of criteria. Even though, looking at these results, one would be tempted to disregard the criteria used to define a true condition as theoretically inappropriate, these are not uncommon criteria; they all have been used in previous research, sometimes with the SAS-SV scale, sometimes with another one of the other 78 scales assessing problematic smartphone use reviewed by Harris et al. [7].

3.3. Two Possible Rules to Choose Cutoff Points

Given that our 13 datasets and 15 potential definitions of problematic smartphone use profiles produced up to 195 potential cutoff points, we need criteria to discriminate between such a diverse set of scenarios.
The usual rule to pick one cutoff point among many, within the same sample, is to maximize the area under the ROC curve, which is the same as picking the cutoff point with the highest Youden Index [9]. The main advantage of this rule is that it provides the cutoff point with the best balance between sensitivity and specificity, i.e., the cutoff point that minimizes misclassification of individuals; the allocation of ‘addicted’ participants in the ‘non-addicted’ category, and vice-versa. Its main problem is that it might lead to a definition of different true conditions across samples. One scale, like the SAS-SV, would classify individuals in the ‘addicted’ and ‘non-addicted’ categories based on diverse criteria; for example, in one sample the true condition would be defined by having severe anxiety and excessive use of social media (i.e., Chile), while in another sample the true condition would be defined by having severe anxiety, depression, and stress, and using social media extensively (i.e., Spain 2, Colombia, Ecuador, El Salvador, México).
Table 5 further illustrates this point; it represents the criterion that maximizes the Youden Index, together with the recommended cutoff value, and the proportion of the general population that would be defined as ‘addicted’, in each of our 13 samples. We find up to five possible criteria: (1) SM: severe stress, excessive use of social media; (2) AM: severe anxiety, excessive social media use; (3) DSM: severe depression, severe stress, excessive social media use; (4) ASM: severe anxiety, severe stress, excessive social media use; and (5) ADSM: severe anxiety, severe depression, severe stress, and excessive social media use (this was the most common solution, found in five of our thirteen samples).
Following this rule, our resulting cutoff points would vary between 26 in Guatemala and 38 in Argentina; the percentage of ‘addicted’ individuals would vary in the opposite direction (as expected, since higher cutoff values mean more stringent definitions of ‘addiction’), with 52.7% of ‘addicted’ youth in Guatemala, and only 16.5% of ‘addicted’ youth in Argentina.
A second possible rule, beyond picking the true condition with the highest Youden Index in each sample, is to pick a theoretically relevant definition of the true condition and apply it to all samples, regardless of whether it leads you to pick the option with the highest Youden Index. Even if this rule would lead to instances in which the classification error is higher than it might otherwise be, it has the advantage of unifying the criteria for the definition of a true condition, making it possible to compare results sensibly across samples.
Table 6 illustrates this rule, by showing the solutions provided by the most stringent definition of the true problematic condition: a profile in which the individual presents symptoms of severe anxiety, severe depression, severe stress, as well as excessive use of social media (over 4 h per day). If we were to follow this second rule (a common definition of the true condition), cutoff values would range from 26 in Guatemala to 40 in Argentina, leading us to determine that 52.7% of Guatemalan youth are ‘addicted’ to their smartphone, compared to only 14.2% of Argentinians.
One final option to be considered would be the creation of a common cutoff point across all Spanish-speaking countries, by combining all samples into one. This would perhaps facilitate comparability across samples, but it might also mask important cultural differences. We explored this option. Using the total sample of 6937 youth aged 18 to 39, the results provided an overall SAS-SV cutoff score of 33, regardless of the rule employed (highest Youden Index, or using ADSM as the true condition). This is not far from the original Korean study which established a cutoff value of 31 for men and 33 for women [9] and the Spanish adaptation which established a common cutoff point of 32 for both men and women [10]. This common cross-sample cutoff point would result in an overall rate of 23% ‘addicted’ youth, but the percentages would vary by sample as follows: 23% in Spain 1, 20% in Spain 2, 18% in Spain 3, 20% in Argentina, 28% in Chile, 26% in Colombia, 22% in Ecuador, 33% in El Salvador, 22% in Guatemala, 21% in Mexico, 27% in Perú, 23% in Uruguay, and 27% in Venezuela.

4. Discussion

The smartphone revolution has put powerful, multipurpose devices in the hands of youth across the globe, leading many to wonder about the potential negative consequences of these technologies on mental health. As a result, the research community has provided many competing assessment tools, many of which attempt to identify individuals with a higher risk of developing unhealthy habits in the use of their smartphones. These individuals, it is argued, can be identified because they cross some theoretically meaningful quantifiable threshold; a diagnostic cutoff score within the range of possible values given by a scale.
The present study aimed to illustrate the difficulty in establishing ‘true’ conditions and objective cutoff points, by applying the Spanish SAS-SV to 13 different Spanish-speaking samples. We explored the psychometric properties of this scale, reliability (Cronbach’s Alpha) and concurrent validity (i.e., based on the variety of criteria), and the resulting cutoff points aimed at estimating users with problematic use of their smartphone within Hispanophone countries.
Regarding the main psychometric properties, the Spanish SAS-SV obtained similar internal consistencies, such as the original one ([9]), i.e., 0.81 to 0.87 (vs. 0.91 of the original SAS-SV and the 0.88 Spanish SAS-SV [10]). This therefore shows the stability of this measure alongside time and space, even during the pandemic.
Concerning criterion-related validity, the SAS-SV shows significant correlations with anxiety, depression, and stress, across all thirteen samples (most of them in the 0.3 to 0.5 range). Regarding social media time, the scale also presents significant associations in all samples. Thus, the SAS-SV proves a reliable estimate which varies along with measures of psychological distress and is associated with social media time. Similar findings have been observed regarding depression and anxiety as predictors of smartphone addiction [64], although other mental health constructs also seem to be related, such as insomnia, social phobia, or loneliness [66]. Problematic smartphone use has been associated with a variety of outcomes, aside from those explored here. Those include personality traits, such as impulsiveness and the ability to self-control [64,67,68]. An individual’s prevalent attachment style can also be related to her use of the smartphone; secure attachment prevents excessive use [69,70], whereas avoidant and insecure attachment increases excessive use [71,72,73], in what Billieux and colleagues [74] described as the reassurance-seeking pathway to problematic smartphone use.
However, the most original contribution of the present study is to compute all possible cutoff scores resulting from the 15 smartphone problematic profiles derived from all possible combinations between the four related constructs (i.e., depressive, anxious, stressed, and spending excessive time on social media), in all 13 samples. To the best of the authors’ knowledge, the field of technological behavioral addiction lacks an evidence-based, consistent strategy to compute cutoff values of a cross-cultural validated scale, such as the SAS-SV. This study is a first attempt to explore the complexity of this problem.
We obtained numbers that vary enormously across Spanish countries. The cutoff points range from 26 to 40, resulting in proportions of ‘addicted’ individuals that vary between 14% and 58%. Even when we try to impose the same decisions across all samples, we get proportions of ‘addicted’ youth that can be as low as 18% and as high as 33%.This might correspond to reality; there is no reason to believe that all countries must have similar proportions of youth engaging in problematic smartphone use. Nonetheless, it proves our point: the decisions and rules deployed by researchers might be highly consequential for the results obtained.
These findings provide a warning and a silver lining. First, the warning is that, given that most researchers do not have (as we do) alternative samples to assess the robustness of their results, they should be very clear about their decisions, making sure that their choices are not only methodologically correct, but also theoretically grounded—as recent literature reviews have stated repeatedly [5,6,7]. We also find, in our data, a silver lining. When we compare the final results obtained by applying the two decision rules (highest Youden; common true condition) sample by sample, we obtain the same cutoff points and percentages of ‘addicted’ youth in 11 out of our 13 samples; different cutoff values are found only in Argentina (38 vs. 40, respectively) and Peru (36 vs. 33). This means that, while one could not guarantee similar results when applying different criteria and rules, there is the possibility that, all in all, they tend to produce converging results. In this case, it would be up to the researchers to justify their decision only in the few cases in which there are dissimilar results for the same sample or country.
This research is not without limitations. First, only two of our samples are nationally representative; all other samples are online convenience samples, with some of them having been taken in countries where internet connectivity is limited in regard to large segments of the population. Thus, most of our samples over-represent individuals with higher education and social class, with a prevalence of women over men. Second, we did not carry out objective clinical assessments, which is the ideal tool to determine the true ‘addicted’ condition of individuals; this limitation is present in the study design, given that most scale adaptations and validations do not use this gold standard either. Our goal was, precisely, to warn about the heterogeneity of results that one might obtain when building that ‘true’ condition lacking a clinical assessment. Absent of this gold standard, researchers could try different specifications and explore the robustness of their results. In the end, decisions can be both clinically and methodologically sound, empirically robust, and theoretically meaningful.
Establishing cutoff points, therefore, needs to have a gold standard, which should emerge from clinical assessment with a methodological basis. As stated in the title, this clinical control is difficult because researchers have no gold standard to compare with the result of SAS-SV. Each participant’s clinical assessment is necessary in future studies to reach the cutoff point for smartphone addiction subsequently.
Finally, there is no reason to expect that cutoff points will remain stable over time. When it comes to the behavior of youth and rapidly evolving technologies, one should expect changes in the types and motivations of use of smartphones. Thus, cutoff points might prove poor classification tools after only a few years, as smartphones become better tools to carry out more and more useful daily tasks. For this reason, cutoff points should be reassessed and updated with some frequency.

5. Conclusions

In this paper, we use the SAS-SV. Available in at least 10 languages covering most of the world population, the SAS-SV is one of the most widely used assessment tools in this area of research. We deployed a similar questionnaire, containing the SAS-SV and other relevant outcomes, in 13 samples in 11 Spanish-speaking countries, including Spain and 10 Latin-American nations. With these data, we illustrated the difficulties in establishing a true condition to evaluate and estimate a cutoff point for scales seeking to classify individuals, by their smartphone use, as ‘addicted’ or ‘not-addicted’ to their devices.
We were able to show how and why the decisions made by the researcher matter in the conclusions one draws about the prevalence of problematic smartphone uses across samples and countries. In consequence, we recommend transparency when reporting the decisions taken in the process of classifying individuals into categories. Hence our warning: researchers seeking to validate or adapt a scale or seeking to establish cutoff points for existing scales that measure clinical conditions, should either find a way to assess individuals with clinical assessment tools, or be very careful about the decisions they make to compensate for the lack of that gold standard—a clinical assessment. It seems to us that many studies have perhaps been unaware of these difficulties, applying tools across countries, languages, and samples without a careful consideration of the consequences of their decisions, not to mention other cultural and socioeconomic issues which must also be addressed, but fall outside the scope of this paper.
It is our hope that this study will help assess the adaptation and validation of other scales beyond the SAS-SV, and revisit some of the evidence drawn from studies that resorted uncritically to cutoff points to classify individuals in strict categories, leading perhaps to the misclassifications of people as ‘addicted’ or ‘problematic’.

Author Contributions

Conceptualization, J.G.-M.; methodology, J.G.-M. and C.S.; formal analysis, J.G.-M. and C.S.; investigation, J.G.-M., C.L.-M.; data curation, C.S.; writing—original draft preparation, J.G.-M., C.S. and C.L.-M.; writing—review and editing, J.G.-M. and O.L.-F.; supervision, O.L.-F. and C.S.-C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of Universidad de Navarra (protocol code 2020.087).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data is not publicly available due to ongoing analyses. However, the data presented in this study can be available upon request to the corresponding author.

Acknowledgments

We would like to acknowledge the universities that collaborated in this study: Universidad Austral (Argentina), Pontificia Universidad Católica de Chile (Chile), Universidad de La Sabana (Colombia), Universidad de Las Américas (Ecuador), Escuela Superior de Economía y Negocios (El Salvador), Universidad de Navarra (España), Universidad del Istmo (Guatemala), Universidad Panamericana (México), Universidad de Piura (Perú), Universidad de Montevideo (Uruguay), and Universidad Monteávila (Venezuela).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table
Table A1. Complete list of the items in English and Spanish of the Smartphone Addiction Scale Short Version (SAS-SV).
Table A1. Complete list of the items in English and Spanish of the Smartphone Addiction Scale Short Version (SAS-SV).
EnglishSpanish
1Missing planned work due to smartphone useDebido al uso del smartphone he perdido tareas/actividades/trabajos/etc. previamente planificados
2Having a hard time concentrating in class, while doing assignments, or while working due to smartphone useDebido al uso del smartphone he tenido problemas de concentración (en clase, en el trabajo, etc.), mientras hacía mis tareas (deberes, etc.) o mientras trabajaba
3Feeling pain in the wrists or at the back of the neck while using a smartphoneDebido al uso del smartphone he sentido dolor en alguna de mis muñecas o detrás del cuello (por ejemplo, en la nuca), etc.
4Will not be able to stand not having a smartphoneNo puedo estar sin mi smartphone
5Feeling impatient and fretful when I am not holding my smartphoneMe siento impaciente e inquieto cuando no tengo mi smartphone
6Having my smartphone in my mind even when I am not using itTengo mi smartphone en mente incluso cuando no lo uso
7I will never give up using my smartphone even when my daily life is already greatly affected by itNo dejaré de usar mi smartphone incluso si mi vida cotidiana está realmente afectada por éste
8Constantly checking my smartphone so as not to miss conversations between other people on Twitter or FacebookEstoy comprobando constantemente mi smartphone para no perderme conversaciones con otras personas en las redes sociales (como Twitter, Facebook, etc.)
9Using my smartphone longer than I had intendedUso mi smartphone más de lo que había previsto inicialmente
10The people around me tell me that I use my smartphone too muchLa gente de mi alrededor me dice que uso demasiado mi smartphone

References

  1. CTIA Wireless Industry Survey 2020. Available online: https://www.ctia.org/news/report-2020-annual-survey-highlights (accessed on 19 October 2021).
  2. Ericsson Mobility Visualizer. Available online: https://www.ericsson.com/en/mobility-report/mobility-visualizer?f=1&ft=1&r=1&t=8&s=1&u=1&y=2011,2026&c=1 (accessed on 21 October 2021).
  3. Anderson, M.; Jiang, J. Others Teens, social media & technology 2018. Pew Res. Res. Cent. 2018, 31, 1673–1689. [Google Scholar]
  4. López-Fernández, O.; Honrubia-Serrano, M.L.; Freixa-Blanxart, M. Spanish adaptation of the “Mobile Phone Problem Use Scale” for adolescent population. Adicciones 2012, 24, 123–130. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Lopez-Fernandez, O.; Honrubia-Serrano, L.; Freixa-Blanxart, M.; Gibson, W. Prevalence of problematic mobile phone use in British adolescents. Cyberpsychol. Behav. Soc. Netw. 2014, 17, 91–98. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Pedrero Pérez, E.J.; Rodríguez Monje, M.T.; Ruiz Sánchez De León, J.M. Mobile phone abuse or addiction. A review of the literature. Adicciones 2012, 24, 139–152. [Google Scholar] [CrossRef] [Green Version]
  7. Harris, B.; Regan, T.; Schueler, J.; Fields, S.A. Problematic Mobile Phone and Smartphone Use Scales: A Systematic Review. Front. Psychol. 2020, 11, 672. [Google Scholar] [CrossRef]
  8. Ellis, D.A.; Davidson, B.I.; Shaw, H.; Geyer, K. Do smartphone usage scales predict behavior? Int. J. Hum. Comput. Stud. 2019, 130, 86–92. [Google Scholar] [CrossRef]
  9. Kwon, M.; Kim, D.-J.; Cho, H.; Yang, S. The smartphone addiction scale: Development and validation of a short version for adolescents. PLoS ONE 2013, 8, e83558. [Google Scholar] [CrossRef] [Green Version]
  10. Lopez-Fernandez, O. Short version of the Smartphone Addiction Scale adapted to Spanish and French: Towards a cross-cultural research in problematic mobile phone use. Addict. Behav. 2017, 64, 275–280. [Google Scholar] [CrossRef]
  11. Andrade, A.L.M.; Kim, D.-J.; Caricati, V.V.; Martins, G.D.G.; Kirihara, I.K.; Barbugli, B.C.; Enumo, S.R.F.; De Micheli, D. Validity and reliability of the Brazilian version of the Smartphone Addiction Scale-Short Version for university students and adult population. Estudos de Psicologia. 2020, 37, e190117. [Google Scholar] [CrossRef] [Green Version]
  12. Sfendla, A.; Laita, M.; Nejjar, B.; Souirti, Z.; Touhami, A.A.O.; Senhaji, M. Reliability of the Arabic Smartphone Addiction Scale and Smartphone Addiction Scale-Short Version in Two Different Moroccan Samples. Cyberpsychol. Behav. Soc. Netw. 2018, 21, 325–332. [Google Scholar] [CrossRef]
  13. Haug, S.; Castro, R.P.; Kwon, M.; Filler, A.; Kowatsch, T.; Schaub, M.P. Smartphone use and smartphone addiction among young people in Switzerland. J. Behav. Addict. 2015, 4, 299–307. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. De Pasquale, C.; Sciacca, F.; Hichy, Z. Italian Validation of Smartphone Addiction Scale Short Version for Adolescents and Young Adults (SAS-SV). Psychology 2017, 8, 1513–1518. [Google Scholar] [CrossRef] [Green Version]
  15. Luk, T.T.; Wang, M.P.; Shen, C.; Wan, A.; Chau, P.H.; Oliffe, J.; Viswanath, K.; Chan, S.S.-C.; Lam, T.H. Short version of the Smartphone Addiction Scale in Chinese adults: Psychometric properties, sociodemographic, and health behavioral correlates. J. Behav. Addict. 2018, 7, 1157–1165. [Google Scholar] [CrossRef] [PubMed]
  16. Nikolic, A.; Bukurov, B.; Kocic, I.; Sipetic, S. The Validity and Reliability of the Serbian Version of the Smartphone Addiction Scale—Short Version. Res. Sq. 2021. [Google Scholar] [CrossRef]
  17. Kwon, M.; Lee, J.-Y.; Won, W.-Y.; Park, J.-W.; Min, J.-A.; Hahn, C.; Gu, X.; Choi, J.-H.; Kim, D.-J. Development and validation of a smartphone addiction scale (SAS). PLoS ONE 2013, 8, e56936. [Google Scholar]
  18. Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett. 2006, 27, 861–874. [Google Scholar] [CrossRef]
  19. Elhai, J.D.; Levine, J.C.; Hall, B.J. The relationship between anxiety symptom severity and problematic smartphone use: A review of the literature and conceptual frameworks. J. Anxiety Disord. 2019, 62, 45–52. [Google Scholar] [CrossRef] [PubMed]
  20. Demirci, K.; Akgönül, M.; Akpinar, A. Relationship of smartphone use severity with sleep quality, depression, and anxiety in university students. J. Behav. Addict. 2015, 4, 85–92. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Chen, B.; Liu, F.; Ding, S.; Ying, X.; Wang, L.; Wen, Y. Gender differences in factors associated with smartphone addiction: A cross-sectional study among medical college students. BMC Psychiatry 2017, 17, 341. [Google Scholar] [CrossRef] [Green Version]
  22. Contractor, A.A.; Frankfurt, S.B.; Weiss, N.H.; Elhai, J.D. Latent-level relations between DSM-5 PTSD symptom clusters and problematic smartphone use. Comput. Hum. Behav. 2017, 72, 170–177. [Google Scholar] [CrossRef]
  23. Enez Darcin, A.; Kose, S.; Noyan, C.O.; Nurmedov, S.; Yılmaz, O.; Dilbaz, N. Smartphone addiction and its relationship with social anxiety and loneliness. Behav. Inf. Technol. 2016, 35, 520–525. [Google Scholar] [CrossRef]
  24. Hawi, N.S.; Samaha, M. Relationships among smartphone addiction, anxiety, and family relations. Behav. Inf. Technol. 2017, 36, 1046–1052. [Google Scholar] [CrossRef]
  25. Rozgonjuk, D.; Levine, J.C.; Hall, B.J.; Elhai, J.D. The association between problematic smartphone use, depression and anxiety symptom severity, and objectively measured smartphone use over one week. Comput. Hum. Behav. 2018, 87, 10–17. [Google Scholar] [CrossRef]
  26. Elhai, J.; Levine, J.; Dvorak, R.; Hall, B. Non-social features of smartphone use are most related to depression, anxiety and problematic smartphone use. Comput. Hum. Behav. 2017, 69, 75–82. [Google Scholar] [CrossRef]
  27. Alhassan, A.; Alqadhib, E.; Taha, N.; Alahmari, R.; Salam, M.; Almutairi, A. The relationship between addiction to smartphone usage and depression among adults: A cross sectional study. BMC Psychiatry 2018, 18, 148. [Google Scholar] [CrossRef]
  28. Andreassen, C.S.; Billieux, J.; Griffiths, M.D.; Kuss, D.J.; Demetrovics, Z.; Mazzoni, E.; Pallesen, S. The relationship between addictive use of social media and video games and symptoms of psychiatric disorders: A large-scale cross-sectional study. Psychol. Addict. Behav. 2016, 30, 252–262. [Google Scholar] [CrossRef] [Green Version]
  29. Rasmussen, E.E.; Punyanunt-Carter, N.; LaFreniere, J.R.; Norman, M.S.; Kimball, T.G. The serially mediated relationship between emerging adults’ social media use and mental well-being. Comput. Hum. Behav. 2020, 102, 206–213. [Google Scholar] [CrossRef]
  30. Wolniewicz, C.A.; Tiamiyu, M.F.; Weeks, J.W.; Elhai, J.D. Problematic smartphone use and relations with negative affect, fear of missing out, and fear of negative and positive evaluation. Psychiatry Res. 2018, 262, 618–623. [Google Scholar] [CrossRef]
  31. Choksi, S.T. A study to find out the correlation of mobile phone addiction with anxiety, depression, stress and sleep quality in the college students of Surat city. Int. J. Curr. Res. Rev. 2021, 13, 137–142. [Google Scholar] [CrossRef]
  32. Dharmadhikari, S.P.; Harshe, S.D.; Bhide, P.P. Prevalence and Correlates of Excessive Smartphone Use among Medical Students: A Cross-sectional Study. Indian J. Psychol. Med. 2019, 41, 549–555. [Google Scholar] [CrossRef]
  33. Andreassen, C.S.; Pallesen, S.; Griffiths, M.D. The relationship between addictive use of social media, narcissism, and self-esteem: Findings from a large national survey. Addict. Behav. 2017, 64, 287–293. [Google Scholar] [CrossRef] [Green Version]
  34. Jiang, Z.; Zhao, X. Self-control and problematic mobile phone use in Chinese college students: The mediating role of mobile phone use patterns. BMC Psychiatry 2016, 16, 416. [Google Scholar] [CrossRef] [Green Version]
  35. Horwood, S.; Anglim, J. Problematic smartphone usage and subjective and psychological well-being. Comput. Hum. Behav. 2019, 97, 44–50. [Google Scholar] [CrossRef] [Green Version]
  36. Vannucci, A.; Flannery, K.M.; Ohannessian, C.M. Social media use and anxiety in emerging adults. J. Affect. Disord. 2017, 207, 163–166. [Google Scholar] [CrossRef]
  37. Coyne, S.M.; Stockdale, L.; Summers, K. Problematic cell phone use, depression, anxiety, and self-regulation: Evidence from a three year longitudinal study from adolescence to emerging adulthood. Comput. Hum. Behav. 2019, 96, 78–84. [Google Scholar] [CrossRef]
  38. Matar Boumosleh, J.; Jaalouk, D. Depression, anxiety, and smartphone addiction in university students—A cross sectional study. PLoS ONE 2017, 12, e0182239. [Google Scholar]
  39. Elhai, J.; Dvorak, R.; Levine, J.; Hall, B. Problematic smartphone use: A conceptual overview and systematic review of relations with anxiety and depression psychopathology. J. Affect. Disord. 2017, 2017, 251–259. [Google Scholar] [CrossRef] [PubMed]
  40. Guo, N.; Luk, T.T.; Wang, M.P.; Ho, S.Y.; Fong, D.Y.T.; Wan, A.; Chan, S.S.-C.; Lam, T.H. Self-Reported Screen Time on Social Networking Sites Associated With Problematic Smartphone Use in Chinese Adults: A Population-Based Study. Front. Psychiatry 2020, 11, 614061. [Google Scholar] [CrossRef] [PubMed]
  41. Kumar, V.A.; Chandrasekaran, V.; Brahadeeswari, H. Prevalence of smartphone addiction and its effects on sleep quality: A cross-sectional study among medical students. Indian Psychiatry J. 2019, 28, 82–85. [Google Scholar]
  42. Randjelovic, P.; Stojiljkovic, N.; Radulovic, N.; Stojanovic, N.; Ilic, I. Problematic Smartphone Use, Screen Time and Chronotype Correlations in University Students. Eur. Addict. Res. 2021, 27, 67–74. [Google Scholar] [CrossRef] [PubMed]
  43. Romero-Rodríguez, J.-M.; Aznar-Díaz, I.; Marín-Marín, J.-A.; Soler-Costa, R.; Rodríguez-Jiménez, C. Impact of Problematic Smartphone Use and Instagram Use Intensity on Self-Esteem with University Students from Physical Education. Int. J. Environ. Res. Public Health 2020, 17, 4336. [Google Scholar] [CrossRef]
  44. Laurence, P.G.; Busin, Y.; da Cunha Lima, H.S.; Macedo, E.C. Predictors of problematic smartphone use among university students. Psicol. Reflex. Crit. 2020, 33, 8. [Google Scholar] [CrossRef] [PubMed]
  45. Cocoradă, E.; Maican, C.I.; Cazan, A.-M.; Maican, M.A. Assessing the smartphone addiction risk and its associations with personality traits among adolescents. Child. Youth Serv. Rev. 2018, 93, 345–354. [Google Scholar] [CrossRef]
  46. Kraut, R.; Patterson, M.; Lundmark, V.; Kiesler, S.; Mukophadhyay, T.; Scherlis, W. Internet paradox: A social technology that reduces social involvement and psychological well-being? Am. Psychol. 1998, 53, 1017–1031. [Google Scholar] [CrossRef]
  47. Winstone, L.; Mars, B.; Haworth, C.M.A.; Kidger, J. Social media use and social connectedness among adolescents in the United Kingdom: A qualitative exploration of displacement and stimulation. BMC Public Health 2021, 21, 1736. [Google Scholar] [CrossRef]
  48. Twenge, J.M.; Spitzberg, B.H.; Campbell, W.K. Less in-person social interaction with peers among U.S. adolescents in the 21st century and links to loneliness. J. Soc. Pers. Relat. 2019, 36, 1892–1913. [Google Scholar] [CrossRef]
  49. Boer, M.; van den Eijnden, R.J.J.M.; Boniel-Nissim, M.; Wong, S.-L.; Inchley, J.C.; Badura, P.; Craig, W.M.; Gobina, I.; Kleszczewska, D.; Klanšček, H.J.; et al. Adolescents’ Intense and Problematic Social Media Use and Their Well-Being in 29 Countries. J. Adolesc. Health 2020, 66, S89–S99. [Google Scholar] [CrossRef]
  50. Houghton, S.; Lawrence, D.; Hunter, S.C.; Rosenberg, M.; Zadow, C.; Wood, L.; Shilton, T. Reciprocal Relationships between Trajectories of Depressive Symptoms and Screen Media Use during Adolescence. J. Youth Adolesc. 2018, 47, 2453–2467. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Twenge, J.M.; Campbell, W.K. Associations between screen time and lower psychological well-being among children and adolescents: Evidence from a population-based study. Prev. Med. Rep. 2018, 12, 271–283. [Google Scholar] [CrossRef] [PubMed]
  52. Loid, K.; Täht, K.; Rozgonjuk, D. Do pop-up notifications regarding smartphone use decrease screen time, phone checking behavior, and self-reported problematic smartphone use? Evidence from a two-month experimental study. Comput. Hum. Behav. 2020, 102, 22–30. [Google Scholar] [CrossRef]
  53. Stiglic, N.; Viner, R.M. Effects of screentime on the health and well-being of children and adolescents: A systematic review of reviews. BMJ Open 2019, 9, e023191. [Google Scholar] [CrossRef] [Green Version]
  54. Beyens, I.; Pouwels, J.L.; van Driel, I.I.; Keijsers, L.; Valkenburg, P.M. The effect of social media on well-being differs from adolescent to adolescent. Sci. Rep. 2020, 10, 10763. [Google Scholar] [CrossRef] [PubMed]
  55. Orben, A.; Przybylski, A.K. The association between adolescent well-being and digital technology use. Nat. Hum. Behav. 2019, 3, 173–182. [Google Scholar] [CrossRef] [Green Version]
  56. Coyne, S.M.; Rogers, A.A.; Zurcher, J.D.; Stockdale, L.; Booth, M. Does time spent using social media impact mental health?: An eight year longitudinal study. Comput. Hum. Behav. 2020, 104, 106160. [Google Scholar] [CrossRef]
  57. Lovibond, P.F.; Lovibond, S.H. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav. Res. Ther. 1995, 33, 335–343. [Google Scholar] [CrossRef]
  58. Lovibond, S.H.; Lovibond, P.F. Manual for the Depression Anxiety Stress Scales; Psychology Foundation of Australia: Sydney, Australia, 1996. [Google Scholar]
  59. Sözbilir, F.; Kürşad, M. Does Social Media Usage Threaten Future Human Resources by Causing Smartphone Addiction? A Study on Students Aged 9–12. Addicta 2018, 5, 185–203. [Google Scholar] [CrossRef]
  60. Cha, S.-S.; Seo, B.-K. Smartphone use and smartphone addiction in middle school students in Korea: Prevalence, social networking service, and game use. Health Psychol. Open 2018, 5, 2055102918755046. [Google Scholar] [CrossRef]
  61. Sfendla, A.; Martinsson, B.; Filipovic, Y.; Senhaji, M.; Kerekes, N. Psychological distress in a sample of Moroccan prisoners with drug-dependence. Int. J. Offender Ther. Comp. Criminol. 2021, 21, 306624X211010286. [Google Scholar] [CrossRef]
  62. Tao, R.; Huang, X.; Wang, J.; Zhang, H.; Zhang, Y.; Li, M. Proposed diagnostic criteria for internet addiction. Addiction 2010, 105, 556–564. [Google Scholar] [CrossRef]
  63. Geng, Y.; Gu, J.; Wang, J.; Zhang, R. Smartphone addiction and depression, anxiety: The role of bedtime procrastination and self-control. J. Affect. Disord. 2021, 293, 415–421. [Google Scholar] [CrossRef] [PubMed]
  64. Aker, S.; Şahin, M.K.; Sezgin, S.; Oğuz, G. Psychosocial Factors Affecting Smartphone Addiction in University Students. J. Addict. Nurs. 2017, 28, 215–219. [Google Scholar] [CrossRef]
  65. Tateno, M.; Kim, D.-J.; Teo, A.R.; Skokauskas, N.; Guerrero, A.P.S.; Kato, T.A. Smartphone addiction in Japanese college students: Usefulness of the Japanese version of the Smartphone Addiction Scale as a screening tool for a new form of internet addiction. Psychiatry Investig. 2019, 16, 115–120. [Google Scholar] [CrossRef] [Green Version]
  66. Enez Darcin, A.; Noyan, C.; Nurmedov, S.; Yilmaz, O.; Dilbaz, N. Smartphone addiction in relation with social anxiety and loneliness among university students in turkey. Eur. Psychiatry 2015, 30, 505. [Google Scholar] [CrossRef]
  67. West, R.; Ash, C.; Dapore, A.; Kirby, B.; Malley, K.; Zhu, S. Problematic smartphone use: The role of reward processing, depressive symptoms and self-control. Addict. Behav. 2021, 122, 107015. [Google Scholar] [CrossRef]
  68. Xiang, M.-Q.; Lin, L.; Wang, Z.-R.; Li, J.; Xu, Z.; Hu, M. Sedentary Behavior and Problematic Smartphone Use in Chinese Adolescents: The Moderating Role of Self-Control. Front. Psychol. 2019, 10, 3032. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  69. Parent, N.; Dadgar, K.; Xiao, B.; Hesse, C.; Shapka, J.D. Social Disconnection During COVID-19: The Role of Attachment, Fear of Missing Out, and Smartphone Use. J. Res. Adolesc. 2021, 31, 748–763. [Google Scholar] [CrossRef] [PubMed]
  70. Yousaf, A.; Adil, A.; Hamza, A.; Ghayas, S.; Niazi, S.; Khan, A. Relationship between Attachment styles and Social Media addiction among young adults: Mediating role of Self-esteem. ISRA Med. 2021, 13, 48–51. [Google Scholar]
  71. Balta, S.; Jonason, P.; Denes, A.; Emirtekin, E.; Tosuntaş, Ş.B.; Kircaburun, K.; Griffiths, M.D. Dark personality traits and problematic smartphone use: The mediating role of fearful attachment. Personal. Individ. Differ. 2019, 149, 214–219. [Google Scholar] [CrossRef]
  72. Jimeno, M.V.; Ricarte, J.J.; Toledano, A.; Mangialavori, S.; Cacioppo, M.; Ros, L. Role of Attachment and Family Functioning in Problematic Smartphone Use in Young Adults. J. Fam. Issues 2021, 43, 0192513X21993881. [Google Scholar] [CrossRef]
  73. Remondi, C.; Compare, A.; Tasca, G.A.; Greco, A.; Pievani, L.; Poletti, B.; Brugnera, A. Insecure Attachment and Technology Addiction Among Young Adults: The Mediating Role of Impulsivity, Alexithymia, and General Psychological Distress. Cyberpsychol. Behav. Soc. Netw. 2020, 23, 761–767. [Google Scholar] [CrossRef]
  74. Billieux, J. Problematic Use of the Mobile Phone: A Literature Review and a Pathways Model. Curr. Psychiatry Rev. 2012, 8, 299–307. [Google Scholar] [CrossRef] [Green Version]
Table
Table 1. Characteristics of the SAS-SV in nine different language adaptations.
Table 1. Characteristics of the SAS-SV in nine different language adaptations.
LanguageReferenceN% FemaleMean AgeCutoff Value
(Gender)		Percentage “Addicted”
(Gender)		Clinical Assessment
KoreanKwon et al., 2013 [9]54036.514.531 (men)
33 (women)		20.6%
16.6% (men)
26.6% (women)		Yes
Spanish (Spain)López-Fernández, 2017 [10]28180.125.6132 (unique) a12.8%
15.2% (men)
10.2% (women)		Symptoms b
French (Belgium)López-Fernández, 2017 [10]14468.829.1132 (unique) a21.5%
20% (men)
22% (women)		Symptoms b
Portuguese (Brazil)Andrade et al., 2020 [11]718 c63.522.1
35.2		31 (men)
33 (women)		39.4%
37.7% (men)
40.4% (women)		Symptoms b
MoroccanSfendla et al., 2018 [12]31050.623.131 (men)
33 (women)		55.8%
(sex differences not specified)		No
German (Switzerland)Haug et al., 2015 [13]151951.81731 (men)
33 (women)		16.9%
14.1% (men)
19.4% (women)		No
ItalianDe Pasquale et al., 2017 [14]63355.41831 (men)
33 (women)		NANo
ChineseTsun Luk et al., 2018 [15]321154.743.331 (men)
33 (women)		37.7%
35.3% (men)
40.3% (women)		No
SerbianNikolik et al., 2021 [16]32369.02131 (men)
33 (women)		19.5%
14% (men)
22% (women)		No
a Calculated new cutoff points; all other adaptations used the same cutoff points given by Kwon et al. (2013). b The symptoms assessed were loss of control, disruption, disregard, withdrawal, preoccupation, and tolerance. Each had a score of 1 to 6, and the average scores had to be equal to or greater than 4 points to be considered as present. c The sample was part university students (N = 387, M age = 22.1) part adults (N = 331, M age = 35.2).
Table
Table 2. Samples, collection information, demographic characteristics, and SAS-SV scores.
Table 2. Samples, collection information, demographic characteristics, and SAS-SV scores.
SampleN Total SampleN Sample Age < 30Fieldwork (Start–End) [dd/mm/yy]% FemaleAverage Age (sd)% with Higher EducationSAS-SV Scores 2 Mean (sd)
Spain 1 11200120027/02/20–18/06/2050.020 (1.5)13.827 (9.8)
Spain 2 1120012001/03/21–16/04/2150.021 (1.5)13.225.2 (9.9)
Spain 398639330/03/20–29/07/2060.822.6 (3.3)39.726.3 (9.6)
Argentina54721231/03/20–20/06/2077.823.2 (2.9)50.929.7 (9.4)
Chile5041801/04/20–15/09/2065.622.5 (4.2)41.131.16 (10.6)
Colombia77027631/03/20–3/09/2080.122.4 (3.8)39.130.6 (10.3)
Ecuador7664331/04/20–30/07/2067.223.8 (3.2)48.028.5 (10.2)
El Salvador6414081/04/20–3/06/2058.821.8 (3.2)27.532.2 (10.5)
Guatemala16197922/04/20–29/08/2054.222.5 (3.5)42.728.7 (9.4)
México108447530/03/20–3/08/2069.922.7 (3.3)43.629.3 (10.2)
Perú10837202/02/20–24/09/2065.322 (3.5)37.230.1 (9.9)
Uruguay8764051/04/20–29/09/2070.422.5 (3.6)36.829.3 (10.1)
Venezuela6242438/04/20–23/06/2071.622.2 (3.4)28.030.4 (10.6)
1 Nationally representative samples administered via phone interview, with quota sampling of the Spanish territory and a margin of error of ±2.5% for a sample size of 1200; the rest are online convenience samples; we set a minimum sample size of 500 per country, for the general population, to guarantee some statistical power with margins of error circa ±5%. 2 SAS-SV scores range from 6 to 60.
Table
Table 3. Reliability and concurrent validity of SAS-SV.
Table 3. Reliability and concurrent validity of SAS-SV.
SampleReliability (Cronbach α)Pearson’s Correlation with
AnxietyDepressionStressSocial Media Use
Spain 10.830.42 ***0.36 ***0.42 ***0.34 ***
Spain 20.840.36 ***0.32 ***0.40 ***0.31 ***
Spain 30.850.35 ***0.36 ***0.39 ***0.31 ***
Argentina0.810.23 ***0.24 ***0.32 ***0.30 ***
Chile0.860.39 ***0.47 ***0.47 ***0.31 ***
Colombia0.840.30 ***0.40 ***0.38 ***0.15 *
Ecuador0.870.41 ***0.37 ***0.41 ***0.24 ***
El Salvador0.860.28 ***0.37 ***0.39 ***0.21 ***
Guatemala0.830.27 ***0.30 ***0.33 ***0.20 ***
México0.850.36 ***0.39 ***0.40 ***0.19 ***
Perú0.860.41 ***0.42 ***0.45 ***0.23 ***
Uruguay0.840.37 ***0.40 ***0.44 ***0.32 ***
Venezuela0.850.45 ***0.40 ***0.51 ***0.23 ***
* p < 0.05; *** p < 0.001.
Table
Table 4. Five profiles of problematic youth: potential criteria for the definition of a true condition for ‘addictive’ smartphone use.
Table 4. Five profiles of problematic youth: potential criteria for the definition of a true condition for ‘addictive’ smartphone use.
Percentage of Young People with Symptoms of Each Construct
SampleSevere AnxietySevere DepressionSevere StressUsing Social Media 4 + HoursProblematic in All Four Criteria
Spain 128.517.218.822.24.0
Spain 224.115.619.818.52.5
Spain 322.422.922.114.84.6
Argentina19.318.423.121.71.9
Chile38.333.338.332.87.2
Colombia37.327.934.4339.4
Ecuador39.525.228.237.48.3
El Salvador37.533.333.331.47.1
Guatemala29.824.026.326.45.3
Mexico30.323.429.330.55.5
Peru41.533.935.832.59.0
Uruguay20.522.524.023.04.0
Venezuela37.926.332.138.79.1
Table
Table 5. True conditions, cutoff values, and percentage of ‘addicted’ youth when highest Youden Index is used as the decision rule.
Table 5. True conditions, cutoff values, and percentage of ‘addicted’ youth when highest Youden Index is used as the decision rule.
SampleYouden IndexTrue Condition 1Cutoff Value% Addicted
Spain 10.462DSM3036.0
Spain 20.485ADSM3123.2
Spain 30.546ASM3127.2
Argentina0.659SM3816.5
Chile0.469AM3537.8
Colombia0.418ADSM2858.3
Ecuador0.432ADSM3524.3
El Salvador0.445ADSM3634.6
Guatemala0.318SM2652.7
México0.566ADSM3235.0
Perú0.393DSM3628.8
Uruguay0.656DSM3432.6
Venezuela0.478SM3336.2
1 True conditions are combinations of: A: severe anxiety; D: severe depression; S: severe stress; M: more than 4 h per day on social media.
Table
Table 6. Youden Index, cutoff values, and percentage of ‘addicted’ youth, when the same definition is given for the true problematic condition across samples.
Table 6. Youden Index, cutoff values, and percentage of ‘addicted’ youth, when the same definition is given for the true problematic condition across samples.
SampleYouden IndexTrue Condition 1Cutoff Value% Addicted
Spain 10.458ADSM3036.0
Spain 20.485ADSM3123.2
Spain 30.508ADSM3127.2
Argentina0.601ADSM4014.2
Chile0.398ADSM3537.8
Colombia0.418ADSM2858.3
Ecuador0.432ADSM3524.3
El Salvador0.445ADSM3634.6
Guatemala0.297ADSM2652.7
México0.566ADSM3235.0
Perú0.378ADSM3334.2
Uruguay0.637ADSM3432.6
Venezuela0.320ADSM3336.2
1 True conditions are combinations of: A: severe anxiety; D: severe depression; S: severe stress; M: more than 4 h per day on social media.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

García-Manglano, J.; López-Madrigal, C.; Sádaba-Chalezquer, C.; Serrano, C.; Lopez-Fernandez, O. Difficulties in Establishing “Truth” Conditions in the Assessment of Addictive Smartphone Use in Young Adults. Int. J. Environ. Res. Public Health 2022, 19, 358. https://doi.org/10.3390/ijerph19010358

AMA Style

García-Manglano J, López-Madrigal C, Sádaba-Chalezquer C, Serrano C, Lopez-Fernandez O. Difficulties in Establishing “Truth” Conditions in the Assessment of Addictive Smartphone Use in Young Adults. International Journal of Environmental Research and Public Health. 2022; 19(1):358. https://doi.org/10.3390/ijerph19010358

Chicago/Turabian Style

García-Manglano, Javier, Claudia López-Madrigal, Charo Sádaba-Chalezquer, Cecilia Serrano, and Olatz Lopez-Fernandez. 2022. "Difficulties in Establishing “Truth” Conditions in the Assessment of Addictive Smartphone Use in Young Adults" International Journal of Environmental Research and Public Health 19, no. 1: 358. https://doi.org/10.3390/ijerph19010358

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop