Next Article in Journal
Post-Traumatic Growth in Volunteers Following the 2023 Kahramanmaraş Earthquakes
Previous Article in Journal
Factors Associated with the Enjoyment of Physical Education Class for Children with Special Educational Needs
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 22
Issue 5
10.3390/ijerph22050698
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Ability of the Hopkins Symptom Checklist-5 to Identify Generalized Anxiety Disorder and Major Depressive Disorder in the General Population

by
Benedicte Kirkøen
1,*,
Ragnhild Elise Ørstavik
1,
Anne Reneflot
1,
Jens Christoffer Skogen
2,3,4,
Børge Sivertsen
2,5 and
Ann Kristin Skrindo Knudsen
6
1
Department of Mental Health, Norwegian Institute of Public Health, 0473 Oslo, Norway
2
Department of Health Promotion, Norwegian Institute of Public Health, 5015 Bergen, Norway
3
Centre for Evaluation of Public Health Measures, Norwegian Institute of Public Health, 0473 Oslo, Norway
4
Alcohol and Drug Research Western Norway, Stavanger University Hospital, 4068 Stavanger, Norway
5
Department of Research & Innovation, Helse-Fonna HF, 5525 Haugesund, Norway
6
Department of Disease Burden, Norwegian Institute of Public Health, 5015 Bergen, Norway
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2025, 22(5), 698; https://doi.org/10.3390/ijerph22050698 (registering DOI)
Submission received: 2 December 2024 / Revised: 10 February 2025 / Accepted: 25 April 2025 / Published: 28 April 2025
(This article belongs to the Section Behavioral and Mental Health)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background: The Hopkins Symptom Checklist (HSCL) is a widely used measure of anxiety and depression symptoms. The short form HSCL-5 is especially suitable for large population-based studies, but its ability to detect mental disorders in the general population remains unknown. The aim of the study was to assess how well the HSCL-5 identified cases of generalized anxiety disorder (GAD) and major depressive disorder (MDD) measured by the Composite International Diagnostic Interview (CIDI) 5.0 and to find the optimal sex-specific cut-off levels of the HSCL-5. Methods: Participants from the population-based Trøndelag Health Study (HUNT) in Norway were recruited for the current study. Between April and September 2020, 1343 participants (64% women) aged 20–65 years completed the CIDI, followed by the HSCL-5. The overall agreement between the HSCL-5 and GAD or MDD measured by CIDI was examined with Receiver Operator Characteristics (ROC) analysis. Sensitivity, specificity, positive predictive value (PPV), and negative predictive values (NPV) for different cut-off levels were assessed. Results: The area under the curve for GAD or MDD was 0.90 (CI 95% = 0.85–0.95) for women and 0.85 (CI 95% = 0.68–1.00) for men. For women, a cut-off level of ≥1.80 had the best balance between sensitivity (85%) and specificity (84%), while the corresponding numbers were ≥2.00, 73%, and 93% for men. The global PPV was 21%, while the NPV was 99%. Conclusions: The HSCL-5 has high sensitivity and specificity for identifying cases of GAD or MDD. In the current study, the positive predictive value of HSCL-5 was low.

1. Introduction

Mental disorders are a leading cause of disability worldwide [1,2]. Recently, an increase in the prevalence of symptoms of anxiety and depression has been found in many countries, especially among young adults [3,4,5,6,7]. This observation has sparked a debate as to whether an increase in self-reported symptoms also reflects an increase in the true prevalence of mental disorders.
For the planning of health promotion and work to prevent mental disorders, accurate prevalence estimates of mental disorders are crucial. Structured diagnostic interviews conducted in population-based surveys represent the gold standard for assessing the prevalence of mental disorders, but such assessments are costly and time-consuming. Hence, there is a global lack of such studies [8], while shorter questionnaires assessing symptoms of mental disorders have become increasingly important in monitoring the mental health of the population [9]. To approximate prevalence estimates of disease burden, cut-off values have been developed to identify possible cases of mental disorders from questionnaires. A valid and reliable assessment of the case-finding abilities of these questionnaires is of the utmost importance in epidemiological research settings. In Norway, large population-based health surveys have been conducted on a regular basis for several decades. A common questionnaire used in these surveys is the Hopkins Symptom Checklist (HSCL) [10].
Since its development in the 1950s, the HSCL [10,11] has become established in clinical research as a measure of symptoms of mental disorders. The first versions of 40 and 90 items were later reduced to HSCL-25, which captures the two most frequent mental disorders: anxiety (measured by 10 items) and depression (15 items). Whether HSCL-25 can discriminate between the two is, however, still under debate [12,13]. The case-finding abilities of HSCL-25 have been tested against structured diagnostic interviews in several studies since the 1990s, indicating that HSCL-25 has good abilities to detect depression but varying results for the detection of anxiety [12,14,15].
In epidemiological surveys, long questionnaires may cause lower response rates, which compromise the generalizability of results [16,17]. Thus, shorter but still valid questionnaires are in demand. In a Norwegian sample, Tambs and Moum identified five questions from HSCL-25 and showed that this short form had good psychometric properties [18]. While studies show that the HSCL-5 has demonstrated good validity when compared to the total HSCL-25 [19,20], to our knowledge, only one study has compared HSCL-5 to a structured diagnostic interview [21]. The authors showed that HSCL-5 is a valid tool for detecting depression in primary care. To our knowledge, no study has validated HSCL-5 against a structured diagnostic interview in the general population or evaluated the ability of HSCL-5 to detect anxiety measured by a structured diagnostic interview. Therefore, the ability of HSCL-5 to detect anxiety and depression in the general population remains unknown. This knowledge would be an important contribution to the literature, as several large population-based studies use HSCL-5 (e.g., [22,23,24,25,26,27]). Moreover, various studies compare HSCL to different subgroups of anxiety and depressive disorders. A Norwegian study found that HSCL-25 correlated most strongly with depression, panic disorder, and generalized anxiety disorder (GAD) [12]. While phobias are more prevalent among anxiety disorders, the study showed that HSCL-25 did not predict phobias well. Major depressive disorder (MDD) is the most prevalent among depression disorders [28,29], and when HSCL-25 is reduced to the short form HSCL-5, items relating to panic disorder are excluded. Consequently, the current study aims to test the ability of HSCL-5 to detect GAD and MDD.
Further, different studies recommend different cut-off levels for HSCL-5 (an average score on the HSCL-5 of ≥1.80 or ≥2.00). Additionally, men and women report different levels of anxiety and depression symptoms (e.g., [3]), which may indicate differences in their experience and reporting of symptoms. Consequently, a recent Spanish study suggests that optimal cut-of levels of HSCL should be stratified by sex [14], a finding further supported by a study of Norwegian students [30]. This highlights the need for new knowledge on appropriate cut-off levels for HSCL-5 in both sexes.
The aim of the current study was to test the ability of HSCL-5 to identify generalized anxiety disorder (GAD) and major depressive disorder (MDD) through the Composite International Diagnostic Interview (CIDI) [31,32] in a population-based sample and determine the sensitivity, specificity, positive predictive value, negative predictive value. Further, the study aimed to identify optimal cut-off levels of the HSCL-5 for women and for men.

2. Materials and Methods

Sample. Participants in the Trøndelag Health Study (HUNT) were invited to participate in a structured diagnostic interview survey. HUNT is a longitudinal, population-based health study in Nord-Trøndelag and Trøndelag (last wave only), two counties in mid-Norway, inviting every county resident above 20 years to participate (for more information, see [33]). Invitees to the current study were sampled among HUNT participants aged 20 to 65 years from Trondheim city. HUNT in Trondheim had a participation rate of 43% [33]. The diagnostic interview survey is registered at ClinicalTrials.gov (identifier: NCT04661228) and approved by the Regional Committee for Medical Research Ethics (2017/28/REK-midt). Participants invited to the diagnostic interview between April to September 2020 (n = 4300) were also asked to complete the HSCL-5, and 1379 (32%) completed both the interview and HSCL-5. All participants gave written informed consent.

3. Procedure and Measures

Procedure. The invitees received postal letters with information about the project, followed by an SMS that informed them how to sign up for the diagnostic interview. One SMS was sent to remind those not responding, and among those who registered for the study, up to four attempts were made to schedule the interview (three by phone and one final by SMS). All participants received a GBP 30 (NOK 300) gift card. The interviews were conducted face-to-face (n = 278) or by telephone (n = 1101) by trained and certified interviewers. Participants completed the diagnostic interview, followed by the questionnaire.
CIDI. Participants completed the Composite International Diagnostic Interview 5th version (CIDI) [31,32], a standardized interview developed by the World Health Organization (WHO). CIDI 5.0 assesses the 30-day, 12-month, and lifetime prevalence of several mental and substance use disorders according to the criteria in the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5) [34] and the International Classification of Diseases 10th edition (ICD-10) [35]. It has demonstrated high validity and reliability for mental disorder assessment in the general population [31,36]. The mental disorders were operationalized according to diagnostic algorithms developed by the World Mental Health Survey (WMHS) for use on CIDI 5.0. In the current study, we aimed to compare CIDI-derived diagnoses with the HSCL-5. The HSCL-5 has a 2-week timeframe, and in order to approximate a comparable timeframe, only the prevalence of diagnoses based on the last 30 days from CIDI was utilized.
Case definitions. The 30-day mental disorder was defined as the presence of a mental disorder during the 30 days prior to the interview (yes/no). As HSCL-5 was designed to measure symptoms of anxiety and depression, the following mental disorders were included: generalized anxiety disorder (GAD) and major depressive disorder (MDD) as individual diagnoses, and GAD or MDD combined, any anxiety or depressive disorder (including generalized anxiety disorder, panic disorder, specific phobia, agoraphobia, social anxiety disorder, major depressive disorder, bipolar type I and II disorders).
HSCL-5. The Hopkins Symptom Checklist (HSCL-5) was used to assess symptoms of anxiety and depression. The HSCL-5 consists of 5 items (see Table 1) scored on a Likert scale from 1 (‘not bothered’) to 4 (‘very bothered’). Participants are asked to rate to which degree they are bothered by the different symptoms for the past two weeks. An average score on the HSCL-5 of ≥1.80 or ≥2.00 is commonly used as a cut-off for identifying possible cases of mental disorders and corresponds well with the cut-off of ≥1.75 on the 25-item version (HSCL-25) [19,21].
Sex, age, level of education, and whether the respondent was living with a partner were reported by participants in the questionnaire. Participants’ level of education was categorized as (a) high school level, (b) lower degree of higher education (≤four years at a university or college), and (c) higher degree of higher education—(≥four years at a university or college). Living with a partner was coded “yes” if the participant reported to be married or living with someone in a marriage-like relationship and “no” otherwise.

4. Statistical Analyses

Factor analyses. We conducted an explorative factor analysis and an orthogonal varimax rotation to determine whether the HSCL-5 was best understood as one or more factors in our sample.
Criterion validity. The overall agreement between the HSCL-5 and the criterion standard CIDI generalized anxiety disorder (GAD) or major depressive disorder (MDD), or GAD and MDD separately, were examined with Receiver Operator Characteristics (ROC) analysis. The ROC curve represents sensitivity and specificity for all different cut-off levels and was used to determine the optimal cut-off level overall for women and men separately and for participants below and above 30 years. Sensitivity is the ability of HSCL-5 to detect a true positive case, and specificity is an expression of how well the test detects a true negative case [37]. In combination, sensitivity and specificity indicate the diagnostic accuracy of HSCL-5 compared to CIDI and are therefore important markers of criterion validity. The area under the ROC curve (AUC) is an index of the goodness of the scale. The AUC was estimated with 95% CI. Positive and negative predictive values for different cut-off levels were also assessed. All analyses were conducted in STATA, version 17.0.

5. Results

Descriptives. In total, 1379 participants completed both the CIDI interview and HSCL-5. Participants included and excluded from the analyses are shown in Figure 1. Four participants had missed only one item in HSCL-5. For these, the mean score was calculated based on the four valid items. Three participants had missing information on bipolar type II disorder. These three participants were categorized as having “no affective disorder”, as they also had no MDD or bipolar type I disorder. Twenty participants had missing diagnostic information on any anxiety disorder and were categorized as “no”, as they had no positive diagnosis on the remaining other anxiety disorders. The final sample consisted of 1343 participants (31% of the invitees). The demographics of the sample and the number of people with different CIDI diagnoses are shown in Table 1.
Factor analyses. Prior to performing factor analysis, the suitability of data for factor analysis was assessed. The Kaiser–Meyer–Olkin value was 0.86, and Bartlett’s test of sphericity was significant, p < 0.01, both indicating the suitability of data for factor analysis. The explorative factor analyses showed the two first eigenvalues of 2.85 and 0.05, yielding a ratio between them of 57, indicating essential unidimensionality. Only the first factor had an eigenvalue above 1, meaning that according to Kaiser’s criterion (eigenvalues > 1), HSCL-5 is best understood as one factor. Consequently, HSCL-5 is treated as one factor in all further analyses. Table 2 shows the factor loadings for all items on one factor. The internal consistency of the whole HSCL-5 scale was high (α = 0.87; with all items contributing to increase the estimated scale reliability).
Criterion validity. The mean HSCL-5 score was 1.32 (SD = 0.47) for the whole sample, 2.32 (SD = 0.68) among those with GAD, and 2.28 (SD = 0.84) among those with MDD. Measured by HSCL-5, 222 individuals (16.5%) scored above ≥1.80 (19% of the women and 12% of the men). Of the 222 with a score above ≥1.80, 46 (21%) had a GAD or MDD diagnosis. Out of those with a GAD or MDD diagnosis in CIDI, 46 (82%) had an HSCL-5 score above ≥1.80 (83% of those with a GAD diagnosis, 77% of those with MDD diagnosis).
The ROC curves for GAD or MDD are presented separately for men and women in Figure 2. The point-estimate for AUC was marginally better for women (AUC= 0.90, CI 95% = 0.85–0.95) than for men (AUC = 0.85, CI 95% = 0.68–1.00). The separate ROC curves for GAD and MDD are presented in Figure 3 and Figure 4.
The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of different cut-off levels of HSCL-5 are shown in Table 3, for GAD or MDD and for GAD and MDD separately (for all cut-off levels, see Table S1 in Supplementary Material; for other diagnoses, see Table S2 in Supplementary Material). In the current sample, no one scored at 1.75, and that cut-off level is therefore not reported. Overall, HSCL had the best balance between sensitivity (82%) and specificity (86%) for GAD or MDD using ≥1.80 as the cut-off point. For women, a cut-off level of ≥1.80 had the best balance between sensitivity (85%) and specificity (84%). For men, a cut-off level of ≥2.00 had the best balance between sensitivity (73%) and specificity (93%) (see Supplementary Table S3). For participants ≤30 years old (n = 434), a cut-off level of ≥2.00 had the best balance between sensitivity (83%) and specificity (88%) (see Supplementary Table S4). For participants >30 (n = 909), a cut-off level of ≥1.60 had the best balance between sensitivity (85%) and specificity (80%).
Sensitivity and specificity for GAD and MDD separately are presented in Table 3. The sensitivity for any affective or anxiety disorder was 0.45, while the specificity was 0.88 (for more details, see Table S2, Supplementary Material).

6. Discussion

The importance of the validity and reliability of measures of mental health are receiving renewed interest as recent studies show an increase in the prevalence of young people reporting symptoms of anxiety and depression [3,4,5,6,7]. To the best of our knowledge, the current study is the first to address to what degree HSCL-5 captures common mental disorders (generalized anxiety disorder (GAD) or major depressive disorder (MDD)) diagnosed with a structured interview (CIDI) in the general population. Our results indicate that HSCL-5 discriminates well between people with and without GAD or MDD, with HSCL-5 showing high sensitivity and specificity, supporting the criterion validity of the scale. The negative predictive value was also found to be high. However, the positive predictive value of HSCL-5 in the general population was low.
Our results confirm that HSCL-5 may be best understood as a unidimensional measure [12,13]. This may not necessarily reflect the quality of the questionnaire per se but rather reflect that symptoms of anxiety and depression often co-occur and may be, to some extent, overlapping [38,39]. The area under the ROC curve above 0.80 indicates that HSCL-5 discriminates well between people with and without GAD or MDD and between people with or without GAD. For MDD, the area under the ROC curve was 0.86 for women but 0.75 for men. While the latter is lower, all levels above 0.75 are considered to be of clinical utility [40].
In the study by Rodríguez-Barragán and colleagues (2023), the sensitivity and specificity for depression were 78% and 73% for HSCL-5 [21], similar to the sensitivity found in the current study for the detection of MDD (77%). However, we found a higher specificity (85%). Previous studies comparing the 25-item version to structured diagnostic interviews report a sensitivity between 70% and 88% and a specificity of between 77% and 85% for depression or mood disorders, and a sensitivity of 43–50% and a specificity of 83% for anxiety disorders [12,14,15]. We show a similar sensitivity and specificity for depression and specificity for anxiety compared to previous studies of the full-scale HSCL-25. However, our results show a higher sensitivity for HSCL when compared to GAD, while the sensitivity when HSCL was compared to any anxiety disorder was low. This might indicate that HSCL-5 discriminates well between people with and without generalized anxiety disorder (GAD) but is less effective in detecting people with other anxiety disorders. Importantly, the different studies are not directly comparable as they include different diagnoses of anxiety and depression.
For the total study population in the current study, the optimal cut-off level was ≥1.80, which provides a good balance between sensitivity (0.82) and specificity (0.86) for GAD or MDD. This means that HSCL-5 correctly identifies 82% of the people diagnosed with GAD or MDD according to CIDI and correctly identifies 86% of the people with no diagnoses. Importantly, we identify different optimal cut-off levels for women and for men. For men, the cut-off level of ≥2 provided the best balance between sensitivity (0.73) and specificity (0.93). The cut-off for HSCL-5 was first recommended to be ≥2, based on a study comparing HSCL-5 to the full HSCL scale [19]. However, a recent study comparing HSCL-5 to a structured diagnostic interview in a clinical setting found that the cut-off of ≥1.80 fitted well for the total patient population and ≥2 for men for the detection of depression [21]. These sex-specific cut-off values are confirmed in the current study, for the prediction of both GAD and MDD in the general population. This provides confidence in our results. One should note, however, that in the current study, no participants had a score of 1.75, which is the recommended cut-off level for HSCL-25. Consequently, we were unable to investigate whether 1.75 or 1.80 received the best balance. We observed that a higher cut-off level had the best balance for participants below 30 years old, and a lower level were most suitable for participants above 30 years. This is interesting to note as many studies report an increase in the prevalence of symptoms of anxiety and depression among young adults. Future studies of young adults should consider to take this into account.
Our choice of optimal cut-off level is based on the use of HSCL-5 for epidemiological purposes, and therefore, the cut-off with the best balance between sensitivity and specificity was chosen [41]. However, which cut-off level is considered optimal depends on the purpose for its use. If HSCL-5 is used with the aim of identifying all people with a mental disorder, one may choose a lower cut-off level with higher sensitivity. On the other hand, if HSCL-5 were to be used for screening in the general population, with limited resources available for follow-up diagnostic interviews, a cut-off level with higher specificity (at the cost of lower sensitivity) may be preferable.
The ability of a questionnaire to detect a disorder depends both on the characteristics of the questionnaire and on the prevalence of the disease in the population of interest. While the sensitivity, specificity, and negative predictive value of HSCL-5 were all high in our study, the positive predictive value (PPV) was low. This means that when a person scores above the cut-off on HSCL-5, the probability of having an actual diagnosis of GAD or MDD, according to CIDI, is low (21%). Our results indicate that if 100 people in a population are tested with HSCL-5, 16 of them will score above the cut-off of ≥1.80, but only 3 out of the 16 (21%) will have a diagnosis of GAD or MDD. To put it differently, the remaining 13 persons who score above the cut-off are unlikely to be diagnosed with GAD or MDD (even though their symptoms might be burdening). The current study offers an important perspective on the interpretation of the observed increases in people scoring above the cut-off for anxiety and depression using shorter questionnaires.
Importantly, PPV (and NPV) is highly connected to the actual prevalence of mental disorders in the population. Using a test with the same sensitivity and specificity, the PPV will increase with increasing prevalence in the population tested. This is illustrated in Table 4, which shows the PPV and the NPV in the current sample as well as estimates of two hypothetical samples with higher prevalence of GAD and MDD, of 8.4% and 16%, respectively. As can be seen in Table 4, in a population with a prevalence of 8.4% or 16% of GAD or MDD, PPV tells us that 36% and 53% of people who score above the cut-off are likely to have a diagnosis of GAD or MDD. Thus, in the latter example, the test result is more correct, even though the characteristics of the test are the same. Consequently, HSCL-5 will have higher positive predictive value in populations with a higher prevalence of mental disorders (e.g., primary care settings) compared to the general population.

7. Strengths and Limitations

To the best of our knowledge, this is the first study to address to what degree HSCL-5 captures common mental disorders (GAD or MDD) diagnosed with a structured interview (CIDI) in the general population. Our study has several strengths: First, the data were collected as part of an internationally recognized population-based health study (HUNT). Second, while previous studies examining the corresponding abilities of HSCL have only referred participants with a score above the cut-off level to the diagnostic interview—the current study invited all participants to complete both CIDI and the HSCL. Thus, the results can be used to determine to what degree this short questionnaire can be applied to determine the true prevalence of mental disorders in a population. Third, both examinations were completed on the same day, ensuring partly overlapping time frames for symptom assessments.
The results must be interpreted in light of six limitations: First, although this study was population-based, there was substantial attrition (participation rate is 31%). While attrition in follow-up studies is a substantial problem when examining exposure–outcome associations and prevalence rates, it might be less problematic for cross-sectional association studies such as those applied in the current project [42]. The prevalence of GAD (2.7%) and MDD (2.3%) was lower than what has been reported in previous cross-sectional studies from Norway [43], but this might be due to the limited time frame (30 days) we applied in the current study. Second, while the HSCL-5 asks participants to report symptoms for the past 2 weeks, CIDI asks participants to report symptoms from the past month. Thus, the two measures do not have the same timeframe, which may influence the results. Third, the CIDI interviews were conducted by phone and face-to face. Previous studies have shown, however, that this is unlikely to have any effects of the results [44]. Fourth, data were collected during the COVID-19 pandemic. This might have had some implications, although we believe that any effects would be small as most of the data collections were conducted by phone, and this part of Norway suffered smaller consequences with regard to disease severity, lockdowns, etc. Fifth, interviews and questionnaires were administered on the same day and always in this order (interviews first). Thus, subjects might be influenced by their answers in the interview when filling out the HSCL questionnaire. Sixth, these results based on a Norwegian sample may not be fully generalizable to other regions and cultural settings.

8. Conclusions

The current study is the first to evaluate the case-finding abilities of Hopkins Symptom Checklist (HSCL)-5 against a diagnostic interview in a general population. The results show that HSCL-5 has high sensitivity and specificity and thus identifies people with diagnoses of GAD or MDD well in the general population. The optimal cut-off level for HSCL-5 in the current study was ≥1.80 (and ≥1.80 for women and ≥2.00 for men). The ability of a questionnaire to detect a disorder depends on the characteristics of the test as well as the prevalence of the disease in the population of interest. In the general population, the positive predictive value of HSCL-5 was low.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijerph22050698/s1, Table S1. Sensitivity, specificity, positive- and negative predictive value for all cut-off levels for Generalized anxiety disorder (GAD) or Major Depressive Disorder (MDD); Table S2. Sensitivity and specificity of Hopkins Symptom Checklist-5 at different cut-off levels for different CIDI-diagnoses (30 days); Table S3. Sensitivity and specificity for different cut-off levels for Generalized anxiety disorder (GAD) or Major Depressive Disorder (MDD), for women and for men; Table S4. Sensitivity and specificity for different cut-off levels for Generalized anxiety disorder (GAD) or Major Depressive Disorder (MDD), for participants below and above 30 years.

Author Contributions

A.K.S.K. and A.R. contributed to the initiation, planning, and design of the structured diagnostic interview survey. B.K. performed the data analyses, and B.K. and R.E.Ø. led the writing of the manuscript. J.C.S., B.S., A.K.S.K., A.R., B.K. and R.E.Ø. contributed to input on the analytical plan, interpretation of results, and critical revision of the manuscript and analyses. All authors have read and agreed to the published version of the manuscript.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Institutional Review Board Statement

The study was approved by the Regional Committee for Medical Research Ethics (2017/28/REK-midt).

Informed Consent Statement

Informed consent was obtained from all individual participants included in the study.

Data Availability Statement

Norwegian data protection regulations and GDPR impose restrictions on the sharing of individual participant data. However, researchers may gain access to survey participant data by contacting the publication committee (anne.reneflot@fhi.no). Approval from the Norwegian Regional Committee for Medical and Health Research Ethics (https://helseforskning.etikkom.no, accessed on 10 February 2025) is a pre-requirement for access to the data. The dataset is administrated by the HUNT databank, and guidelines for access to data are found at https://www.ntnu.edu/hunt/data (Accessed on 10 February 2025). The study protocol and the informed consent form are available on the two homepages of the project: https://www.ntnu.no/hunt/forskningsprosjekt (Accessed on 10 February 2025) and https://www.fhi.no/cristin-prosjekter/aktiv/diagnosebasert-undersokelse-psykiske-lidelser-og-ruslidelser/ (Accessed on 10 February 2025).

Acknowledgments

We would like to thank all participants in the survey for their time and efforts spent in the study. Further, we are very grateful to the coordinators and interviewers who made an extraordinary effort to conduct the data collection under changing and challenging circumstances.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

HSCLThe Hopkins Symptom Checklist
GADgeneralized anxiety disorder
MDDmajor depressive disorder
CIDIComposite International Diagnostic Interview
HUNTTrøndelag Health Study
ROCReceiver Operator Characteristics
PPVpositive predictive value
NPVnegative predictive values
WHOWorld Health Organization
DSM-5Diagnostic and Statistical Manual of Mental Disorders 5th edition
ICD-10International Classification of Diseases 10th edition
WMHSWorld Mental Health Survey
AUCarea under the ROC curve

References

  1. GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: A systematic analysis for the Global Burden of Disease Study 2019. Lancet Psychiatry 2022, 9, 137–150. [Google Scholar] [CrossRef] [PubMed]
  2. Rehm, J.; Shield, K.D. Global burden of disease and the impact of mental and addictive disorders. Curr. Psychiatry Rep. 2019, 21, 10. [Google Scholar] [CrossRef]
  3. Krokstad, S.; Weiss, D.A.; Krokstad, M.A.; Rangul, V.; Kvaløy, K.; Ingul, J.M.; Bjerkeset, O.; Twenge, J.; Sund, E.R. Divergent decennial trends in mental health according to age reveal poorer mental health for young people: Repeated cross-sectional population-based surveys from the HUNT Study, Norway. BMJ Open 2022, 12, e057654. [Google Scholar] [CrossRef] [PubMed]
  4. Knapstad, M.; Sivertsen, B.; Knudsen, A.K.; Smith, O.R.F.; Aaro, L.E.; Lonning, K.J.; Skogen, J.C. Trends in self-reported psychological distress among college and university students from 2010 to 2018. Psychol. Med. 2021, 51, 470–478. [Google Scholar] [CrossRef] [PubMed]
  5. Twenge, J.M.; Cooper, A.B.; Joiner, T.E.; Duffy, M.E.; Binau, S.G. Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005–2017. J. Abnorm. Psychol. 2019, 128, 185. [Google Scholar] [CrossRef]
  6. Calling, S.; Midlöv, P.; Johansson, S.-E.; Sundquist, K.; Sundquist, J. Longitudinal trends in self-reported anxiety. Effects of age and birth cohort during 25 years. BMC Psychiatry 2017, 17, 119. [Google Scholar] [CrossRef]
  7. Gagné, T.; Schoon, I.; McMunn, A.; Sacker, A. Mental distress among young adults in Great Britain: Long-term trends and early changes during the COVID-19 pandemic. Soc. Psychiatry Psychiatr. Epidemiol. 2021, 57, 1261–1272. [Google Scholar] [CrossRef]
  8. Baxter, A.J.; Scott, K.M.; Vos, T.; Whiteford, H.A. Global prevalence of anxiety disorders: A systematic review and meta-regression. Psychol. Med. 2013, 43, 897–910. [Google Scholar] [CrossRef]
  9. Kessler, R.C.; Andrews, G.; Colpe, L.J.; Hiripi, E.; Mroczek, D.K.; Normand, S.-L.; Walters, E.E.; Zaslavsky, A.M. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol. Med. 2002, 32, 959–976. [Google Scholar] [CrossRef]
  10. Derogatis, L.R.; Lipman, R.S.; Rickels, K.; Uhlenhuth, E.H.; Covi, L. The Hopkins Symptom Checklist (HSCL): A self-report symptom inventory. Behav. Sci. 1974, 19, 1–15. [Google Scholar] [CrossRef]
  11. Parloff, M.B.; Kelman, H.C.; Frank, J.D. Comfort, effectiveness, and self-awareness as criteria of improvement in psychotherapy. Am. J. Psychiatry 1954, 111, 343–352. [Google Scholar] [CrossRef]
  12. Sandanger, I.; Moum, T.; Ingebrigtsen, G.; Dalgard, O.S.; Sørensen, T.; Bruusgaard, D. Concordance between symptom screening and diagnostic procedure: The Hopkins Symptom Checklist-25 and the Composite International Diagnostic Interview I. Soc. Psychiatry Psychiatr. Epidemiol. 1998, 33, 345–354. [Google Scholar] [CrossRef] [PubMed]
  13. Skogen, J.C.; Øverland, S.; Smith, O.R.; Aarø, L.E. The factor structure of the Hopkins Symptoms Checklist (HSCL-25) in a student population: A cautionary tale. Scand. J. Public Health 2017, 45, 357–365. [Google Scholar] [CrossRef] [PubMed]
  14. Rodríguez-Barragán, M.; Fernández-San-Martín, M.I.; Clavería-Fontán, A.; Aldecoa-Landesa, S.; Casajuana-Closas, M.; Llobera, J.; Oliván-Blázquez, B.; Peguero-Rodríguez, E. Validation and Psychometric Properties of the Spanish Version of the Hopkins Symptom Checklist-25 Scale for Depression Detection in Primary Care. Int. J. Environ. Res. Public Health 2021, 18, 7843. [Google Scholar] [CrossRef]
  15. Veijola, J.; Jokelainen, J.; Läksy, K.; Kantojärvi, L.; Kokkonen, P.; Järvelin, M.-R.; Joukamaa, M. The Hopkins Symptom Checklist-25 in screening DSM-III-R axis-I disorders. Nord. J. Psychiatry 2003, 57, 119–123. [Google Scholar] [CrossRef]
  16. Guo, Y.; Kopec, J.A.; Cibere, J.; Li, L.C.; Goldsmith, C.H. Population survey features and response rates: A randomized experiment. Am. J. Public Health 2016, 106, 1422–1426. [Google Scholar] [CrossRef] [PubMed]
  17. Rolstad, S.; Adler, J.; Rydén, A. Response burden and questionnaire length: Is shorter better? A review and meta-analysis. Value Health 2011, 14, 1101–1108. [Google Scholar] [CrossRef]
  18. Tambs, K.; Moum, T. How well can a few questionnaire items indicate anxiety and depression? Acta Psychiatrica Scandinavica 1993, 87, 364–367. [Google Scholar] [CrossRef]
  19. Strand, B.H.; Dalgard, O.S.; Tambs, K.; Rognerud, M. Measuring the mental health status of the Norwegian population: A comparison of the instruments SCL-25, SCL-10, SCL-5 and MHI-5 (SF-36). Nord. J. Psychiatry 2003, 57, 113–118. [Google Scholar] [CrossRef]
  20. Schmalbach, B.; Zenger, M.; Tibubos, A.N.; Kliem, S.; Petrowski, K.; Brähler, E. Psychometric properties of two brief versions of the Hopkins Symptom Checklist: HSCL-5 and HSCL-10. Assessment 2021, 28, 617–631. [Google Scholar] [CrossRef]
  21. Rodríguez-Barragán, M.; Fernández-San-Martín, M.I.; Clavería, A.; Le Reste, J.Y.; Nabbe, P.; Motrico, E.; Gómez-Gómez, I.; Peguero-Rodríguez, E. Measuring depression in Primary Health Care in Spain: Psychometric properties and diagnostic accuracy of HSCL-5 and HSCL-10. Front. Med. 2023, 9, 1014340. [Google Scholar] [CrossRef]
  22. Marti, A.R.; Degerud, E.; Sterud, T. Onset of work-life conflict increases risk of subsequent psychological distress in the Norwegian working population. Int. J. Environ. Res. Public Health 2022, 19, 13292. [Google Scholar] [CrossRef] [PubMed]
  23. AarØ, L.E.; Knapstad, M.; Nilsen, T.; Skogen, J.C.; Nes, R.B.; Leino, T.; Riise, I.A.; Johansen, R.; GrØtvedt, L.; Vedaa, Ø. The Norwegian Counties Public Health Survey (NCPHS): A design study. Scand. J. Public Health 2024, 14034948241287853. [Google Scholar] [CrossRef] [PubMed]
  24. Rangul, V.; Holmen, T.L.; Langhammer, A.; Ingul, J.M.; Pape, K.; Fenstad, J.S.; Kvaløy, K. Cohort Profile Update: The Young-HUNT Study, Norway. Int. J. Epidemiol. 2024, 53, dyae013. [Google Scholar] [CrossRef] [PubMed]
  25. Liu, Y.; Liu, L.; Shi, Z. Exposure, perceived risk, and psychological distress among general population during the COVID-19 lockdown in Wuhan, China. Front. Psychiatry 2023, 14, 1086155. [Google Scholar] [CrossRef]
  26. Lu, L.; Hannigan, L.J.; Brandlistuen, R.E.; Nesvåg, R.; Trogstad, L.; Magnus, P.; Unnarsdóttir, A.B.; Valdimarsdóttir, U.A.; Andreassen, O.A.; Ask, H. Mental Distress Among Norwegian Adults During the COVID-19 Pandemic: Predictors in Initial Response and Subsequent Trajectories. Int. J. Public Health 2023, 68, 1606164. [Google Scholar] [CrossRef]
  27. Schoon, I.; Henseke, G. Social inequalities in young people’s mental distress during the COVID-19 pandemic: Do psychosocial resource factors matter? Front. Public Health 2022, 10, 820270. [Google Scholar] [CrossRef]
  28. Hasin, D.S.; Sarvet, A.L.; Meyers, J.L.; Saha, T.D.; Ruan, W.J.; Stohl, M.; Grant, B.F. Epidemiology of adult DSM-5 major depressive disorder and its specifiers in the United States. JAMA Psychiatry 2018, 75, 336–346. [Google Scholar] [CrossRef]
  29. Nübel, J.; Guhn, A.; Müllender, S.; Le, H.D.; Cohrdes, C.; Köhler, S. Persistent depressive disorder across the adult lifespan: Results from clinical and population-based surveys in Germany. BMC Psychiatry 2020, 20, 1–13. [Google Scholar] [CrossRef]
  30. Sivertsen, B.; Skogen, J.C.; Reneflot, A.; Knapstad, M.; Smith, O.R.F.; Aarø, L.E.; Kirkøen, B.; Lagerstrøm, B.O.; Knudsen, A.K.S. Assessing Diagnostic Precision: Adaptations of the Hopkins Symptom Checklist (HSCL-5/10/25) Among Tertiary-Level Students in Norway. Clin. Psychol. Eur. 2024, 6, e13275. [Google Scholar] [CrossRef]
  31. Kessler, R.C.; Üstün, T.B. The world mental health (WMH) survey initiative version of the world health organization (WHO) composite international diagnostic interview (CIDI). Int. J. Methods Psychiatr. Res. 2004, 13, 93–121. [Google Scholar] [CrossRef] [PubMed]
  32. Kessler, R.; Üstün, T. The World Health Organization composite international diagnostic interview. In The WHO World Mental Health Surveys: Global Perspectives on the Epidemiology of Mental Disorders; World Health Organization: Geneva, Switzerland, 2008; pp. 58–90. [Google Scholar]
  33. Åsvold, B.O.; Langhammer, A.; Rehn, T.A.; Kjelvik, G.; Grøntvedt, T.V.; Sørgjerd, E.P.; Fenstad, J.S.; Heggland, J.; Holmen, O.; Stuifbergen, M.C.; et al. Cohort Profile Update: The HUNT Study, Norway. Int. J. Epidemiol. 2022, 52, e80–e91. [Google Scholar] [CrossRef]
  34. Edition, F. Diagnostic and statistical manual of mental disorders. Am. Psychiatric. Assoc. 2013, 21, 591–643. [Google Scholar]
  35. World Health Organization. The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines; World Health Organization: Geneva, Switzerland, 1992; Volume 1. [Google Scholar]
  36. Haro, J.M.; Arbabzadeh-Bouchez, S.; Brugha, T.S.; De Girolamo, G.; Guyer, M.E.; Jin, R.; Lepine, J.P.; Mazzi, F.; Reneses, B.; Vilagut, G. Concordance of the Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO World Mental Health surveys. Int. J. Methods Psychiatr. Res. 2006, 15, 167–180. [Google Scholar] [CrossRef]
  37. Trevethan, R. Sensitivity, specificity, and predictive values: Foundations, pliabilities, and pitfalls in research and practice. Front. Public Health 2017, 5, 307. [Google Scholar] [CrossRef] [PubMed]
  38. Hiller, W.; Zaudig, M.; Bose, M.V. The overlap between depression and anxiety on different levels of psychopathology. J. Affect. Disord. 1989, 16, 223–231. [Google Scholar] [CrossRef]
  39. Zbozinek, T.D.; Rose, R.D.; Wolitzky-Taylor, K.B.; Sherbourne, C.; Sullivan, G.; Stein, M.B.; Roy-Byrne, P.P.; Craske, M.G. Diagnostic overlap of generalized anxiety disorder and major depressive disorder in a primary care sample. Depress. Anxiety 2012, 29, 1065–1071. [Google Scholar] [CrossRef]
  40. Polo, T.C.F.; Miot, H.A. Use of ROC Curves in Clinical and Experimental Studies; SciELO: São Paulo, Brasil, 2020; Volume 19. [Google Scholar]
  41. Vilagut, G.; Forero, C.G.; Barbaglia, G.; Alonso, J. Screening for depression in the general population with the Center for Epidemiologic Studies Depression (CES-D): A systematic review with meta-analysis. PLoS ONE 2016, 11, e0155431. [Google Scholar] [CrossRef] [PubMed]
  42. Knudsen, A.K.; Hotopf, M.; Skogen, J.C.; Øverland, S.; Mykletun, A. The Health Status of Nonparticipants in a Population-based Health Study: The Hordaland Health Study. Am. J. Epidemiol. 2010, 172, 1306–1314. [Google Scholar] [CrossRef]
  43. Ørstavik, R.E.; Kendler, K.S.; Czajkowski, N.; Tambs, K.; Reichborn-Kjennerud, T. The relationship between depressive personality disorder and major depressive disorder: A population-based twin study. Am. J. Psychiatry 2007, 164, 1866–1872. [Google Scholar] [CrossRef]
  44. Knudsen, A.K.S.; Stene-Larsen, K.; Gustavson, K.; Hotopf, M.; Kessler, R.C.; Krokstad, S.; Skogen, J.C.; Øverland, S.; Reneflot, A. Prevalence of mental disorders, suicidal ideation and suicides in the general population before and during the COVID-19 pandemic in Norway: A population-based repeated cross-sectional analysis. Lancet Reg. Health-Eur. 2021, 4, 100071. [Google Scholar] [CrossRef] [PubMed]
Ijerph 22 00698 g001
Figure 1. Flowchart of participants in the study.
Figure 1. Flowchart of participants in the study.
Ijerph 22 00698 g001
Ijerph 22 00698 g002
Figure 2. Receiver Operator Characteristics (ROC) curve showing how well Hopkins Symptom Checklist-5 compares to generalized anxiety disorder (GAD) or major depressive disorder (MDD) for men and women.
Figure 2. Receiver Operator Characteristics (ROC) curve showing how well Hopkins Symptom Checklist-5 compares to generalized anxiety disorder (GAD) or major depressive disorder (MDD) for men and women.
Ijerph 22 00698 g002
Ijerph 22 00698 g003
Figure 3. Receiver Operator Characteristics (ROC) curve showing how well Hopkins Symptom Checklist-5 compares to generalized anxiety disorder (GAD) for men and women.
Figure 3. Receiver Operator Characteristics (ROC) curve showing how well Hopkins Symptom Checklist-5 compares to generalized anxiety disorder (GAD) for men and women.
Ijerph 22 00698 g003
Ijerph 22 00698 g004
Figure 4. Receiver Operator Characteristics (ROC) curve showing how well Hopkins Symptom Checklist-5 compares to major depressive disorder (MDD) for men and women.
Figure 4. Receiver Operator Characteristics (ROC) curve showing how well Hopkins Symptom Checklist-5 compares to major depressive disorder (MDD) for men and women.
Ijerph 22 00698 g004
Table 1. Descriptives of the sample.
Table 1. Descriptives of the sample.
Demographic Information
Women, n = (%)860 (64%)
Age, mean 39
(SD = 12, range 20–66)
Higher education,
(lower or higher degree of higher education)		1036 (77%)
Diagnoses From CIDI, 30 Days
GAD or MDD *, n = (%)56 (4.2%)
Generalized anxiety disorder36 (2.7%)
Major depressive disorder32 (2.3%)
Any affective or anxiety disorder **172 (13%)
Any anxiety disorder154 (11.5%)
Any affective disorder36 (2.7%)
CIDI = Composite International Diagnostic Interview. * Generalized anxiety disorder (GAD) or major depressive disorder (MDD). ** Including generalized anxiety disorder, panic disorder, specific phobia, agoraphobia, social anxiety disorder, major depressive disorder, bipolar type I and II disorders.
Table 2. Factor loadings of the 5 items in Hopkins Symptom Checklist-5.
Table 2. Factor loadings of the 5 items in Hopkins Symptom Checklist-5.
ItemFactor Loading
Feeling fearful0.83
Nervousness or shakiness inside0.83
Feeling hopeless about the future0.77
Feeling blue0.78
Worry too much about things0.85
Table 3. Sensitivity and specificity of Hopkins Symptom Checklist-5 (HSCL) at different cut-off levels for different CIDI diagnoses (30 days).
Table 3. Sensitivity and specificity of Hopkins Symptom Checklist-5 (HSCL) at different cut-off levels for different CIDI diagnoses (30 days).
CIDI Diagnoses
GAD or MDD *GADMDD
HSCL cut-off ≥ 1.80Sensitivity0.820.830.77
Specificity0.860.850.85
PPV0.210.130.10
NPV0.990.990.99
HSCL cut-off ≥ 2.00Sensitivity0.770.780.74
Specificity0.920.910.90
PPV0.290.190.16
NPV0.980.990.99
HSCL cut-off ≥ 2.25Sensitivity0.430.440.38
Specificity0.970.970.96
PPV0.410.270.20
NPV0.970.980.98
CIDI = Composite International Diagnostic Interview. * Generalized anxiety disorder (GAD) or major depressive disorder (MDD), PPV = positive predictive value, NPV = negative predictive value.
Table 4. Positive predictive value and negative predictive value in the current sample and two hypothetical populations with different prevalences of generalized anxiety disorder (GAD) or major depressive disorder (MDD).
Table 4. Positive predictive value and negative predictive value in the current sample and two hypothetical populations with different prevalences of generalized anxiety disorder (GAD) or major depressive disorder (MDD).
Current SampleHypothetical Population IHypothetical Population II
Prevalence of GAD or MDD, % (n=)4.2%
(56)		8.4%
(113)		16%
(215)
Sensitivity of HSCL-5 cut-off ≥ 1.800.820.820.82
Specificity of HSCL-5 cut-off ≥ 1.800.860.860.86
Positive predictive value (PPV)0.210.360.53
Negative predictive value (NPV)0.990.980.96
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kirkøen, B.; Ørstavik, R.E.; Reneflot, A.; Skogen, J.C.; Sivertsen, B.; Knudsen, A.K.S. The Ability of the Hopkins Symptom Checklist-5 to Identify Generalized Anxiety Disorder and Major Depressive Disorder in the General Population. Int. J. Environ. Res. Public Health 2025, 22, 698. https://doi.org/10.3390/ijerph22050698

AMA Style

Kirkøen B, Ørstavik RE, Reneflot A, Skogen JC, Sivertsen B, Knudsen AKS. The Ability of the Hopkins Symptom Checklist-5 to Identify Generalized Anxiety Disorder and Major Depressive Disorder in the General Population. International Journal of Environmental Research and Public Health. 2025; 22(5):698. https://doi.org/10.3390/ijerph22050698

Chicago/Turabian Style

Kirkøen, Benedicte, Ragnhild Elise Ørstavik, Anne Reneflot, Jens Christoffer Skogen, Børge Sivertsen, and Ann Kristin Skrindo Knudsen. 2025. "The Ability of the Hopkins Symptom Checklist-5 to Identify Generalized Anxiety Disorder and Major Depressive Disorder in the General Population" International Journal of Environmental Research and Public Health 22, no. 5: 698. https://doi.org/10.3390/ijerph22050698

APA Style

Kirkøen, B., Ørstavik, R. E., Reneflot, A., Skogen, J. C., Sivertsen, B., & Knudsen, A. K. S. (2025). The Ability of the Hopkins Symptom Checklist-5 to Identify Generalized Anxiety Disorder and Major Depressive Disorder in the General Population. International Journal of Environmental Research and Public Health, 22(5), 698. https://doi.org/10.3390/ijerph22050698

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop