Validation of the Italian Version of the Web Screening Questionnaire for Common Mental Disorders

Background: The ever-increasing spread of Internet-based systems for common mental disorders has generated the need for brief online screening methods. This study aims to test the psychometric properties of the Web Screening Questionnaire (WSQ) to examine its suitability for screening for common mental health problems among a community sample of Italian adults. Methods: A total of 1282 subjects (F = 819; mean age = 42.05) answered the WSQ. Its discriminant characteristics were examined with other validated selected scales for measuring mental health widely used in the Italian population using sensitivity, specificity, and area under the curve (AUC), as well as positive (PPV) and negative predictive values (NPV). Results: Most of the WSQ subscales exhibited moderate to high specificity values. Specifically, the scales of ‘agoraphobia’ (0.947; 95%CI [0.934, 0.960]), ‘anxiety’ (0.959; 95%CI [0.946, 0.970]), and ‘panic disorder’ (0.973; 95%CI [0.964, 0.981]) showed the highest values whilst the ‘obsessive-compulsive’ dimension had the lowest value at 0.838, 95%CI [0.815, 0.861]. With exceptions observed for ‘depression’ (0.716; 95%CI [642, 798]) and ‘alcohol abuse’ (0.760; 95%CI [560, 920]), instead, the WSQ demonstrated critical sensitivity values (<0.6) in all dimensions. Conclusions: The WSQ was appropriate for discriminating between people with and without a psychiatric condition, as it helps to confirm the absence of disorders. However, further diagnostic procedures are required, in case of a positive WSQ screening result.


Introduction
A growing workforce and mental health services shortage paired with an enormous demand for mental health services has led to increased interest in diverse digital solutions [1].These include online consultations, mobile applications accessed via smartphones or tablets, and text messaging services designed to improve the psychological health of users, among others.Furthermore, research has shown that Internet-based interventions can be as effective as face-to-face psychotherapy for a variety of psychological issues [2][3][4][5]-such as anxiety [6,7], depression [8], eating disorders [9], and adjustment disorders [10]-not only in controlled trials but also in routine clinical care [11,12].
These interventions can be delivered with minimal or no therapist input [9] and have established cost-effectiveness compared with face-to-face treatment.
Since the WSQ is currently not available for use among the Italian population, the purpose of the present study is to examine its psychometric properties among adults recruited from the general population in Italy.

Materials and Methods
A cross-sectional research approach was used to examine the validity of the Italian version of the WSQ among adults from the general population.

Translation and Cultural Adaptation
International guidelines [31,32] for translating the WSQ were followed.First, two experienced clinical psychologists independently translated the questionnaire from English to Italian.To maintain consistency, an independent translator performed a back-translation.The final version of the WSQ was tested on a sample of 30 individuals from the general population, to assess the clarity of the items.No additional modifications were required.

Sample Size Determination
The sample size was planned a priori and the following formula [33] was used: where N minimum is the minimum sample size required to correctly estimate the sensitivity of the WSQ, Z α/2 is equal to 1.96, P is a pre-determined value of sensitivity that is determined by previous published data, d is the margin of error, and PREV is the prevalence of the disorder.A sensitivity of 0.80, a margin of error equal to 0.07, and a prevalence of 0.10 were considered [34].A minimum of 1255 participants were guaranteed.

Participants
The participants were recruited from the general Italian population.The research included individuals who met the following criteria: (1) being over 18 years old, (2) fluent in Italian, and (3) providing their informed consent to participate.Those with visual or cognitive impairments that hindered the completion of the questionnaire were excluded.Participation was voluntary and no financial compensation was provided to the participants.

Procedures
This study was completed solely online and hosted by the questionnaire tool Qualtrics.The recruitment ad included a link placed on the main social networks (i.e., Instagram, Facebook, Twitter).Furthermore, in line with previous studies [35][36][37], personal invitations and advertisements were placed on university campuses, cafés, and libraries in Milan and Padua, Italy.
The initial page contained a detailed description of the study, inclusion, and exclusion criteria along with any potential risks that may occur as a result of participation.The subjects were then asked to acknowledge that they had read the terms and conditions and were aware of any potential risks by signing an online informed consent form.
Following informed consent, participants were asked to complete a survey consisting of 10 sets of questions: demographic questions, the WSQ, the short form of the Post-Traumatic Symptom Questionnaire (PTSQ) [38][39][40], as well as selected dimensions of both the Patients Health Questionnaire (PHQ) and the 90-Revised Symptom Checklist (SCL-90-R).
Upon completion of the survey, participants were provided access to a debriefing page containing information about the study objectives and methodology, along with contact details for support services.

Measures
Demographic information included gender, age, education, civil status, and employment status.The participants were asked to self-report if they had ever been diagnosed with a mental disorder and, in the event of a positive answer, to specify which one(s) (multipleanswer question).Furthermore, participants were asked to self-report if they have ever sought help for a mental health problem and to which professional(s) (multiple answer question), and if they have ever taken or are taking psychotropic drugs (yes/no answer).
In addition, Italian validation of the following self-report measures was administered: The Patient Health Questionnaire (PHQ) [41,42] evaluates the presence of eight psychological disorders, divided into threshold disorders (those that met specific DSM-IV diagnoses: major depressive disorder, panic disorder, other anxiety disorder, and bulimia nervosa), and subthreshold disorders (those whose criteria encompass fewer symptoms than are required for any specific DSM-IV diagnoses: other depressive disorder, probable alcohol abuse/dependence, somatoform, and binge eating disorder).For this study, the 9-item depression module (known as PHQ-9), the 7-item anxiety module (known as GAD-7), the 7-item panic disorder, and the 5-item alcohol abuse dimension were administered.
The diagnosis of major depression requires the presence of 5 or more of the 9 criteria of depressive symptoms for at least "more than half the days" in the past 2 weeks, with one of the symptoms being depressed mood or anhedonia.Calculating depression severity involves assigning scores of 0, 1, 2, and 3 to the response categories "not at all", "several days" "more than half the days", and "nearly every day", respectively, for each of the nine elements of the PHQ-9.The total PHQ-9 score, ranging from 0 to 27, is derived by summing these assigned scores.Furthermore, the severity of depression is then interpreted based on the total score, with cut-off points established at scores of 5, 10, 15, and 20.Specifically, scores of 5, 10, 15, and 20 signify thresholds for mild, moderate, moderately severe, and severe depression, respectively.
The diagnosis of generalized anxiety requires that you answer more than half of the days to the first item and at least three other items.Calculating anxiety severity using the GAD-7 involves assigning scores of 0, 1, 2, and 3 to the response categories "not at all", "several days", "more than half the days", and "nearly every day", respectively, for each of the seven items.The total GAD-7 score, ranging from 0 to 21, is obtained by adding these assigned scores.In this context, scores of 5, 10, and 15 serve as cut points for categorizing the severity of anxiety, representing mild, moderate, and severe anxiety, respectively.Panic disorder is identified if at least 8 of 11 symptoms (Yes/No answer) are present over the past month.
Alcohol problems are detected if at least one of the 5 alcohol-related risk behaviors in the last six months is present during the past 6 months (Yes/No answer).
The 90-Revised Symptom Checklist (SCL-90-R) [43] measures the severity of a wide range of psychological symptoms and is widely used in clinical practice and research.It consists of 90 items scored on a 5-point Likert scale from 0 (not at All) to 4 (Extremely) specifying how much each statement has bothered the respondents over the past 7 days.The SCL-90-R assesses nine scores along the primary symptom dimensions (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism).For the present study, the 10 elements comprising the obsessive-compulsive subscale, the 9 elements creating the interpersonal sensibility subscale, and the 7 elements creating the phobic anxiety subscale were used.According to the literature, a cut-off score equal to 1 (=presence of disorder) was used for all subscales.
The Post-traumatic Symptom Questionnaire (PTSQ) [38][39][40] is a 12-item self-report scale that measures the presence of subjective distress after exposure to traumatic events.The PTSQ comprises three dimensions: intrusion, avoidance, and hyperarousal measured with a Likert-type response format ranging from 1 (not at all) to 5 (extremely) with a cut-off score indicating the absence/presence of post-traumatic stress [38].For the present study, the brief version (6 items; PTSQ-SF) was used [40].

Statistical Analysis
The presence/absence of depression measured by the WSQ has been related to the presence/absence of depression as measured by PHQ9.The presence/absence of anxiety measured by the WSQ has been related to the presence/absence of anxiety as measured by GAD7.The presence/absence of social phobia as measured by the WSQ has been related to the presence/absence of anxiety as measured by the dimension of interpersonal sensibility of the SCL-90-R.The presence/absence of panic disorder, panic with agoraphobia, and agoraphobia as measured by the WSQ has been related to the presence/absence of panic disorder, panic with agoraphobia, and agoraphobia as measured by the panic disorder module of the PHQ.The presence/absence of specific phobia as measured by the WSQ has been related to the presence/absence of specific phobia as measured by the phobic anxiety dimension of the SCL-90-R.The presence/absence of OCD as measured by the WSQ has been related to the presence/absence of OCD as measured by the obsessivecompulsive subscale of the SCL-90-R.The presence/absence of PTSD as measured by the WSQ has been related to the presence/absence of PTSD as measured by the phobic anxiety dimension of the SCL-90-R.The presence/absence of OCD as measured by the WSQ has been related to the presence/absence of OCD as measured by the PTSQ-SF.The presence/absence of alcohol abuse/dependence as measured by the WSQ has been related to the presence/absence of alcohol abuse/dependence as measured by the alcohol abuse dimension of the PHQ.
The general accuracy-validity of the WSQ was assessed with the area under the ROC curve (AUC; 5000 stratified bootstrap resamples).Swets' benchmarks were used to interpret the AUC [44][45][46]: AUC = 0.50, null; AUC from 0.51 to 0.70, small; AUC from 0.71 to 0.90, moderate; AUC from 0.91 to 0.99, large; and AUC = 1.00, perfect accuracy.Furthermore, the sensitivity (SEN), specificity (SPE), and accuracy (ACC) of each subscale of the WSQ were tested against the corresponding disorder.Specifically, SEN is the probability that a person with a disorder is positive at the screening, while SPE is the probability that a person without a disorder is negative at the screening.ACC is the ability of a test to correctly classify subjects (true positives and true negatives) against a criterion.Additionally, positive predicted values ((PPV) probability of a positive diagnosis after positive screening) and negative predicted values ((NPV) probability of a negative diagnosis after negative screening) were also calculated.
Lastly, to determine the strength of the difference between those who scored positive on the WSQ and those who screened negative, a series of t tests for each screening instrument were performed separately.Specifically, the diagnosis obtained on the WSQ was used as the independent variable, while the score obtained on the questionnaire corresponding to the disorder of interest was used as the dependent variable.The strength of the difference was interpreted using Hedge's g and its benchmarks [47]: null (<0.20), small (from 0.20 to 0.49), moderate (0.50 to 0.79), and large (>0.80).Also, the separation index (1 − η) index was used to quantify the magnitude of a difference between groups [48,49] -also with non-normal distributions and of samples with different sizes.The separation index is the opposite of the overlapping index (that is, the 1-overlapping index): it ranges from 0 (=perfect overlap) to 1 (=perfect separation) and should be interpreted as a normalized effect size [48].

Sample Characteristics
The sample of this study was made up of 1282 participants: 463 men (36.1%) and 819 females (63.7%), in the age range of 18 to 82 years (mean = 42.05,SD = 14.317).The descriptive statistics are reported in Table 1.

Assessing the Strength of Differences
Table 3 and Figure 2 showed that respondents who voted yes for a specific WSQ "diagnosis" showed markedly higher means (p < 0.001) on the associated validation questionnaire in comparison to those who voted no for the same WSQ "diagnosis".Moreover, all Hedges' g were higher than 0.9 suggesting a strong difference between the two groups.Accordingly, the separation index (1 − η) suggests that the two WSQ diagnosis-related groups exhibit a notable separation.

Discussion
This study evaluated the psychometric properties of the WSQ to detect the presence of mood, anxiety, obsessive-compulsive, post-traumatic stress disorders, and alcohol abuse among a community sample of Italian adults.

Principal Findings
Overall, the WSQ was successful in discriminating between individuals with and without a psychiatric disorder.On average, response of subjects who tested positive for the diagnosis of mental disorders in WSQ consistently showed higher values compared with those without WSQ diagnosis.For example, individuals with depression exhibited

Discussion
This study evaluated the psychometric properties of the WSQ to detect the presence of mood, anxiety, obsessive-compulsive, post-traumatic stress disorders, and alcohol abuse among a community sample of Italian adults.

Principal Findings
Overall, the WSQ was successful in discriminating between individuals with and without a psychiatric disorder.On average, response of subjects who tested positive for the diagnosis of mental disorders in WSQ consistently showed higher values compared with those without WSQ diagnosis.For example, individuals with depression exhibited markedly elevated mean scores compared with those without a depression diagnosis measured by the WSQ, with an extremely high effect size.Similarly, mean WSQ scores for anxiety were significantly higher among those presenting this psychiatric condition compared who did not test positive for anxiety problems at the WSQ with significantly substantial impact magnitude.
Respondents with PTSD also showed significantly higher mean WSQ post-traumatic stress scores than those without diagnosis of PTSD according to the WSQ and high effect size.
Furthermore, those who presented alcohol abuse disorder according to the WSQ showed moderately higher mean values than those who did not test positive for this disorder according to the WSQ criteria, with an increased effect size.
A similar trend was observed for the remaining phobic-related problems, namely social phobia, panic, panic with agoraphobia, agoraphobia, specific phobia, and obsessivecompulsive: subjects who tested positive for the diagnosis through the WSQ consistently exhibited significantly higher means scored in these dimensions compared with respondents without WSQ diagnosis.
This indicates that the WSQ can distinguish between people with or without mental disorders so powerfully that those with a diagnosis appear to be markedly pathological.
In addition, the findings on the sensitivity, specificity, and AUC values suggest that the WSQ has some advisable screening characteristics.
Its high specificity suggests that it may help to confirm the absence (true negative) of all psychiatric conditions, meaning that it cannot rule out the disorders.This is especially important when people who are identified as having a condition should undergo a more detailed screening for the presence and intensity of mental disorders.
However, the low sensitivity for most of the disorders measured by the WSQ was somewhat surprising, given the higher values reported in a previous sample, which demonstrate both the high sensitivity and high specificity of the tool [29].The sensitivity reported in another research was similar to that found in this study [50].
The difference could in part be due to the sample and the measuring methods.In fact, in this study, we only employed self-report screening questionnaires and did not conduct clinical diagnostic interviews to assess the presence of a current psychiatric diagnosis [29,50].Another possible explanation for the low sensitivity of the WSQ could lie in the proposed cut-off criteria.

Strengths and Limitations
To our knowledge, this is the only valid measure of common psychiatric conditions available in Italy for planning and conducting an online survey-the WSQ.
A strength of this research is the extensive inclusion of participants, constituting a well-balanced and representative sample.
Another crucial aspect of this study is the use of rigorous statistical methods to test hypotheses and identify patterns and relationships in the data, leading to increased objectivity and credibility in the results and drawing meaningful conclusions that can inform the work in the field.To this end, only psychometric instruments validated in Italian were used to inform the results.
However, the generality of this finding is limited to Italian-speaking respondents who are Internet-literate.Furthermore, the sensitivity and specificity of the Italian version may not apply to clinical samples.Therefore, the present results need to be confirmed in other populations.Furthermore, the low prevalence of disorders in respondents devalues the predictive value of the tool, in addition to the fact that the clinical value of self-reported diagnosis remains debatable.
Lastly, since no clinical interviews were conducted to confirm (or disconfirm) the diagnosis made by self-report tools, and the results of this work are based solely on the single validation study available in the literature [29], it was not possible to explore, adjust, and propose alternative cutoffs for the use of the WSQ in the Italian population.This would have improved the screening capacity and accuracy of the tool; therefore, its diagnostic performance for clinical and research purposes in both clinical and non-clinical samples.
Future studies should parallel the WSQ with a validated clinical semi-structured interview to gather more in-depth information on participant attitudes, thoughts, and actions and generate confirmatory results on the absence of mood, anxiety, obsessivecompulsive, post-traumatic stress disorders, and alcohol abuse among respondents.
Despite its limitations, the WSQ remains a valuable and rapid online screening tool (it takes about two/three minutes to complete) for common mental disorders.
It offers an accessible and convenient means of assessing mental health conditions, particularly in non-clinical samples who may not have easy access to traditional mental health services.
The WSQ can be easily disseminated through online platforms, reaching a wide range of individuals regardless of their geographic location or socio-economic status.
This population-level approach allows identification of trends and patterns in mental health within communities, facilitating the development of targeted prevention and intervention strategies [51,52] to address identified individual risk factors.
This targeted approach ensures that healthcare resources are directed towards those who need them the most, thereby improving efficiency within the healthcare system.
The provision of valuable insights into the prevalence and characteristics of mental health problems within non-clinical samples can information public health policies and initiatives aimed at promoting mental well-being and preventing mental illness at a population level.

Conclusions
The WSQ functions as a concise and easy to administer survey designed to identify conditions such as depression, GAD, panic disorder (with or without agoraphobia), social phobia, specific phobia, OCD, PTSD, and alcohol abuse or dependence.Its validity in the Italian general population has been established by comparing it to the standardized self-repost measure of mental illnesses.The findings suggest that the WSQ could potentially serve as a cost-effective and brief online screening tool for common psychiatric conditions, particularly in primary care settings.In fact, it could aid healthcare providers in pre-consultation screening, with positive screens prompting more thorough diagnostic evaluations.By facilitating early detection, personalized intervention thought telehealth services, and population-level screening the WSQ has the potential to improve diagnostic accuracy and reduce the healthcare expenditures.

Figure 2 .
Figure 2. Boxplot representing differences in mean scores on the screening questionnaires between subjects who have received a diagnosis through the WSQ (positive) and those who have not.

Figure 2 .
Figure 2. Boxplot representing differences in mean scores on the screening questionnaires between subjects who have received a diagnosis through the WSQ (positive) and those who have not.

Author Contributions:
Conceptualization and methodology G.P.; formal analysis, A.A.R.; data curation, M.S. and G.M.; writing-original draft preparation, G.P. and A.A.R.; writing-review and editing, S.M., G.A. and G.C.; supervision, G.C.All authors have read and agreed to the published version of the manuscript.Funding: This research received no external funding.Institutional Review Board Statement: The study was conducted following the Declaration of Helsinki, and approved by the Institutional Ethics Committee of the University of Padua, Padova, Italy (protocol code: 4849).

Table 2 .
Agreement between the screening questionnaires and the WSQ.

Table 3 .
Differences in mean scores on the screening questionnaires between subjects who have received a diagnosis through the WSQ and those who have not.