Inventory of Attitudes toward Seeking Mental Health Services: Psychometric Properties among Adolescents

Abstract

Mental health is an essential element of wellness; however, some populations are more vulnerable to mental health problems. Suicide is the second-highest cause of death in young people and help-seeking unwillingness is a significant obstacle to mental health interventions. Young people are especially reluctant to seek help and negative attitudes stand out as help-seeking barriers, highlighting the importance of evaluating these constructs. The Inventory of Attitudes toward Seeking Mental Health Services (IASMHS) is a measure that evaluates help-seeking attitudes; nevertheless, it lacks applicability in Portuguese youth. This study aims to explore the psychometric characteristics of the IASMHS in 750 Portuguese adolescents (52.3% female, M_age = 14.67), through exploratory and confirmatory factor analysis, and reliability and validity analyses. Results suggest that a three-factor structure was adequate, those factors being help-seeking propensity, indifference to stigma, and psychological openness. The IASMHS was shortened to 20 items and revealed satisfactory properties regarding internal consistency, convergent/discriminant validity, and temporal stability. These findings validate the IASMHS as an adequate tool in the assessment of young people’s help-seeking attitudes.

1. Introduction

Mental health is a fundamental element of general health and wellness, and it is influenced by several factors (World Health Organization (WHO) 2013). Therefore, some populations are at higher risk of experiencing mental-health-related difficulties (World Health Organization (WHO) 2013).

Preadolescents and adolescents are in life stages susceptible to the onset of mental health problems (Rapee et al. 2020), making it fundamental to destigmatize mental health assistance, facilitate access and convince them to start therapy, given its long-lasting beneficial effects (Galambos et al. 2004; Rasing et al. 2021).

In particular, during adolescence, several biopsychosocial changes occur, making suicide one of the most troubling problems to arise from this life stage (Grande et al. 2020). Consequently, suicide prevention is prioritized to highlight that young people are especially susceptible. Indeed, suicide is the second-highest cause of death in young people (World Health Organization (WHO) 2013).

Youth suicide is a taboo, the exploration of which is essential to prevent fatalities. (Kuczynski 2014). However, one of the biggest obstacles for prevention/intervention in mental health is people’s unwillingness to seek professional help (Rickwood and Thomas 2012), with distressed young people being especially hesitant (Biddle et al. 2004).

Thus, there is a significant amount of research in the field of help-seeking (Rickwood and Thomas 2012), which is a potential protective factor against suicide (Reynders et al. 2014).

1.1. Help-Seeking and Attitudes

Help-seeking entails an effort to acquire external support to work through a perturbing event or a problem, by relying on formal or informal sources (Rickwood et al. 2005).

This construct encompasses three general aspects: attitudes, intentions, and behaviors (Gulliver et al. 2012). Attitudes toward seeking professional help stand out as one of the main barriers to help-seeking (Rickwood et al. 2005). This is understandable since negative help-seeking attitudes and a sense of self-reliance discourage adolescents from partaking in treatment or exposing their symptoms, even when the symptomatology is considerable (Labouliere et al. 2015).

Since attitudes toward help-seeking and personal beliefs can explain the relationship between suicidal ideation and intention to seek professional help (Wilson et al. 2005b), it is fundamental to promote positive attitudes and reduce stigma to enable help-seeking, by taking attitudinal factors as central variables in suicide prevention (Reynders et al. 2014).

Nevertheless, the validity of help-seeking interventions for adolescents may be thwarted by the usage of age-inappropriate instruments in research (Divin et al. 2018). Hence, it is imperative to invest in the study of help-seeking assessment tools, to foster accurate knowledge about adolescent help-seeking and ensure adequate interventions.

1.2. Inventory of Attitudes toward Seeking Mental Health Services (IASMHS)

The IASMHS is a scale developed by Mackenzie et al. (2004) which assesses attitudes toward seeking formal help. Its origin was founded on Fischer and Turner’s (1970) Attitudes Toward Seeking Professional Psychological Help Scale (ATSPPHS) which comprised several conceptual and methodological constraints (Mackenzie et al. 2004).

The ATSPPHS was adapted to create the IASMHS, including a new five-point scale and various wording changes. Subsequently, the IASMHS was extended, creating a tool consisting of 29 items from the ATSPPHS and 12 new items (Mackenzie et al. 2004).

The preliminary version of the tool was first assessed in 208 participants from the general population (48.1% female; M_age = 45.6; SD = 17.8), resulting in the final version of the IASMHS (α = 0.87) comprising 24 items and three factors: psychological openness (α = 0.82), which measures the degree to which a person is open to the recognition of psychological problems and to the prospect of seeking professional help for such problems; help-seeking propensity (α = 0.76), which evaluates the degree to which a person considers they are capable of seeking and personally inclined to seek professional psychological help; and indifference to stigma (α = 0.79), which appraises the degree to which a person worries about what others might think, if they found out that the person was seeking professional help for psychological concerns (Mackenzie et al. 2004).

Subsequently, the IASMHS was employed in 297 undergraduate students (51.5% female; M_age = 21.0; SD = 2.7), verifying that the three-factor structure was adequate. The inventory presented satisfactory indicators of validity, correlating positively with intention to seek professional help, and to the actual use of such help previously. Discriminant validity was also demonstrated since, globally, the scale correlated more strongly with intentions to seek help from formal sources, rather than informal sources (Mackenzie et al. 2004).

Finally, the tool was used in 23 second-year undergraduate students (82.6% female; M_age = 21.1; SD = 1.0), attesting its temporal stability (total scale: r = 0.85, p < 0.01; psychological openness: r = 0.86, p < 0.01; help-seeking propensity: r = 0.64, p < 0.01; and indifference to stigma: r = 0.91, p < 0.01) (Mackenzie et al. 2004).

Furthermore, the IASMHS has already been validated in different countries. In Ireland, the tool was evaluated in 331 police officers (39.0% female; M_age = 28.41; SD = 8.63), confirming that the abovementioned three-factor structure presented the best suitability (Hyland et al. 2014). Adequate validity was shown since the psychological openness and the help-seeking propensity factors were significant predictors of intention to seek psychological counseling. Satisfactory reliability results were also obtained (psychological openness: ρc = 0.70; help-seeking propensity: ρc = 0.76; indifference to stigma: ρc = 0.77) (Hyland et al. 2014).

In Russia, the tool was assessed in 153 adults from a community sample (76.5% female; M_age = 31.2; SD = 11.0), suggesting a four-factor model as being the most acceptable, and also revealing variable reliability results (total scale: α = 0.85; recognition of need for psychological help: α = 0.63; confidence in mental health professionals: α = 0.65; help-seeking propensity: α = 0.77; indifference to stigma: α = 0.84) (Weinstein et al. 2015).

In France, the IASMHS was evaluated in 702 subjects (57.5% female; M_age = 36.0; SD = 12.12), again suggesting the suitability of a three-factor structure. In this study the inventory was correlated with several variables such as gender and familiarity with psychological help, and its reliability was satisfactory (psychological openness: α = 0.71; help-seeking propensity: α = 0.76; indifference to stigma: α = 0.76) (Lheureux 2015).

In Austria, the tool was used to examine 220 adult survivors of institutional abuse (40.0% female; M_age = 57.9; SD = 9.55) (Kantor et al. 2017). The three-factor model was again revealed as most appropriate; however, some items of the psychological openness dimension loaded weakly on their expected factors (items 1, 4, 7, and 14) and item 23 loaded poorly on its expected factor (indifference to stigma). Help-seeking propensity, along with PTSD-intrusions and depression, contributed to the prediction of current use of mental health services. The inventory revealed satisfactory reliability (total scale: α = 0.84; psychological openness: α = 0.67; help-seeking propensity: α = 0.80; indifference to stigma: α = 0.77) (Kantor et al. 2017).

The IASMHS was also employed in a study of 200 Chinese Canadian immigrants (68.5% female; M_age = 72.99; SD = 10.12), again confirming the adequacy of the three-factor model, though with a reduction in the scale to 20 items (items 1, 7, 12, and 14 were problematic) (Tieu et al. 2018). The reliability analysis revealed acceptable results (total scale: α = 0.75; help-seeking propensity: α = 0.79; indifference to stigma: α = 0.77) except for the psychological openness factor (α = 0.59). Moreover, the help-seeking propensity factor correlated positively with intentions to use mental health services (Tieu et al. 2018).

In the Philippines, two versions of the IASMHS were used, one in English (psychological openness: α = 0.63; help-seeking propensity: α = 0.71; indifference to stigma: α = 0.84) and another in Filipino (psychological openness: α = 0.58; help-seeking propensity: α = 0.76; indifference to stigma: α = 0.75) (Tuliao et al. 2019). The English version was assessed by a study of 359 college students (52.1% female; M_age = 17.69; SD = 0.97) and the Filipino version studied 374 Filipino migrant workers (51.6% female; M_age = 33.19; SD = 7.72). Both versions were reduced to 20 items (items 1, 4, 7, and 23 were problematic), positively correlating, in most factors, with social support (Tuliao et al. 2019).

Lastly, a Portuguese version of the IASMHS, composed of 24 items, was used in a sample of 175 women in the perinatal period (100% female; M_age = 29.84; SD = 4.72) (Fonseca et al. 2017). It was found that the three-factor model revealed the best fit, displaying satisfactory reliability for the total scale (α = 0.83) as well as for the help-seeking propensity (α = 0.75) and indifference to stigma (α = 0.83) factors. The psychological openness factor (α = 0.63) exhibited more unsatisfactory results. The IASMHS presented suitable convergent validity indicators and displayed numerous correlations, particularly between intention to seek formal help and the factors of help-seeking propensity and indifference to stigma (Fonseca et al. 2017).

1.3. The Present Study

Fonseca et al. (2017) remark that the sample of the Portuguese version of the IASMHS did not allow the generalization of the results, it displayed a limited size, and temporal stability was not investigated, suggesting that the inventory’s properties should be studied in diversified populations, encompassing the analysis of temporal stability.

Recalling that adolescence is a changeful period of life (Grande et al. 2020) and given that the age-appropriateness of help-seeking measures is important (Divin et al. 2018), the aim of the present study is to assess the psychometric properties of the Portuguese version of the IASMHS in a sample of Portuguese adolescents, intending specifically (i) to determine its factor structure, (ii) to evaluate its reliability and its convergent and discriminant validity, and (iii) to explore gender differences.

2. Materials and Methods

2.1. Sample

The sample included 750 Portuguese adolescents (female: n = 392; 52.3%), enrolled in the third cycle of basic education (74.8%) or high school (25.2%). Ages ranged from 12 to 19 years (M_age = 14.67; SD = 1.85).

For factor analysis, the samples were split into two randomized halves. The first half, consisting of 375 participants (M_age = 14.69, SD = 1.88; 185 males) was allocated to exploratory factor analysis. The second half, consisting of another 375 participants (M_age = 14.66, SD = 1.83; 173 males), was allocated to confirmatory factor analysis.

The retest sample consisted of the original sample reduced to a total of 697 adolescents due to a dropout rate of approximately 7.1%.

2.2. Measures

Impulse, Self-harm and Suicide Ideation Questionnaire for Adolescents (ISSIQ-A; Carvalho et al. 2015a): Containing 56 items, the ISSIQ-A assesses impulse (module A), self-harm (module B), self-harm functions (module C) and suicidal ideation (module D), through four-point scales from 0 (Never) to 3 (Always), and a dichotomic scale (yes or no) (Carvalho et al. 2015a). In the present study, only module D was used, to measure suicidal ideation. This module exhibited satisfactory reliability in the original study (α = 0.82) (Carvalho et al. 2015a), and in the present study (α = 0.85).

Positive and Negative Suicide Ideation Inventory (PANSI; Osman et al. 1998): Consisting of 14 items, the PANSI evaluates positive (positive ideation) and negative (negative ideation) thoughts regarding suicidal ideation, using a five-point scale from 1 (None of the time) to 5 (Most of the time) (Osman et al. 1998). Adequate reliability results were obtained in the original study (negative ideation: α = 0.91 e 0.93; positive ideation: α = 0.80 e 0.82) (Osman et al. 1998) as well as in the present study (negative Ideation: α = 0.95; positive ideation: α = 0.83). With the author’s (Peter Gutierrez) consent, this measure was translated for the context of the current investigation, according to procedures grounded in the literature (e.g., Fortin 2003; Gutierrez and Osman 2008). The translation process entailed the translation of the measure from English to Portuguese, by two independent individuals. Both translations were compared and mutually adjusted to create a unified Portuguese version. Afterwards, the Portuguese version was back-translated by two different individuals, and both back-translations were compared with each other and with the original version of the PANSI. Lastly, the final version of the measure was filled in by monolingual Portuguese subjects to obtain feedback regarding the comprehensibility and interpretation of the translated items.

Center for Epidemiologic Studies Depression Scale for Children (CES-DC; Weissman et al. 1980; Carvalho et al. 2015b): Containing 20 items, the CES-DC measures depressive symptomatology in children/adolescents, through a four-point scale from 0 (not at all) to 3 (a lot) (Weissman et al. 1980). The Portuguese version (α = 0.90) includes three factors labeled as: humor (α = 0.90); interpersonal (α = 0.87) and happiness (α = 0.57) (Carvalho et al. 2015b). In the present study, suitable reliability was shown for the total scale (α = 0.93) and its factors (humor: α = 0.92; interpersonal: α = 0.85; happiness: α = 0.70).

Satisfaction with Social Support Scale (SSSS; Pais-Ribeiro 1999): Encompassing 15 items, the SSSS appraises satisfaction with perceived social support, through a five-point scale from A (totally agree) to E (totally disagree) (Pais-Ribeiro 1999). Subdivided into four factors, it presents suitable reliability in the original study (total scale: α = 0.85; satisfaction with friends: α = 0.83; intimacy: α = 0.74; satisfaction with family: α = 0.74; social activities: α = 0.64) (Pais-Ribeiro 1999). In the present study the reliability was also adequate (total scale: α = 0.87; satisfaction with friends: α = 0.85; satisfaction with family: α = 0.88), excepting in the intimacy (α = 0.60) and social activity (α = 0.57) factors.

The Inventory of Attitudes toward Seeking Mental Health Services (IASMHS; Mackenzie et al. 2004; Fonseca et al. 2017): For a full description of this tool, refer to the introduction section. In the present study, the measure’s reliability results were heterogeneous: total scale: α = 0.80; psychological openness: α = 0.54; help-seeking propensity: α = 0.81; indifference to stigma: α = 0.82.

General and Actual Help-Seeking Questionnaire (GHSQ; Wilson et al. 2005a; Fonseca and Canavarro, ongoing psychometric studies): Measuring help-seeking intentions and behaviors, the first section of the GHSQ assesses help-seeking intentions through a seven-point scale, from 1 (extremely unlikely) to 7 (extremely likely) (Fonseca et al. 2017). The second section analyses help-seeking behaviors, also including four questions about previous contacts with mental health professionals (Silva 2015).

In the original study adequate reliability was demonstrated in the first section of the measure (α = 0.70) (Wilson et al. 2005a). In the absence of a published Portuguese validation study, it is not possible to the attest the original psychometric data, nonetheless, Fonseca et al. (2017), attested acceptable reliability (α = 0.63) in Portugal, similarly to the present study (α = 0.75).

2.3. Procedure

2.3.1. Data Collection

Authorizations were solicited from the Regional Directorate for Education and from the agencies responsible for the schools on São Miguel Island. In the seven schools that authorized the execution of the study, meetings were scheduled, data collection sessions were programed and permission sheets for parents/guardians were provided.

Data collection was conducted on two separate occasions. On the first occasion, informed consent was obtained from the participants and all the assessment tools were filled in. On the second occasion, after 3 to 4 weeks, the IASMHS was reapplied for retest purposes. Participants were asked to write a numeric code on the questionnaire protocol, making it possible to pair the test and retest of the same adolescent while maintaining confidentiality. Furthermore, the assessment measures were arranged in different orders and, when finished, were inserted into an opaque container.

2.3.2. Data Analysis

SPSS 26.0 was used for most of the analysis, while AMOS 26.0 was employed in confirmatory factor analysis, G*Power (Faul et al. 2009) was utilized to calculate Cohen’s d, and Monte Carlo PCA for Parallel Analysis (Watkins 2000) was applied to run the eigenvalue parallel analysis.

Exploratory factor analysis was performed using varimax (orthogonal rotation) according to the following criteria: factors with eigenvalues larger than 1 (Kaiser criteria) and higher than presented in Monte Carlo parallel analysis were retained; items with factor loadings higher than 0.30 (ideally 0.40) were retained; items with a communality lower than 0.20 were removed; items with cross-loadings were assessed on a case-by-case basis, based on the communalities, the intensity of cross-loadings and the impact of the item on the scales’ internal consistency.

In the confirmatory factor analysis, the correlations between factors were freely estimated, covariances between error terms freely estimated and the factors were normed (the loading of the first item set to 1), cross-loadings were not allowed, and the items were assigned to respective factors, according to the results of EFA. The factor model fit was considered adequate when: the Chi-square over degrees-of-freedom ratio (χ²/df) was lower than 2; the comparative fit index (CFI) and the Tucker–Lewis Index (TLI) were higher than 0.90; the goodness-of-fit index (GFI) was also higher than 0.90; the root mean squared error of approximation (RMSEA) was lower than 0.06; the standardized root mean residual (SRMR) was lower than 0.08 and the parsimonious normed fit index (PNFI) was close to the unit (Blunch 2016).

The following were also evaluated: internal consistency (Cronbach’s alpha and McDonald’s omega coefficients); temporal stability (Pearson correlation coefficient and Student’s t test for paired samples); convergent and discriminant validity (Pearson correlation coefficient); and gender differences (Student’s t test for independent samples).

Regarding convergent validity, the constructs used in the analysis were formal help-seeking intentions, depression, social support, and suicidal ideation. The hypothesized correlations between constructs entailed positive correlations between help-seeking attitudes and help-seeking intentions as well as social support. Negative correlations between help-seeking attitudes and depression as well as suicidal ideation were also hypothesized

Finally. The indicator of discriminant validity was evaluated employing the same constructs as Mackenzie et al. (2004), on the hypothesis that the IASMHS should be more correlated with formal help-seeking intentions than with informal help-seeking intentions.

3. Results

3.1. Exploratory Factor Analysis (EFA)

The EFA’s viability was confirmed, as the value noted in the Kaiser–Meyer–Olkin (KMO) test for sampling adequacy was 0.84, which is very good, and the Bartlett’s test of sphericity (ꭓ²(276) = 2095.63, p < 0.001) was significant (Field 2018).

Through the assessment of the Kaiser criterion, the scree plot, and the Monte Carlo parallel analysis, it was revealed that the IASMHS comprised three factors, explaining 38.85% of the variance, as described in

Table 1. Exploratory Factor Analysis of the IASMHS.

Item	F1	F2	F3	h2
Factor 1–Help-seeking propensity
Item 8	0.73			0.57
Item 15	0.69			0.51
Item 19	0.69		0.33	0.59
Item 13	0.65			0.46
Item 10	0.64			0.48
Item 22	0.63			0.40
Item 5	0.59			0.35
Item 23	0.49			0.26
Item 2	0.45			0.33
Item 4 a	−0.32			0.17
Factor 2–Indifference to stigma
Item 11 a		0.74		0.57
Item 16 a		0.71		0.52
Item 20 a		0.69		0.52
Item 24 a		0.63		0.41
Item 6 a		0.63		0.40
Item 3 a		0.59		0.43
Item 17 a		0.58		0.37
Factor 3–Psychological openness
Item 14 a		0.36	0.36	0.26
Item 21 a			0.65	0.45
Item 12 a			0.60	0.41
Item 18 a			0.45	0.28
Item 9 a			0.40	0.20
Item 7 a		0.31	0.37	0.23
Item 1 a			0.36	0.15
Eigenvalue	4.64	3.32	1.37
Variance explained (%)	19.33	13.84	5.69
Total variance explained (%)	38.85

Note. N = 375; h² = Communalities. ^a Reverse coded item.

.

Items 7 (“It is probably best not to know everything about oneself”), 14 (“There are experiences in my life I would not discuss with anyone”), and 19 (“If I believed I were having a mental breakdown, my first inclination would be to get professional attention”) exhibited cross-loadings. Items 7 and 14 were removed since the difference between the highest factor loadings was lower than 0.20 and the communalities were lower than 0.30. Item 19 was retained as its removal would decrease the scale’s internal consistency (e.g., Mun et al. 2015), its communality was high and the difference between the highest factor loadings was above 0.20 (Tabachnick and Fidell 2019).

Items 1 (“There are certain problems which should not be discussed outside of one’s immediate family”) and 4 (“Keeping one’s mind on a job is a good solution for avoiding personal worries and concerns”) were also removed for having “very low” communalities (<0.20) (Child 2006, p. 47) and for lessening the scale’s internal consistency (e.g., Mun et al. 2015).

At the end of the EFA, the IASMHS encompassed 20 items and three factors: help-seeking propensity (items 8, 15, 19, 13, 10, 22, 5, 23, and 2); indifference to stigma (items 11, 16, 20, 24, 6, 3, and 17) and psychological openness (items 21, 12, 18, and 9). Item 23 loaded on the help-seeking propensity factor, which was not conceptually expected.

3.2. Confirmatory Factor Analysis (CFA)

In the CFA, both a one-factor model and a three-factor model were tested. The one-factor model revealed a notably poor fit since none of the indices achieved the established cut-off values. Contrarily, the proposed three-factor model exhibited a satisfactory model fit (

Table 2. Fit indices for the Tested Models (One-Factor and Three-Factor).

Model	χ2	df	χ2/df	RMSEA	SRMR	GFI	TLI	CFI	PNFI
One-factor model (20 items) (N = 375)	1122.67	170	6.60	0.12	0.14	0.65	0.44	0.50	0.42
Three-factor model (20 items) (N = 375)	315.30	167	1.89	0.05	0.05	0.92	0.91	0.92	0.75

Note: χ² = Chi-Square; df = Degrees of Freedom; χ²/df = Chi-square over Degrees-of-Freedom Ratio; CFI = Comparative Fit Index; GFI = Goodness-of-Fit Index; RMSEA = Root Mean Square Error of Approximation; TLI = Tucker–Lewis Index; PNFI = Parsimonious Normed Fit Index; SRMR = Standardized Root Mean Square Residual.

).

In the three-factor model, the Chi-square over degrees-of-freedom ratio was shown to be adequate although it was observed that the Chi-square test was significant. However, despite being an undesirable result, this is not definitive since this test may fail in large samples (N > 300), as described by Kline (2016). The CFI, TLI, and GFI all had satisfactory results, respecting the cut-off criteria, while the RMSEA, SRMR and PNFI outcomes were also suitable (Blunch 2016).

3.3. Reliability Analysis

Through the assessment of the Cronbach’s alpha and McDonald’s omega coefficients, it was shown that the total scale (α = 0.80; Ω = 0.79) as well as two of the three factors (help-seeking propensity: α = 0.81; Ω = 0.83; indifference to stigma: α = 0.82; Ω = 0.83) had satisfactory internal consistency. However, the psychological openness factor (α = 0.54; Ω = 0.57) displayed the lowest internal consistency.

A temporal stability analysis (N = 697) was also run indicating that positive correlations were established between the test and the retest (total scale: r = 0.72, p < 0.001; help-seeking propensity: r = 0.64, p < 0.001; indifference to stigma: r = 0.61, p < 0.001; psychological openness: r = 0.50, p < 0.001), and that there were no significant mean differences (total scale: t(696) = 0.25, p = 0.803; help-seeking propensity: t(696) = 1.01, p = 0.313; indifference to stigma: t(696) = −0.58, p = 0.564; psychological openness: t(696) = −0.39, p = 0.699), suggesting the existence of suitable temporal stability.

3.4. Validity Analysis

As presented in

Table 3. Convergent and Discriminant Validity of the IASMHS.

	R
	GHSQ—F	GHSQ—I	CES-DC	SSSS	QIAIS	PANSI-N
Total scale	0.42 ***	0.32 ***	−0.38 ***	0.45 ***	−0.29 ***	−0.30 ***
Help-seeking propensity	0.46 ***	0.31 ***	−0.23 ***	0.34 ***	−0.14 ***	−0.13 ***
Indifference to stigma	0.18 ***	0.22 ***	−0.39 ***	0.40 ***	−0.32 ***	−0.36 ***
Psychological openness	0.13 ***	0.04	−0.11 **	0.11 **	−0.14 ***	−0.13 **

Note. N = 750. R = Pearson correlation coefficient; GHSQ—F = Formal-help-seeking intentions; GHSQ—I = Informal-help-seeking intentions; CES-DC = Depression; SSSS = Satisfaction with social support; QIAIS = Suicidal ideation; PANSI-N = Negative suicidal ideation. ** p < 0.01; *** p < 0.001.

, a validity analysis was carried out by correlating the IASMHS with several variables and assessing the associations between them.

To explore convergent validity, the IASMHS was correlated with different constructs, namely: intention to seek formal help, depression, social support, and suicidal ideation. Positive correlations were displayed between the IASMHS and intention to seek formal help, and also between the IASMHS and social support. Negative correlations were found between the IASMHS and depression, as well as between the IASMHS and suicidal ideation, whether it was measured by the ISSIQ-A or by the PANSI.

Discriminant validity was assessed by examining whether the IASMHS would correlate more strongly with intention to seek formal help than with intention to seek informal help (Mackenzie et al. 2004). The total scale correlated more strongly with intention to seek formal help rather than informal help. Similarly, in the help-seeking propensity factor the correlation was also stronger with formal sources, compared to informal sources. Additionally, the psychological openness factor did not correlate with the intention to seek informal help and the indifference to stigma correlated more strongly with informal than formal sources.

3.5. Gender Differences

No gender differences were found, neither in the total scale (t(748) = −1.11, p = 0.270), nor in the factors (help-seeking propensity: t(748) = −1.56, p = 0.119; indifference to stigma: t(748) = 0.71, p = 0.481; psychological openness: t(748) = −1.59, p = 0.112).

4. Discussion

Attitudes constitute a significant barrier to help-seeking (Rickwood et al. 2005), and it is important to promote positive attitudes which are essential in suicide prevention (Reynders et al. 2014). Mackenzie et al. (2004) developed the IASMHS to assess help-seeking attitudes. Knowing that young people in distress are reluctant to seek help (Biddle et al. 2004) the present study aimed to explore the psychometric properties of the IASMHS, in a sample of Portuguese adolescents.

The IASMHS was subjected to EFA, validating a three-factor structure, comprised of the following dimensions: help-seeking propensity, indifference to stigma and psychological openness. These results not only corroborated the original study of the inventory (Mackenzie et al. 2004) but also the Portuguese study (Fonseca et al. 2017) as well as numerous other international studies (e.g., Hyland et al. 2014; Kantor et al. 2017; Tieu et al. 2018).

Due to a number of concerns, particularly cross-loadings and low communalities, four items were removed from the scale (items 1, 4, 7, and 14), reducing it to 20 items. Item 23 loaded on the help-seeking propensity factor, which was not expected, however it was retained for exhibiting adequate characteristics. The reduction of the scale to 20 items has already been observed in other studies (Tieu et al. 2018; Tuliao et al. 2019). Regarding the items with limitations, it is observed that these seem to be constant in the literature (Kantor et al. 2017; Tieu et al. 2018; Tuliao et al. 2019) since items 1, 4, 7 and/or 14 have frequently been shown to be problematic and item 23 has often revealed factor loading problems.

Regarding item 23, when analyzing its content (“Had I received treatment for psychological problems, I would not feel that it ought to be ‘covered up’”) it can be inferred that the said content could be subjectively placed under the definition of help-seeking propensity as well as that of indifference to stigma (refer to the introduction section for definitions). Therefore, this ambiguity regarding the item’s content might be a contributing factor for its not loading on the theoretically expected factor.

The help-seeking propensity factor kept all the items of the original (Mackenzie et al. 2004) and Portuguese (Fonseca et al. 2017) studies; however, item 23 was added to this dimension. The indifference to stigma factor encompassed all the expected items, excepting item 23 which migrated to the abovementioned factor. The psychological openness factor maintained all the original items, except items 1, 4, 7, and 14 which were excluded.

Subsequently, the CFA was run with both a one-factor and a three-factor model, with the 20-item three-factor model exhibiting the best fit. It is noteworthy that the observed index values revealed in the three-factor model are similar to those found in the original study (Mackenzie et al. 2004) as well as in other studies (e.g., Hyland et al. 2014; Kantor et al. 2017; Lheureux 2015), having, in some indices, achieved a greater fit. Overall, the model fit is revealed to be slightly lower than that attested by Fonseca et al. (2017) which might be explained by the differences between the samples included in each study.

Concerning reliability, the total scale as well as the help-seeking and the indifference to stigma factors presented satisfactory internal consistency. The psychological openness factor revealed a value (α = 0.54; Ω = 0.57) which was not desirable; however, in scales with a reduced number of items, such as this one, we should expect internal consistency values to be inferior, since an increased number of items leads to increased internal consistency (Field 2018). Furthermore, the reduced internal consistency observed in the psychological openness factor is a well- corroborated occurrence (e.g., Fonseca et al. 2017; Kantor et al. 2017; Tieu et al. 2018), since, in several studies, this factor’s Cronbach’s alpha has tended frequently to be below 0.70 and lower than the value achieved on the total scale and/or on the other factors.

Several authors have put forward explanations for this factor’s poor reliability, based on such valid considerations as sample characteristics (e.g., Kantor et al. 2017; Tieu et al. 2018); however, considering the frequent reports of the psychological openness factor’s low reliability results, it might be argued that future research concerning this measure should focus on promoting a better understanding of this one factor. Regarding Portuguese adolescents specifically, future research could aim to strengthen and clarify the reduction of this measure’s items on this sample, to possibly enhance its global and factorial reliability. This goal could be reached through specific methodology such as a focus group.

Temporal stability was also confirmed in the present study, corroborating Mackenzie et al. (2004) supporting the idea that the data collected with the IASMHS remain stable, even when set apart in time.

Regarding convergent validity, the assessment measure corresponded to the theoretically expected correlations, attesting that more positive attitudes were associated with greater social support and intention to seek formal help as well as lower levels of depression and suicidal ideation, corroborating what is indicated in the literature (e.g., Fonseca et al. 2017; Kantor et al. 2017; Tuliao et al. 2019; Wilson et al. 2005b).

For discriminant validity, the results obtained by Mackenzie et al. (2004) were also corroborated as, in general, the IASMHS correlated more strongly with intention to seek formal help. However, the indifference to stigma factor correlated more strongly with intention to seek informal help. Regarding this last result and despite it being initially hypothesized that all factors would be more correlated with intention to seek formal help, when comparing the results to the study of Mackenzie et al. (2004) it is possible to observe that among all of the factors, indifference to stigma is the one most strongly correlated with informal sources, when compared to the other two factors and to the total scale. Therefore, although this result on the indifference to stigma factor is the only one which refutes the study of Mackenzie et al. (2004), it is understandable considering that in both studies the aforementioned factor was shown to be the most correlated with informal sources.

Finally, gender differences were analyzed, suggesting that attitudes did not differ between boys and girls, thereby refuting the results attained by Mackenzie et al. (2004).

Considering the present study, two main limitations should be mentioned: the measurements were taken in a school context which might have influenced the responses of some adolescents, and the duration/extension of the measure protocol might have contributed to demotivation in some participants.

Nonetheless, the study also encompasses several strengths by contributing to the existing knowledge about help-seeking and associated attitudes in Portugal; by promoting the validation of an assessment tool in adolescents; by considering the recommendations made by Fonseca et al. (2017); by considering the analysis of temporal stability and investigating the IASMHS in a diversified sample; by attesting satisfactory psychometric properties in the assessed measure; and by conducting the investigation using a wide-ranging sample (N = 750).

In conclusion, the results in the present study validate the use of the IASMHS and contribute to the enhancement of its applicability in research and clinical practice contexts.