Translation, Adaptation and Assessment of the Psychometric Properties of the Mental Health Knowledge Questionnaire in a Sample of Higher Education Students in Portugal

A significant part of the worldwide population is affected by some mental disorder. Previous research conducted with the general population has revealed poor knowledge when it comes to mental health. Therefore, it is imperative to assess mental health literacy using robust assessment tools. Thus, this study aimed to translate, adapt and assess the psychometric properties of the Mental Health Knowledge Questionnaire in a sample of higher education students in Portugal. This study used a sample consisting of 2887 participants. Internal consistency for the psychometric study was estimated using Cronbach’s alpha coefficient. Construct validity was tested using exploratory and confirmatory factor analysis, convergent validity, and discriminant validity. After data analysis, the final Portuguese version of the Mental Health Knowledge Questionnaire was composed of 14 items. The goodness-of-fit indices were adequate, confirming the quality of the model’s fit to the empirical data (χ²/df = 2.459, GFI = 0.983, CFI = 0.969, RMSEA = 0.032, RMR = 0.023, SRMR = 0.032). This assessment tool is valid and reliable to assess higher education students’ mental health literacy in Portugal. Analyses to confirm the scale’s external validity, measurement equivalence and replicability are still required.


Introduction
The definition of mental health literacy has evolved over the last few years. The concept was first introduced by Jorm et al. [1], who defined it as "knowledge and beliefs about mental disorders which aid their recognition, management or prevention". More recently, Kutcher et al. [2] gave a broader definition of mental health literacy as "understanding how to obtain and maintain positive mental health; understanding mental disorders and their treatments; decreasing stigma related to mental disorders; and enhancing help-seeking efficacy (knowing when and where to seek help and developing competencies designed to improve one's mental health care and self-management capabilities)".
Understanding the importance of health literacy should be a cross-cutting issue for every generation. However, this awareness should be an even greater concern when dealing with younger age groups as they represent the next generations of adults. Data from the WHO World Mental Health International College Student Initiative [3] indicated that only 24.6% of first-year students would seek help in case of an emotional problem. Furthermore, only 16.4% of them received healthcare, despite the high prevalence of mental health disorders and their well-documented consequences [4]. That way, the focus may be placed on new generations, such as higher education students, since there are approach to analyse how the scale would perform in a Portuguese sample, which led to a change in its response format that later would need to be tested in a different sample.
Therefore, despite that preliminary study, the MHKQ was not validated for the Portuguese population and, most specifically, for higher education students in Portugal. Therefore, to fill that gap, this study aimed to translate, adapt and validate the MHKQ in a sample of higher education students in Portugal.

Design, Participants and Setting
A psychometric study was carried out since this is the most commonly used approach to validate self-report instruments to measure latent constructs. This type of research uses quantitative procedures in order to obtain the validity, reliability and standardisation of the measurement instrument [17].
This study was based on a non-probability convenience sample comprising 2887 higher education students from several universities in mainland Portugal and the Islands. Most participants were single (90.9%) with a mean age of 23.13 years (±6.51). Most respondents attended undergraduate studies (90.8%) in higher education institutions located in the North and Centre of the country, and 72.1% attended healthcare courses.
The psychometric study was conducted with a sample randomly divided into two groups (each group consisted of approximately 50% of the participants) using the option "data, select case, random sample case" in IBM SPSS Statistics. One of the groups was used to perform exploratory factor analysis, while the other was used to perform confirmatory factor analysis.
Data collection took place between October 2019 and March 2020. The only inclusion criterion defined was that the participants had to be students attending a higher education institution who had freely accepted to participate in the study once the final informed consent was validated. All participants were provided with the necessary information and asked to complete and validate the informed consent form.

Instrument
Past psychometric testing of the MHKQ has reported internal consistency of Cronbach's alpha coefficients ranging from 0.57 to 0.73 and a 2-week test-retest reliability of 0.68, as measured by intraclass correlation coefficients [18]. The exploratory factor analysis performed by Yu et al. [16] yielded a three-factor solution: knowledge of the characteristics of mental health and mental disorders encompassing items 1, 2, 3, 5, 7, 8, 11, 12, 15 and 16, belief in the epidemiology of mental disorders, which included items 4, 6, 9, 10, 13 and 14, and awareness of mental health promotion activities, comprising items 17, 18, 19 and 20. Cronbach's alpha coefficients ranged from 0.62 to 0.67.

Procedures Followed to Adapt the Mental Health Knowledge Questionnaire
The translation and adaptation of the MHKQ for European Portuguese were based on: (1) technical review and semantic analysis adapted to a Likert-type scale; (2) content validation; and (3) administration of the pretest to a sample of the target population to assess respondents' level of understanding.
Thus, firstly, the existing dichotomous scale of the MHKQ items was converted into a Likert-type scale in which each item was given five different response options: (1) strongly disagree, (2) disagree, (3) neither agree nor disagree, (4) agree, and (5) strongly agree. Items 2, 4, 6, 9, 10, 13, 14 and 15 are reverse-scored. Items 17,18,19 and 20 are exceptions since they maintained their initial categorisation ("Yes" = 1 point; "No" = 0 points). In doing so, linguistic equivalence was ensured, an outcome that had already been achieved in the study conducted with postpartum women, as was conceptual and psychometric equivalence. Conceptual equivalence was obtained through expert consultation, and psychometric equivalence was established through the revalidation of the instrument and the analysis of the psychometric properties.
To successfully adapt the document to a new format (Likert-type scale), it was submitted to a review conducted by a panel of experts composed of an expert in psychometrics, a healthcare worker experienced in mental health situations, a nursing professor, the translators who translated and back-translated the assessment tool (MHKQ) and a member of the research team (C.C.), who was responsible for assessing the semantic and idiomatic equivalence, the conceptual equivalence, and the cultural equivalence of this new format.
Once the necessary changes were made, a pretest was administered to a sample of 50 higher education students to assess the respondents' level of understanding. The qualitative assessment showed that the questions were clear and that there was a high level of agreement among the respondents.

Data Analysis Procedures
After being adapted to a Likert-type scale written in Portuguese, the MHKQ was validated using multivariate exploratory and confirmatory factor analysis techniques.
The exploratory factor analysis was conducted to uncover the underlying structure in a data matrix and determine the number and nature of the latent variables (factors) that best represent a set of observed variables [19,20]. This statistical technique allows a more accurate exploration of the underlying dimensions, constructs or latent variables of the observed variables.
On the other hand, confirmatory factor analysis is a procedure whose purpose is to analyse the relationships between a set of indicators or observed variables and one or more latent variables or factors. It is usually used when there is already previous information on the factorial structure that needs to be confirmed [20].
The psychometric properties of the MHKQ were assessed based on its reliability and validity, as these are the most appropriate procedures to ensure the informative quality of the data. Reliability is crucial in determining the internal consistency or homogeneity of the items, namely by determining the Pearson correlation coefficient between the different questions and the overall score and Cronbach's alpha coefficient. To achieve a good item definition, some authors [20,21] only consider the items or variables with correlation coefficients (r) higher than 0.2 with the overall score when this score does not include that specific item.
Regarding validity, only concept or construct validity and hypothetic-deductive validity [22], which provide information on the quality of the items, were considered. Construct validity involves a statistical approach that includes factor analysis of the items and the results. This method consists of an exploratory data analysis technique whose purpose is to discover and analyse the structure of a set of interrelated variables to build a measurement scale [20]. Several methods can be used to achieve that aim, but the "principal components analysis" is mostly adopted. The varimax orthogonal rotation was the factor rotation method selected.
To achieve a good factor definition, the items with loading levels (r) higher than or equal to 0.4 were considered. Eigenvalues higher than 1 (one) and the scree plot were considered to determine which factors should be retained. IBM SPSS Statistics version 25 for Windows was used for data processing. Additionally, through the Factor software, the analysis was implemented using a polychoric matrix and Robust Diagonally Weighted Least Squares (RDWLS) extraction method [23]. The decision on the number of factors to be retained was performed based on the Parallel Analysis technique with a random permutation of the observed data [24]. Model fit was assessed using the root mean square error of approximation (RMSEA), comparative fit index (CFI) and Tucker-Lewis Index (TLI) fit indices. Then, the other half of the sample was used in a confirmatory factor analysis to assess the goodness-of-fit of the measurement models of the subscales. Maximum likelihood estimation method was combined with the AMOS software version 25. Data analysis was conducted using several statistical procedures deemed to be the most adequate, namely: (i) item distribution, assessed using skewness (Sk) and kurtosis (Ku), and all items with absolute skewness values higher than 3 and kurtosis higher than 7 were removed; (ii) construct validity, assessed using factor validity, convergent validity and discriminant validity; and (iii) goodness-of-fit of the factor model, assessed based on the following indices and reference values: χ 2 /df (the ratio between chi-square and degrees of freedom)-the model's fit is considered good if the ratio (χ 2 /df) is lower than 2, acceptable when it is lower than 5 and unacceptable when it is higher than 5; root mean square residual (RMR)-the lower it is, the better the model's fit (e.g., RMR = 0 indicates a perfect fit); standardized root mean square residual (SRMR)-the standardized difference between the observed and the predicted correlation (a zero value indicates a perfect fit and a value lower than 0.08 is generally considered a good fit); goodness-of-fit index (GFI)-values higher or close to 0.95 are recommended, but any value above 0.90 is considered a good fit; comparative fit index (CFI)-is an additional fit index (values lower than 0.90 indicate a poor fit, values between 0.90 and 0.95 are evidence of a good fit, and values higher than 0.95 reflect a very good fit (this index can be a reference regardless of the size of the sample); root mean square error of approximation (RMSEA)-reference values between 0.05 and 0.08 represent a good fit and a very good fit occurs when the index is lower than 0.05; local goodness of fit assessed by factor loadings and by individual item reliability-reference values are 0.5 and 0.25, respectively, although in early studies factor loading values of 0.40 or close can be accepted.
The model fit was performed drawing on the modification indices proposed by AMOS, and trajectories with modification indices above 11 were adjusted; construct reliability was assessed using composite reliability (CR), which indicates the degree to which the items are consistent manifestations of the latent factor (CR ≥ 0.7 indicates appropriate construct reliability).
The convergent validity of each factor was assessed using average variance extracted (AVE). Convergent validity was achieved when AVE was higher than 0.5 [18]. Factor discriminant validity was obtained by squaring the correlation between the factors. There is evidence of discriminant validity when the value obtained is lower than the AVE for each factor.
All factors were standardized by setting their variances at 1.00 (one) and keeping them inter-correlated.

Ethical Procedure
The study protocol was approved by the Ethics Committee of the Nursing School of Porto (protocol code: 2019/1945). All participants were provided with the necessary information and asked to sign the informed consent form. Anonymity, data confidentiality, and autonomy were ensured. Participants were also informed that they were free to participate in the study and could withdraw at any time. They were informed that they would not receive any compensation for participating and withdrawal from the study would not result in any harm/loss. Table 1 presents the statistics (mean and standard deviation), item-total correlation, Cronbach's alpha, and McDonalds' omega coefficients of the 20 items of the scale based on the responses of the 1443 sample participants. The average indices in all items are higher than the expected average score, which means that the data are well-centred.

Exploratory Factor Analysis
The Cronbach's alpha coefficients of the items are barely reasonable, ranging between 0.661 (item 4-"All mental disorders are caused by external stressors") and 0.710 (item 15-"Individuals with a bad temperament are more likely to have mental problems"), with an overall Cronbach's alpha of 0.691. The value of the split-half reliability index is slightly lower than the overall alpha for both the first half (α = 0.563) and the second half (α = 0.469). The McDonalds' omega coefficients are more consistent, presenting the lowest value on item 11 (ω = 0.695) and the highest value on item 15 (ω = 0.740). The overall McDonalds' omega coefficient is 0.726.
The factor analysis of the scale was conducted and included all the items. The Kaiser-Meyer-Olkin (KMO) test for sampling adequacy, which assesses the model's adequacy, revealed a value of 0.809, suggesting the existence of a good correlation between the items. Bartlett's Test of Sphericity (specificity test) was x 2 = 4559.4980; p = 0.001, which indicates that the items are correlated. These test results support the use of exploratory factor analysis with these data.
Factors were then extracted using principal component analysis and varimax orthogonal rotation, and eigenvalues greater than 1 and item loading levels higher than 0.40 were considered for factor retention [20]. A factorial structure comprising five factors, which explained 48.74% of the total variance, was then obtained. However, the scree plot showed a three-factor structure according to the inflection point of the curve (Figure 1). This finding, combined with two factors that only included two items each, led to a three-factor rotation.

McDonalds' ω
Overall 0.726 The factor analysis of the scale was conducted and included all the items. The Kaiser-Meyer-Olkin (KMO) test for sampling adequacy, which assesses the model's adequacy, revealed a value of 0.809, suggesting the existence of a good correlation between the items. Bartlett's Test of Sphericity (specificity test) was x 2 = 4559.4980; p = 0.001, which indicates that the items are correlated. These test results support the use of exploratory factor analysis with these data.
Factors were then extracted using principal component analysis and varimax orthogonal rotation, and eigenvalues greater than 1 and item loading levels higher than 0.40 were considered for factor retention [20]. A factorial structure comprising five factors, which explained 48.74% of the total variance, was then obtained. However, the scree plot showed a three-factor structure according to the inflection point of the curve (Figure 1). This finding, combined with two factors that only included two items each, led to a three-factor rotation. The results obtained with this new procedure led to the removal of items 6 and 9 because they showed loading scores lower than 0.40. There was a decrease in the The results obtained with this new procedure led to the removal of items 6 and 9 because they showed loading scores lower than 0.40. There was a decrease in the explained variance (40.87%), while the KMO value increased (0.815). The anti-image matrix indicated the existence of high correlations ranging between r = 0.649 and r = 0.884.
Factor 1 ended up with eight items (1, 3, 5, 7, 8, 11, 12 and 16) that explained 18.53% of the total variance. This factor was referred to as the knowledge of mental health characteristics and mental disorders and presented an eigenvalue of 3.336.
Factor 2 comprised six items (2, 4, 10, 13, 14 and 15) that explained 11.79% of the total variance. It was called belief in the epidemiology of mental disorders and presented an eigenvalue of 2.212.
Factor 3 included four items (17, 18, 19 and 20) that explained 10.55% of the total variance. It was dubbed awareness of health promotion activities and presented an eigenvalue of 1.900. Table 2 presents the loadings of the items per factor and reveals that all items had factor loadings higher than 0.40. Based on these results, a confirmatory factor analysis was then conducted.
The parallel analysis reinforced the existence of three factors since the percentages of the actual variance explained in the three factors were higher than those obtained by the resampling procedures.
On the other hand, the goodness-of-fit indices obtained with the exploratory factor analysis indicate a quite adequate adjustment (RMSEA = 0.026; CFI = 0.982; GFI = 0.985).

Confirmatory Factor Analysis
The hypothesized three-factor model was submitted to confirmatory factor analysis using the maximum likelihood method. In assessing item sensitivity, we observed skewness and kurtosis values that suggested the existence of symmetry and normokurtosis in most of the items. Items 1 and 8 were the exceptions, as they were slightly skewed to the right but perfectly flat. In subsequent tests, these items were removed. Mardia's coefficients of multivariate skewness and kurtosis (1.410) were lower than the reference value (5.0). Table 3 presents the critical ratios and the items' loading values. Critical ratios showed that all items were statistically significant with the corresponding factor. Some items had very low loading values, so they were removed from subsequent analyses. Figure 2 represents the graphical output of the early model with the trajectories between the items and their respective factors, the factor loadings and the individual reliability of the items. The model presented an adequate overall goodness-of-fit for all indices (χ 2 /df = 3.117, GFI = 0.940, RMSEA = 0.058, RMR = 0.053, and SRMR = 0.056), except for CFI (=0.864).  The model was refined as the most problematic items were removed, and the trajectories were adjusted using the modification indices suggested by AMOS. The results are presented in Figure 3. In factor 1, items 1 and 8 were removed, and so were items 2 and 15 in factor 2. The modification indexes highlight the association between errors 9 and 12, included in factor 2, and errors 16   The model was refined as the most problematic items were removed, and the trajectories were adjusted using the modification indices suggested by AMOS. The results are presented in Figure 3. In factor 1, items 1 and 8 were removed, and so were items 2 and 15 in factor 2. The modification indexes highlight the association between errors 9 and 12, included in factor 2, and errors 16 and 18 in factor 3. The overall fit indices revealed values that are appropriate for all indices under analysis: χ 2 /df = 2.459, GFI = 0.983, CFI = 0.969, RMSEA = 0.032, RMR = 0.023, SRMR = 0.032. tories were adjusted using the modification indices suggested by AMOS. The results are presented in Figure 3. In factor 1, items 1 and 8 were removed, and so were items 2 and 15 in factor 2. The modification indexes highlight the association between errors 9 and 12, included in factor 2, and errors 16 and 18 in factor 3. The overall fit indices revealed values that are appropriate for all indices under analysis: χ²/df = 2.459, GFI = 0.983, CFI = 0.969, RMSEA = 0.032, RMR = 0.023, SRMR = 0.032.  Even though the correlations obtained between factors were low, especially between factors 1 and 3 and between factors 2 and 3, a second-order model was carried out. The results indicated that the model did not suggest a second order factor, as factor 1 explains 111% of the global scale ( Figure 4).

Int. J. Environ. Res. Public Health 2023, 20, x FOR PEER REVIEW 11 of 18
Even though the correlations obtained between factors were low, especially between factors 1 and 3 and between factors 2 and 3, a second-order model was carried out. The results indicated that the model did not suggest a second order factor, as factor 1 explains 111% of the global scale ( Figure 4).

Convergent Validity and Composite Reliability
Internal consistency of the scale ranged between low and barely acceptable depending on the results of the composite reliability for the different factors of the scale. On the other hand, AVE values suggested a lack of convergent validity as the indices found were higher than 0.50. The discriminant validity of the factors, assessed by comparing the AVE with the squared correlation, appeared only in factors 1 and 3 and between factors 2 and 3 (Table 4). Table 4. Composite reliability, convergent validity and discriminant validity.

Convergent Validity and Composite Reliability
Internal consistency of the scale ranged between low and barely acceptable depending on the results of the composite reliability for the different factors of the scale. On the other hand, AVE values suggested a lack of convergent validity as the indices found were higher than 0.50. The discriminant validity of the factors, assessed by comparing the AVE with the squared correlation, appeared only in factors 1 and 3 and between factors 2 and 3 (Table 4). To conclude the validation study of the MHKQ, we calculated the internal consistency of the remaining items for each subscale.
In Factor 1 (knowledge of the characteristics of mental health and mental disorders), Cronbach's alpha coefficients indicated a barely acceptable internal consistency: the lowest value (α = 0.628) was found in item 16, and the highest value (α = 0.670) was reported in item 3, with an overall alpha of 0.693. In Factor 2 (belief in the epidemiology of mental disorders), Cronbach's alpha coefficients presented a tendentially low internal consistency: the lowest value (α = 0.510) was obtained in item 14 and the highest (α = 0.536) in item 10, with an overall alpha of 0.597. In Factor 3 (awareness of mental health promotion activities), the internal consistency of Cronbach's alpha coefficients was also tendentially low: indices ranged between 0.513 in item 19 and 0.558 in item 20, with an overall alpha of 0.602. Thus, Cronbach's alpha threshold of 0.70 was not reached for any of the factors (Table 5).  Table 6 presents the convergent/divergent validity of the items. The presented outcomes highlight the existence of convergent validity between the different items, and the factor to which they correspond since the correlational value presented in bold is higher with the subscale to which it belongs and the second one with the total factor of the scale. Table 6. Convergent/divergent validity of the items.

Items
Factor 1 Factor 2 Factor 3 Total Scale Finally, the correlation matrix between the three factors and the global scale is presented in Table 7. All correlations were positive and significant. The correlation between factor 1 and factor 3 (r = 0.345) is the highest and explains 11.90%. The variability found for the global scale is higher and ranges between 0.338-with factor 3 explaining 11.42%-and 0.817, with a variability of 66.74%. The statistics of the different factors were determined based on the sum of the items composing each factor. To compare the results obtained in the three subscales, since they are not composed of the same number of items, they were converted into the same level of magnitude on a scale ranging from 0 to 100, using the following formula: ((scale raw score-minimum score)/range) × 100. The final results are expressed in Table 8, with factors ranging between 0% and 100% and scores for the global scale ranging between 4.55% and 100%. The average indices showed that the lowest index was found in factor 3 (mean = 60.62 ± 30.97), while the highest average index was found in factor 1 (mean = 84.40 ± 13.14). The coefficients of variation indicated low dispersion for factor 1 and moderate dispersion for the remaining factors and the global scale. In turn, skewness and kurtosis values tended to show leptokurtic curves skewed to the right for factors 2 and 3 and skewed to the left for factor 1 and the global scale. Cut-off values were defined for each factor and the overall value of the sample based on the 25th and 75th percentiles, and the participants' level of knowledge was classified as poor, moderate and good. Factor 3 was an exception since only poor and good levels of knowledge are reflected. As presented in Table 9, a good level of knowledge can be found in factor 1 (32.2%), while in factor 2 and the global scale, moderate knowledge obtained the highest score, while in factor 3, the percentage of respondents with poor and good knowledge is almost identical (50.5% and 49.5%, respectively).

Discussion
The translation and adaptation of the MHKQ to European Portuguese were based on (1) a technical review and semantic assessment adapted to a Likert-type scale; (2) content validation; and (3) administration of a pretest to a sample of the target population, to assess the participants' degree of understanding. In the psychometric study of the scale, internal consistency was estimated using Cronbach's alpha coefficient (α) and construct validity was assessed using exploratory and confirmatory factor analysis, convergent validity, and discriminant validity. Once the 20 statements were adapted to a 5-point Likert scale, exploratory factor analysis led to a three-factor structure that explained 40.86% of the total variance. Then, items 6 and 9 were removed since they showed loading levels lower than 0.40. Confirmatory factor analysis also led to the removal of items 1, 2, 8 and 15 since they all showed low loading levels. The overall goodness-of-fit indices were fairly adequate, confirming the quality of the model's fit to the empirical data. The final Portuguese version of the MHKQ was composed of 14 items (Table S1).
The final refined model with modification indices was obtained after correlated errors between item 4 ("All mental disorders are caused by external stressors") and item 10 ("Even for severe mental disorders [e.g., schizophrenia], medications should be taken only for a given period"). The same method was applied to item 19 ("Have you heard about the International Day for Suicide Prevention?") and item 20 ("Have you heard of World Sleep Day?"). We considered that the errors might be correlated for items 4 and 10 as they are both related to knowledge specifically focused on the causes and treatment of mental disorders, which are usually analysed comprehensively in studies about knowledge toward mental disorders and their treatment [25]. On the other hand, we considered that the errors might be correlated for items 19 and 20 because both concern to relatively recent mental health promotion days/activities, as they were celebrated for the first time in 2003 and 2008, respectively (while the International Mental Health Day and the International Day against Drug Abuse and Illicit Drug Trafficking are celebrated since 1992 and 1987, respectively).
Notably, we found low correlations between Factor 3 and the other factors in the final refined model with modification indices. That may be explained by two major reasons: (1) all items for Factor 3 were measured dichotomously, whereas the rest of the items (for Factors 1 and 2) were measured continuously; and (2) all items on Factor 3 relate to simple facts knowledge on mental health promotion days/activities, while the items on Factors 1 and 2 relate to respondents' stereotypes, beliefs, and attributions.
Our findings also pointed out to a barely acceptable internal consistency of the MHKQ Factor 1, but low internal consistencies of Factors 2 and 3. This might be explained by the low number of questions of Factors 2 and 3 (four questions each) or the item content heterogeneity. However, this will need to be further explored.
It is interesting to compare and contrast the psychometrics of the MHKQ with the ones of other mental health literacy assessment tools, which have already been validated in Portugal. One of those tools is the Questionnaire for Assessment of Mental Health Literacy (QuALiSMental) [26]. That assessment tool presents acceptable levels of reliability and a factor structure consistent with the theoretical components of mental health literacy. However, this assessment tool is not easy to score, and it is mostly focused on knowledge about mental disorders.
The Mental Health Literacy Scale (MentaHLiS) [27] is a mental health literacy assessment tool aiming to assess mental health literacy in adolescents. The tool is composed of three scales. Each scale is divided into five subscales (recognition of the disorder; helpseeking recourses and options; help-seeking, perceived barriers and facilitators; first aid intentions and beliefs; and lifestyles and health behaviours). All subscales present acceptable reliability coefficients (0.705 to 0.811), better than the MHKQ. However, the tool also focusses on mental disorders, specifically depression, anxiety, and alcohol abuse.
The Mental Health Literacy Scale (MHLS) [14] presents a three-factor structure (attitudes towards mental illness; knowledge about mental illness; and ability to recognize symptoms), which explains 35% of the total variance (a value lower than the MHKQ). On the other hand, the internal reliability of the factors tends to be better than in the MHKQ (ranging from 0.66 and 0.89). However, as previously stated, this assessment tool tends to focus on the knowledge about mental disorders, such as social anxiety disorder, generalized anxiety disorder, or depression.
Finally, the Mental Health Literacy questionnaire (MHLq) was developed in 2016 and validated in a sample of adolescents [28]. Then, in 2018 [29], it was adapted for the young adult population. The final version of the questionnaire included 29 items, with good total internal consistency (α = 0.84), organized into four factors: knowledge of mental health problems; erroneous beliefs/stereotypes; help-seeking and first-aid skills; and self-help strategies. This assessment tool presents better psychometric properties than the MHKQ; however, it is a longer tool (29 items) compared to the MHKQ (14 items).
Thus, compared to the other mental health literacy assessment tools already validated in Portugal, the MHKQ presents acceptable psychometric properties, even though other tools present better psychometrics. Nonetheless, the MHKQ presents two important advantages: (1) it does not focus solely on the knowledge about mental disorders; and (2) it is a short (14 items) and easy to fill assessment tool.
This study enabled to translate, adapt and validate the MHKQ in a sample of higher education students in Portugal. This study has some limitations, even though the aim of the study was achieved. The most relevant of those limitations is that it was not possible to assess the external validity, measurement equivalence and reproducibility of the tool. Another potential limitation lies in the convenience sampling technique, as it limits the generalizability of results. Moreover, the MHKQ concurrent validity alongside an established mental health literacy instrument was not assessed, which can also be viewed as a limitation of this study. Finally, the low internal consistency of the MHKQ should also be pointed out as a limitation of the assessment tool itself.

Conclusions
The MHKQ was translated and adapted to European Portuguese in the form of a Likert-type scale. the produced results during content validation, verbal comprehension and internal consistency were rather positive. The factor model found in this study is somehow different from the theoretical proposal offered by the original version of the tool and the three-factor structure obtained in a study previously carried out in a sample of postpartum women. However, this version may still be considered valid and barely reliable for assessing students' knowledge of mental health. Nevertheless, further research is still required to assess the external validity, measurement equivalence and reproducibility of the MHKQ. Author Contributions: Conceptualization, C.C., J.D. and C.S.; methodology, C.C., J.D. and C.S.; formal analysis, J.D.; investigation, C.C.; writing-original draft preparation, C.C.; writing-review and editing, F.S., J.C., J.D. and C.S.; supervision, C.S.; project administration, C.S.; funding acquisition, C.S. All authors have read and agreed to the published version of the manuscript.
Funding: This work was financed by national funds through FCT Fundação para a Ciência e a Tecnologia, I.P., within the scope of the project "RISE-LA/P/0053/2020".

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of Nursing School of Porto (protocol code; 2019/1945 date of approval: 23 July 2019).

Informed Consent Statement:
Informed consent was obtained from all subjects involved in the study. Data Availability Statement: Not applicable.