Effectiveness of Mental Health Literacy Programs in Primary and Secondary Schools: A Systematic Review with Meta-Analysis

In recent years, there has been an increase in studies evaluating the effectiveness of mental health literacy programs within the context of education as a universal, preventive intervention. A systematic review and meta-analysis regarding the effectiveness of mental health literacy interventions in schools, from 2013 to the present, on mental health knowledge, stigma, and help-seeking is conducted. Of the 795 identified references, 15 studies met the inclusion criteria. Mental health knowledge increased after the interventions (standardized mean difference: SMD = 0.61; 95% CI (0.05, 0.74)), at two months (SMD = 0.60; 95% CI (0.4, 1.07)) and six months (SMD = 0.39; 95% CI (0.27, 0.51)). No significant differences were observed between stigma and improving help-seeking. Mental health literacy interventions are effective in augmenting mental health knowledge, but not in reducing stigma or improving help-seeking behavior.


Introduction
Most mental health problems in individuals commence between 12 and 25 years of age, with 20% of adolescents being affected [1,2]. Moreover, it is calculated that 50% of such issues appear before the age of 14 [3]. In spite of this elevated incidence, 70-80% of young people and adults do not receive adequate mental healthcare [4]. Due to the importance of mental health during adolescence, international bodies such as the European Union (EU) and the World Health Organization (WHO) are promoting policies and interventions to prevent mental health disorders and raise awareness. They are highlighting the need to analyze and diffuse good practice, for instance, with the EU-Compass for action on Mental health and Well-being [5], and encourage multi-sectorial strategies to provide integral support for mental health [6]. The WHO is currently carrying out its Comprehensive Mental Health Action Plan 2013-2020 and its subsequent update until 2030. This plan aims to ensure that 80% of countries will have at least three functioning national, multisectoral mental health promotion and prevention programs by 2030. These plans have to be targeted at vulnerable groups using, as an example, programs like mental health awareness/anti-stigmatization or school-based mental health prevention and promotion [7,8]. Within this framework lies the concept of mental health literacy (MHL), which has been described by Jorm [9] as "the knowledge and beliefs about mental health problems that help in their recognition, management, and prevention". This definition includes the capacity to recognize and inform oneself about mental disorders and their risk factors and how to seek treatment and professional support [9]. In recent years, the notion of MHL has been undergoing changes and has become an integral, multifactorial concept that recognizes the need to reduce mental health stigma and foster first aid [10][11][12]. Its programs are focused on the capacity of individuals to understand and make correct use of healthcare information so as to better understand mental health disorders, improve treatment compliance, and be aware of how and when to look for assistance. The effectiveness of MHL interventions has been demonstrated in systematic revisions such as those by Wei [2] and Seedaket [13]. The former included 27 studies carried out up to 2012 in the field of education with 17,643 individuals aged 12 to 25 years. It reported an increase in knowledge concerning mental health and a decrease in stigma both in the short-term and during follow-up. The latter was made up of seven randomized clinical trials (RCTs) performed up to 2020 in a population aged 10 to 19 years and had results that were similar to Wei's review. All the interventions improved mental health knowledge: 71% and 25% for stigma and help-seeking, respectively. As neither of the two systematic reviews included a meta-analysis, the effect size of this kind of intervention could not be established. The objective of this systematic review with meta-analysis is to evaluate the effectiveness of MHL interventions on mental health knowledge, stigma, and help-seeking in young, school-enrolled populations from the beginning of the international Comprehensive Mental Health Action Plan 2013-2030 to the present.

Materials and Methods
This review was carried out according to the protocol Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) [14]. The review protocol was pre-registered on the International Prospective Register of Systematic Reviews (PROSPERO) [15] prior to data extraction (CRD42021281178, access date on 10 October 2021 in https://www.crd. york.ac.uk/prospero/display_record.php?RecordID=281178).

Inclusion Criteria
Inclusion criteria for studies regarding the efficacy/effectiveness of MHL interventions were the following: (1) population aged between 10-19 years; (2) research carried out in primary and/or secondary schools; (3) interventions with presence-based modality (school setting); (4) RCT and quasi-experimental (QE) design; (5) outcomes: mental health knowledge, stigma, and/or help-seeking; (6) studies performed from commencement of the WHO 2013-2030 Action Plan on Mental Health [7,8] and the set-up of programs of mental health prevention and promotion such as MHL; and (7) no language limits. Exclusion criteria were: (1) qualitative studies; (2) determined populations, for instance, university students and students with specific mental health or drug abuse issues; (3) studies concerning mental health first aid; (4) MHL interventions focused on a specific disorder (depression, schizophrenia...); and (5) MHL interventions that only assess the concept of stigma.

Study Search and Selection Strategy
The search was performed with the databases Pubmed, Psycinfo, Web of Science, and Cinahl from January 2013 to July 2020. The bibliographic search in the three databases included the following terms in the widest sense possible: ("mental health literacy" AND ("program*" OR "intervention*") AND "young") OR ("mental health literacy" AND school AND adolesc*). The secondary strategy involved examining the references of the included studies and relevant reviews. Duplicates and studies that did not fulfill the inclusion criteria were eliminated, and only article-type publications were selected. After revising the study abstracts, those that were related to the theme of the present research were accepted. Finally, a hand search was conducted. The studies were included according to peer review, and the following information extracted: author, year, country, design, educational stage, age, number of subjects, follow-up, type of intervention, duration of intervention, professional who delivers intervention, measurement tools, and level of evidence.
The Scottish Intercollegiate Guidelines Network (SIGN) [16] was employed to establish the identified studies' level of scientific evidence. We completed a cross-tabulation of each article selected by I.A.R. response categories to obtain frequencies and percentages of each variable of interest in the study. Inter-rater agreement was performed among two team members (blinded to peer review) to assess acceptance by determining the grade of evidence and practice guideline levels. The two members extracted the data from the included investigations and coded the studies considering the variables of our research questions. Concordance was considered adequate if it met at least 80% [17]. Seven studies (46, 6%) were randomly selected, and the investigators independently extracted the information and entered it into a codebook. The consistency of these data among the two evaluators was evaluated by calculating the proportion of agreements and disagreements in the different codes. Interrater agreement was 97.4%. Cohen's Kappa was not calculated due to the low variability of agreement between the evaluators [18]. The SIGN model is based on three sections (internal validity, general evaluation, and study description) and classifies the research into 8 levels of evidence. These range from 1++ the maximum available evidence (high-quality meta-analyses, systematic reviews of clinical trials, and clinical trials with a reduced risk of bias) to 4 (expert opinions) [16] (Appendix A, Table A1).

Analysis
A meta-analysis of the main variables (mental health knowledge, stigma, and helpseeking) was performed employing standard mean differences (SMDs) due to the different quantitative scales employed. The pooled estimates were evaluated post-intervention and at 2-and 6-months follow-up. A meta-analysis of random effects was carried out in order to obtain pooled estimates as differences were observed among the studies. These Heterogeneity was measured by the statistical test I 2 . A value of 0-40% indicated low heterogeneity; 40-75% moderate; 75-100% considerable [19]. Publication bias was assessed with a funnel plot (scatter plot with treatment effectiveness against study sample size) [20]. Review Manager (RevMan) software was employed for the meta-analysis, and the program Stata was employed for the meta-regression.
Approximately half of the selected study interventions were carried out by teaching professionals in the educational centers (n = 8; 53.3%). Some disparity was observed regarding the length of the interventions, which varied from 45 min to 12 h. One of them did not specify its duration [27].  26.67%). All the RCTs had a level of evidence between 1++ and 1−, whilst the QEs were classified between 2++ and 2+.

Evaluation of Mental Health Knowledge
Of the 15 selected studies, 1 of them pending publication [26], 6 presented statistically significant improvements at the moment of post-intervention and 10 at follow-up (between 1 and 12 months). In only one study was a statistically significant improvement observed at follow-up, but not at the moment of post-intervention [27].
With respect to the analysis of change in knowledge, we included eight studies (six RCTs and two QEs) at the moment of post-intervention and obtained, broadly speaking, a significant increase (SMD = 0.61; 95% CI (0.17, 1.04)) with marked heterogeneity I 2 = 97% ( Figure 2). If we analyze it according to the type of study, a significant increase was observed in both QEs (SMD = 1.26; 95% CI (0.59, 1.93)) and in RCTs (SMD = 0.40; 95% CI (0.05, 0.74)). At the two-month follow-up, we included three studies (one RCT and two QEs) and observed a significant rise in knowledge (SMD = 0.60; 95% CI (0.14, 1.07); I 2 = 89%) ( Figure 3). If we analyzed it according to the type of study, a significant increase was observed in the QEs (SMD = 0.85; 95% CI (0.72, 0.99)) but not in the RCTs (SMD = 0.08; 95% CI (−0.24, 0.41) p = 0.61). At the six-month follow-up, three RTCs were incorporated, which also showed a significant increase in knowledge (SMD = 0.39; 95% CI (0.27, 0.51)) without heterogeneity I 2 = 0% ( Figure 4). The QEs studies presented a greater SMD than that observed for the RCTs at both post-intervention and the two-month follow-up. After performing the meta-regression, we did not observe a significant relationship between the hours of intervention and the effect on knowledge (p = 0.450) ( Table 2).   observed in the QEs (SMD = 0.85; 95% CI (0.72, 0.99)) but not in the RCTs (SMD = 0.08; 95% CI (−0.24, 0.41) p = 0.61). At the six-month follow-up, three RTCs were incorporated, which also showed a significant increase in knowledge (SMD = 0.39; 95% CI (0.27, 0.51)) without heterogeneity I 2 = 0% (Figure 4). The QEs studies presented a greater SMD than that observed for the RCTs at both post-intervention and the two-month follow-up. After performing the meta-regression, we did not observe a significant relationship between the hours of intervention and the effect on knowledge (p = 0.450) ( Table 2).

Evaluation of Help-Seeking
Assessment of help-seeking for mental health was present in nine studies, five of which reported a significant improvement at the moment of post-intervention [24,27,31,33,35].
With respect to the analysis of change in help-seeking, we included three studies (three RCTs) at the moment of post-intervention and did not obtained a significant increase (p = 0.24; SMD = 0.20; 95% CI (−0.13, 0.52)) with heterogeneity I 2 = 90% (Figure 7). At the six-month follow-up we included two studies (two RCTs) and not statistically significant differences were observed after the interventions (SMD = 0.02; 95% CI (−0.14, 0.19) I 2 = 0%) (Figure 8). After performing the meta-regression, we did not observe a significant relationship between the hours of intervention and the effect on help-seeking (p = 0.392) ( Table 4). = 0%) ( Figure 8). After performing the meta-regression, we did not observe a significant relationship between the hours of intervention and the effect on help-seeking (p = 0.392) ( Table 4).   The funnel plots were calculated to assess publication bias (Appendix B, Figures A1-A7), although statistical tests were not applied due to having less than 10 studies.

Discussion
This review examines the effectiveness of mental health literacy interventions for young people in educational centers from the beginning of the international Comprehen- = 0%) ( Figure 8). After performing the meta-regression, we did not observe a significant relationship between the hours of intervention and the effect on help-seeking (p = 0.392) ( Table 4).   The funnel plots were calculated to assess publication bias (Appendix B, Figures A1-A7), although statistical tests were not applied due to having less than 10 studies.

Discussion
This review examines the effectiveness of mental health literacy interventions for young people in educational centers from the beginning of the international Comprehen-  The funnel plots were calculated to assess publication bias (Appendix B, Figures A1-A7), although statistical tests were not applied due to having less than 10 studies.

Discussion
This review examines the effectiveness of mental health literacy interventions for young people in educational centers from the beginning of the international Comprehensive Mental Health Action Plan 2013-2030 to the present. It assesses how such programs influence knowledge regarding mental health, the stigma surrounding it, and help-seeking. Most of them displayed positive effects, although only the variable of knowledge showed significant improvement in the short-and long-term.
With respect to knowledge, we observed that after analysis, the studies with the best post-intervention results were two QEs (Yamaguchi and Bella-Awusah) [28,35], followed by one RCT from Campos [24]. These findings were maintained at two months in Yamaguchi and Bella-Awusah [28,35] and at six months in Campos [24]. As shown in the comparison of results, QE studies tended to overestimate intervention effectiveness (lack of randomization leading to a greater probability of bias). The three interventions lasted less than 3 h and differed with respect to the characteristics of the individual responsible for carrying them out (teacher, researcher, and expert psychologist). In general, we observed a statistically significant improvement in knowledge for the studies analyzed both post-intervention (SMD = 0.61) and at the six-month follow-up (SMD = 0.39) compared to the usual programs.
Regarding stigma, the studies by Campos [24] and Perry [21] were the only ones that demonstrated statistically significant improvements post-intervention at the meta-analysis. While both were RCTs, they differed with respect to duration (3 h and 10 h, respectively) and professionals delivering the intervention (expert psychologists and teachers, respectively). Both studies did not present contact with a person who has suffered from mental disorders in comparison with the studies by Casañas [26], Pinto-Foltz [25], and Chisholm [23], who obtained worse results in this variable. Milin s study [22] did not present direct contact, but it did have video interviews and cases with young people with mental illnesses. The QE study of Bella-Awusah [25] analyzed intervention carried out by a psychologist and without direct contact but did not present post-intervention improvements. Future research should assess whether a first-person experience is an unfavorable factor in reducing stigma or if educational sessions with specialists or teachers are more effective in reducing stigma. We observed that Campos [24] employed the same tool to measure knowledge and stigma in contrast to the other studies, which used different scales, this could be a bias in this study, and we should take these results with caution. After analyzing the studies, we did not find a significant improvement post-intervention (SMD = 0.06; p = 0.72) nor at the six-month follow-up (SMD = 0.12; p = 0.68).
In the help-seeking variable, the three studies included in the meta-analysis were RCTs, with a different duration between them (10 h [22], 4 h 45 min [23], and 3 h [24]), as those in charge of carrying out the interventions (teachers [22], a researcher together with teachers [23], and an expert psychologist [24]). We observed that Campos [24] presents a significant improvement in the short term in the meta-analysis but not at 6 months. In general, no significant improvements were found in this variable, neither post-intervention (SMD = 0.2; p = 0.24), nor at 6 months (SMD = 0.02; p = 0.79).
Regarding the analysis over time, worse results were observed in the three main variables as time progressed. This may be due to the fact that over time if knowledge is not reinforced, it is progressively forgotten, as Ebbinghaus [36] and other current studies postulate [37]. In addition, no significant time-dependent relationship was observed in the hours of intervention with the three main variables (knowledge, stigma, and help-seeking), as can be seen in the meta-regression in Tables 2-4.
These results are similar to the results of the Wei and Seedaket [2,13] reviews both in terms of knowledge, stigma, and help-seeking at a qualitative level. However, because they did not perform their respective meta-analyses, we cannot compare them. There was considerable variability amongst the interventions with respect to format, content, and measurement tools which hindered comparison. Moreover, their duration ranged from 45-50 min, as was the case of the Short MHL program [31,33], up to 12 h for The Guide [29,30]. In addition, MHL domains were not consistent; some studies included them in their broadest sense, for instance, Perry [21], Campos [24], and Pinto-Foltz [23]; whilst others were only concerned with the detection of one or two specific mental health disorders such as Ojio [31,33].
Our findings concur with the systematic review performed by Mansfield [38], in which the lack of homogeneity regarding the concept of MHL was highlighted. There was also a tendency to focus solely on the identification of mental health disorders rather than examining the issue from a broader, more positive perspective. The review of Nobre [39] provided us with plausibility to the search carried out since they present similar results in the selected articles, despite having different objectives and selection criteria. The most recent review, Freitan [40], also assesses knowledge and stigma but not help-seeking. Freitan's analysis differs from ours by including participants with mental illness, the time stage of the included studies, and the interventions outside the school context. Both studies present after the meta-analysis improvement in knowledge but not in stigma in the follow-up phase.
Our review presents some limitations. Firstly, the design of the studies was heterogeneous, and we observed a greater weighting in the case of the QEs when calculating the intervention effect. Moreover, the three variables (knowledge concerning mental health, stigma, and help-seeking behavior) were assessed with different scales, and one-third of the studies employed their own unvalidated tools to measure help-seeking. Secondly, regarding publication bias, it should be noted that the statistical analysis could not be performed due to having fewer than 10 studies. We have to emphasize that only one study [28] belongs to an economically depressed region. Instead, we found descriptive studies in those locations like Vietnam, Sub-Saharan Africa, and Zambia [41][42][43], but not the evaluation of interventions. This can cause the results to be irreproducible in these regions. Thirdly, it should be noted as a limitation that studies prior to 2013 were not included. In this study, we were interested in assessing the studies carried out during the period of implementation of the WHO Comprehensive Mental Health Action Plan 2013-2020 [7] and the promotion of this plan in mental health prevention and promotion interventions such as MHL in the adolescent population. We have to add that our data search was mainly in health databases. This could have caused the loss of information from specialized education databases.
Our results should be interpreted with caution. In spite of the short-and mediumterm clinical benefits regarding the improvement in knowledge, we lack sufficient data to evaluate the efficacy of these interventions long-term (12 months follow-up). Further studies with more robust methodology, for instance, RCTs, with interventions that share the same MHL concept domains and employ the same measurement tools (particularly with respect to help-seeking) are warranted. More studies are needed that focus on the conditions in which the study is carried out, such as the content of the intervention, the person responsible for delivering the intervention, or the way of carrying out the intervention (use of videos, contact with first-person experience, talking, etc.). Subsequent studies should assess the following time frame until 2030 to study the evolution of the effectiveness of interventions based on MHL in the classroom after the end of the WHO plan in that year.

Conclusions
Mental health literacy interventions in a young school-age population improve mental health knowledge. Considering the limited data meta-analyzed, we cannot conclude that MHL interventions are effective on stigma and help-seeking outcomes. Even so, the results of individual studies on such outcomes are encouraging. There is considerable variability in the concept of mental health literacy and the instruments used to evaluate these interventions. Future research should go further on the effectiveness of MHL intervention considering stigma and help-seeking outcomes and, in addition, analyze other factors associated with the effectiveness of MHL interventions, such as the length of follow-ups and the type of professional in charge of carrying it out.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.
Acknowledgments: This research has been carried out within the framework of the Biomedical and Public Health Research Methodology doctoral program at the Barcelona Autonomous University (UAB). We would like to acknowledge the collaboration of Jorge Piqueras Marques from the Universitat Internacional de Catalunya (UIC).

Conflicts of Interest:
The authors declare no conflict of interest. Table A1. SIGN Grading System.

1++
High-quality meta-analyses, systematic reviews of RCTs, or RCTs with a very low risk of bias 1+ Well-conducted meta-analyses, systematic reviews, or RCTs with a low risk of bias 1− Meta-analyses, systematic reviews, or RCTs with a high risk of bias 2++ High-quality systematic reviews of case control or cohort or studies High-quality case control or cohort studies with a very low risk of confounding or bias and a high probability that the relationship is causal 2+ Well-conducted case control or cohort studies with a low risk of confounding or bias and a moderate probability that the relationship is causal 2− Case control or cohort studies with a high risk of confounding or bias and a significant risk that the relationship is not causal