Improving Early Detection of Refugee-Related Stress Symptoms: Evaluation of an Inter-Professional and Inter-Cultural Skills Training Course in Sweden

Abstract

Twenty-three of 26 participants, mainly women from six local agencies involved in the reception of refugees, completed a university course titled “Refugee-related stress and mental health—local cooperation”, which was spread over seven days in 2011. The course was based on evidence and clinical experience and was commissioned to serve as competency training by Stockholm County Council and Södertälje Municipality. It received funding from the Swedish National Board of Health and Welfare. It was a continuation of an earlier one-week full-time university course from 2010 with the same title. As a result of a new law relating to refugee reception, which led to organizational change, the participants requested a continuation of the original course. The learning objectives were met (5.4 on a 6-point scale; 1 = strongly disagree, 6 = strongly agree). The general assessment of the course as a whole by the participants was 5.7 (on a 6-point scale, 1 = very unsatisfied, 6 = very satisfied). The participants thought that their skills had increased, and their perception was that they had significantly better control of their work situation following completion of the course. The most important findings were that participants from different agencies at the local level: (1) perceived that they had developed the sense that there was a local inter-cultural and inter-professional inter-agency collaboration in the reception of newly arrived refugees and (2) will continue efforts to stabilize and develop this together. This method of teaching, in terms of skills training, is not a “quick fix.” It is a process, and it needs support from those in power in order to continue.

1. Introduction

One of the unfortunate consequences of war is the creation of refugees. Because many refugees are damaged by traumatic life events not only physically, but also spiritually and emotionally, host countries must make special efforts to deal with their needs. However, responsibility for the reception of refugees may be divided among agencies, which lack the tools or experience to collaborate with each other. This inability to collaborate properly could severely diminish the agencies’ ability to care for refugees. In order to offer the best care possible, it is therefore important that we find ways to improve inter-agency collaboration. The results of the following case study shed light on one way to develop effective collaboration.

In 2011, a university course titled “Refugee-related stress and mental health—local collaboration”, 10 ECTS (1.5 ECTS/week) was taught at Karolinska Institutet, Stockholm, the only medical university in Sweden. The course was based on evidence and clinical experience and was evaluated by the three authors. One of the aims of the course was for the participants to produce new policy documents for their respective organizations, regarding collaboration in the local reception of refugees. The aim of this article is to present the results of the evaluation of the training that took place in 2011. The main question is whether inter-professional and inter-cultural competence skills training can promote local collaboration in the reception of refugees.

The structure of the article is as follows. We start with an introduction of our theoretical framework, a discussion of refugees as a group at risk of mental illness and our pedagogic concept. The next section presents our methodology, including the context, analysis of data and ethical considerations. The following section covers the participants, as well as the quantitative and qualitative results. In the discussion section, we take up the results and methodology. The article concludes with a summary and the implications of our findings.

1.1. Theoretical Framework

In recent years, collaboration has become more important in the welfare system, due to increased specialization among staff and the increased professionalization of different groups within the workforce [1]. In this study, the concept of collaboration is defined according to two frameworks [2,3]. Danermark [2] divides collaboration into four levels. The first level is where people from different disciplines meet and discuss a question in order to exchange knowledge and ideas with each other. The second level consists of collaboration, taking into account different interventions in a structured way, in order to make the work more efficient and to produce better results. The third level starts with a specific problem, and through collaboration, new working practices are created. The fourth and final level occurs when two or more agencies develop one unit together. This level is the most significant level of collaboration. Danermark [2] also discusses the importance of empowerment among the different groups of professionals who participate in the process of learning about collaboration and notes that the basis for a successful collaboration is not the interpersonal chemistry between the participants, but the responsibility of their managers.

According to Fridolf [3], it is important for the agencies that come into contact with vulnerable groups, such as newly arrived refugees, to efficiently organize the empowerment of those vulnerable groups. In order to succeed, an inter-agency collaboration is required and the agencies must take the responsibility to collaborate. The challenges include legislative barriers, short-term and time-limited projects and financial constraints, which increase the odds that such collaboration will fail. Furthermore, support from politicians and the management of the agencies involved are necessary for such collaboration over the long-term.

1.2. Refugees: A Group at Risk of Mental Illness

The international literature indicates that there is a correlation between migration and mental health. Refugees are a group, which is vulnerable to the risk factors for both physical and mental disorders. They have been exposed to traumatic life events before their arrival and post-migration stress after their arrival in the reception country [4,5], as well as the social determinants of health, which must be considered in order to promote mental health [6]. Swedish epidemiological data shows that immigrants and, particularly, refugees have poorer mental health than native Swedes. The relative risk of a depressive disorder following unemployment is highest among immigrant women. Refugee men have a higher mortality risk for cardiovascular disease and external causes of death than do non-refugees [7].

1.3. Pedagogic Concept

The present university course was the continuation of an earlier course, which had the same title, but was only worth 7.5 ECTS points (1.5 ECTS/week) [4]. The inter-professional and inter-cultural course was performed to train staff involved in the reception of refugees in Södertälje Municipality and was commissioned by the Stockholm County Council and Södertälje Municipality [8]. The course was included as part of the second year of grant funding from the Swedish National Board of Health and Welfare. The law on introduction activities for certain newly arrived immigrants [9], introduced on 1 December, 2010, requires that the Swedish Employment Service take responsibility for coordinating the newcomers’ establishment in Sweden, a role previously undertaken by municipalities. Participants requested a continuation of the course in 2011 as a result of this change, as it had an effect on the interaction procedures for inter-agency collaboration regarding the reception of new arrivals. The learning outcomes after completing the course were that participants would be able to have:

1.3.1. Knowledge and Understanding

—Define key concepts and describe mental health and its social determinants among refugees in the local reception.

—Describe the theoretical background to case methodology in inter-agency collaboration.

—Describe the theory behind cognitive behavioral therapy (CBT) (clinicians) and evidence-based innovative methods (clinicians).

1.3.2. Skills and Abilities

—Plan and implement interventions, which is quality assured to promote mental health in the inter-agency collaboration of refugee reception.

—Apply knowledge of case methodology, theories and frameworks related to inter-agency collaboration.

—Develop the ability to identify and plan for inter-agency collaboration in the development of refugee reception that can promote mental health and compliance.

1.3.3. Ethics and Ability to Reflect

—Develop a flexible and problem solving approach in the inter-agency collaboration.

—Develop the ability to critically reflect on and evaluate their own knowledge and the need for increased knowledge to develop and deepen their skills.

—Develop the ability to contribute to the sharing of knowledge within the inter-agency collaboration by discussing their own and others’ roles in and around the mental health field.

The curriculum was completed and approved by Karolinska Institutet. The course program was divided into seven themes and took place on seven occasions (Table 1).

Table 1. The course covered seven themes on seven occasions in 2011.

Day	Date	Themes
1	April 7	1. Feedback on the basic course that took place in 2010 and the introduction about the current local agencies in asylum and refugee reception
2	May 12	2. Introduction and maintenance support in Södertälje Municipality
3	August 18	3. The primary care perspective—asylum and newly arrived refugees
4	September 15	4. Primary care—new arrivals
5	October 20	5. Specialist care (psychiatry)
6	November	6. Children: specialist medical care: child and adolescent psychiatry (BUP)
7	December 15	7. Evaluation. Workshop: the next steps in co-operation

Table 1. The course covered seven themes on seven occasions in 2011.

Day	Date	Themes
1	April 7	1. Feedback on the basic course that took place in 2010 and the introduction about the current local agencies in asylum and refugee reception
2	May 12	2. Introduction and maintenance support in Södertälje Municipality
3	August 18	3. The primary care perspective—asylum and newly arrived refugees
4	September 15	4. Primary care—new arrivals
5	October 20	5. Specialist care (psychiatry)
6	November	6. Children: specialist medical care: child and adolescent psychiatry (BUP)
7	December 15	7. Evaluation. Workshop: the next steps in co-operation

The course consisted of theoretical studies, lectures, reflection groups, case methodology and cognitive behavioral therapy (CBT) (for clinicians) under supervision.

Education in the workplace: in the municipality (maintenance of support), primary health care, specialist care in psychiatry, employment service under supervision.

Mandatory participation: participation in lectures, group activities and education in the workplace (every agency was responsible for half a day in the afternoon) over the course of a seven-day spread over the year (2011), homework and individual written feedback under supervision over a period of five and a half days and a half-day workshop on the last day of the course.

Clinical experience shows that, in terms of inter-professional learning, the most successful adult learning, being more mindful of the larger non-specialist audience, has as its basis the participants’ own experiences [10]. According to this concern, relevant issues in the various professions’ day-to-day work and teaching can be addressed.

The case method is used, mainly, for management training and training in how to make decisions in critical, usually ambiguous, situations. The method is action-oriented and fits well in terms of individual skills and the development of collaborative learning, where different participants’ knowledge, experience and positions are utilized [11]. Through homework assignments that were completed between the training days, the participants were given the opportunity to have valuable inter-professional discussions, reflection and evaluation at the end of each day of the course, which reinforced comprehension from a reflective pedagogical point of view [12].

In short, the learning process is based on the key ingredients of the course that were relevant to the participants, such as the perception of increased empowerment and participation in developing a local inter-agency collaboration for the reception of new arrivals, in terms of the agencies involved, inter-cultural communication, inter-professional competencies and shared values. The course included a regular and recurring theory during the morning of each training day and a group discussion in the afternoons. These discussions were about one or more cases, case methodology visits to the various local agencies, which had prepared anonymous cases for group discussion and which ended with ethical discussions. In between the training days, the participants had homework, which was completed individually, in the form of cases in which all relevant agencies were involved in the task. These were then presented to the whole group on the next training day.

2. Methods

The course was evaluated with so-called quality assurance in the following three ways:

(1) A questionnaire was developed in the first year that the project was commissioned [4]; and this was adapted and used on two measurement occasions, one before and one after the course. The questionnaire was distributed to the participants before the first day of the course and again on the final day. The questionnaire was completed voluntarily. Each participant received an identification number and a label, determined by the respective participants, in order to compare the two measurements, but without course management being able to identify the individual learner. Questions, which had closed response options (Likert scale 1–5; 1 = low, 5 = very high) covered the areas regarding how challenging the participants thought the course would be and how challenging it actually was. Another question concerned how demanding the course was expected to be and how demanding the course actually was. There was also a question about the demands placed on the participants and how in control they felt in their work situation.

In the follow-up questionnaire, the participants were asked how much knowledge they thought that they had gained in the subject. The evaluation of the first course indicated that the participants had difficulty remembering what they had written prior to the course; we, therefore, chose to examine these issues together at the end. The open qualitative questions covered the expectations participants had of the course and whether these were met after the course. Another question was about policy. Documents about the participants’ work and how they could see the new knowledge gained during the course were analyzed.

(2) The KI evaluation questionnaire was used, as recommended by Karolinska Institutet. The first eight questions in the evaluation questionnaire had closed answers in a Likert scale (1 = strongly disagree/very unsatisfied/very impossible to 6 = strongly agree/very satisfied/very possible).

(3) At the end of the morning of the last day of the course (day seven), a workshop took place. The participants were divided into four small groups (about 5–6 participants/group), with each group containing a mixture of participants from the different local agencies. The project had received further funding from the Swedish National Board of Health and Welfare for 2012, and the topics discussed included proposals for solutions in 2012. Themes and contact strategies for each agency, so as to maintain the local inter-agency collaboration, were reported.

2.1. The Context

The evaluation uses the refugee reception program in Södertälje Municipality [13], not far from, Stockholm, as a case study. Södertälje Municipality and Stockholm County Council commissioned the present competency training, worth 10.0 ECTS (1.5 ECTS/week full-time course), from Karolinska Institutet (the first author was in charge of the course) as part of a two-year project titled “Health promoting strategies in the reception of refugees with mental disorders and disabilities—an inter-agency collaboration.” The participants who encountered the target group were invited and participated voluntarily. The training took place over seven full days spread over the year 2011, with homework in between. The participants had an obligatory list of references to read and homework during the course. A diploma was awarded if the participant attended the seven full days and had done the homework between the course days.

2.2. Analysis of Data

The evaluation was primarily a qualitative/explorative study. Quantitative data was descriptively analyzed with SPSS version 19.0. The answers to the open questions were analyzed based on themes in a simplified qualitative content analysis [14]. The anonymous quotes from the participants provide an exploration of the quantitative results.

2.3. Ethical Considerations

Ethical considerations are important when conducting interviews. This study was not subject to the Swedish ethical law, as it was an evaluation study. However, we conducted the study so as to protect the integrity of the research in the following four areas: (1) demands for information, (2) confirmation, (3) confidentiality and (4) applicability. The study was consistent with these as all participants received information (verbal and written) about the aim of the evaluation, participation was voluntary, participants could withdraw from answering the questionnaires in the evaluation before and after the course without having to provide a reason and the material would be used anonymously [15].

3. Results

3.1. Participants

Twenty-three of the 26 participants who signed up completed the course and provided their signed consent. At the beginning of the course, three participants reported that they would be unable to complete the homework, due to a lack of time. Eleven of the 23 participants reported that they had, during the previous year, passed the basic course worth 7.5 ECTS (1.5 ECTS/week). The majority of the participants was women and was working at one of six local agencies involved in the reception of refugees in the municipality (Table 2). After finishing the course, 19 of the 23 participants (83%) who participated on the final day completed the evaluation questionnaire and received the course certificate.

Table 2. Participants from each of the agencies who completed the course.

Participants’ workplace	Profession	Total Number	Total Men	Total women
Municipality (introduction, maintenance of support)	Social worker (counsellor, social secretary)	7	-	7
Swedish for immigrants (adult education teachers)	Teachers and guidance counsellors	4	1	3
Employment Service	Employment service advisors	3	1	2
Primary care/Stockholm County Council	Registered nurse, medical doctor, counsellor	3	1	2
Specialist care/SLL adult psychiatry	Psychiatrists, psychologist, nursing assistant	4	1	3
Other: NGO	Consultant behaviorist, public health worker	2	-	2
Total participants		23	4	19

Table 2. Participants from each of the agencies who completed the course.

Participants’ workplace	Profession	Total Number	Total Men	Total women
Municipality (introduction, maintenance of support)	Social worker (counsellor, social secretary)	7	-	7
Swedish for immigrants (adult education teachers)	Teachers and guidance counsellors	4	1	3
Employment Service	Employment service advisors	3	1	2
Primary care/Stockholm County Council	Registered nurse, medical doctor, counsellor	3	1	2
Specialist care/SLL adult psychiatry	Psychiatrists, psychologist, nursing assistant	4	1	3
Other: NGO	Consultant behaviorist, public health worker	2	-	2
Total participants		23	4	19

3.2. Quantitative Results

Table 3 provides a summary of the mean values of the closed responses, before and after the course, in terms of perceived challenges, competence skills, control and demands. Participants felt that their competence skills had increased slightly, and they felt that they had significantly more control in their work situation after the course (p < 0.007).

Table 3. Mean values for perceived challenge, competence, control and demands, before and after the course (1–5, 1 = little, 5 = very much).

Areas	Mean (SD) before the course (N = 19)	Mean (SD) after the course (N = 19)	p-value, z-value
Challenge	3.58 (1.121)	3.58 (1.170)	p < 1.000, z = 0.000
Competence skills	4.06 (0.873)	4.17 (0.786)	p < 0.782, z = −0.277
Control *	3.63 (0.761)	4.11 (0.737)	p < 0.007, z = −2.714
Demands	4.42 (0.838)	4.47 (0.838)	p < 0.655, z = −0.447

Table 3. Mean values for perceived challenge, competence, control and demands, before and after the course (1–5, 1 = little, 5 = very much).

Areas	Mean (SD) before the course (N = 19)	Mean (SD) after the course (N = 19)	p-value, z-value
Challenge	3.58 (1.121)	3.58 (1.170)	p < 1.000, z = 0.000
Competence skills	4.06 (0.873)	4.17 (0.786)	p < 0.782, z = −0.277
Control *	3.63 (0.761)	4.11 (0.737)	p < 0.007, z = −2.714
Demands	4.42 (0.838)	4.47 (0.838)	p < 0.655, z = −0.447

The learning outcomes were addressed in the answers to Karolinska Institutet’s evaluation questionnaire. The participants’ responses evaluated the course very highly on its content, aim and general assessment (Table 4).

Table 4. Responses to Karolinska Institute’s evaluation questionnaire after the course.

Content	Average (range)	N (%)
CONTENT AND AIM
1. The level of the course correlates with the description of the course (1 = strongly disagree; 6 = strongly agree)	5,6 (4–6)	17 (74%)
2. The aims of the course were fulfilled (1 = strongly disagree; 6 = strongly agree)	5,4 (4–6)	17 (74%)
3. The level of satisfaction with the format of the course (1 = very unsatisfied; 6 = very satisfied)	5,7 (4–6)	18 (78%)
GENERAL ASSESSMENT
4. The teachers facilitated the achievements of the learning outcomes in an adequate way (1 = strongly disagree; 6 = strongly agree)	5,8 (5–6)	18 (78%)
5. I would recommend this course to others enrolled in similar profession (1 = strongly disagree; 6 = strongly agree)	5,6 (3–6)	18 (78%)
6. The level of satisfaction with the practical arrangements (1= very unsatisfied; 6 = very satisfied)	5,8 (4–6)	18 (78%)
7. The general level of satisfaction with the course (1= very unsatisfied; 6 = very satisfied)	5,7 (4–6)	15 (65%)
8. The level of possibility for application of the new knowledge at my office (1= very impossible; 6= very possible)	5,4 (2–6)	18 (78%)

Table 4. Responses to Karolinska Institute’s evaluation questionnaire after the course.

Content	Average (range)	N (%)
CONTENT AND AIM
1. The level of the course correlates with the description of the course (1 = strongly disagree; 6 = strongly agree)	5,6 (4–6)	17 (74%)
2. The aims of the course were fulfilled (1 = strongly disagree; 6 = strongly agree)	5,4 (4–6)	17 (74%)
3. The level of satisfaction with the format of the course (1 = very unsatisfied; 6 = very satisfied)	5,7 (4–6)	18 (78%)
GENERAL ASSESSMENT
4. The teachers facilitated the achievements of the learning outcomes in an adequate way (1 = strongly disagree; 6 = strongly agree)	5,8 (5–6)	18 (78%)
5. I would recommend this course to others enrolled in similar profession (1 = strongly disagree; 6 = strongly agree)	5,6 (3–6)	18 (78%)
6. The level of satisfaction with the practical arrangements (1= very unsatisfied; 6 = very satisfied)	5,8 (4–6)	18 (78%)
7. The general level of satisfaction with the course (1= very unsatisfied; 6 = very satisfied)	5,7 (4–6)	15 (65%)
8. The level of possibility for application of the new knowledge at my office (1= very impossible; 6= very possible)	5,4 (2–6)	18 (78%)

3.3. Qualitative Results

3.3.1. Expectations were Met

Before the course, the majority responded that they expected to learn more about the target group, to learn how to work more professionally in their encounters with refugees and to learn about how to improve collaboration between different local agencies in the municipality.

After the course, all responded that their expectations of the course had been met, and some wrote that the course was better than they had expected. In particular, participants highlighted the importance of exchanging information with colleagues from other agencies involved in the local reception of refugees. They usually met the same client, but had different responsibilities based on structural organization.

—Yes, I have learned a lot by exchanging information about the activities of the other participants.

3.3.2. Homework after Work

Before the course, the majority described their work situation as tough and demanding and many replied that they had been informed by their employers that they were allowed to participate in the course during working hours. However, their homework had to be done after work, and some commented that it was only possible to arrange this if they were able to plan the time they spent on the course during working hours.

Many of the participants had a high workload, even after the course, but most thought that they had managed to find time for their competence training and learning, even if it had to be done outside of working hours. This was because the content of the course was rewarding, and they had met colleagues in similar situations and so they were able to support each other.

—Sometimes, time has been a bit scarce…extra work from the boss, but the time spent on the course has been very rewarding and worthwhile.

—O.K., as the homework was sometimes done as a group, it became easier taking the time to do it as a result of this co-operation.

3.3.3. The Content of the Course was Relevant

After the course, almost all participants had gained relevant knowledge.

—Absolutely, the health perspective has become a natural part of my work with job-seekers (newly arrived refugees).

—Yes, it did. I've learned a lot about how to analyze difficult cases. I have learned more about the different agencies day-to-day work.

All participants responded that they brought with them different skills that they can use in their encounters with the target group. The course not only presented new knowledge and information about other agencies’ responsibilities and policies, but it also allowed the participants to become more aware of their own background and to have more realistic expectations of and respect for the target group’s background and needs, but also their strengths.

—Listen, support them as if they are strong people and not as if they are sick. The main thing that I have learned is which of the various agencies is the most relevant to contact to suit the requirements of the newcomer.

—In my everyday life, I have re-learned what to think about when meeting with newly arrived individuals, to consider the difficulties they encounter and the many agencies they may come in contact with.

In Karolinska Institutet’s evaluation survey, including open questions, the participants' responses indicated the core strengths of the competence training. These core strengths were that the course had developed their understanding of how other agencies work, the networking opportunity the course provided, “face the people”, theory, case methodology and knowledge sharing, homework, the course venue and the course’s broad approach. The weaknesses of the course, which was also confirmed during the final day’s workshop, were that certain topics were not included, such as the child perspective, that other key agencies should not only be helping to teach the course, but should also be involved as participants (e.g., the Swedish Migration Board, the Swedish Social Insurance Agency) and that staff in management positions must also be involved, as well as a higher number of the local primary health care clinics. These weaknesses can be resolved in the future. The participants felt that the training should be made permanent and be mandatory for all professionals working with the target group in the municipality.

3.3.4. Continued Collective Development of the Local Platform Model

All of the participants' answers indicated that they wanted to work actively in various ways to keep the local inter-agency collaboration model, developed during the two courses, intact. They were certain that they would continue to work together to further develop this.

—I shared knowledge with my colleagues and will maintain contacts with the other participants. Together, we have a collaborative model, a collaboration model, which works.

—Three agencies (at least two agencies and the newcomer) in the early stages. Continuing the collaboration meetings and also invite other agencies.

4. Discussion

4.1. Discussion of the Results

This university course received a positive evaluation from the quantitative and qualitative results of the participants. According to Danermark [2], the aim of the first level of collaboration is that different professionals meet to exchange knowledge and to gain an understanding of each other’s competencies and roles. The results from the evaluation show that this level was attained, as the participants had the opportunity to communicate with staff from different agencies.

Collaborations on levels two and three deals with efforts to collaborate in different activities in a structured way and to collaborate in dealing with a case [2]. These two levels of collaboration had also been attained, according to the answers from the participants. According to Fridolf [3], collaboration creates better prerequisites for individual staff who are in contact with several agencies, as collaboration strengthens their empowerment. The knowledge and the tools (i.e., skills), which the participants gained during the course, will help them support newly arrived refugees, and the participants will develop a similar local frame in the collaboration. During this course, the participants perceived the changes in detection and intervention, but the aim was not to assess this in a more concrete way. It was not relevant, due to the aim during the year to build up sustainability indicators within the local agencies. No information regarding how some participants problem-solved getting their bosses more invested or supportive of the work was collected. Besides these limitations, the collaboration in the reception of newly arrived refugees will be improved, as the skills training will be evidence based.

However, the evaluation also indicates that the participants do not receive support from their managers, with regard to the local collaboration. There is a need for ongoing activity in order to maintain the local inter-agency collaboration, and the participants were worried about how this would happen. As noted by Danermark [2] and Fridolf [3], in order to maintain a collaboration over the long-term, management must provide support to their staff. Furthermore, according to the same authors, there is a limitation in the case of a project, which is time-limited for collaboration, but which needs continuity. Both international and national data show that refugees are a group, which is more at risk of severe mental illness, compared with other groups of immigrants and the majority of the population [5,7,16]. There is a risk that newly arrived refugees are unable to meet the demands of the reception to be able to work, but they do not have access to active rehabilitation, which is connected to the social insurance scheme, because they have not worked in Sweden previously. Instead, they will be referred to the income support service of the municipality. An interview study examined how newly arrived refugees, due to limitation of their work capacities (at least 25 percent of full time to stay in the reception program for two years) were transferred from the Swedish Employment Service to the municipal social welfare, perceived their health, care situation and employment opportunities. The qualitative content analysis of the interviews reflected poor living conditions, physical and mental illness and alienation [17].

A State Treasury report [18] on the agencies’ implementation of the reform introduced on 1 December, 2010 [9], shows, among other things, that “…the activities are not sufficiently adapted to the target group’s needs and requirements” (page 8). Therefore, it is high time that the knowledge, skills and attitudes to inter-cultural communication and inter-professional training among staff in the health sector, especially mental health, starts to be passed on to the Employment Service, so that they can adapt their activities to newly arrived refugees’ conditions and requirements and prevent them from becoming marginalized.

4.2. Discussion of the Method

In this evaluation, a mixed method was used in the form of questionnaires. As the aim of this evaluation was to discover information about the participants’ experiences, thoughts and attitudes towards the university course, qualitative methods are especially relevant [14] and have, therefore, been the focus. We also wanted to use quotes from the open-ended answers in order to explore the quantitative answers, i.e., to increase the trustworthiness of the results. When using quantitative data, it is important to study the validity and reliability of the results. In qualitative methods, the results are evaluated according to credibility, reliability and transformability. In order to increase the credibility we presented the results from the first seminar to the participants and to the steering group, which was founded after the course due to the project being approved for a third year of project funding from the Swedish National Board of Health and Welfare. To further increase the validity of the results, we used a relevant theoretical framework to analyze the results. Regarding transformability of the results, there is too little material to be used for generalization. However, as the local agencies involved in the study encompassed several disciplines, there is some generalization. In the quantitative part of the evaluation, the validity appears to be high, as the questions in the questionnaire provided information about the issue we evaluated. Further reflections are reported in Ekblad and Forsström [4].

4.3. Implications

The results of this evaluation indicate the significance of collaboration in the field of refugee reception and that collaborative learning in the field of inter-cultural and inter-professional communication is of importance in order to create a local inter-agency collaboration, which increases the empowerment of a vulnerable group. This pedagogic concept may be fruitful for training courses targeting other disciplines and subjects, and further research in this area is required. The Interactive Systems Framework of Wandersman et al. [19] could be used to further guide finding implications, as well as next steps in creating a more sustainable collaboration for the work.

5. Conclusions

The results of the evaluation of the present course indicate that it is possible to create an interdisciplinary local inter-agency collaboration in the reception of newly arrived refugees with mental illness in a municipality and that the content of the course was relevant to the participants. However, due to it being project-based, the continuation of the course requires the support of those who have responsibility for the finances (e.g., of the reception of refugees), which often constitutes a barrier, according to Danermark [2] and Fridolf [3]. Collaboration requires both resources and structures in order to be maintained, and there is a need for an external coordinator who can facilitate and conduct regular evaluation.