In 2017, 68.5 million of the world’s people were considered displaced from their homes due to violent conflict, setting a new record for the fifth straight year. Only a very small percentage will be resettled in another country (United Nations High Commissioner for Refugees (UNHCR 2018a
)). The number accepted to the U.S. has markedly decreased, but the global refugee crisis and successful integration of new arrivals are nonetheless of great interest to nationwide stakeholders.
Since 2002, the city of Oshkosh, Wisconsin, has welcomed the second-largest number of refugees in the state, behind Milwaukee and ahead of Madison. In 2015, at the initial meeting of the University of Wisconsin Oshkosh (UWO) Department of Sociology’s community advisory board, the facilitator of a local resettlement task force suggested it conduct applied research to help service providers move beyond anecdotes to a data-driven understanding of regional resettlement (e.g., what works, what does not, what gaps in service remain, what stories from resettled people need to be told).
The study began as a semester-long project for undergraduate students in the new Applied Sociology course at UWO in spring 2016. The idea was to have students learn sociology through community-engaged research (CER). Launching the resettlement study in partnership with the task force was a strong fit. Recruiting residents with refugee background (RRBs) as subjects proved difficult and the project challenging for students. However, by the end of the semester, initial data had been gathered and relationships developed between the university project team, task force, and area RRBs.
The university project team subsequently received a grant to continue research and expand it into a comparative study of resettlement in the region around Oshkosh (the Fox Valley), as well as Milwaukee and Madison. This made for an exciting boost to the study. It also added layers of complexity through broader geography and larger team size.
In this article, we share experiences, reflections, and lessons learned from CER implementation within the realm of refugee resettlement. Our focus is the process of this CER effort. To provide relevant context, we present a brief background on the dynamics of refugee resettlement in the U.S. and Wisconsin, as well as literature and theory on CER in sociology and on this topic.
Results and discussion center on our experiences carrying it out, which are analyzed via a framework drawn from the literature and grounded in our experiential analysis of the applied project, along with implications for similar efforts. To enhance understanding of the community-engaged elements of this project, some project material is also incorporated. One of the paper’s primary contributions is to share experiences—both successes and failures—in a transparent and unvarnished fashion. We hope it will enhance understanding of CER on marginalized populations and lead to more impactful initiatives in the future.
3. Methods and Material
We write this article as key members of the CER team for this project. This includes the project leader, who began the effort with his students in an undergraduate sociology class, colleagues who helped design the methods, several key researchers from multiple study sites—including a student who contributed to the project during the class and after it concluded—and a photographer who collaborated with the team on the public exhibit.
The origin of this project was clearly in the research-teaching-service nexus. It began with a request for data. A strategic plan for the UW Oshkosh Department of Sociology that stemmed from an external review of the program resulted in, among other things, the creation of a new course in Applied Sociology and a community advisory board. The latter gathers representatives of area nonprofits, government agencies, and other stakeholders in an informal body that would meet once or more annually to learn more about the department’s activities and students, share information, and discuss possible collaborations.
One goal was to learn about ways to provide further service to local communities through CER, using a participatory, inclusive inquiry process rooted in the needs and activities of local stakeholders. The specific goal was to identify a project for the Applied Sociology course through which students could do applied research (research done at the request of an outside entity, as opposed to originating from the interests of the researcher). We had hoped to demonstrate the value of sociology while building social capital for the lasting benefit of the program, its partners, and students.
At the first meeting, we asked the roughly 25 attendees to consider possible research needs. One local nonprofit leader, who had been facilitating the work of the resettlement task force for several years, suggested we work with that coalition to produce data about refugee resettlement in the area (e.g., what is and is not working, what gaps in services exist, what RRB stories need to be more widely told) and a better understanding of RRBs themselves. This would be done by gathering socioeconomic information as well as stories about their lives. At this point, neither the professor primarily responsible for this effort, his department, or the university in general had any significant engagement with the task force or RRBs of the area. The project was viewed as an opportunity to begin building stronger relationships.
The research team, based at UWO, agreed that the project was a good fit and moved forward with it, eventually adding personnel in each case city. We collected surveys from RRBs to gather background information, quantitative data about their livelihoods, health, well-being, and open-ended responses about their experiences. RRBs were recruited through appeals via local media, referrals from service providers, or meetings/events attended by research team members. Snowball sampling then occurred in some cases.
To delve more deeply into experiences and stories of resettlement, we also conducted in-depth interviews with RRBs, utilizing translators when appropriate. For a number of RRBs, we used participant-driven photo elicitation (PDPE). Interested participants were asked to take, or instruct researchers to take, photos that illustrate specific ideas from the RRB perspective:
Who they are
Where they are from
How they live now
The most positive thing about living where they now live
The most difficult thing about living where they now live
Their hope for the future.
Beyond asking RRBs to explain the resulting photos, we asked additional questions, following an interview guide. For RRBs who chose not to involve photos, we simply followed the interview guide. As a token of our appreciation, RRBs were given a $10 grocery store gift card for their participation in each step of the process (i.e., one for completing the survey and another for the interview).
We also interviewed people who provide services to RRBs via their work with RRAs, other nonprofit or government organizations, or because they simply want to help. Most service providers were recruited through networking and direct appeals made at local resettlement task force meetings and public events involving RRBs. The interview guide for service providers included questions such as:
What do you think it means to be successfully resettled in a new place?
What do you feel are the most important contributions that you/your organization provide for successful resettlement?
What do you feel are the most difficult barriers to providing services for successful resettlement?
4. Results and Discussion
Results are presented in terms of process. In this section, we discuss our experiences with the study of refugee resettlement as compared to CER desired process outcomes. Each of the six elements derives from arguments presented in Section 2.2
, and is elaborated upon further in this section. Based upon this literature, in comparison to traditional research, CER has the potential to be more:
The combination of these desired process outcomes creates a framework to aid in project analysis from a CER perspective. We present our experiences as related to the six elements of this framework below.
Participation in CER hinges upon who is involved and how. Historically, according to Stoecker
), participatory “research emphasized grassroots participation and critical analysis” (p. 387). Generally, CER is concerned with local residents and local organizations being involved, with the latter broken down into community-based organizations (CBO, defined as being controlled by affected local residents, i.e., constituents) and nonprofit organizations (NPO, not constituent-controlled). Stoecker
) found that local “residents/constituents are rarely involved in helping to define the research question or design the methods” (p. 393), suggesting a lack of willingness to share power and allow subjects to have control over research about them. Meaningful participation by an inclusive CBO at every step of the CER process is recommended.
Both RRBs and service providers should be considered constituents of our project. Their involvement in each step is detailed below. While we collaborated with constituents to varying degrees through the process, our core research personnel included three academics who are immigrants to the U.S. and multiple other members of racial or ethnic minorities, including one with extensive experience working with RRA and a practicing Muslim, another with fluency in Arabic from time spent living in the Middle East, and finally, a collaborator on the dissemination phase who recently returned to Wisconsin after a decade living and working in East Africa, where, among other things, he documented refugee crises.
4.1.1. Choosing a Question
According to Stoecker
), NPOs are not necessarily CBOs, and although CBOs are led by residents, they do not always adequately represent constituents affected by their work. A grassroots coalition of multiple organizations and individuals providing services to RRBs, Oshkosh Resettlement Task Force (RTF) met monthly to share information and resources, and occasionally collaborate on events, all with the goal of serving RRBs more effectively. Though the local United Way facilitated its meetings and often hosted them, RTF had no staff or official leadership, and as such did not consider itself an organization. This lack of capacity is one reason we were approached to do the work.
While RRBs—generally people who have lived here long enough adapt to life in the area, are relatively well-educated, with strong English skills—are involved in the RTF to an extent, they represent a small, but important minority in a group of 60+ people from CBOs, NPOs, churches, state agencies, and academic institutions. Some of these are likely CBOs, but organizations key to RTF, such as the local United Way, World Relief, a local literacy council, a local community action agency, and the branch of a statewide workforce development group would be considered NPOs based on their board membership. While meetings in Oshkosh sometimes attracted 20+ attendees, and the twice-annual joint meetings with an RTF based in the Appleton area generally attracted 50+, more typical attendance at an Oshkosh RTF meeting was 5–10, perhaps including 1–2 RRBs.
In this case, research questions were developed by the RTF. They included, How did RRBs end up here? What are their stories? What factors help predict their success? What gaps in resettlement services exist? While our data will allow us to pursue scholarly angles, our research questions were essentially RTF’s questions. Participation at this stage by local stakeholders in the Fox Valley was clearly meaningful, though people from other research sites were not included, as they were added to the project after it had been underway for a year. Further, the RTF does not clearly fall into the category of CBO or NPO, though it was largely community-based it is likely closer to the latter.
4.1.2. Designing Methods
Consistent with CER tenets, our survey was based on themes and questions developed by RTF members, who were clear from the start that they wanted a survey, hoping to move past their own anecdotes to reliable data on resettlement. Collaboration around research design involved our team members attending RTF meetings to discuss the concept behind the project, possible methods, potential questions, parameters around who should be included, and how best to recruit them.
Inherent in participation is inclusivity. In collaboration with RTF we decided upon RRBs as our primary constituency, allowing potential participants to decide whether they belonged in that category. That meant adult children of refugees and other immigrants who felt their experiences fit the definition of a refugee could participate.
Once we agreed to do a survey, the RTF created a subcommittee to develop topics and questions, which we later converted into suitable draft survey questions. We then facilitated a methods workshop at UWO as part of a monthly, two-hour RTF meeting. It included a report by the director of the Wisconsin state refugee office, so there was much to discuss, but the workshop proved fruitful.
Attendees were presented with the draft questions and asked to create theme-based groups related to their field (e.g., education, health) to help refine them. We used results to create a 56-question survey divided into seven thematic sections: Basic Demographic and Background Questions; Migration to and Resettlement in the study area; Services; Economic Factors; Health; About Children (if applicable); and Experiences Related to Moving and Resettling. For example, survey respondents were asked for their level of agreement with statements such as “I felt welcomed upon my arrival here” or “My own culture is accepted and understood here”, how their standard of living compares to their pre-resettlement situation, and what services have been most beneficial.
Our primary survey collection method was in-person, but we also created an online version, which was used not only for completion by participants with strong English skills, but also for storage of data from paper surveys. In addition, we agreed to conducted qualitative, in-depth interviews with an optional PDPE component based on a precursor project (Van Auken et al. 2016
). Our design also included interviews with service providers, including informal ones, about their experiences with, and perspectives on, resettlement. One key direct service provider we interviewed expressed appreciation for having been involved in survey design because it reflected information they “felt was important to disseminate from the research.”
4.1.3. Collecting Data
Resettlement service providers were not only research collaborators, but also interviewees. This dual role caused a bit of confusion on the part of some, who required additional follow-up and confirmation that we were not only interested in learning about potential RRB participants through them, but also in their roles as service providers. In the end, we were successful, interviewing 35+ people representing formal and informal service providers.
We were most concerned about RRB participation. Though we did recruit a number of RRBs through direct engagement at events, our own networks, and social media, we also relied on service providers to help make connections. Aside from a handful of informal service providers who were involved with the research team on a limited basis, service providers did not help collect the data from RRBs directly. This would have blurred the lines, raising ethical concerns about coercion while also raising questions about validity (due to heightened social desirability bias).
We had strong participation during the development of the project, but few service providers contributed to data collection in terms of helping us recruit RRB participants. Perhaps we did not make it clear enough that we sought their partnership. The key service provider quoted above highlighted complications when agencies were asked to assist recruiting RRBs for the survey, arguing that in “some cases, this could compromise the refugee/service provider relationship.”
Recruitment proved complicated for myriad reasons. According to a facilitator of the RTF, “Coordinating times during the day was difficult as many refugees are working; finding a neutral place to conduct the interviews could be challenging (respondents’ home or the library, etc.); the level of English the participant possessed,” and other barriers. Beyond logistics, a common theme within recruitment had much to do with trust development, wherein service providers could have made the connection between RRB participants and research team members. This service provider anticipated local agencies would be more forthcoming with referrals to overcome these issues and later expressed disappointment that they were not.
Ownership of the project by service providers during the collection phase was limited and marked by instances of friction, though there were examples of productive collaboration with a particular RRA and others. We counted on identifying shared goals to achieve buy-in. For example, an informal service provider in Milwaukee suggested we connect with a leader of a smaller-scale, service-providing organization who spoke a key language. Communication via email died off, but was revived when the researcher (a male outsider) attended a women’s event led by these two individuals. It was largely attended by Muslim RRBs from Syria and Burma, who networked with one another and had activities for their children. While the researcher was there, he was able to simply be seen. He had already become a familiar face at other events, and a handful of women walked by, offered a greeting, and asked through others who he was. This is when he was introduced to the key contact. During introductions his aim was to identify overlapping goals. In the end, an appointment was made to interview her the next day, along with another, female researcher.
Overall, the lack of participation in this stage proved challenging. Data collection included a small number of research team members who were themselves RRBs, so there was a limited amount of direct participation by our constituents. The bulk of RRB participation in data collection was as research subjects. Our goal was to incorporate a diverse (e.g., country of origin, length of time in the United States, current city of residence, and gender) cross-section of RRBs, though it was a struggle to recruit participants, causing us to fall short of our targeted numbers. Ultimately, we made contact with 155+ people who expressed interest in participating, and surveyed and/or interviewed 100+ RRBs.
RRBs were born in 20 different countries, including the United States. Congolese, Hmong, South Sudanese, Syrian, and Burmese participants were most common. There were slightly more males than females in the sample, which was relatively young, with a median age in the mid-30s. For some, the journey from displacement to their new residence in Wisconsin took as little as two years, while for others it was much longer—10, 15, or 20+ years—with much of it spent in a refugee camp. Most participants were from the original research location in the Fox Valley, with a substantial number from Milwaukee, and fewer from Madison.
RRB engagement was enhanced through photo elicitation. PDPE—pairing photos taken by members of the population of interest (and/or by researchers as directed by RRBs in this case) with semi-structured, in-depth interviews about those photos—can be an effective CER tool. PDPE encourages active participation and spurs thinking about the topics of interest prior to the interview. It produces tangible, multi-dimensional stimuli for rich conversation (Van Auken et al. 2010
). Photos also help illustrate key points and bring stories to life. This project yielded numerous RRB-derived photos (see Figure 1
and Figure 2
), as well as others taken by researchers to provide additional visual context.
While we were able to incorporate a solid number of RRBs into the CER process, we had contact with a number of others who did not participate. Language barriers were difficult to overcome in some cases. We also experienced challenges related to gender (where researcher/participant gender identity needed to match, but this was not always communicated in advance), the multi-faceted nature of our process, and the fact that RRBs, and particularly recent arrivals, have complex lives that do not always allow for easy scheduling. A common thread was barriers stemming from lack of social power in various forms. Participation in this stage was mixed, but the process was dynamic and we were able to gather useful information.
4.1.4. Analyzing Data
At the time of this writing, data collection is complete but analysis is ongoing. In this paper, we focus on reflections about the process, with substantive data analysis to be presented in subsequent papers. Notably, because this project began as an applied project where we planned to deliver a product to our “clients” (RTF), we had planned to conduct the analysis “in-house,” with stakeholders not being involved again until the final stage. But, because this is an iterative process, and we have learned a great deal about CER, we are discussing ways in which participation can be enhanced in this stage: validity testing of qualitative data with participants and discussion of survey results at gatherings comprised of both RRBs and service providers. These efforts could illuminate why certain findings emerged and help build social capital.
The majority of our reporting and action will occur in the near future. We will consider how to incorporate participation into the dissemination of our data and other action. Because Action-Oriented is an element of the CER desired process outcomes framework, however, it is expounded on below.
In addition to being more participatory, CER should be more respectful to stakeholders than traditional methods (Stoecker 2009
). Respect in this context is based largely on intercultural knowledge and competence, and levels of power. RRBs are a vulnerable population, often with limited power. All of our methods were closely scrutinized and approved by the Institutional Review Board (IRB), which evaluates all proposed research on human subjects at UWO for design soundness and particularly for ethical practices and respect for participants. Following negotiation with the IRB and discussion with RTF collaborators, we agreed to take steps beyond standard informed consent to protect RRBs, facilitate respectful treatment, and provide resources.
We decided to only incorporate RRB participants who had been in the United States at least eight months, to ensure basic resettlement had already taken place and reduce the risk of RRBs feeling pressure to participate. We also created extra steps for screening potential participants, beyond the informed consent process. Prior to data collection, we asked RRBs to complete a one-page pre-participation form asking for demographic information and confirmation of RRB status, US residency of at least eight months, and that they understood we would like to meet to ask questions (some of which would be sensitive) to better understand them. It gave RRBs an opportunity to list preferences for meeting location, gender of the researcher, use of a translator, and whether/how they wished to complete a survey and/or interview. We also agreed to work with a local RRA to screen for individuals with known mental health concerns, and to distribute local resource guides. On the survey, potentially sensitive questions did not go into explicit detail, especially when pertaining to their migration history and any related trauma, giving participants power over their story.
We provided RRBs with grocery store gift cards as a token of appreciation for their time and energy. Further, in our informed consent process we stressed that underlying goals for this project were to facilitate the telling of their stories and produce information that would improve resettlement experiences for other RRBs. Many participants clearly appreciated this aspect, due to the desire to be helpful stated by many, but also perhaps because it suggested they had some power to influence outcomes important to them.
Most RRB participants lived with significant social power differentials compared to service providers, researchers, and even student assistants, all well-socialized into this culture with relatively high levels of education. Most team members also had relatively high standards of living compared to RRBs, especially those who were resettled recently, which describes the majority in our study. The vast majority were also people of color and though most reported feeling welcomed and few incidents of direct racial animus were discussed, in a society where race and ethnicity continue to directly influence life chances, this was another key element of overarching power dynamics. Such differentials were difficult to overcome, but when we were successful in getting RRBs to the table, it seems we were also successful in showing proper respect and establishing rapport, aided by targeted training in cultural competence and research methods for all team members.
Power in a particular society also hinges in no small part on the mastery of the dominant language, essential to many aspects of participation. To accommodate participants with limited English comprehension, material was translated into several common languages. The research team included a native Swahili speaker and people fluent in Arabic and Hebrew. Others served as cultural bridge builders, helping open doors and make connections as people known to RRBs, which was designed to show respect and build rapport. RRBs with strong English skills had the option to complete a paper or online survey on their own time. But, in most cases, they conducted the surveys in person with research team members and the help of translators as needed. Completing the surveys in person was helpful in overcoming the language barrier, as questions could be repeated and clarified. It also allowed for rapport development between researcher and participant, and produced a richer learning experience for all involved. It was time-consuming, however. In the field, a survey could take over an hour to complete, when it took our students 17–20 min during trial runs.
Language barriers became a stumbling block to RRB recruitment, as well as successful completion of surveys and interviews in some cases. Adding translation of documents and the need to coordinate the schedules for not only the RRB and researchers, but also translators/bridge builders proved daunting. Our community-engaged efforts to design culturally responsive instruments were generally successful, but some RRBs found the survey too long and the language cumbersome, as even highly educated RRBs did not understand certain questions the first time through. Some had difficulty grasping the PDPE concept, but on the other hand, for those who did, flipping the dynamic to some degree by allowing RRBs to take the lead on the photo choices—assuming the role of teacher of their story while the researcher became the student—was clearly powerful for some.
Other lessons learned include that one cannot spend too much time carefully crafting and testing survey questions and other design elements for marginalized populations such as RRBs. Students in the initial Applied Sociology course pre-tested the survey to give us a general sense for what issues we might have with wording, proper ordering of questions, and how long it might take to complete. Given the compressed nature of the process—again, initially this was an applied project to be completed during a semester—and difficulty we experienced in recruiting RRBs, we elected to have students test the survey, which was also effective training for them (i.e., survey implementation and learning details and nuances of the instrument). Best practices would clearly suggest that pre-testing should have been conducted with RRBs themselves, however.
Further, while we created a project handbook and did customized training with all researchers, we should have provided more guidance and practice (e.g., clarifying the meaning of questions or probing). One successful adjustment made in the process towards the end of the data collection period was for cases where the English ability of RRBs was sufficient (according to them, which we attempted to verify in pre-meeting interactions) to take the survey on their own, was to start the research meeting with the researcher going through the completed survey with the RRB, while doing an audio recording. This was a positive way to build rapport by simply reiterating some key points about their backgrounds, as well as an effective and respectful way to fill in blank spots, ask for clarification, and produce elaboration. The interviews that followed then tended to be even more open-ended and participant-driven, since much of the background had already been discussed and elucidated.
We were somewhat surprised at what proved more challenging in the vein of respect and spirit of collaboration. Service provider interactions and formal interviews were anticipated to accomplish several tasks. First, as these organizations are a main contact for new refugees, they could provide contact information for potential interviewees willing to share their stories with researchers. Second, interviews with service providers could provide additional insight about the landscape of resettlement. Finally, results of both types of interviews could then be used by service providers to better understand their own organization’s strengths and weaknesses, and identify gaps in the resettlement service continuum. For example, if recently resettled refugees discussed a lack of knowledge about how to enroll children in public schools, the organizations could adjust their programs to account for this oversight and ensure students gained access. Thus, this research path was seen as a mutually beneficial arrangement to both scholars and those invested in the resettlement process.
Such anticipated benefits to both the research team and resettlement organizations were recognized by many service providers, who were willing to be interviewed and helpful in other ways. A number of requests for service provider interviews or interactions, however, were met with skepticism, with some often voicing apprehension about sharing information on their organization or those receiving its services. From a safety standpoint, this skepticism about the motives of anyone requesting information about RRBs is understandable. Some of those dependent on resettlement services may have fled violence in their home countries due to their ethnic group membership, political stances, sexual orientation, etc. As such, limiting information available about these individuals could be the difference between life-or-death for these refugees, especially if they suspected their whereabouts were still being sought by people who would do them harm.
This well-founded concern aside, we also argue that there are significant structural obstacles to completing this field research that do not depend directly on the service population’s safety. In our case, as successful resettlement cases can be used to justify an organization maintaining their current funding levels or acquiring additional resources, limiting the amount of information about organizational process makes competitive sense. Even though all interview data is kept confidential, some may have believed we intended to collect information about them. Risks associated with an interviewer sharing an organization’s experiences of shortcomings with another service provider, a funder, or a suddenly more suspicious general public—however inadvertent the disclosure—was a significant risk to not only the organization and its employees, but also its RRB constituents. Therefore, it may have been more beneficial to the organization to withhold information, even if it could help improve their service provision, than risk showing weakness.
Added to this complicated mix were the changing dynamics in the landscape of resettlement after the 2016 presidential election. World Relief had to lay off some staff in the Fox Valley and one long-time RRA closed its resettlement operations in Milwaukee. Now in the political crosshairs, it was clear why some of our original partners exhibited heightened insecurity and increased guardedness in working with us. This may help to explain why we had great difficulty in establishing working relationships with staff from formal RRAs outside the Fox Valley, where we had less time to establish trust. The presence of multiple RRAs may have created stronger turf boundaries, and there seemed to be research fatigue on the part of some organizations and RRBs alike. It may also help to explain why many of our go-to collaborators were informal service providers. This included a woman who taught fitness classes to RRBs on her own, had connections with many of them, and a passion to help. Such individuals could collaborate without the restrictions and stress that counterparts at formal RRAs had to deal with.
The dynamics of uncertainty manifested in a concern about our work that seemed to come out of the blue (and based upon no particular incident, it turned out) from an original partner two years into the project. Other NPOs that provided various services to RRBs showed little interest in working with us for reasons that were not quite clear, though in one case an NPO manager at a meeting where we presented preliminary themes from the data seemed offended we would share that a participant was critical of local efforts to help well-educated RRBs find jobs consistent with their qualifications. Two research team members, both well-trained students who were highly involved in the project, experienced direct conflict with representatives of NPOs in two different cities.
One publicly lambasted our project at a meeting in which the undergraduate was our representative, arguing that we were having a negative impact upon RRBs, without any detail or substantiation, and made several other claims that were clearly false. In the other case, the team member arrived at what she thought would be a service provider interview with the executive director, only to find herself in a room with four staff people, all of whom had a list of questions for our project representative written down in advance. While she appreciated their apparent passion for protecting their RRB clients, the researcher was shaken by what she considered an interrogation. At the time of this writing we do not have a complete understanding of why these incidents occurred, but the damage was done. More importantly, lessons were learned.
Much of this may have been avoided through clearer, more direct, and consistent communication on our part, as this is key to successful collaboration of any kind. When social friction develops, it typically stems from a communication breakdown of some sort, and it is possible some service providers felt a disconnect. The key direct service provider remarked how she perceived tension “around the means to getting an interview.”
The RTF facilitator acknowledged that we “went to great lengths to ask permissions and be respectful of participants,” but also that investing more time at the beginning of the process (i.e., identifying potential participants and increasing agency buy-in) would have resulted in greater success. The same interviewee also conceded that RTF partners, too, could have assuaged the process: “What we also could have done better was continuous evaluation to see if we needed to shift directions or change our approach. I feel that this [was put] on you and your team and we didn’t participate or have input the way we should have.”
It has been argued that if the project does well in terms of being participatory and respectful, it has the potential to be more valid as well (Gilhooly and Lynn 2015
; Van Auken et al. 2010
). In our case, we used mixed methods (survey and interviews) because we wanted to reach as many RRBs as possible and go deeper with a sub-set. Given the population's characteristics, we knew that this would require separate approaches that pay attention to the sensitivity of the subject matter, lack of power on the part of RRBs, and their safety concerns. Surveys are often the most effective and efficient method to reach a broader population, particularly in situations of limited time and resources, and can provide important information pertaining to patterns in respondents’ lives. Producing this type of relatively “thin” data was a very specific aim of the stakeholders with whom we were collaborating and this expectation structured our work. We therefore sought to achieve the highest possible N for our survey, to enhance our ability to make statistical inferences about the resettlement experiences of RRBs. For a variety of reasons, however, we knew it would not be possible to produce a random sample and that we would have a difficult time achieving a large N, so our expectations for the quantitative side of things were tempered by such limitations from the outset.
Further, while the survey was viewed as an important first step, the RTF was interested in having us gather RRB stories and probe into the resettlement process from the service provider perspective. Interviews are superior to surveys for this purpose, so we incorporated interviews for “thicker” descriptions of RRB settlement and adaptation, an approach with a different purpose than the survey, as well as an alternative framework of validity and reliability (Noble and Smith 2015
). Because of the open-ended nature of in-depth, qualitative interviews, and the additional time it takes to see them through, the overall goal is to help illuminate what is happening and why, from the point of view of those involved.
This element of the project was highly consistent with the CER’s aim to make an impact beyond traditional research. As noted, while we did achieve a solid sample of RRB survey respondents and interviewees, and a strong sample of service provider interviewees, the complicated nature of this CER project posed some risks to validity, while other elements seemed to enhance validity, understanding that in social research it is impossible to produce “the truth” with certainty. It can be argued, in at least some ways, that CER should be more valid if successful in removing sources of error that occur in traditional research, such as researchers not being “close” enough to the topic or population of interest, leading to a lack of rapport and trust, the crafting of questions that are not meaningful to participants, power dynamics that produce high levels of social desirability bias, and more.
Active stakeholder involvement in our project may have helped lessen some of this error. People tend to tell stories that fit embedded, institutionalized narrative expectations (Carter 2017
). In this case, social identities are influenced by public policies and organizations that seek to help populations seen as vulnerable and needy (Loseke 2007
). When RRBs are being interviewed by an academic, out-group researcher, they are likely to tell a story based on the expected narrative they think the researcher wants to hear, producing social desirability bias. Sometimes the responses appear “canned” to fulfill the expected narrative. With the inclusion of cultural bridge builders and informal service providers, the skewing of responses for this reason may have been mitigated, as their presence often seem to help participants feel more at ease, lessening the drive to fulfill narrative expectations. As discussed, however, inter-group power dynamics likely had similar effects on some RRBs.
Our research was often driven by project collaborators, without whom we would not have had access to certain RRBs. The most obvious examples were when they aided in translating survey questions. When a service provider who spoke a key language (spoken by potential participants from multiple countries of origin) began working with our team, she became the leader of our efforts to recruit RRBs who otherwise would have been virtually inaccessible to us. During interviews with these RRBs, there was less control by the interviewers and time spent on the process was often lengthened. Increased duration was due to repeating and clarifying survey questions, the less structured survey style, and most importantly, free-flowing dialogue (i.e., not boxed in). There appear to have been less attempts by the RRB to fit into expected narratives.
Further, the PDPE element appeared to work well for those RRBs who elected to engage in it. Since first being used in the 1950s, incorporating photos has been shown to help produce deep interviews and shift the power from researcher to participant, yielding arguably more ethical research and valid data (Van Auken et al. 2010
) including from socially marginalized groups [such as homeless people (Klitzing 2004
; Packard 2008
)], of which RRBs would certainly be a prime example.
Another example would be a recently resettled RRB from the Middle East who was illiterate in any language and had difficulty understanding certain concepts, such as culture. This led to a very long survey and interview process conducted in Arabic and spread over two meetings. He understood the idea behind PDPE, though, and used photos he asked the researcher to take to make important points, including one of him posing by his vehicle with a disabled parking pass dangling from the rearview mirror, which illustrated a level of accommodation and respect he did not feel in his home country. In another instance, a relatively young Hmong RRB born in a refugee camp though raised mostly in the U.S. was enthusiastic about PDPE. He took a number of photos he had thoughtfully considered and composed, and came to the interview meeting ready to tell stories and share perspectives. Indeed, there is hardly any dead air in the one-hour recording; he likely could have filled it with only his voice.
Alternatively, CER projects like ours may be more complicated and introduce different sources of error than traditional methods. Validity of the survey data may have suffered somewhat if participants found it cumbersome and difficult to understand, which was at least partly due to the applied/community-based nature of it, as we set out to answer all of the key questions from our partners. Of course, we pursued this project because we believed the benefits outweighed such concerns, but our experience illustrates the double-edged sword that is CER: We feel compelled to ask questions partners want answered, which could introduce additional sources of error (i.e., respondent fatigue if the survey is too long).
Though this kind of concern applies to any study with multiple researchers, the incorporation of 40+ students over three years, and several teams of people working largely independently in different locales, and not always following the protocols from our training and handbook, likely caused some error. Finally, the complication caused by needing to rely upon cultural bridge builders and translators, who may not have translated everything that was said or all of it precisely, along with some occasionally offering their own perspectives or probing RRBs for answers outside “the script,” introduced additional sources of error. This is not ideal, but it seems that we must take the bad with the good in CER, while striving to apply lessons to the next iteration.
We practiced triangulation for many of our RRB participants, who completed both surveys and interviews, some including photos of their choosing, which provided multiple sources of data. Using PDPE involved a form of member checking, as researchers engaged in in-depth discussion of each chosen photo and later carefully confirmed that the photos did indeed help illustrate their stories from their perspective, we asked for express permission to confidentially utilize each photo in publication or public display. We also used multiple investigators, with most surveys and interviews being completed by pairs of researchers, and in any case working together in small teams (per study site), and as a larger team across all sites, to increase reflexivity. Finally, while we did create and utilize a study handbook outlining the various steps and elements of the study process, and research log, where we recorded specifics about potential and actual study participants, we would have benefited by taking more detailed notes about all of the steps we followed in the process of study design, data collection, and analysis, to create a more robust audit trail, thereby enhancing the trustworthiness and validity of our data.
We should also attempt to understand the action orientation of our effort. Stoecker
) presents four basic types of research output, explaining that, a “community change process begins with diagnosing some condition, prescribing an intervention, implementing that intervention and evaluating its effectiveness” (p. 390). Viewed from this perspective, our CER project is currently in the first stage of working towards community change. The applied research we agreed to execute was designed to diagnose the state of refugee resettlement, to identify successes and challenges, strengths and gaps to fill, and point towards possible interventions (the second stage).
) further notes that another way to consider action orientation is through the most common types of proposed outputs by CER projects: reports, organizing, meetings, websites, advocacy, programs, plans, and educational efforts. Regardless of the type of organization proposing the CER work, Stoecker found most propose no action at all or only a report of some kind, and even “when we look at the other action categories, what is defined by applicants as action is often just another form of reporting. A meeting is simply where the research results are presented orally” (p. 391).
Although we designed action into our CER, we were also guilty of settling for reports and meetings as key outputs. We have presented about our process and preliminary findings at multiple RTF meetings, both local and regional, and will share a final written report with our collaborators upon completion. Given that the project started as a relatively traditional applied research project, we largely sustained elements of that inherited structure throughout, which yielded benefits and drawbacks. We did, however, also incorporate plans—“using the research to outline some proposed program or intervention” (p. 391)—and education, one of RTF’s explicit goals.
First, we encouraged site stakeholders to use the term “RRB” instead of “refugee,” which constituted an educational outcome in and of itself, and will perhaps yield a small shift in power away from those who would use a pejorative label to stigmatize, and towards individuals to allow them to define themselves. Another key goal was for us to produce data about resettlement outcomes and quality of service to educate RTF in the hopes of yielding the best possible service. We also plan to use our findings to develop a brief primer for service providers and volunteers in cities resettling refugees, as well as a written vision for a one-stop shop to provide services to RRBs and build intercultural competence and community among RRBs and other local residents.
RTF also hoped to educate the general public about RRBs. We are doing so via a traveling exhibit of photographs and narratives that debuted at UWO and was also displayed as part of a major international festival in Milwaukee shortly before we wrote this manuscript. Included in the exhibit were some photos, as illustrated in Figure 3
, that were taken by one of the authors when he worked abroad documenting humanitarian crises professionally.
Along with data, photos, and narratives from RRB research participants, the thrust of the exhibit, however, is portraits of RRBs taken by project collaborator Colin Crowley for this purpose, separately from the research process. It was very important to him that portraits did not reinforce negative stereotypes of refugees. He wanted images that went against the stereotype of the destitute, hopeless refugee and instead presented the public with visions of strong, accomplished, and cultured individuals. Our collective hope is that the members of the public who view these photographs come away feeling that, regardless of our backgrounds, our futures are inextricably tied together, and that these neighbors have more to offer than some may have previously understood (see Figure 4
, Figure 5
, Figure 6
and Figure 7
Initial feedback about the exhibit from some project constituents has been positive. A number of RRBs attended the opening reception and clearly enjoyed it, posing for photos and spending a good deal of time there. Sticky notes collected at the end asked, among other things, what exhibit attendees learned from it. Answers included, “What they went through!” The key direct service provider noted in a follow-up email,
Our staff visited the exhibit a week ago and appreciated the efforts of your team. One noteworthy comment was regarding the diversity of the refugees who were included in the survey. We felt including Holocaust survivors, the Hmong population and recent arrivals was inclusive and reflective of resettlement as a whole. Thank you! Also, all of the pictures were powerful but (Colin’s portraits of the four) individuals made the exhibit ‘personal’.
Finally, another, more informal service provider who spends a great deal of time with RRBs shared via a public post on social media after the opening reception,
It takes wisdom and stamina to complete academic research of interest to the academy and a wider public. It is characteristic of the UWO Sociology Department to stand at that crossroads at which academic excellence and relevance to the wider community meet—the Wisconsin idea! […] Visit the exhibit yourself if you can. I deem it a great honor to count (names of some RRBs featured in the exhibit) among my friends.
The hope is that the outcomes from our CER project will not only point to potential interventions that service providers and policy makers can make to improve resettlement experiences, but also that our stakeholders and the general public will learn more about this important slice of our social and political landscape. In an interview, one service provider said in response to our question about barriers to successful resettlement, “Barriers? Knowledge […] what is a refugee? What are the cultural backgrounds of refugees?”
The result of continued action around education may be reduction in stereotypes, enhanced intercultural knowledge and competence, increased social interaction, and eventually community building between these groups. As for all criteria for this CER desired process outcomes framework, we seem to have at least partially met some of the benchmarks and fallen short in others. The point, of course, is for us to learn from the process and share what we have learned with other scholars and constituents.
It is argued that CER has potential to be more impactful than traditional research (Stoecker 2009
; Van Auken et al. 2010
; Gilhooly and Lynn 2015
; McMorrow and Saksena 2017
; Im 2018
), but the question of impact logically depends upon the goals of a CER project. In our case, time will tell whether it is truly impactful in meeting the goals of the stakeholders. We have, however, discussed apparent and potential impact via process-oriented goals above. These are important considerations in CER, where the process or means of engagement itself, regardless of the “tangible outcomes”, can have meaningful impact upon participants (including organizations in this case), as well as researchers.
Connecting back to Stoecker
), however, it is unclear to what extent we helped build local constituents’ capacity to create knowledge that will lead to lasting change. We did relatively well in the other points of this desired process outcomes framework, but the lack of buy-in by service providers, and RRBs’ roles largely as respondents and less as partners suggest that we will fall short. We do have the ability to address such shortcomings in the final stages of the project, however, and apply these lessons to future projects.
A final point to consider in terms of impact is whether CER like this helps build community. Since this is a contested term often used without precision, we would suspect most CER does not explicitly propose building community as an outcome (ours did not) and therefore will not be highly impactful, as it requires concerted effort over time. We do, however, consider social capital, and particularly bridging capital—produced as people make connections across divisions based upon class, race, and social power, and which can be converted into resources—as a building block of community.
We did observe elements of such capital being built and potential for such connections and purposive interactions to come together in community, strengthening capacity to produce long-term change in the future. Some examples included a researcher connecting an RRB looking to pursue an MBA with a business professor he knows from a regular pick-up basketball game, another being invited to join the game, the referral of a recently arrived RRB with interest in becoming a veterinary technician to a college admissions counselor, the RRB and project team member playing music together (which has occurred multiple times beyond the exhibit), students gaining internships and other relevant experience due to the project, and, despite the lack of participation described earlier, the academic department establishing new collaborative relationships and demonstrating its value to numerous stakeholders. If service providers, university students, and typical residents start to feel more knowledgeable, confident, and informed about RRBs, who simultaneously become better integrated into local society, such constituents will feel more motivated to reach out to RRBs and vice versa, and more opportunities for regular interaction will develop.
As with any social research endeavor involving direct interaction with participants, our project was filled with a considerable number of relationships that must be addressed throughout the research process. In short, researchers must be reflexive in considering power dynamics and the manner in which other participants in the process were treated (Berger 2015
), and this is particularly true in CER.
Often, the methodology for a research project is clear and clean. Execution instructions are simple and efficient. However, when working with RRBs in several cities, we found that the data could not always be collected in such a streamlined fashion. The data emerged in unexpected ways—the stories of RRBs presented themselves outside the lines, hid in the margins, jumped off the page. In short, collecting data with resilient populations can be somewhat messy.
One assumption we make as researchers, for instance, is that an interview session can be scheduled ahead of time and that we will have an approximate idea of how long a session will take. We know when we will be home for dinner. CER, however, does not always work this way. Indeed, we came to understand that our western assumptions regarding the twenty-four hour clock were flawed. Often, our research team would find ourselves in someone’s home for an interview and be presented with a meal or tea (an enjoyable tangent indeed), and that we would be interviewing more than one person that day, or more than one at a time. As sessions progressed, before any interview had begun, we often found ourselves knee-high in data as talk suddenly took a turn from the weather, the food, a television show, etc., to discussing concerns over community organizing, politics, resettlement issues, etc. As guests at the table, we became flies on the wall. We would discreetly text our family and friends to cancel plans and begin to pick up the data as it fell around us. In essence, CER, regardless of outlined methodology, is a form of ethnography.
When conducting research, it typically makes sense to work in an organized fashion. A leads to B, which leads to C, and so forth. This orderly approach often did not work with the RRB population. At one event where we provided food to a group learning English, we were pleasantly surprised to collect a large number of pre-participation forms from RRBs. We then had to search for a translator, and sometimes found it difficult to enlist one. As a result, we lost a number of RRBs who were willing to be interviewed. Had we worked backwards with a translator lined up first, we would not have faced this situation, but because they needed to be paid and the IRB process is time-consuming, it was illogical to do so unless we knew we had willing participants from that group.
Trust was another area in which our team encountered messiness. Often, the vulnerable populations with whom we engaged brought unannounced people, such as spouses or adult children, to the research meeting, or took steps to limit the involvement of those we assumed would be present for interviews. For instance, one respondent changed the planned venue from his home to a café last minute so as to avoid having his parents, with whom he lived, hear our session. Other examples could be found with elderly respondents who had their children watching us, we suspect at the insistence of the children, to ensure their parents’ safety. Sometimes we had spouses who refused to leave the room and offered unwanted commentary throughout the interview, which led to the couple bickering throughout.
Several interviewees simply preferred to be interviewed together and refused to participate alone. One example that stands out is that of two elderly RRBs who had been through the Holocaust together, in concentration camps together, and who moved to their current city together. They did not really see their refugee experience as their own—it was a shared experience. In all these cases (once ensuring that IRB protocol was still being followed), our choice was to either cancel the interviews or to collect the data, blurry as it was, and hear the stories of our respondents. We chose the latter.
The survey was designed to gather largely quantitative and relatively straight forward information from RRBs, but they often had a great deal of commentary on each question, going far beyond the places for them to fill in the blanks. We were then able to use these comments as starting points to inform the qualitative part of the interview session. Also, their answers to the survey questions, while so absolute once entered into Qualtrics, were less so while being formed. Often respondents were torn between two numbers on a Likert scale or wanted to answer more than the question was asking. At times, RRBs were confused as to what community they were supposed to answer the survey question about. Many Jewish RRBs were confused as to whether the question was asking about their Jewish culture or the culture of their country of origin. Sometimes there would be confusion as to whether culture referred to the culture of their home countries at the time they left, or the richer, more complex, historical culture of that same place, to which they felt a personal connection. Each RRB dealt with these questions in their own way, rendering the data complex.
Even the planning of the research was chaotic at times. Often, the path to participants was unclear. One case in particular stands out: We met an RRB working at a hair salon (where a team member was getting her hair done). This RRB refused to be interviewed, but told team members we could interview their parents, also RRBs. We were given instructions to go into an unfamiliar area of the city and find their parents’ store. Once we arrived we found that the parents spoke very little English and did not wish to be involved. They mentioned their other child, an RRB who worked as a service provider. Months later, while at an event that was attended by many service providers, we happened to meet this other child. We explained how we had met their sibling and parents, creating instant rapport. They were there working, busily handing out food to RRBs. However, they had forgotten to pack plastic cutlery. Research team members immediately set off to the nearest mall on a quest to find plastic cutlery. Once our mission was complete, and cutlery delivered, we were offered food and became fast friends. Not only did this RRB grace us with an interview, but introduced us to other RRBs and service providers. What seemed earlier to be a waste of time (salons and stores and cutlery quests), was actually important in terms of process. Often one needs to wade through the shallow end in order to get to deeper waters.
To arrive at realizations like these, an attempt to minimize power disparities and maintain an awareness of how biases could influence our results, reflexivity was built into our process. Investigators were tasked with writing frequent reflections on a variety of issues and attending regular research meetings, including three retreat-like sessions held once in each of the three study sites during the grant period for the expanding project. Not only did these meetings involve presentations related to enhancing cultural competence and methodological soundness, team members were asked to reflect on their experiences with the project to that point. Robust discussions ensued, as team members reflected on the most interesting things they had learned from a respondent/interviewee, what such stakeholders noted as the most pressing problems during the resettlement process, how participating in the project, itself, affected team members personally, and, most importantly, what challenges team members were facing and how they could be addressed. Similarly, research meetings ensured all investigators were maintaining standards set forth for the study and provided a space for team members to become acquainted with one another and learn how other sites were progressing.
In one case, the team meeting corresponded with a special “Write Your Life” workshop that helped bring RRB youth together with several team members, who acted as transcribers of their stories, which proved empowering for the youth and highly impactful to team members. The workshop also allowed team members to interact with service provider stakeholders. Otherwise, these reflection-oriented team meetings were limited to research personnel and, in hindsight, would have been even more effective with greater inclusion of RRB and service provider stakeholders.
As noted, we have had subsequent discussions with some stakeholders that touched on our level of reflexivity. When asked to examine our power/position in the equation of the study process, the key direct service provider we interviewed noted, “I appreciate you asking that question as I feel I need to be reminded and challenged of the lens that I am using and responding [with] in my work and interactions with RRBs.”