Abstract
Despite the high prevalence of adverse health and trauma-related outcomes associated with intimate partner violence (IPV), help-seeking and service utilization among survivors is low. This study is part of a larger mixed-methods and survivor-centered validation study on the Icelandic Barriers to Help-Seeking for Trauma (BHS-TR) scale, a new barriers measure focused on trauma recovery. A mixed-methods legitimation strategy of integration was employed to evaluate the BHS-TR structure in samples of IPV survivors. The merging of qualitative (n = 17) and quantitative (n = 137) data through a joint display analysis revealed mainly complementarity findings, strengthening the scale’s overall trustworthiness and validity evidence. Divergent findings involved items about mistrust, perceived rejection, stigmatization, fearing vulnerability, and safeguarding efforts that were significant help-seeking barriers in the survivors’ narratives, whereas factor analysis indicated their removal. These BHS-TR items were critically evaluated in an iterative spiraling process that supported the barriers’ influence, illuminated core issues, and guided potential refinements. This work contributes to the growing field of mixed methods instrument validation placing equal status on qualitative and quantitative methods and emphasizing integration to provide more complete insights. Moreover, the study’s findings highlight the added value of further exploring divergence between two sets of data and the importance of giving attention to the voices of the target population throughout the validation process.
1. Introduction
1.1. Gender-Based Violence
Gender-based violence (GBV) is a public health crisis of pandemic proportions and one of the most prevalent human rights violations worldwide [1,2]. GBV refers to harmful acts directed against an individual because of their biological sex or gender identity. It is deeply rooted in gender inequality and cultural norms that uphold male domination and female subordination [2,3]. While people of all genders may experience GBV, women-identifying individuals are disproportionately impacted by it, and the most common form of GBV is intimate partner violence (IPV) against women [2,3,4]. IPV has been defined as any current or former intimate partner behavior that causes physical, sexual, or psychological harm, including physical aggression, sexual coercion, emotional abuse, and controlling actions [4]. Global estimates from the World Health Organization indicate that almost one-third (27%) of women have been subjected to violence by their intimate partners [1].
1.2. Violence against Women in the Nordic Countries
The Nordic countries are often considered the most gender-equal countries in the world [5,6], and according to the Global Gender Gap Index, Iceland had in 2021 closed 89.2% of its overall gender gap, holding the top spot for 12 years in a row [7]. While critical domains are used to measure gender equality (e.g., health and survival, educational access and attainment, economic participation, political power, and time allocation), these indices do not take GBV into account [5,6,7]. Previous research has reported a high prevalence of violence against women in the Nordic countries [8,9,10]. One of the most extensive studies in this regard (n = 42,000 women) was conducted by the European Union (EU) Agency for Fundamental Rights, which found lifetime prevalence rates of IPV in the Nordic countries to be among the highest of the 28 EU Member States. With an EU average of 22%, ranging between 13% and 32%, the rates were 28% in Sweden, 30% in Finland, and 32% in Denmark [11]. In Iceland (a non-EU member), a national survey estimated the lifetime prevalence of IPV against women at 22.4% [12].
1.3. Help-Seeking for Trauma Recovery
Numerous studies have documented the severe impact of IPV, a form of interpersonal trauma, on survivors’ health and well-being, showing increased risk of depression, post-traumatic stress disorder (PTSD), anxiety, somatic symptoms, substance abuse, and suicidal ideation [13,14,15,16]. This suffering is associated with functional impairment, low sense of coherence (SOC), and substantially reduced quality of life [17,18,19], even years after leaving the abusive relationship [20,21].
Despite these adverse outcomes related to IPV, previous research has shown that help-seeking among survivors is low in most countries. Some never seek help, and those who do mainly choose informal sources of help, usually from their family or friends and are less likely to seek formal help, such as from shelters, healthcare services, or the police [22,23,24,25]. It should be noted that a significant body of literature suggests that women with a history of IPV use healthcare services more than non-abused women, especially primary care and emergency departments [26,27,28]. Still, these women rarely disclose the violence and often do not receive the support and appropriate care they need [29,30,31].
The IPV help-seeking literature is primarily focused on escaping the violence and attending to the immediate harm caused. While these often first steps are critical, there is a need for an increased focus on survivors’ pathways for trauma recovery [32,33,34]. Help-seeking after IPV is a complex journey involving a series of meaning-making judgments and socially engaged and culturally informed actions [35,36,37], and the road to recovery is often challenging [38,39].
Findings of low help-seeking rates are consistent with other studies reporting that IPV survivors are faced with a wide range of barriers on sociocultural, structural, interpersonal, and individual levels, e.g., normalization of violence, access challenges, fearing consequences of disclosure, and self-blame [30,40,41,42]. Moreover, studies have indicated that survivors with depression, PTSD, and low SOC face even more significant barriers to help-seeking, such as symptom burden, fearing mental illness stigmatization, and a weak sense of manageability and meaning, making it more challenging to take action [21,36,43,44].
1.4. Use of Mixed Methods for Instrument Validation
In a widely used definition based on a review of definitions, mixed methods research is defined as a “type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration [45] (p. 123)”. In the mixed methods research field, meta-inferences refer to inferences stemming from both qualitative (Qual) and quantitative (Quan) findings integrated into a coherent whole [46,47].
One of the earliest examples of using multiple research methods for validation dates to the 1950s, with Campbell and Fiske’s [48] framework giving rise to methodological triangulation and arguing that the convergence of findings derived from more than one method would strengthen the evidence of validity. However, as innovative and valuable their framework has been, it is first and foremost Quan. To date, in the instrument development literature, construct validation is often conceived as mainly a Quan endeavor [45,49,50,51,52]. When Qual data are used, it is usually only granted a supplementary role to Quan data, and often the methods are utilized in isolation rather than fully integrated [50,52,53]. Still, there is a growing literature on mixed methods validation. A few frameworks have been developed that place equal value on Quan and Qual methods, focusing on validity and trustworthiness, and emphasizing the integration or “mixing” of findings from both databases to inform validation evidence for a measure [49,51,52,54].
The “fit” of data integration refers to the coherence of Quan and Qual findings [55]. Such assessment is likely to lead to four possible outcomes: Confirmation is when the findings are consistent with each other, supporting drawing the same conclusion from each. Complementarity is when the findings tell different but nonconflicting stories (reflecting different sides of the same coin). Expansion is when the findings diverge to a certain degree but, when combined, can expand insights. Discordance is when the findings are inconsistent, contradictory, or disagree with each other [55,56].
The term legitimation [46] has been recommended to refer to validity and quality in mixed methods studies, as it considers both Quan and Qual research paradigms [47,57]. A legitimation typology was developed to guide researchers to ensure design quality and interpretive rigor to yield high-quality meta-inferences. The typology includes several legitimation types and is grounded on the notion that legitimation is not only an outcome but a process occurring throughout all stages of the research [46,47,57,58]. Recently a new type of legitimation was proposed to be added to the typology, “Divergent Findings Legitimation” [59], stemming from the challenge researchers face when Quan and Qual findings diverge [55,59,60,61]. Failure to follow up on divergence can threaten legitimation, and this new type is about the extent to which researchers explore and learn from such findings [59].
1.5. Validation of the Barriers to Help-Seeking for Trauma Scale
The Barriers to Help-Seeking for Trauma (BHS-TR) scale was developed from an existing mental health barriers measure [62] focusing on service use for mental disorders. Based on an international literature review about barriers to seeking help after trauma and findings from focus groups and individual interviews with American and Irish GBV survivors, the original scale was adapted for GBV survivorship [21,63]. The scale added new items about normalization, shame, mistrust, perceived rejection, fearing the consequences, and feeling frozen, making the measure more trauma-specific and survivor-centered. The BHS-TR consists of seven barrier subscales that can be grouped into two structural and internal barriers indices. The scale was psychometrically sound in a sample of American GBV survivors [63].
The BHS-TR was translated and cross-culturally adapted into the Icelandic language and context [64] and initially validated in a mixed-methods study among IPV survivors in Iceland [65], creating the first Icelandic trauma-specific measure that assesses help-seeking barriers. An essential part of this work was qualitatively evaluating the scale through cognitive interviewing, resulting in the development of new barrier items based on the survivors’ lived experiences. Using these findings, we utilized building to adapt the BHS-TR scale and then carried out a psychometric evaluation of the whole instrument with the additional items. Both Qual and Quan phases provided evidence that the Icelandic BHS-TR is relevant, reliable, and valid [64,65]. Nevertheless, there was a noticeable mismatch between the Qual and Quan findings regarding several items on the scale. Primarily, items that were significant barriers to help-seeking in the survivors’ narratives were problematic in the exploratory factor analysis technique, mainly due to cross-loadings with different factors. According to the factor analysis procedure, these cross-loaded items would be dropped and hereafter referred to as the dropped items [65]. This mismatch between the participants’ narratives and the factor analysis results points to a legitimation issue, demonstrating the need for further systematic assessment of the coherence of barriers to help-seeking Qual and Quan findings.
1.6. The Current Study
This study is part of a larger mixed-methods and survivor-centered validation study on the Icelandic BHS-TR scale. The more extensive study followed well-established international guidelines and best practices for cross-culturally adapting and validating measures [66,67,68]. An exploratory sequential mixed methods design [69] was used, conducted in two phases (Qual → Quan) with equal emphasis given to both phases. The primary aim of this present study was to use a mixed-methods legitimation strategy of integration to evaluate the BHS-TR structure by merging the Qual and Quan findings and examining meta-inferences on divergence.
Specifically, this study:
- Integrated the Qual and Quan data to discover convergent (confirmation or complementarity) and divergent (expansion or discordance) validation evidence for the BHS-TR.
- Critically evaluated divergent items by examining the following:
- influence of the barriers on the survivors’ help-seeking,
- factor cross-loadings for the most strongly endorsed items,
- mean score differences for survivors’ subsamples (depression, PTSD, and SOC).
- Made recommendations about the benefits of this integration to BHS-TR refinements.
2. Materials and Methods
2.1. Study Design, Integration, and Legitimation
This study used a mixed methods spiraling design [49,56,70] that employed integration strategies at the design, methods, interpretation, and reporting levels [55] through merging and joint display. We implemented nine types of legitimation strategies to ensure design quality and interpretive rigor to generate high-quality meta-inferences: sample integration, inside–outside (emic–etic), weakness minimization, sequential, commensurability approximation, multiple validities, integration, pragmatic, and divergent findings legitimation [57,58,59].
2.2. Data Collection
A sequential parallel sampling design [71] was used for the Qual and Quan phases. The inclusion criteria for both phases were to self-identify as a woman, be 18 years old or older, live in Iceland, speak Icelandic, and have experienced IPV. Since what constitutes IPV can vary from person to person, we let participants self-identify as survivors based on the definition and examples of types of IPV provided in the introductory material, including physical, sexual, emotional, financial, and cyber violence. Additionally, for ethical reasons and the participants’ security, they had to have been out of the abusive relationship for at least a year.
2.2.1. Qualitative Phase
The Qual data were comprised of in-person cognitive interviews [72,73] with 17 Icelandic IPV survivors; a total of 25 interviews was conducted between August 2019 and January 2020. Participants were recruited using purposive sampling from survivors’ centers and services in North and South Iceland. During recruitment, attempts were made to select a diversity of IPV survivors. A detailed description of the procedures has been previously reported [64].
2.2.2. Quantitative Phase
The Quan data were gathered using an anonymous online survey advertised through social media posts and flyers distributed at various services for violence survivors in Iceland, located all over the country. Participants accessed the survey by following a link hosted on the Icelandic Directorate of Equality’s website, and a voluntary response sampling was applied. Qualtrics, a secure online software package, was used to build the survey, including the BHS-TR scale, demographics, help-seeking history, mental health, and trauma-recovery related outcome measures (detailed below). The data were collected between February and October 2020, and our sample included 137 IPV survivors in Iceland. A detailed description of the procedures has been previously reported [65].
2.3. Participants
The Qual sample included 17 women IPV survivors in Iceland, ages 18 to 64 years (M = 37.4, SD = 12.2). Most of the women were employed (70.6%), with about half having received a university degree (52.9%), and most had children (70.6%). Most of our sample self-reported a current mental or physical diagnosis (64.7%), and all had been faced with physical, emotional, and social health effects because of their traumatic experiences. The women in the Qual sample had all sought help because of the violence, especially when leaving the abusive relationship, hence the recruitment strategy. However, when asked explicitly about seeking help for trauma recovery, only a few (35.3%) said they had done so, and about half of them had received mental healthcare at some point in their life (52.9%).
The Quan sample included 137 women IPV survivors in Iceland, ages 19 to 76 years (M = 40.7, SD = 11.7). Most of the women were employed (64.2%), with over half having received a university degree (59.9%), and most had children (76.7%). Most of our Quan sample self-reported a current mental or physical diagnosis (67.9%), and most had received mental healthcare at some point in their life (81.8%). When asked about needing help for trauma recovery in the last 12 months, 75.9% said that they had needed help in general, of which 26.3% listed as needing it most of the time or very necessary. Furthermore, 70.1% said they had especially required mental health treatment, of which 29.2% listed as needing it most of the time or very necessary. Nonetheless, almost half of the women (45.3%) reported not seeking the professional help they needed in the last 12 months. Further information on the characteristics of both Qual and Quan samples can be found in Table 1.
Table 1.
Characteristics of participants in both phases.
2.4. Measures
2.4.1. Barriers to Help-Seeking for Trauma Scale
Help-seeking barriers were measured with the BHS-TR scale, which asks about barriers to seeking help for trauma recovery in the last 12 months. Respondents answer on a 4-point Likert scale (1 = “Did not influence me”, 2 = “Influenced me a little”, 3 = “Influenced me somewhat”, 4 = “Strongly influenced me”), with a higher total score indicating more barriers [63]. The current Icelandic version of the scale has 26 items divided into two indices (Structural and Internal Barriers), each of which has four subscales (see Supplementary Table S1). Initially, the scale included 41 items, and a list of the 15 dropped items can be found in Supplementary Table S2. The 26-item BHS-TR was found to have sound psychometric properties in our previous work [65]. There were no missing data for the BHS-TR, and the Cronbach’s α in this study was 0.87.
2.4.2. Patient Health Questionnaire-8
Depression was assessed with the Patient Health Questionnaire-8 (PHQ-8) [74]. It is a widely used and valid measure consisting of eight items based on the DSM-IV diagnostic criteria for depression. PHQ-8 is identical to the PHQ-9 without the suicidal thoughts item, making it more suitable for general survey use. Respondents are asked to assess the frequency of symptoms in the past two weeks on a 4-point response scale from 0 (“Not at all”) to 3 (“Nearly every day”), with a total score range from 0 to 24. A clinical cut-off score of ≥10 has been recommended to indicate probable current depression [74,75]. There were no missing data for the PHQ-8, and the Cronbach’s α in this study was 0.87.
2.4.3. Post-Traumatic Stress Disorder Checklist for DSM-5
Post-traumatic stress disorder was assessed with the PTSD Checklist for DSM-5 (PCL-5) [76]. It is a widely used and validated measure consisting of 20 items corresponding to the DSM-5 symptom criteria for PTSD. Respondents are asked to rate how bothered they have been by the symptoms in the past month on a 5-point response scale from 0 (“Not at all”) to 4 (“Extremely”), with a total score range from 0 to 80. A clinical cut-off score of ≥31 has been recommended to indicate probable current PTSD [76,77]. There were no missing data for the PCL-5, and the Cronbach’s α in this study was 0.96.
2.4.4. Sense of Coherence Scale-13
SOC was measured with the shortened version of the Sense of Coherence Scale (SOC-13), also known as the Orientation to Life Questionnaire [78]. It comprises 13 items about how people view their lives, measuring SOC’s three main dimensions: comprehensibility, manageability, and meaningfulness. Respondents rate their level of agreement or disagreement on a 7-point semantic differential scale that has two anchoring responses tailored to each item. The total score range is from 13 to 91, with a higher score indicating a stronger SOC. It has been proposed that a low SOC corresponds to scores of 13–57, medium SOC corresponds to scores of 58–74, and a high SOC corresponds to scores of 75–91 [79]. The measure has been shown to be reliable and valid in multiple studies conducted across many cultures [80]. Five participants had missing values on the SOC-13, and the Cronbach’s α in this study was 0.85.
2.5. Ethical Considerations
The National Bioethics Committee in Iceland approved all procedures and materials (Qual phase: VSNb2019060009/03.01; Quan phase: VSNb2019090016/03.01), and the study was reported to the Icelandic Data Protection Authority (Qual phase: 19–119; Quan phase: 19–166). After receiving detailed information about the study, all participants voluntarily gave informed consent to participate. Written consent was obtained from the interview participants, while the survey participants provided their consent by answering the survey. All participants received a list of local referral resources with the introduction and after participation. Moreover, they were all offered support from a psychologist if difficult emotional reactions had emerged.
2.6. Data Analysis
The data analysis for this study occurred in two stages, each with several steps employed in an iterative spiraling process [49,56,70]. First, we integrated the Qual and Quan findings to assess the coherence of the scale structure. Based on those findings, the next stage focused on divergent findings at the dropped item level.
2.6.1. Mixed Methods Analysis
Coherence of the findings was determined by merging [55] Qual and Quan findings using joint display analysis [81,82]. As noted, there are four possible outcomes of comparing data and drawing conclusions about the fit: Confirmation, Complementarity, Expansion, and Discordance [55,56]. Only the expansion and discordant items were explored in the second analytic stage.
2.6.2. Examination of Expansion and Discordant Findings
The examination of the expansion or discordant dropped items was guided by reconciliation and additional validation strategies [59,60], where we re-analyzed existing data with a new perspective and evaluated these items. This analysis was driven by a critical examination of Quan data based on Qual findings. All statistical analyses were performed using the SPSS Statistics software package (Version 27.0; IBM Corp., Armonk, NY, USA, 2020), and significance was assessed at the p ≤ 0.05 level.
Level of Influence. We determined that higher mean scores were a metric indicating a stronger level of influence of items on survivors’ help-seeking. We, therefore, rank-ordered all the BHS-TR items by mean scores and examined the frequency of survivors who endorsed those items at 3 (“Influenced me somewhat”) or 4 (“Strongly influenced me”). Only items that met the threshold of a mean score above 2.00 were included. The strongly endorsed and dropped items were retained for further evaluation in the subsequent step.
Cross-Loadings. Dropped items that cross-loaded at or above 0.35 were compared with retained items to evaluate whether the dropped items were conceptually distinct. Cross-loaded items that were conceptually covered by a retained item were not included in further analysis. Conceptually distinct items were evaluated in the subsequent step.
Known-Groups Validity. We evaluated whether influential and conceptually distinct dropped items could differentiate between groups of survivors using independent sample t-tests. We created “probable depression”, “probable PTSD”, and “low SOC” subsamples by using the PHQ-8 clinical cut-off score of ≥10, the PCL-5 clinical cut-off score of ≥31, and the SOC-13 scores of 13–57. Based on the relevant literature (e.g., [21,36,43,44]) and results from prior studies on the BHS-TR [63,65], it was expected that survivors with depression, PTSD, and low SOC would, on average, score higher on the barrier items than survivors without depression or PTSD and with high SOC.
3. Results
3.1. Merging of Qualitative and Quantitative Findings
The joint displays linking the Qual and Quan findings are shown in Table 2 (Structural Barriers) and Table 3 (Internal Barriers), revealing evidence of complementarity, expansion, and discordance. To illuminate the lived experiences of the barriers, we chose exemplar quotations from the survivors, reported in the Qual column. The items referred to (using their original numbers) in the Quan column can be found in Supplementary Tables S1 and S2.
Table 2.
Joint display of the coherence of findings for structural barriers to help seeking.
Table 3.
Joint display of the coherence of findings for internal barriers to help seeking.
3.1.1. Complementarity Evidence
Most of the Qual and Quan findings were congruent, reinforced one another, were deemed complementary, and strengthened the overall validation evidence of the BHS-TR. This complementarity applied to the BHS-TR Structural and Internal Barriers indices, as well as for the following specific factors of barriers: Financial Concerns, Unavailable/Not Helpful, External Constraints, Problem Management Beliefs, Frozen/Confused, and Shame. Therefore, these items were not evaluated in any subsequent analyses.
3.1.2. Expansion Evidence
Two factors (Inconvenience and Discrimination) and one category (Reveals Weakness) were found to need expansion. First, the Inconvenience factor revealed in the Quan phase was missing a key inconvenience item identified in the Qual phase. That factor comprised of only two items, likely affecting the poor internal consistency, and needs to be expanded. Secondly, when combined, the Qual and Quan findings for the Discrimination factor indicates the need for further development and evaluation of these items in a more diverse sample. Moreover, the expansion should involve additional items specifically on the prejudice and stigmatization of GBV survivors. Thirdly, the Reveals Weakness category expanded into the Weakness/Vulnerability factor but was missing a vulnerability barrier that had been central in the survivors’ narratives. This new BHS-TR factor deserves continuing attention and better understanding of why the one dropped weakness item did not work on the factor.
3.1.3. Discordance Evidence
One factor (Mistrust/Rejection) and one category (Safeguard Yourself) were revealed to be discordant. First, despite the Qual evidence of relevance, face validity, and content validity, the Quan phase showed that all the items on the Mistrust/Rejection factor should be dropped. The concept of mistrust for BHS-TR requires additional examination. Secondly, despite the support for three new items from the Qual phase in the Safeguard Yourself category, two of them were dropped in the Quan phase. The concept of safeguarding yourself requires additional examination.
3.1.4. Integration Implications from Joint Display
Taken together, ten items that were dropped in the factor analysis belong to an expansion or discordant factor or category and need further analysis. Expansion is required for one inconvenience item (#9), all the discrimination items (#20, 21, and 23), and one weakness item (#38). Discordance was identified for all the mistrust items (#31, 32, and 33) and two safeguard items (#36 and 37). These items were focused on in the second analysis stage.
3.2. Examination of Expansion and Discordant Findings
3.2.1. Level of Influence
The BHS-TR items above 2.00 were ranked by mean score (see Table 4). Of the ten expansion and discordance dropped items from the coherence joint displays, seven were above the 2.00 threshold and are indicated in bold. Of these items, one was in the top 10, and five were in the top 20. For most of these items, more than 50% of survivors reported the barrier as somewhat or strongly influencing them in not seeking help (range 35.8% to 62.8%).
Table 4.
Ranking of the items after influence (n = 137).
Integration Implications from Influence Analysis. The inconvenience item and two of the discrimination items did not reach the mean influence 2.00 threshold and were therefore omitted from further analysis in this study. Two expansion items (Weakness #38 and Discrimination #23) and five discordance items (Mistrust #31, 32, and 33 and Safeguard #36 and 37) had mean influences over 2.00 and were included in subsequent analyses.
3.2.2. Cross-Loadings
Table 5 shows cross-loadings onto BHS-TR factors for the seven influential dropped items. The specific inferences drawn from this analysis are also listed in the table. None of the items were conceptually similar to retained BHS-TR items.
Table 5.
Cross-loadings of the dropped items.
Integration Implications from Cross-Loading Analysis. This analysis revealed issues with the wording of the safeguarding yourself items. We discovered that the concepts of feeling mistrustful and frozen/confused overlap. Similarly, there was a significant overlap between the feeling of revealing weakness and shame. The analysis further supported that the expansion of the discrimination factor needs a GBV specific items. These conceptual overlaps and cultural nuances need additional data gathering and analysis to understand. Because these items were all identified as conceptually distinct, all seven were kept for the final analysis step in this study.
3.2.3. Known-Groups Validity
We used t-tests to compare the mean scores for the influential and conceptually distinct items for subgroups of probable depression, probable PTDS, and low SOC survivors (see Table 6). The mean scores for the mistrust and safeguard items were all significantly higher for the probable depression, probable PTSD, and low SOC groups. However, there were no significant differences in the mean scores for the weakness item, as all the survivors’ groups scored high on average. The mean score for the discrimination item was significantly higher for the depression group only.
Table 6.
Known-groups validity results on the item level.
Integration Implications from Known-Groups Analysis. Five out of the seven remaining items were statistically significant for the distress sub-groups and the low SOC subgroup. While the mistrust and safeguard yourself items need refinements, this analysis provided further validity evidence for these two concepts of barriers. The indications of ceiling effects for the weakness item might be related to its shame concept overlap, demonstrating the need for further evaluation. The performance of the discrimination item supports making it more specific about prejudice or discrimination related to GBV, which might enhance its ability to distinguish between different groups.
4. Discussion
This mixed-methods study presents the integration process of evaluating the structure of BHS-TR, the first trauma-specific measure on help-seeking barriers for GBV survivors in Iceland. Our study is part of a larger mixed-methods validation study [64,65] that places equal value on the Qual and Quan phases, focusing on credibility/trustworthiness and reliability/validity. Traditionally, if mixed methods are used, instrument development and validation designs place more emphasis on the Quan phase [49,69]. The merging of the Qual and Quan findings provided more complete insights into the construct validation evidence than could be gained by either alone. A majority of our Qual and Quan findings were complementarity, telling different but congruent stories, strengthening the overall legitimation evidence of the BHS-TR.
The added value of the integration also involved identifying our expansion and discordant findings. Few mixed methods studies examine details of divergence [59,60], and in the past, conflicting evidence between Qual and Quan data have often led researchers to overlook or dismiss Qual findings [83]. It is essential to acknowledge and respect the value of divergence as a possible generative of unanticipated insights [84]. Our survivor-centered approach applying an iterative spiraling process helped illuminate the core issues and inform potential refinements. At the end of this spiraling analysis, we identified seven dropped BHS-TR items that deserved further attention, representing barriers that had strongly influenced the survivors from seeking help yet are not covered by the scale. These items belonged to four critical factors/categories of barriers (Mistrust/Rejection, Safeguard Yourself, Reveals Weakness, and Discrimination) that still require work to accurately capture what they are supposed to be measuring and become BHS-TR subscales.
The findings showed that the mistrust items were conceptually overlapping with items related to feeling frozen and confused, indicating the need to rewrite these critical items about the notion that people would not understand them or could not be trusted to help them. These barriers are often impacted by past encounters and are therefore central to the survivors’ help-seeking experience [21,30,41]. Harmful responses (e.g., disbelieving, pathologizing, or blaming survivors) from the helpers and helping services are likely to prevent additional help-seeking [35,41,85,86]. Further, the need to expand the discrimination factor to capture the stigma and prejudice associated with GBV were discovered in this study, consistent with previous studies demonstrating that fear of stigmatization is a major barrier among survivors [30,32]. For example, the IPV Stigmatization Model [40] describes how three components of stigma, cultural, internalized, and anticipated, hinder help-seeking behaviors.
The perception of harmful responses from helpers was also related to our findings on the safeguarding items, highlighting the importance of trauma-informed responses and services. Wanting to protect oneself from further pain, facing one’s experience, and being re-traumatized were significant deterrents to help-seeking for the participants in this study. A recent systematic review among trauma survivors shows similar results [85]. However, we detected potential issues involving the items being either written too broadly or too specific. Additional analysis will reveal if the reworded items load, along with the third safeguard item currently belonging to the Weakness/Vulnerability subscale, onto a new Safeguarding Efforts subscale.
The weakness item was the most hindering barrier of the dropped items and was in the top 10 influence rank order for all BHS-TR items. We did not detect a significant problem with the wording. Yet, this item loaded onto the Weakness/Vulnerability and Shame factors, which are related but still conceptually distinct, at about the same strength. What might be causing this is how the item is more related to feeling weak by opening up while the other items focus on being seen as weak by others. Interestingly, this item performed worst in the final validity testing step, as it could not differentiate between any of the groups. On average, all the survivors scored high regardless of distress symptoms or their SOC. That might be related to how intertwined beliefs about staying strong and not revealing weakness are to Icelandic culture. As with many other individualistic countries, value is placed on independence and self-reliance [87,88,89]. Additionally, a new study examining cultural effects on trauma recovery processes of GBV survivors showed that fearing vulnerability and being perceived as weak were the most significant barriers among the American participants [90].
While the findings of this study are essential, they need to be considered alongside some limitations, such as the relatively small Qual and Quan samples selected using non-probability sampling methods. Probability sampling was unfeasible, as is true for many other studies among hidden and vulnerable populations [91,92]. Moreover, with the Qual data collection occurring before the Quan one, using identical or nested samples [71] was not possible, which can be a legitimation threat to the meta-inferences generated [57]. Still, legitimation strategies were applied to minimize these threats. Our samples were drawn from the same underlying population and comparable to most demographics collected. We had less information on the Quan sample, which is a limitation, and a noticeable difference between the samples was concerning age and prevalence of receiving mental healthcare. In addition, the Qual data collection was conducted before the COVID-19 pandemic, while the Quan data collection occurred in the first months of the pandemic in Iceland. The study’s cross-sectional design furthermore did not allow for longitudinal analysis of the relationships between the survivors’ distress symptoms, SOC, help-seeking barriers, and trauma recovery actions.
Moreover, we used a largely Quan approach to examine the divergent findings because we were trying to understand the issues revealed in the smaller Qual dataset in the larger Quan sample, allowing us to potentially generalize our findings. While balanced Quan and Qual sample sizes would be desirable, mixed methods research often has smaller Qual samples. What the mixing methods does, however, as demonstrated in this study, is use the findings from each to validate or legitimate the findings from the other. Our previous work had provided strong Qual evidence for the significance of these barriers [21,64,65], and therefore we decided to focus on the Quan data in this stage. The level of influence step was performed to gain information on the significance of the barriers from a Quan lens. As noted, these Quan data are cross-sectional, providing a point-in-time estimate, and additional Qual data can give us a story of a journey for deeper understanding. Research examining the help-seeking journey of Icelandic survivors is underway. Furthermore, it needs to be acknowledged that “mainstream” responses do not account for individual experiences, and the voices of those who have significantly different experiences of life to most of the population may be lost.
Finally, the work presented here illustrates that the factor analysis of requiring items to load only onto one factor in order to be considered “valid” may sometimes be troubling, promoting an overly reductionist approach that can oversimplify the complexities of social phenomena. A person’s view of the world might not align with the way instruments are developed, and we believe that the use of mixed methods, especially the legitimation strategy of integration illustrated here, can help move the field toward the critical examination of including participants’ voices in our instrument development and validation.
Future work on the construct validation of the Icelandic BHS-TR should further examine and develop the critical expansion and discordant items identified in this study. Additional interviewing with survivors to gain in-depth insights into the conceptual overlaps and cultural nuances of these barriers are needed to inform BHS-TR refinements further. In addition, assessing floor-and-ceiling effects for all items, examining the BHS-TR structure using confirmatory factor analysis, and evaluating response consistency via test–retest reliability measurement using large and representative samples is required. Finally, future studies with and about the BHS-TR would be greatly enhanced by utilizing recruitment procedures targeting diverse groups of survivors, including gender identity, ethnic background, socioeconomic status, forms of GBV, and help-seeking attempts or experiences. We hope the BHS-TR can serve as a valuable tool in future help-seeking research among survivors of IPV and other types of GBV. Along with creating the Icelandic BHS-TR, our research group, the Multicultural Study of Trauma Recovery network (https://mistory-traumarecovery.org/, accessed on 22 March 2022), is currently working on adapting the BHS-TR to various cultures and contexts.
5. Conclusions
The current study focused on mixed methods integration to inform validation evidence for a new help-seeking barriers instrument centered on trauma recovery. The overall findings indicated that the BHS-TR is a trustworthy and valid measure but deserves continuing attention for refinements. Expanding our measure based on the lived experience of survivors can help BHS-TR better capture the significant hindrances faced and the immense amount of effort survivors often take to seek help. Measuring the various barriers to seeking help for trauma recovery among survivors can move the IPV help-seeking literature into a more holistic and survivor-centered direction. This study adds to the growing literature supporting the benefits of using mixed methods for instrument development and validation. Moreover, its findings highlight the importance of giving attention to the voices of the target population throughout the validation process. Finally, the authors hope that the current study can serve as an exemplar to encourage mixed methods researchers faced with divergent findings to embrace the possibility of expanding insights from additional analyses and development of innovative follow-up strategies for a deeper, more nuanced understanding of complex phenomena.
Supplementary Materials
The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijerph19074297/s1, Table S1: The Barriers to Help-Seeking for Trauma (BHS-TR) scale; Table S2: Items removed from the BHS-TR scale.
Author Contributions
Conceptualization, K.B.T. and D.M.S.A.; methodology, K.B.T. and D.M.S.A.; validation, K.B.T., S.H. and D.M.S.A.; formal analysis, K.B.T.; writing—original draft preparation, K.B.T.; writing—review and editing, K.B.T., S.H. and D.M.S.A.; visualization, K.B.T.; supervision, S.H. and D.M.S.A.; project administration, S.H.; funding acquisition, K.B.T., S.H. and D.M.S.A. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by the Icelandic Gender Equality Fund (No. 190036-0551 and No. 200197-5501), the Fulbright Commission, the University of Akureyri Research Fund (No. R2219), and the Institute for Research on Women and Gender, University of Michigan.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the National Bioethics Committee in Iceland (VSNb2019060009/03.01, 25 June 2019; and VSNb2019090016/03.01, 19 November 2019).
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data presented in this study are available on reasonable request from the corresponding author. The data are not publicly available due to ethical and privacy reasons.
Acknowledgments
The authors would like to express their sincere thanks to the women participating in this study for sharing their experiences and perspectives. We gratefully acknowledge the valuable contributions of Rhonda M. Johnson and Sigrún Sigurðardóttir for providing their constructive feedback and support. We also thank our MiStory colleagues and colleagues at the University of Michigan Mixed Methods Program for their valuable insights.
Conflicts of Interest
The authors declare no conflict of interest. The funders had no role in the study’s design, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.
References
- World Health Organization. Violence against Women Prevalence Estimates, 2018: Global, Regional and National Prevalence Estimates for Intimate Partner Violence against Women and Global and Regional Prevalence Estimates for Non-Partner Sexual Violence against Women; WHO: Geneva, Switzerland, 2021; ISBN 978-92-4-002225-6. [Google Scholar]
- European Institute for Gender Equality. What is Gender-Based Violence? Available online: https://eige.europa.eu/gender-based-violence/what-is-gender-based-violence (accessed on 22 February 2022).
- UN Women. Gender Equality—Women’s Rights in Review 25 Years after Beijing; UN Women: New York, NY, USA, 2020; ISBN 978-92-1-127072-3. [Google Scholar]
- World Health Organization. Understanding and Addressing Violence against Women: Intimate Partner Violence; WHO: Geneva, Switzerland, 2012. [Google Scholar]
- European Institute for Gender Equality. Gender Equality Index 2021: Health; Publications Office of the European Union: Luxembourg, 2021; ISBN 978-92-9482-941-2. [Google Scholar]
- United Nations. Human Development Report 2020—The Next Frontier: Human Development and the Anthropocene; United Nations Development Programme: New York, NY, USA, 2020; ISBN 978-92-1-126442-5. [Google Scholar]
- World Economic Forum. Global Gender Gap Report 2021; World Economic Forum: Geneva, Switzerland, 2021; ISBN 978-2-940631-07-0. [Google Scholar]
- Ministry of Welfare. Male Violence against Women in Intimate Relationships in Iceland; Ministry of Welfare: Reykjavík, Iceland, 2012; ISBN 978-9979-799-48-1. [Google Scholar]
- Andersson, T.; Heimer, G.; Lucas, S. Violence and Health in Sweden: A National Prevalence Study on Exposure to Violence among Women and Men and Its Association to Health; The National Centre for Knowledge on Men’s Violence against Women: Uppsala, Sweden, 2014; ISSN 1654-7195. [Google Scholar]
- Official Statistics of Finland. Statistics on Offenses and Coercive Measures; Statistics Finland: Helsinki, Finland, 2020; ISSN 2342-9178. [Google Scholar]
- European Union Agency for Fundamental Rights. Violence against Women: An EU-Wide Survey—Main Results; Publications Office of the European Union: Luxembourg, 2014; ISBN 978-92-9239-999-3. [Google Scholar]
- Arnalds, Á.; Karlsdóttir, E. Research on Violence against Women—The Experience of Women Aged 18–80 in Iceland; Centre for Children and Family Research: Reykjavík, Iceland, 2010; ISBN 978-9979-9859-9-0. (In Icelandic) [Google Scholar]
- Lutwak, N. The Psychology of Health and Illness: The Mental Health and Physiological Effects of Intimate Partner Violence on Women. J. Psychol 2018, 152, 373–387. [Google Scholar] [CrossRef]
- Dutton, M.A.; Green, B.L.; Kaltman, S.I.; Roesch, D.M.; Zeffiro, T.A.; Krause, E.D. Intimate Partner Violence, PTSD, and Adverse Health Outcomes. J. Interpers. Violence 2006, 21, 955–968. [Google Scholar] [CrossRef] [PubMed]
- García-Moreno, C.; Pallitto, C.; Devries, K.; Stöckl, H.; Watts, C.; Abrahams, N. Global and Regional Estimates of Violence against Women: Prevalence and Health Effects of Intimate Partner Violence and Non-Partner Sexual Violence; WHO: Geneva, Switzerland, 2013; ISBN 978-92-4-156462-5. [Google Scholar]
- Bacchus, L.J.; Ranganathan, M.; Watts, C.; Devries, K. Recent Intimate Partner Violence against Women and Health: A Systematic Review and Meta-Analysis of Cohort Studies. BMJ Open 2018, 8, e019995. [Google Scholar] [CrossRef]
- Alsaker, K.; Moen, B.E.; Morken, T.; Baste, V. Intimate Partner Violence Associated with Low Quality of Life—A Cross-Sectional Study. BMC Women’s Health 2018, 18, 148. [Google Scholar] [CrossRef] [PubMed]
- Hisasue, T.; Kruse, M.; Raitanen, J.; Paavilainen, E.; Rissanen, P. Quality of Life, Psychological Distress and Violence among Women in Close Relationships: A Population-Based Study in Finland. BMC Women’s Health 2020, 20, 85. [Google Scholar] [CrossRef] [PubMed]
- Daneshvar, S.; Shafiei, M.; Basharpoor, S. Group-based Compassion-focused Therapy on Experiential Avoidance, Meaning in Life, and Sense of Coherence in Female Survivors of Intimate Partner Violence with PTSD: A Randomized Controlled Trial. J. Interpers. Violence 2020, in press. [CrossRef] [PubMed]
- Bonomi, A.E.; Thompson, R.S.; Anderson, M.; Reid, R.J.; Carrell, D.; Dimer, J.A.; Rivara, F.P. Intimate Partner Violence and Women’s Physical, Mental, and Social Functioning. Am. J. Prev. Med. 2006, 30, 458–466. [Google Scholar] [CrossRef] [PubMed]
- Saint Arnault, D.M.; O’Halloran, S. Using Mixed Methods to Understand the Healing Trajectory for Rural Irish women Years after Leaving Abuse. J. Res. Nurs. 2016, 21, 369–383. [Google Scholar] [CrossRef]
- World Health Organization. WHO Multi-Country Study on Women’s Health and Domestic Violence against Women: Initial Results on Prevalence, Health Outcomes, and Women’s Responses; WHO: Geneva, Switzerland, 2005; ISBN 92-4-159358-X. [Google Scholar]
- Cho, H.; Shamrova, D.; Han, J.; Levchenko, P. Patterns of Intimate Partner Violence Victimization and Survivors’ Help-Seeking. J. Interpers. Violence 2020, 35, 4558–4582. [Google Scholar] [CrossRef] [PubMed]
- Goodson, A.; Hayes, B.E. Help-Seeking Behaviors of Intimate Partner Violence Victims: A Cross-National Analysis in Developing Nations. J. Interpers. Violence 2021, 36, 4705–4727. [Google Scholar] [CrossRef] [PubMed]
- Lövestad, S.; Vaez, M.; Löve, J.; Hensing, G.; Krantz, G. Intimate Partner Violence, Associations with Perceived Need for Help and Health Care Utilization: A Population-Based Sample of Women in Sweden. Scand. J. Public Health 2021, 49, 268–276. [Google Scholar] [CrossRef] [PubMed]
- Bonomi, A.E.; Anderson, M.L.; Rivara, F.P.; Thompson, R.S. Health Care Utilization and Costs Associated with Physical and Nonphysical-Only Intimate Partner Violence. Health Serv. Res. 2009, 44, 1052–1067. [Google Scholar] [CrossRef]
- García-Moreno, C.; Zimmerman, C.; Morris-Gehring, A.; Heise, L.; Amin, A.; Abrahams, N.; Montoya, O.; Bhate-Deosthali, P.; Kilonzo, N.; Watts, C. Addressing Violence against Women: A Call to Action. Lancet 2015, 385, 1685–1695. [Google Scholar] [CrossRef]
- Daoud, N.; Kraun, L.; Sergienko, R.; Batat, N.; Shoham-Vardi, I.; Davidovitch, N.; Cohen, A. Patterns of Healthcare Services Utilization Associated with Intimate Partner Violence (IPV): Effects of IPV Screening and Receiving Information on Support Services in a Cohort of Perinatal Women. PLoS ONE 2020, 15, e0228088. [Google Scholar] [CrossRef] [PubMed]
- McCloskey, L.A.; Lichter, E.; Williams, C.; Gerber, M.; Wittenberg, E.; Ganz, M. Assessing Intimate Partner Violence in Health Care Settings Leads to Women’s Receipt of Interventions and Improved Health. Public Health Rep. 2006, 121, 435–444. [Google Scholar] [CrossRef] [PubMed]
- Heron, R.L.; Eisma, M.C. Barriers and Facilitators of Disclosing Domestic Violence to the Healthcare Service: A systematic review of Qualitative Research. Health Soc. Care Community 2021, 29, 612–630. [Google Scholar] [CrossRef] [PubMed]
- Palmieri, J.; Valentine, J.L. Using Trauma-Informed Care to Address Sexual Assault and Intimate Partner Violence in Primary Care. J. Nurse Pract. 2021, 17, 44–48. [Google Scholar] [CrossRef]
- Lelaurain, S.; Graziani, P.; Lo Monaco, G. Intimate Partner Violence and Help-Seeking A Systematic Review and Social Psychological Tracks for Future Research. Eur. Psychol. 2017, 22, 263–281. [Google Scholar] [CrossRef]
- Heywood, I.; Sammut, D.; Bradbury-Jones, C. A qualitative Exploration of ‘Thrivership’ among Women who Have Experienced Domestic Violence and Abuse: Development of a New Model. BMC Women’s Health 2019, 19, 106. [Google Scholar] [CrossRef] [PubMed]
- Sinko, L.; Goldner, L.; Saint Arnault, D.M. The Trauma Recovery Actions Checklist: Applying Mixed Methods to a Holistic Gender-Based Violence Recovery Actions Measure. Sexes 2021, 2, 363–377. [Google Scholar] [CrossRef]
- Liang, B.; Goodman, L.; Tummala-Narra, P.; Weintraub, S. A Theoretical Framework for Understanding Help-Seeking Processes Among Survivors of Intimate Partner Violence. Am. J. Community Psychol. 2005, 36, 71–84. [Google Scholar] [CrossRef] [PubMed]
- Schreiber, V.; Renneberg, B.; Maercker, A. Seeking Psychosocial Care After Interpersonal Violence: An Integrative Model. Violence Vict. 2009, 24, 322–336. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Saint Arnault, D.M. Defining and Theorizing about Culture: The Evolution of the Cultural Determinants of Help-Seeking, Re-vised. Nurs. Res. 2018, 67, 161–168. [Google Scholar] [CrossRef] [PubMed]
- Flasch, P.; Murray, C.E.; Crowe, A. Overcoming Abuse: A Phenomenological Investigation of the Journey to Recovery from Past Intimate Partner Violence. J. Interpers. Violence 2017, 32, 3373–3401. [Google Scholar] [CrossRef]
- Melgar Alcantud, P.; Campdepadrós-Cullell, R.; Fuentes-Pumarola, C.; Mut-Montalvà, E. ‘I Think I Will Need Help’: A systematic Review of Who Facilitates the Recovery from Gender-Based Violence and How They Do So. Health Expect. 2021, 24, 1–7. [Google Scholar] [CrossRef] [PubMed]
- Overstreet, N.M.; Quinn, D.M. The Intimate Partner Violence Stigmatization Model and Barriers to Help Seeking. Basic Appl. Soc. Psych. 2013, 35, 109–122. [Google Scholar] [CrossRef]
- Robinson, S.R.; Ravi, K.; Voth Schrag, R.J. A Systematic Review of Barriers to Formal Help Seeking for Adult Survivors of IPV in the United States, 2005–2019. Trauma Violence Abus. 2021, 22, 1279–1295. [Google Scholar] [CrossRef] [PubMed]
- Rodelli, M.; Koutra, K.; Thorvaldsdottir, K.B.; Bilgin, H.; Ratsika, N.; Testoni, I.; Saint Arnault, D.M. Conceptual Development and Content Validation of a Multicultural Instrument to Assess the Normalization of Gender-Based Violence against Women. Sex. Cult. 2022, 1–22. [Google Scholar] [CrossRef]
- Hien, D.; Ruglass, L. Interpersonal Partner Violence and Women in the United States: An Overview of Prevalence Rates, Psychiatric Correlates and Consequences and Barriers to Help Seeking. Int. J. Law Psychiatry 2009, 32, 48–55. [Google Scholar] [CrossRef][Green Version]
- Fleming, C.E.; Resick, P.A. Help-Seeking Behavior in Survivors of Intimate Partner Violence: Toward an Integrated Behavioral Model of Individual Factors. Violence Vict. 2017, 32, 195–209. [Google Scholar] [CrossRef]
- Johnson, R.B.; Onwuegbuzie, A.J.; Turner, L.A. Toward a Definition of Mixed Methods Research. J. Mix. Methods Res. 2007, 1, 112–133. [Google Scholar] [CrossRef]
- Onwuegbuzie, A.J.; Johnson, R.B. The Validity Issue in Mixed Research. Res. Sch. 2006, 13, 48–63. [Google Scholar]
- Tashakkori, A.; Teddlie, C. Handbook of Mixed Methods in Social and Behavioral Research; Sage Publications: Los Angeles, CA, USA, 2003; ISBN 0-7619-2073-0. [Google Scholar]
- Campbell, D.T.; Fiske, D.W. Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychol. Bull. 1959, 56, 81–105. [Google Scholar] [CrossRef] [PubMed]
- Onwuegbuzie, A.J.; Bustamante, R.M.; Nelson, J.A. Mixed Research as a Tool for Developing Quantitative Instruments. J. Mix. Methods Res. 2010, 4, 56–78. [Google Scholar] [CrossRef]
- Daigneault, P.; Jacob, S. Unexpected but Most Welcome: Mixed Methods for the Validation and Revision of the Participatory Evaluation Measurement Instrument. J. Mix. Methods Res. 2014, 8, 6–24. [Google Scholar] [CrossRef]
- Sankofa, N.L. Transformativist Measurement Development Methodology: A Mixed Methods Approach to Scale Construction. J. Mix. Methods Res. 2021, 1–21. [Google Scholar] [CrossRef]
- Luyt, R. A Framework for Mixing Methods in Quantitative Measurement Development, Validation, and Revision. J. Mix. Methods Res. 2012, 6, 294–316. [Google Scholar] [CrossRef]
- Koskey, K.L.K.; Sondergeld, T.A.; Stewart, V.C.; Pugh, K.J. Applying the Mixed Methods Instrument Development and Construct Validation Process: The Transformative Experience Questionnaire. J. Mix. Methods Res. 2018, 12, 95–122. [Google Scholar] [CrossRef]
- Dellinger, A.B.; Leech, N.L. Toward a Unified Validation Framework in Mixed Methods Research. J. Mix. Methods Res. 2007, 1, 309–332. [Google Scholar] [CrossRef]
- Fetters, M.D.; Curry, L.A.; Creswell, J.W. Achieving Integration in Mixed Methods Designs–Principles and Practices. Health Serv. Res. 2013, 48, 2134–2156. [Google Scholar] [CrossRef] [PubMed]
- Fetters, M.D.; Molina-Azorin, J.F. The Journal of Mixed Methods Research Starts a New Decade: The Mixed Methods Research Integration Trilogy and Its Dimensions. J. Mix. Methods Res. 2017, 11, 291–307. [Google Scholar] [CrossRef]
- Onwuegbuzie, A.J.; Johnson, R.B.; Collins, K.M.T. Assessing Legitimation in Mixed Research: A New Framework. Qual. Quant. 2011, 45, 1253–1271. [Google Scholar] [CrossRef]
- Johnson, R.B.; Christensen, L.B. Educational Research: Quantitative, Qualitative, and Mixed Approaches, 6th ed.; Sage Publications: Los Angeles, CA, USA, 2017; ISBN 978-1483391601. [Google Scholar]
- Perez, A.; Howell Smith, M.C.; Babchuk, W.; Lynch-O’Brien, L. Advancing Quality Standards in Mixed Methods Research: Extending the Legitimation Typology. J. Mix. Methods Res. 2002; under review. [Google Scholar]
- Pluye, P.; Grad, R.M.; Levine, A.; Nicolau, B. Understanding Divergence of Quantitative and Qualitative Data (or Results) in Mixed Methods Studies. Int. J. Mult. Res. Approaches 2009, 3, 58–72. [Google Scholar] [CrossRef]
- Bustamante, C. TPACK and Teachers of Spanish: Development of a Theory-Based Joint Display in a Mixed Methods Research Case Study. J. Mix. Methods Res. 2019, 13, 163–178. [Google Scholar] [CrossRef]
- Boyle, M.H.; Offord, D.R.; Campbell, D.; Catlin, G.; Goering, P.; Lin, E.; Racine, Y.A. Mental Health Supplement to the Ontario Health Survey: Methodology. Can. J. Psychiat. 1996, 41, 549–558. [Google Scholar] [CrossRef] [PubMed]
- Saint Arnault, D.M.; Zonp, Z. Understanding Help-Seeking Barriers after Gender-Based Violence: Validation of the Barriers to Help Seeking-Trauma Version (BHS-TR). Arch. Psychiatr. Nurs. 2022, 37, 1–9. [Google Scholar] [CrossRef] [PubMed]
- Thorvaldsdottir, K.B.; Halldorsdottir, S.; Johnson, R.M.; Sigurdardottir, S.; Saint Arnault, D.M. Adaptation of the Barriers to Help-Seeking for Trauma (BHS-TR) Scale: A Cross-Cultural Cognitive Interview Study with Female Intimate Partner Violence Survivors in Iceland. J. Patient-Rep. Outcomes 2021, 5, 22. [Google Scholar] [CrossRef] [PubMed]
- Thorvaldsdottir, K.B.; Halldorsdottir, S.; Saint Arnault, D.M. Understanding and Measuring Help-Seeking Barriers among Intimate Partner Violence Survivors: Mixed-Methods Validation Study of the Icelandic Barriers to Help-Seeking for Trauma (BHS-TR) Scale. Int. J. Environ. Res. Public Health 2022, 19, 104. [Google Scholar] [CrossRef]
- Wild, D.; Grove, A.; Martin, M.; Eremenco, S.; McElroy, S.; Verjee-Lorenz, A.; Erikson, P. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: Report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005, 8, 94–104. [Google Scholar] [CrossRef] [PubMed]
- Ægisdóttir, S.; Gerstein, L.H.; Çinarbaş, D.C. Methodological Issues in Cross-Cultural Counseling Research: Equivalence, Bias, and Translations. Couns. Psychol. 2008, 36, 188–219. [Google Scholar] [CrossRef]
- Boateng, G.O.; Neilands, T.B.; Frongillo, E.A.; Melgar-Quiñonez, H.R.; Young, S.L. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Front. Public Health 2018, 6, 1–18. [Google Scholar] [CrossRef] [PubMed]
- Creswell, J.W.; Plano Clark, V.L. Designing and Conducting Mixed Methods Research, 3rd ed.; Sage Publications: Los Angeles, CA, USA, 2018; ISBN 9781483344379. [Google Scholar]
- Mendlinger, S.; Cwikel, J. Spiraling Between Qualitative and Quantitative Data on Women’s Health Behaviors: A Double Helix Model for Mixed Methods. Qual. Health Res. 2008, 18, 280–293. [Google Scholar] [CrossRef] [PubMed]
- Collins, K.M.T.; Onwuegbuzie, A.J.; Jiao, Q.G. A Mixed Methods Investigation of Mixed Methods Sampling Designs in Social and Health Science Research. J. Mix. Methods Res. 2007, 1, 267–294. [Google Scholar] [CrossRef]
- Tourangeau, R. Cognitive Sciences, and Survey Methods. In Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines; Jabine, T., Straf, M., Tanur, J., Tourangeau, R., Eds.; National Academy Press: Washington, DC, USA, 1984; pp. 73–100. [Google Scholar]
- Willis, G.B. Research Synthesis the Practice of Cross-Cultural Cognitive Interviewing. Public Opin. Q. 2015, 79, 359–395. [Google Scholar] [CrossRef]
- Kroenke, K.; Strine, T.W.; Spitzer, R.L.; Williams, J.B.W.; Berry, J.T.; Mokdad, A.H. The PHQ-8 as a Measure of Current Depression in the General Population. J. Affect. Disord. 2009, 114, 163–173. [Google Scholar] [CrossRef]
- Wu, Y.; Levis, B.; Riehm, K.E.; Saadat, N.; Levis, A.W.; Azar, M.; Rice, D.B.; Boruff, J.; Cuijpers, P.; Gilbody, S.; et al. Equivalency of the Diagnostic Accuracy of the PHQ-8 and PHQ-9: A Systematic Review and Individual Participant Data Meta-Analysis. Psychol. Med. 2020, 50, 2816. [Google Scholar] [CrossRef] [PubMed]
- Blevins, C.A.; Weathers, F.W.; Davis, M.T.; Witte, T.K.; Domino, J.L. The Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): Development and Initial Psychometric Evaluation. J. Trauma. Stress 2015, 28, 489–498. [Google Scholar] [CrossRef] [PubMed]
- Bovin, M.J.; Marx, B.P.; Weathers, F.W.; Gallagher, M.W.; Rodriguez, P.; Schnurr, P.P.; Keane, T.M. Psychometric Properties of the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (PCL-5) in Veterans. Psychol. Assess. 2016, 28, 1379–1391. [Google Scholar] [CrossRef] [PubMed]
- Antonovsky, A. The Structure and Properties of the Sense of Coherence Scale. Soc. Sci. Med. 1993, 36, 725–733. [Google Scholar] [CrossRef]
- Holmefur, M.; Sundberg, K.; Wettergren, L.; Langius-Eklöf, A. Measurement Properties of the 13-Item Sense of Coherence Scale using Rasch Analysis. Qual. Life Res. 2015, 24, 1455–1463. [Google Scholar] [CrossRef]
- Eriksson, M.; Lindström, B. Validity of Antonovsky’s Sense of Coherence Scale: A Systematic Review. J. Epidemiol. Community Health 2005, 59, 460–466. [Google Scholar] [CrossRef] [PubMed]
- Guetterman, T.C.; Fetters, M.D.; Creswell, J.W. Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays. Ann. Fam. Med. 2015, 13, 554–561. [Google Scholar] [CrossRef] [PubMed]
- Haynes-Brown, T.K.; Fetters, M.D. Using Joint Display as an Analytic Process: An Illustration Using Bar Graphs Joint Displays from a Mixed Methods Study of How Beliefs Shape Secondary School Teachers’ Use of Technology. Int. J. Qual. 2021, 20, 1–14. [Google Scholar] [CrossRef]
- Patton, M.Q. Qualitative Research & Evaluation Methods, 3rd ed.; Sage Publications: Los Angeles, CA, USA, 2002; ISBN 978-0761919711. [Google Scholar]
- Greene, J.C. Mixed Methods in Social Inquiry, 1st ed.; Jossey-Bass: San Francisco, CA, USA, 2007; ISBN 978-0-7879-8382-6. [Google Scholar]
- Coates, L.; Wade, A. Langauge and Violence: Analysis of Four Discursive Operations. J. Fam. Violence 2007, 22, 511–522. [Google Scholar] [CrossRef]
- Wilson, D.; Smith, R.; Tolmie, J.; de Haan, I. Becoming Better Helpers: Rethinking Language to Move Beyond Simplistic Responses to Women Experiencing Intimate Partner Violence. Policy Q. 2015, 11, 25–31. [Google Scholar] [CrossRef][Green Version]
- Kantor, V.; Knefel, M.; Lueger-Schuster, B. Perceived Barriers and Facilitators of Mental Health Service Utilization in Adult Trauma Survivors: A Systematic Review. Clin. Psychol. Rev. 2017, 52, 52–68. [Google Scholar] [CrossRef]
- Olafsson, S. Contemporary Icelanders—Scandinavian or American? Scand. Rev. 2003, 91, 6–14. [Google Scholar]
- Andrade, L.H.; Alonso, J.; Mneimneh, Z.; Wells, J.E.; Al-Hamzawi, A.; Borges, G.; Bromet, E.; Bruffaerts, R.; de Girolamo, G.; de Graaf, R.; et al. Barriers to Mental Health Treatment: Results from the WHO World Mental Health Surveys. Psychol. Med. 2014, 44, 1303–1317. [Google Scholar] [CrossRef]
- Sinko, L.; Burns, C.J.; O’Halloran, S.; Saint Arnault, D.M. Trauma Recovery is Cultural: Understanding Shared and Different Healing Themes in Irish and American Survivors of Gender-Based Violence. J. Interpers. Violence 2021, 36, 7765–7790. [Google Scholar] [CrossRef] [PubMed]
- Shaghaghi, A.; Bhopal, R.S.; Sheikh, A. Approaches to Recruiting ‘Hard-To-Reach’ Populations into Research: A Review of the Literature. Health Promot. Perspect. 2011, 1, 86–94. [Google Scholar] [CrossRef]
- Ellard-Gray, A.; Jeffrey, N.K.; Choubak, M.; Crann, S.E. Finding the Hidden Participant: Solutions for Recruiting Hidden, Hard-to-Reach, and Vulnerable Populations. Int. J. Qual. Methods 2015, 14, 38–48. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).