Next Article in Journal
Quantifying Spatial Heterogeneity of Tumor-Infiltrating Lymphocytes to Predict Survival of Individual Cancer Patients
Next Article in Special Issue
The Communication Chain of Genetic Risk: Analyses of Narrative Data Exploring Proband–Provider and Proband–Family Communication in Hereditary Breast and Ovarian Cancer
Previous Article in Journal
Modeling a Novel Variant of Glycogenosis IXa Using a Clonal Inducible Reprogramming System to Generate “Diseased” Hepatocytes for Accurate Diagnosis
Previous Article in Special Issue
Educational Programme for Cancer Nurses in Genetics, Health Behaviors and Cancer Prevention: A Multidisciplinary Consensus Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Clinical and Psychosocial Outcomes for Women Who Received Unexpected Clinically Actionable Germline Information Identified through Research: An Exploratory Sequential Mixed-Methods Comparative Study

1
Parkville Familial Cancer Centre, Peter MacCallum Cancer Centre, Melbourne, VIC 3000, Australia
2
Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, VIC 3010, Australia
3
Kinghorn Centre for Clinical Genomics, Garvan Institute of Medical Research, Darlinghurst, NSW 2010, Australia
4
Cancer Genetics Laboratory, Peter MacCallum Cancer Centre, Melbourne, VIC 3000, Australia
5
Lifepool, Peter MacCallum Cancer Centre, Melbourne, VIC 3000, Australia
*
Author to whom correspondence should be addressed.
J. Pers. Med. 2022, 12(7), 1112; https://doi.org/10.3390/jpm12071112
Submission received: 6 May 2022 / Revised: 29 June 2022 / Accepted: 4 July 2022 / Published: 7 July 2022
(This article belongs to the Special Issue Genetic Counseling and Genetic Testing in Precision Medicine)

Abstract

:
Background Research identifying and returning clinically actionable germline variants offer a new avenue of access to genetic information. The psychosocial and clinical outcomes for women who have received this ‘genome-first care’ delivering hereditary breast and ovarian cancer risk information outside of clinical genetics services are unknown. Methods: An exploratory sequential mixed-methods case-control study compared outcomes between women who did (cases; group 1) and did not (controls; group 2) receive clinically actionable genetic information from a research cohort in Victoria, Australia. Participants completed an online survey examining cancer risk perception and worry, and group 1 also completed distress and adaptation measures. Group 1 participants subsequently completed a semi structured interview. Results: Forty-five participants (group 1) and 96 (group 2) completed the online survey, and 31 group 1 participants were interviewed. There were no demographic differences between groups 1 and 2, although more of group 1 participants had children (p = 0.03). Group 1 reported significantly higher breast cancer risk perception (p < 0.001) compared to group 2, and higher cancer worry than group 2 (p < 0.001). Some group 1 participants described how receiving their genetic information heightened their cancer risk perception and exacerbated their cancer worry while waiting for risk-reducing surgery. Group 1 participants reported a MICRA mean score of 27.4 (SD 11.8, range 9–56; possible range 0–95), and an adaptation score of 2.9 (SD = 1.1). Conclusion: There were no adverse psychological outcomes amongst women who received clinically actionable germline information through a model of ‘genome-first’ care compared to those who did not. These findings support the return of clinically actionable research results to research participants.

1. Introduction

Access to genetic information has increased as direct-to-consumer models, research initiatives, and private companies offer germline testing to anyone irrespective of their underlying risk. These pathways promote a significant departure from the model of care offered by clinical genetics services (CGS), where access is driven by personal and family history of disease (e.g., breast cancer). CGS have traditionally been the largest provider of genetic information and offer genetic testing nested within the practice of genetic counselling, where pre-and post-test appointments facilitate preparation for, and delivery of, genetic information [1]. The CGS model arose due to the risk of harm to individuals and families receiving genetic information without support [2,3]. Genetic counselling fosters informed consent, promotes empowerment and adaptation, facilitates family communication, and encourages the uptake of cancer risk management strategies [4,5,6].
In contrast to traditional CGS models, the ‘genome-first’ approach takes a more liberal approach to cancer risk assessment, beginning with genotype rather than phenotype [7,8]. For example, taking a genome-first approach to determining familial breast cancer risk means using the genomic investigation as a first-line assessment, rather than starting with personal and/or family history of breast cancer [8]. In research settings, genome-first approaches identify participants with clinically actionable germline information that is “independent of any clinical parameters” [7]. Genome-first approaches may reach far more of the population, likely before the onset of disease, and provide more efficient risk estimation than CGS models [9]. This paradigm nonetheless has implications for disease penetrance estimates, recommendations for risk management, and identification of at-risk family members. Research participants who receive germline information therefore ideally require referral to CGS for genetic counselling [7,10]. The experiences of research participants and outcomes for populations who receive genome-first care, however, are not well described and may have implications for genetic counselling practices.
The lived experiences of many women with a family history of breast cancer are a primary source of knowledge on how to live with cancer and cancer risk, and greatly influence perceptions of cancer risk and risk management [11,12,13,14,15]. An elevated perception of risk may negatively impact some women leading them to adopt inappropriate risk management strategies [12]. In contrast, many research participants may have limited or negligible lived experience of cancer, and their risk perception may differ from those who have a strong family history of disease. As a result, their responses to receiving this genetic information, decisions about cancer risk management, and family communication may be different to the population well characterised in the literature.
In Australia, clinically actionable germline pathogenic variants (PVs) identified through research are often returned to participants via a genetic counsellor-mediated pathway [16,17]. The notification process consists of (1) a letter notifying participants that information relevant to their health has been identified, and (2) follow-up by a genetic counsellor (GC) via a telephone genetic counselling (TGC) service [17]. This process allows research participants to obtain information and support enabling them to make an informed decision about whether or not they wish to receive their results. Participants who receive their research results are subsequently referred to CGS for ongoing management including confirmatory genetic testing and genetic counselling [16,17].
It is important to understand the clinical and psychosocial outcomes for individuals who receive clinically-actionable genetic information through research to inform models of care and service delivery for individuals who have not had formal pre-test genetic counselling and may not have lived experience of the condition [10]. Further, the increasing interest in population genomic testing underlines the need to develop an evidence-based policy for best practice in returning results. Recognising this, this study aimed to examine the clinical and psychosocial outcomes for women in a population-based cohort (lifepool) that returned clinically actionable research results for hereditary breast and ovarian cancer (HBOC) [18] by:
(1)
comparing women’s cancer worry after receiving or not receiving clinically actionable genetic information from lifepool as measured by the cancer worry scale; and
(2)
exploring outcomes including cancer risk perception, uptake of cancer risk management strategies, the experience of receiving and adapting to the information, and how the information is communicated to at-risk family members.

2. Materials and Methods

We used a sequential, comparative mixed method research design consisting of qualitative interviews and an online survey using case and control participant groups [19]. This approach allowed us to identify differences in key psychosocial outcomes (risk perception and cancer worry) between women who did and did not receive genetic information from lifepool, while also qualitatively exploring women’s experiences of receiving this information. Survey and interview data were analysed separately and integrated at the interpretation stage. A separate in-depth qualitative study of women’s experiences has been previously reported [20].

2.1. Setting and Sample Population

Participants were sampled from lifepool, a research cohort recruited from a population-based breast cancer screening program, described elsewhere [18,20]. At the commencement of this study, 55,000 women had been recruited to lifepool, 17,000 had provided a DNA sample for germline sequencing with a custom 11-gene HBOC panel, and 75 women had been notified that a clinically actionable germline PV had been identified in their sample. Notification occurred by a two-step pathway with participants receiving a letter from lifepool, then followed up by a genetic counsellor from the Parkville Familial Cancer Centre at the Peter MacCallum Cancer Centre who provided telephone genetic counselling [18].

2.2. Participants, Sampling, and Recruitment

Participants were recruited as cases or controls in a 1:2 ratio. Cases (group 1 participants) were eligible if they had been identified with a clinically actionable germline PV in genes BRCA1, BRCA2, PALB2, or ATM and notified of this information. Controls (group 2 participants) were also recruited from the lifepool population and were age- and education-matched to group 1. Group 2 women had provided a DNA sample to lifepool, which had been sequenced with no clinically actionable PVs identified and were not notified of this information. All over recruitment numbers of 48 cases and 96 controls were based on an 80% probability of identifying a difference at a 5% significance level (two-sided) with a moderate effect size (0.5) using the primary outcome of cancer worry. Eligible participants were invited from July 2019 to September 2020 by email, which included a URL to access online participant information and consent. Group 1 participants had been notified about the clinically actionable germline PV by lifepool between June 2014 and May 2020.

2.3. Data Collection

After providing consent participants accessed an online survey hosted using REDCap software at the Peter MacCallum Cancer Centre [21,22]. The survey for group 1 included six domains and the survey for group 2 included domains 1 and 3 (see supplementary information for the surveys).
  • Domain 1. Demographics
  • Domain 2. Genetic testing and risk management
  • Domain 3. Cancer risk perception and cancer worry
  • Domain 4. Impact of cancer risk assessment
  • Domain 5. Psychological adaptation
  • Domain 6. Family communication

2.4. Measures

Items in domains 2 and 6 were purpose designed. Affective and cognitive risk perception measures were used to assess the risk perception of developing breast and ovarian cancer (domain 3) [23,24].
The Cancer Worry Scale (CWS) is an eight-item questionnaire that measures concerns about developing cancer (again), worries about family, and future surgery. Higher CWS scores indicate higher worry. The CWS was used to assess concerns about developing breast and/or ovarian cancer (again) (domain 3) [25].
The Multidimensional Impact of Cancer Risk Assessment (MICRA) is a 25-item questionnaire with three subscales: distress, positive thinking, and uncertainty [26]. The MICRA measures sub-clinical symptoms of individuals who receive genetic test results. Higher MICRA scores indicate greater distress, positive thinking, or uncertainty, respectively. The MICRA was used to assess psychological responses after receiving genetic information from lifepool (domain 4).
The Psychological Adaptation Scale (PAS) is a 15-item questionnaire that measures adaptation to a chronic condition or disease risk. Higher PAS scores indicate greater adaptation to the condition. The PAS was used to measure the psychological adaptation of women who had received genetic information from lifepool (domain 5) [4].
For the present sample, the Cronbach’s alpha for each of these scales were: CWS = 0.80, MICRA = 0.82, and the PAS = 0.96.

2.5. Interviews

After completion of the survey, a subset of purposively sampled women from group 1 were interviewed (authors RFS and ET) via telephone or in person. Interviews were guided by a semi-structured schedule that covered experiences of notification, perceived value of and adaptation to notification, cancer risk perception, risk management, and family communication [20]. Interviews were audio recorded and transcribed verbatim. Transcripts were quality controlled, de-identified, and pseudonyms assigned before analysis.

2.6. Data Analysis

2.6.1. Quantitative

Demographics were described per group and compared between groups using chi-squared tests and independent samples t-tests. Logistic regression modelling was used to examine predictors of uptake of risk-reducing surgery by group 1. Risk perception and cancer worry were compared between groups 1 and 2 using independent samples t-tests. In the absence of evidence-based guidelines for the interpretation of between-group differences in the CWS and risk perception scales, the Cohen’s d effect size was calculated to characterise the size of observed differences (small = 0.2; medium = 0.5; and large = 0.8) [27]. Responses to the CWS scale were dichotomised into low (≤13), and high cancer worry (≥14) and groups were compared using chi-squared testing [25]. Data were analysed using Stata BE/17. p values < 0.05 were considered statistically significant.

2.6.2. Qualitative

Team-based, codebook thematic analysis used to generate qualitative findings [28,29], is described previously [20]. Recruitment ceased after reaching requisite informational power to answer the research questions. QSR International NVivo v12.6.0 software (Massachusetts, USA) supported data management and analysis. Participant quotes are used to illustrate the findings with a pseudonym, age at interview in years, and the gene within which the germline PV was identified in parentheses.

3. Results

Sixty-eight women in Group 1 were invited to participate, 10 declined, 8 were uncontactable at follow-up, and 4 agreed to interview only, resulting in 46 survey respondents. One participant was subsequently deemed ineligible due to a partial response, resulting in 45 completed surveys (response rate, 66%). Of these respondents, 31 subsequently participated in an interview (mean age 62.5 years) on average 2.3 years (range 0.3–5.1 years) after result notification. Most interview participants had a BRCA2 PV (n = 22, 71%), detailed demographics have been published [20].
One hundred and fifty-seven women in group 2 were invited and 115 consented. Twelve did not complete the survey, and a further 7 were subsequently deemed ineligible, resulting in 96 respondents (response rate 61%).

3.1. Demographics

There were no differences between groups 1 and 2 in terms of age, education, income, and relationship status. A greater proportion of group 1 had children (93.3%) compared to group 2 (79.2%) (p = 0.03). All women in group 2 were unaffected by cancer, whereas one third of group 1 had a personal cancer diagnosis (Table 1).

3.2. Uptake of Confirmatory Genetic Testing and Cancer Risk Management Strategies

Ninety-six percent of group 1 participants (43/45) had undergone confirmatory genetic testing through a CGS, and 76% (34/45) engaged in breast cancer risk management strategies (Table 2). Participants who had a risk-reducing mastectomy (RRM) were younger (mean 54.2 years, SD 9.2) than participants who had not (mean 66.9 years, SD 6.8; p < 0.001). Multivariate analyses showed no independent predictors (time since genetic testing (years), tertiary education, income, relationship status, or age) were associated with uptake of RRM. Whereas greater time since genetic testing was associated with uptake of bilateral salpingo-oophorectomy (BSO) (OR 3.7, p = 0.02, 95% CI 1.2–11.4) and uptake of tamoxifen (OR 1.8, p = 0.05, 95% CI 1.0–3.1) when controlling for tertiary education, income, relationship status, and age. There were no independent predictors for uptake of breast screening.
While nearly all women had completed confirmatory testing, some described the waiting period between being notified by lifepool and attending their clinical genetic service appointment as a distressing time.
I had to wait nine weeks. I think that was probably the hardest part for me… if people have been kind enough to give their time and their energy to research, I think you probably could expedite that process for them in a way. Rather than just let them wait in the system.
(Bridget, 54 years, BRCA2)
Further, waiting to confirm that they were at high risk of cancer prompted feelings of uncertainty and anxiety, which persisted until they received their clinically confirmed results at an FCC appointment.
Before I told anybody, I had to have a second test done, which took maybe six weeks and I think that was probably the worst time of anxiety for me, knowing, but not saying anything, because I wanted to be a hundred percent sure. That [test] actually took a lot longer and was a lot more badly organised. It was an extra month past the time that it was received by [hospital], to when I was actually advised, so I don’t know why that delay was there? I think that could have been handled better.
(Evie, 51 years, BRCA2)

3.3. Decision-Making and Motivation Regarding Cancer Risk Management

Women’s attitudes and subsequent decision making with regards to risk management were informed by their risk perception, attitudes towards the efficacy and burden of screening, as well as their perception of disease burden.
Women’s attitudes towards risk-reducing surgery for breast cancer were more variable than their attitudes towards risk-reducing surgery for ovarian cancer. For some women, the idea of waiting until a ‘lump’ was identified on a scan provided enough motivation to have an RRM.
I was told I’d need to have scans and breast checks alternatively, six-monthly. Why would you do that for the rest of your life until they actually say, “Oh, yes, now we’ve found a lump”? I mean … that’s crazy to me, sitting on a time bomb going “Oh well, they’ll check me every six months and one day they might find a lump”. You’d say “why didn’t I just get that done at the start?” To me, the choice of having [breast] surgery was a no-brainer.
(Evie, 51 years, BRCA2)
Whereas most other women described breast cancer as more manageable due to being readily identifiable through screening or self-examination and “unlikely to kill me” (Elise, 55 years, BRCA2). This attitude is supported by the high proportion of women engaged in high-risk breast screening.
In contrast, nearly all women perceived their ovarian cancer risk as more threatening, or as Louise (56 years, BRCA1) described, “a time bomb just waiting to go off”. Similarly, Theresa described:
[The doctor’s] summary was, “Well Theresa, you’d be pissed if you didn’t do anything and you got ovarian cancer, wouldn’t you?” And I thought, “That sums it up, doesn’t it?” It’s been brought to my attention and if removing the ovaries is a 100% certainty that you won’t get it… I’ve got an appointment in three weeks to see the gynaecological cancer specialist person about my ovaries.
(Theresa, 59 years, BRCA2)
Participants felt that undergoing RRM was “pretty drastic” (Elise, 55 years, BRCA2) compared to undergoing BSO, which was described as “non-invasive” (Liz, 60 years, BRCA2). Elise exemplified this logic when carefully weighing up her breast and ovarian cancer risk management options:
I looked into [RRM] and thought it was pretty drastic and then the next option was the drug, the chemoprevention and so I looked into that as well and I weighed up the risks and decided not to do that. But I decided, yes, I would have my ovaries removed, because ovarian cancer is very hard to detect, whereas breast cancer is a slow growing and you can feel if there’s a lump, you can have a mammogram, and also, I was post-menopause. I didn’t really need my ovaries.
(Elise, 55 years, BRCA2)
With over half of the women with a BRCA1/2 PV undergoing BSO, their pragmatism towards the BSO was evident in comments like: “They [ovaries] did their job and yeah. They had to go.” (Bridget, 54 years, BRCA2). Many were also menopausal or post-menopausal, which further justified their decision.
Well, I was at that age [where] my ovaries were probably not working that well anyway because I was 55 … I’d started menopause so it was no big deal.
(Allison, 60 years, BRCA2)

3.4. Psychosocial Outcomes of Receiving Clinically Actionable Genetic Information

3.4.1. Breast Cancer Risk Perception after Notification

Group 1 had significantly higher affective (p < 0.001) and cognitive (p < 0.001) breast cancer risk perception compared to group 2 (Table 3).

3.4.2. Ovarian Cancer Risk Perception after Notification

Women who had received BRCA1/2 genetic information from lifepool had significantly higher affective ovarian cancer risk perception compared to group 2 (p = 0.0001), whereas there was no observable difference between groups for cognitive ovarian cancer risk perception (p = 0.1). However, when women who had a BSO were excluded, the remainder of group 1 had a significantly higher cognitive ovarian cancer risk perception (mean 35.5; SD 24.3) compared to group 2 (mean 22.7; SD 17.7; p = 0.02).

3.5. Cancer Worry after Notification

Despite recruitment being less than anticipated, a significant difference was identified in the CWS between the two groups, with group 1 having significantly higher cancer worry than group 2 (Cohen’s d = 0.64; p < 0.001) (Table 4). Furthermore, a greater proportion of group 1 (66.7%) met or exceeded the threshold for experiencing severe levels of worry about cancer occurrence or recurrence compared to group 2 (41.7%) (p = 0.01).

3.6. Women’s Reflections of Their Cancer Risk Perception and Cancer Worry

For many of the women from group 1, the link between their cancer risk perception and worry about developing cancer was evident. Many of these women could recall the numerical, absolute risk of breast or ovarian cancer associated with having a PV and ascribed different understandings of what it meant to them personally:
The percentage of ovarian cancer [risk] is quite high if you are BRCA2 positive. It’s from 11% to 25%. So, it’s quite alarmingly high, one in four. “So, I don’t know whether I’m the one in four, you don’t know. It’s still something that worries you at the back of the mind”.
(Angela, 66 years, BRCA2)
After learning their genetic status, group 1 participants described becoming “more attuned” to their cancer risks as they had become “a real thing”, not just an abstract concept (Sharon, 75 years, BRCA2). The transition from knowing about cancer risks numerically to experiencing them subjectively was described by Sharon:
“It just made me reassess … and even though I get it, even though you know what the [cancer] risk is in numbers, when you actually experience someone going through cancer, it does actually highlight the risk and make your perception of that risk a bit more attuned, I would say. So I think that that was like, “OK I don’t want to go through that.” This is a real thing not just an academic thing”.
(Sharon, 75 years, BRCA2)
Some group 1 participants shared how their heightened cancer risk perception manifested and exacerbated their cancer worry while waiting for risk-reducing surgery.
“That plays on my mind a bit now, every twinge or any pain you get you’re like, “Oh, what if, what if, what if”, and I think I was like that pre getting my breasts removed. [It] was that every lump, everything would just play on my mind. I was always getting checked, and it was almost like a paranoia thing. But once they [my breasts] were gone, that was alleviated. I think once my ovaries are gone, a bit more of that anxiousness will be gone as well”.
(Lucy, 36 years, BRCA2)

3.7. Distress, Uncertainty, and Positive Experiences after Notification

Group 1 completed the MICRA to assess the specific impact of the disclosure of their genetic information conceptualised as distress, uncertainty, and positive experiences. Overall, these women had a total MICRA mean score of 27.4 (SD 11.8, range 9–56; possible range 0–95) (Table 5).

3.8. Adapting to Genetic Information Provided by Lifepool

Group 1 completed the PAS to assess their adaptation to receiving the genetic information (Table 6). The mean PAS score was 2.9 (SD = 1.1), with 5 being the highest possible score.

3.9. Family Communication

All bar one participant in group 1 had informed their family members about the genetic information. Of the 41 group 1 respondents who were parents, 37 (90.2%) had told all their children the genetic information (another 2 had told some children; 4.9%). Of the 41 who had siblings, 35 (85.4%) had told all their brothers and sisters (and another 2 had told some siblings; 4.8%). The next most informed groups were nieces and nephews with 26 of 39 (66.7%) respondents informing all or some.

4. Discussion

The results from this study provide early evidence of the psychological and clinical outcomes for women who attended a population mammographic screening program, enrolled in research, and subsequently received unexpected genetic information about hereditary breast and ovarian cancer risk. A strength of this study was the recruitment of a matched comparator group, enabling the comparison of psychosocial outcomes for women who received this information and women who did not [18]. The women who received this information are among the first to receive cancer genetic information outside of the traditional models of care offered by CGS that includes pre-test genetic counselling [7]. Thus, the findings present the opportunity to examine whether the women who received information about a PV from a research cohort experienced outcomes that differ in comparison to women who receive this information through this traditional model of CGS.
Cancer worry and risk perception are frequently examined psychosocial outcomes after delivery of genetic information, interventions in CGS, or participation in cancer risk management programs [30,31]. These two outcomes are often reported as being correlated and are associated with subsequent decisions to engage in cancer risk management [32,33]. The case-control design of this study demonstrated that receiving clinically actionable genetic information increased women’s cancer worry and risk perception compared to women who did not receive this information [18]. Nevertheless, there is very little difference in cancer worry scores in the lifepool population who received ‘genome-first’ care to populations who receive traditional pre-test genetic counselling and result disclosure in a CGS [30]. Eijzenga et al. (2015) tested a genetic counselling intervention and participants completed the CWS five months after receiving their genetic test results [30]. The mean CWS for the control group, who received standard genetic counselling and testing was 13.27 (95% CI 11.47–15.08), which is similar to group 1 in our study who received genetic information from lifepool. In contrast, Chirico et al. (2021) studied women who had survived breast cancer, documenting their cancer worry 12 months after their primary treatment as substantially higher at 23.4 (SD 5.5) [34]. It is likely that the recency of these women’s breast cancer diagnoses is the primary reason for this much higher worry compared to the participants in our study. It is possible that the mode of notification employed by lifepool incorporating telephone genetic counselling, followed by a referral to a CGS for genetic counselling and testing, works to mitigate any substantially detrimental experiences for participants [16]. Further, our findings illustrate how important timely and efficient access to CGSs are to confirm the research result, manage distress and promote adaptation by supporting participants’ engagement with cancer risk management strategies [10,35].
The previously unknown impact of receiving genetic information from a research study is gradually being elucidated from studies such as this and others that have returned clinically significant research results to participants. The participants in this study had a total MICRA score that was slightly higher (27.4, SD 11.8) compared to research participants in the high-risk melanoma group (24.2, SD 15.2) from Smit et al. (2018) [36]. The quantification of this impact can also be compared to patients who have received their genetic information through a clinical pathway. For example, Hamilton et al. (2021) examined the impact of receiving genetic results from a cancer gene panel for patients who had mainstream genetic testing [37]. While our participants reported slightly higher distress scores than identified by Hamilton at three weeks and three months after receipt of result, our participants had lower uncertainty compared to both time points and scored similarly for positive experiences.
Participants in our study reported that waiting to confirm if they were at high-risk of cancer was a source of distress, which prompted feelings of uncertainty. Health system responsiveness has previously been identified as a barrier to research participants taking up genomic information identified through research [38]. A model of service delivery that prioritises ‘genome-first’ care is required as research participants identified with a PV are highly likely to have this genetic information confirmed through clinical testing. Therefore, they can bypass many of the clinical processes in place for individuals with a personal/family history [7,10]. This may help mitigate distress and uncertainty upon receipt of this information.
How women adapt to this information is also under investigation. Adaptation, as defined by Biesecker et al. (2008), is “a dynamic and multi-dimensional process of coming to terms with the implications of a health threat and the outcomes of that process” [4,39]. Until the development and validation of the Psychological Adaptation Scale, no metric existed that comprehensively measured adaptation to genetic information, and therefore no cross-study comparisons were available [4]. Shapira et al. (2018) used the PAS to measure the adaptation of women with a BRCA1/2 PV finding a mean of 3.04 (SD 1.11) [40] compared to women in our study with a mean PAS of 2.9 (SD 1.1). Shapira et al. also noted that women in their study who had a risk-reducing mastectomy had higher adaptation scores compared to women who had not [40].
Overall, results from our study provide preliminary evidence that receiving unexpected genetic information through research participation can be distressing although not obviously different from receiving similar information from CGS.

Limitations

A limitation of this study is the women who attended lifepool and subsequently participated in this research could be viewed as proactive with regards to health information as they attended a population-based breast screening program, consented to research, and opted in to receive unexpected genomic information. We did not have access to women who do not attend a population-based screening program or those who are less inclined to participate in research. These individuals may have different outcomes. The views of men were also not captured.

5. Conclusions

Our findings demonstrate that the clinical and psychosocial outcomes of individuals who receive cancer risk information through ‘genome-first’ care are not dissimilar to those reported for patients who attend CGS. This alternative avenue of identifying individuals and their families in the population who have a significantly increased risk of cancer does not pose unmanageable risks due to the mechanism of notification involving telephone genetic counselling and follow up genetic counselling through a clinical service. A model of genetic counselling for ‘genome-first’ care is needed that incorporates expedited access to CGS to mitigate feelings of anxiety and uncertainty by confirming participants’ high-risk of cancer and enabling engagement with cancer risk management strategies.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jpm12071112/s1.

Author Contributions

Conceptualization, L.E.F., R.F.S., E.T. and M.-A.Y.; Data curation, L.E.F., R.F.S. and L.D.; Formal analysis, L.E.F., R.F.S., E.T. and A.P.; Funding acquisition, I.C., A.H.T., P.A.J. and M.-A.Y.; Investigation, R.F.S. and E.T.; Methodology, L.E.F., R.F.S. and M.-A.Y.; Project administration, L.E.F., R.F.S., L.D. and M.-A.Y.; Supervision, M.-A.Y.; Writing – original draft, L.E.F., R.F.S. and M.-A.Y.; Writing – review & editing, L.E.F., R.F.S., E.T., I.C., L.D., A.H.T., P.A.J. and M.-A.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by a project grant from the Australian National Health and Medical Research Council (No. APP1126679).

Institutional Review Board Statement

All procedures were approved by the Human Research Ethics Committee of the Peter MacCallum Cancer Centre (LNR/48609/PMCC-2019).

Informed Consent Statement

All participants in this study provided informed consent.

Data Availability Statement

The data presented in this study are not publicly available due to restrictions on maintaining participants’ privacy.

Acknowledgments

The authors thank all the women who participated and kindly gave their time to this study. We also acknowledge the invaluable contributions to this research made by Lyon Mascarenhas, Alexandra Lewis, and Amanda Willis.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Resta, R.; Biesecker, B.B.; Bennett, R.L.; Blum, S.; Hahn, S.E.; Strecker, M.N.; Williams, J.L. A new definition of Genetic Counseling: National Society of Genetic Counselors’ Task Force report. J. Genet. Couns. 2006, 15, 77–83. [Google Scholar] [CrossRef]
  2. Rolland, J.S. Families and genetic fate: A millennial challenge. Fam. Syst. Health 1999, 17, 123–132. [Google Scholar] [CrossRef]
  3. Tibben, A.; Vegter-van der Vlis, M.; Skraastad, M.I.; Frets, P.G.; van der Kamp, J.J.; Niermeijer, M.F.; van Ommen, G.J.; Roos, R.A.; Rooijmans, H.G.; Stronks, D.; et al. DNA-testing for Huntington’s disease in The Netherlands: A retrospective study on psychosocial effects. Am. J. Med. Genet. 1992, 44, 94–99. [Google Scholar] [CrossRef] [PubMed]
  4. Biesecker, B.B.; Erby, L.H.; Woolford, S.; Adcock, J.Y.; Cohen, J.S.; Lamb, A.; Lewis, K.V.; Truitt, M.; Turriff, A.; Reeve, B.B. Development and validation of the Psychological Adaptation Scale (PAS): Use in six studies of adaptation to a health condition or risk. Patient Educ. Couns. 2013, 93, 248–254. [Google Scholar] [CrossRef] [Green Version]
  5. McAllister, M.; Payne, K.; Macleod, R.; Nicholls, S.; Dian, D.; Davies, L. Patient empowerment in clinical genetics services. J. Health Psychol. 2008, 13, 895–905. [Google Scholar] [CrossRef]
  6. Forrest, L.; Delatycki, M.; Curnow, L.; Skene, L.; Aitken, M. Genetic Health Professionals and the Communication of Genetic Information in Families: Practice During and After a Genetic Consultation. Am. J. Med. Genet. Part A 2010, 152, 1458–1466. [Google Scholar] [CrossRef]
  7. Schwartz, M.L.B.; McCormick, C.Z.; Lazzeri, A.L.; Lindbuchler, D.M.; Hallquist, M.L.G.; Manickam, K.; Buchanan, A.H.; Rahm, A.K.; Giovanni, M.A.; Frisbie, L.; et al. A Model for Genome-First Care: Returning Secondary Genomic Findings to Participants and Their Healthcare Providers in a Large Research Cohort. Am. J. Hum. Genet. 2018, 103, 328–337. [Google Scholar] [CrossRef] [Green Version]
  8. Trainer, A.H.; Thompson, E.; James, P.A. BRCA and beyond: A genome-first approach to familial breast cancer risk assessment. Discov. Med. 2011, 12, 433–443. [Google Scholar]
  9. Mirshahi, U.L.; Kim, J.; Best, A.F.; Chen, Z.E.; Hu, Y.; Haley, J.S.; Golden, A.; Stahl, R.; Manickam, K.; Carr, A.G.; et al. A Genome-First Approach to Characterize DICER1 Pathogenic Variant Prevalence, Penetrance, and Phenotype. JAMA Netw. Open 2021, 4, e210112. [Google Scholar] [CrossRef]
  10. Schwartz, M.L.B.; Buchanan, A.H.; Hallquist, M.L.G.; Haggerty, C.M.; Sturm, A.C. Genetic counseling for patients with positive genomic screening results: Considerations for when the genetic test comes first. J. Genet. Couns. 2021, 30, 634–644. [Google Scholar] [CrossRef]
  11. D’Agincourt-Canning, L. The effect of experiential knowledge on construction of risk perception in hereditary breast/ovarian cancer. J. Genet. Couns. 2005, 14, 55–69. [Google Scholar] [CrossRef] [PubMed]
  12. Sivell, S.; Elwyn, G.; Gaff, C.L.; Clarke, A.J.; Iredale, R.; Shaw, C.; Dundon, J.; Thornton, H.; Edwards, A. How risk is perceived, constructed and interpreted by clients in clinical genetics, and the effects on decision making: Systematic review. J. Genet. Couns. 2008, 17, 30–63. [Google Scholar] [CrossRef] [PubMed]
  13. Willis, A.M.; Smith, S.K.; Meiser, B.; James, P.A.; Ballinger, M.L.; Thomas, D.M.; Yanes, T.; Young, M.-A. Influence of lived experience on risk perception among women who received a breast cancer polygenic risk score: ‘Another piece of the pie’. J. Genet. Couns. 2021, 30, 849–860. [Google Scholar] [CrossRef] [PubMed]
  14. Austin, J.C. Re-conceptualizing risk in genetic counseling: Implications for clinical practice. J. Genet. Couns. 2010, 19, 228–234. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Senay, I.; Kaphingst, K.A. Anchoring-and-adjustment bias in communication of disease risk. Med. Decis. Mak. 2009, 29, 193–201. [Google Scholar] [CrossRef]
  16. Forrest, L.E.; Young, M.A. Clinically Significant Germline Mutations in Cancer-Causing Genes Identified Through Research Studies Should Be Offered to Research Participants by Genetic Counselors. J. Clin. Oncol. 2016, 34, 898–901. [Google Scholar] [CrossRef]
  17. Willis, A.M.; Terrill, B.; Pearce, A.; McEwen, A.; Ballinger, M.L.; Young, M.-A. My Research Results: A program to facilitate return of clinically actionable genomic research findings. Eur. J. Hum. Genet. 2021, 30, 363–366. [Google Scholar] [CrossRef]
  18. Rowley, S.M.; Mascarenhas, L.; Devereux, L.; Li, N.; Amarasinghe, K.C.; Zethoven, M.; Lee, J.E.A.; Lewis, A.; Morgan, J.A.; Limb, S.; et al. Population-based genetic testing of asymptomatic women for breast and ovarian cancer susceptibility. Genet. Med. 2019, 21, 913–922. [Google Scholar] [CrossRef]
  19. O’Cathain, A.; Murphy, E.; Nicholl, J. The quality of mixed methods studies in health services research. J. Health Serv. Res. Policy 2008, 13, 92–98. [Google Scholar] [CrossRef]
  20. Forbes Shepherd, R.; Forrest, L.E.; Tutty, E.; Pearce, A.; Devereux, L.; James, P.A.; Campbell, I.G.; Trainer, A.H.; Young, M.-A. Unselected women’s experiences of receiving genetic research results for hereditary breast and ovarian cancer: A qualitative study. Genet. Test. Mol. Biomark. 2021, 25, 741–748. [Google Scholar] [CrossRef]
  21. Harris, P.A.; Taylor, R.; Minor, B.L.; Elliott, V.; Fernandez, M.; O’Neal, L.; McLeod, L.; Delacqua, G.; Delacqua, F.; Kirby, J.; et al. The REDCap consortium: Building an international community of software platform partners. J. Biomed. Inf. 2019, 95, 103208. [Google Scholar] [CrossRef] [PubMed]
  22. Harris, P.A.; Taylor, R.; Thielke, R.; Payne, J.; Gonzalez, N.; Conde, J.G. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inf. 2009, 42, 377–381. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Weinstein, N.D.; Kwitel, A.; McCaul, K.D.; Magnan, R.E.; Gerrard, M.; Gibbons, F.X. Risk perceptions: Assessment and relationship to influenza vaccination. HealthPsychol. 2007, 26, 146–151. [Google Scholar] [CrossRef] [PubMed]
  24. Gurmankin Levy, A.; Shea, J.; Williams, S.V.; Quistberg, A.; Armstrong, K. Measuring Perceptions of Breast Cancer Risk. Cancer Epidemiol. Biomark. Prev. 2006, 15, 1893. [Google Scholar] [CrossRef] [Green Version]
  25. Custers, J.A.; van den Berg, S.W.; van Laarhoven, H.W.; Bleiker, E.M.; Gielissen, M.F.; Prins, J.B. The cancer worry scale: Detecting fear of recurrence in breast cancer survivors. Cancer Nurs. 2014, 37, E44–E50. [Google Scholar] [CrossRef]
  26. Cella, D.; Hughes, C.; Peterman, A.; Chang, C.-H.; Peshkin, B.N.; Schwartz, M.D.; Wenzel, L.; Lemke, A.; Marcus, A.C.; Lerman, C. A brief assessment of concerns associated with genetic testing for cancer: The Multidimensional Impact of Cancer Risk Assessment (MICRA) questionnaire. Health Psychol. 2002, 21, 564. [Google Scholar] [CrossRef]
  27. Cohen, J. Statistical Power Analysis for the Behavioural Sciences, 2nd ed.; Lawrence Erlbaum Associates: New York, NY, USA, 1988. [Google Scholar]
  28. Braun, V.; Clarke, V.; Hayfield, N.; Terry, G. Thematic Analysis. In Handbook of Research Methods in Health Social Sciences; Liamputtong, P., Ed.; Springer: Singapore, 2019; pp. 843–860. [Google Scholar]
  29. MacQueen, K.M.; McLellan, E.; Kay, K.; Millstein, B. Codebook Development for Team-Based Qualitative Analysis. Field Methods 1998, 10, 31–36. [Google Scholar] [CrossRef]
  30. Eijzenga, W.; Bleiker, E.M.; Ausems, M.G.; Sidharta, G.N.; Van der Kolk, L.E.; Velthuizen, M.E.; Hahn, D.E.; Aaronson, N.K. Routine assessment of psychosocial problems after cancer genetic counseling: Results from a randomized controlled trial. Clin. Genet. 2015, 87, 419–427. [Google Scholar] [CrossRef]
  31. Wevers, M.R.; Ausems, M.G.; Verhoef, S.; Bleiker, E.M.; Hahn, D.E.; Brouwer, T.; Hogervorst, F.B.; van der Luijt, R.B.; van Dalen, T.; Theunissen, E.B.; et al. Does rapid genetic counseling and testing in newly diagnosed breast cancer patients cause additional psychosocial distress? results from a randomized clinical trial. Genet. Med. 2016, 18, 137–144. [Google Scholar] [CrossRef] [Green Version]
  32. Watson, M.; Duvivier, V.; Wade Walsh, M.; Ashley, S.; Davidson, J.; Papaikonomou, M.; Murday, V.; Sacks, N.; Eeles, R. Family history of breast cancer: What do women understand and recall about their genetic risk? J. Med. Genet. 1998, 35, 731–738. [Google Scholar] [CrossRef] [Green Version]
  33. Esplen, M.J.; Harrington, S.; Leung, Y.W.; Aronson, M.; Rothenmund, H.; Semotiuk, K.; Wong, J.; Gallinger, S.; Dicks, E.; McLaughlin, J. Telephone versus in-person colorectal cancer risk and screening intervention for first-degree relatives: A randomized controlled trial. Cancer 2019, 125, 2272–2282. [Google Scholar] [CrossRef] [PubMed]
  34. Chirico, A.; Vizza, D.; Valente, M.; Iacono, M.L.; Campagna, M.R.; Palombi, T.; Alivernini, F.; Lucidi, F.; Bruno, F. Assessing the fear of recurrence using the Cancer Worry Scale in a sample of Italian breast cancer survivors. Supportive Care Cancer 2022, 30, 2829–2837. [Google Scholar] [CrossRef]
  35. Athens, B.A.; Caldwell, S.L.; Umstead, K.L.; Connors, P.D.; Brenna, E.; Biesecker, B.B. A Systematic Review of Randomized Controlled Trials to Assess Outcomes of Genetic Counseling. J. Genet. Couns. 2017, 26, 902–933. [Google Scholar] [CrossRef] [Green Version]
  36. Smit, A.K.; Newson, A.J.; Best, M.; Badcock, C.A.; Butow, P.N.; Kirk, J.; Dunlop, K.; Fenton, G.; Cust, A.E. Distress, uncertainty, and positive experiences associated with receiving information on personal genomic risk of melanoma. Eur. J. Hum. Genet. 2018, 26, 1094–1100. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Hamilton, J.G.; Symecko, H.; Spielman, K.; Breen, K.; Mueller, R.; Catchings, A.; Trottier, M.; Salo-Mullen, E.E.; Shah, I.; Arutyunova, A.; et al. Uptake and acceptability of a mainstreaming model of hereditary cancer multigene panel testing among patients with ovarian, pancreatic, and prostate cancer. Genet. Med. 2021, 23, 2105–2113. [Google Scholar] [CrossRef] [PubMed]
  38. Crook, A.; Plunkett, L.; Forrest, L.E.; Hallowell, N.; Wake, S.; Alsop, K.; Gleeson, M.; Bowtell, D.; Mitchell, G.; Australian Ovarian Cancer Study Group; et al. Connecting patients, researchers and clinical genetics services: The experiences of participants in the Australian Ovarian Cancer Study (AOCS). Eur. J. Hum. Genet. 2015, 23, 152–158. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Biesecker, B.B.; Erby, L. Adaptation to living with a genetic condition or risk: A mini-review. Clin. Genet. 2008, 74, 401–407. [Google Scholar] [CrossRef] [Green Version]
  40. Shapira, R.; Turbitt, E.; Erby, L.H.; Biesecker, B.B.; Klein, W.M.; Hooker, G.W. Adaptation of couples living with a high risk of breast/ovarian cancer and the association with risk-reducing surgery. Fam. Cancer. 2018, 17, 485–493. [Google Scholar] [CrossRef]
Table 1. Demographic characteristics comparing groups 1 and 2.
Table 1. Demographic characteristics comparing groups 1 and 2.
DemographicTotalGroup 1
PV
Group 2
No PV
p Value
Age (years)n = 138n = 45n = 93
        Mean (SD)64.4 (6.7)65.2 (8.2)64.1 (5.8)0.4
        Range37–7937–7948–77
Educationn = 140n = 44n = 96
        Non-tertiary86 (61.4)26 (59.1)60 (62.5)0.7
        Tertiary54 (38.6)18 (40.9)36 (37.5)
Income (per annum)n = 140n = 44n = 96
        <$90,000AUD83 (59.3)23 (52.3)60 (62.5)0.3
        >$90,000AUD57 (40.7)21 (47.7)36 (37.5)
Relationshipn = 141n = 45n = 96
        Partnered118 (83.7)39 (86.7)79 (82.3)0.5
        Not partnered23 (16.3)6 (13.3)17 (17.7)
Childrenn = 141n = 45n = 96
        Yes118 (83.7)42 (93.3)76 (79.2)0.03 *
        No23 (16.3)3 (6.7)20 (20.8)
Pathogenic variantn = 141n = 45n = 96
        Not detected96 (68.1)0 (0)96 (100)
         ATM1 (0.7)1 (2.2)0 (0)
         BRCA12 (1.4)2 (4.4)0 (0)
         BRCA224 (17.0)24 (53.3)0 (0)
         PALB215 (10.6)15 (33.3)0 (0)
        HBOC predisposition/Cannot recall3 (2.1) a3 (6.6) a0 (0)
Mean time since notification of genetic information (years)n = 141n = 45n = 96
        Mean (SD)n/a1.7 (1.5)n/a
        Rangen/a0.2–5.1n/a
Cancer historyn = 141n = 45n = 96
        No cancer126 (89.4)30 (66.7)96 (100)
        Breast cancer9 (6.4)9 (20.0)0 (0)
        Ovarian cancer1 (0.7)1 (2.2)0 (0)
        Other cancer5 3.6)5 (11.1)0 (0)
a Two had not had confirmatory clinical genetic testing at the time of the study and one could not specifically recall her result; * Denotes statistically significant result (p < 0.05).
Table 2. Clinical outcomes after notification of genetic information.
Table 2. Clinical outcomes after notification of genetic information.
Clinical Outcomesn (%)
Confirmatory genetic testingn = 45
        Complete43 (95.6)
        Waiting for FCC appointment1 (2.2)
        Undecided1 (2.2)
Uptake of breast cancer risk management strategiesn = 45
        High-risk breast screening27 (60.0)
        Chemoprevention a10 (22.2)
        Risk-reducing mastectomy6 (13.3)
Uptake of ovarian cancer risk management strategiesn = 26 b
        Bilateral salpingo-oophorectomy14 (53.9)
a 9/10 participants taking tamoxifen reported they were also having breast screening; b Participants with a BRCA1/2 PV only.
Table 3. Comparison of breast and ovarian cancer risk perception between groups 1 and 2.
Table 3. Comparison of breast and ovarian cancer risk perception between groups 1 and 2.
Breast Cancer Risk Perception
TotalGroup 1
PV
Group 2
No PV
p Value
Affective (0–3)n = 141n = 45n = 96
Range0–30–30–3Cohen’s d = 1.13
t = 6.2 (df 139)
p < 0.001 *
Mean (SD)0.9 (0.8)1.4 (0.8)0.6 (0.6)
95% CI0.7–1.01.1–1.60.5–0.7
Cognitive (0–100)n = 141n = 45n = 96
Range0–800–800–80Cohen’s d = 0.85
t = 5.0 (df 139)
p < 0.001 *
Mean (SD)31.6 (21.6)43.8 (23.5)25.9 (18.1)
95% CI28.0–35.236.7–50.922.2–29.5
Ovarian Cancer Risk Perception
TotalGroup 1
PV
Group 2
No PV
p Value
Affective (0–3)n = 125n = 29n = 96
Range0–30–30–2Cohen’s d = 0.68
t = 4.1 (df 123)
p < 0.001 *
Mean (SD)0.6 (0.8)1.1 (1.1)0.5 (0.6)
95% CI0.5–0.80.7–1.60.4–0.6
Cognitive (0–100)n = 125n = 29n = 96
Range0–1000–1000–70Cohen’s d = 0.28
t = 1.6 (df 123)
p = 0.1
Mean (SD)24.4 (22.7)30.3 (34.3)22.7 (17.7)
95% CI20.4–28.517.2–43.319.1–26.3
* Denotes statistically significant result (p < 0.05).
Table 4. Comparison of cancer worry between groups 1 and 2.
Table 4. Comparison of cancer worry between groups 1 and 2.
TotalGroup 1
PV
Group 2
No PV
p Value
n = 141n = 45n = 96
Range8–2510–258–22<0.001 *
Mean (SD)13.6 (3.0)14.9 (3.1)13.1 (2.7)
95% CI13.2–14.113.9–15.812.5–13.6
High cancer worry70 (49.7)30 (66.7)40 (41.7)0.01 *
Low cancer worry71 (50.4)15 (33.3)56 (58.3)
* Denotes statistically significant result (p < 0.05).
Table 5. Women’s distress, uncertainty, and positive experiences of receiving clinically actionable genetic information (MICRA).
Table 5. Women’s distress, uncertainty, and positive experiences of receiving clinically actionable genetic information (MICRA).
MICRA Sub-ScaleMean (SD)Reported Range
Distress (0–30)7.2 (6.9)0–26
Uncertainty (0–45)9.8 (6.0)0–24
Positive experiences (0–20)7.3 (4.9)0–20
TOTAL (0–95)27.4 (11.8)9–56
Table 6. Women’s adaptation to receiving clinically actionable genetic information from lifepool.
Table 6. Women’s adaptation to receiving clinically actionable genetic information from lifepool.
PAS Sub-ScaleMean (SD)Reported Range
Coping efficacy (4–20)12.9 (4.4)4–20
Self-esteem (5–25)13.7 (6.0)5–25
Social integration (3–15)8.2 (4.0)3–15
Spiritual wellbeing (3–15)8.8 (4.2)3–15
Total PAS (15–75)43.6 (16.5)15–75
Adaptation score2.9 (1.1)1–5
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Forrest, L.E.; Forbes Shepherd, R.; Tutty, E.; Pearce, A.; Campbell, I.; Devereux, L.; Trainer, A.H.; James, P.A.; Young, M.-A. The Clinical and Psychosocial Outcomes for Women Who Received Unexpected Clinically Actionable Germline Information Identified through Research: An Exploratory Sequential Mixed-Methods Comparative Study. J. Pers. Med. 2022, 12, 1112. https://doi.org/10.3390/jpm12071112

AMA Style

Forrest LE, Forbes Shepherd R, Tutty E, Pearce A, Campbell I, Devereux L, Trainer AH, James PA, Young M-A. The Clinical and Psychosocial Outcomes for Women Who Received Unexpected Clinically Actionable Germline Information Identified through Research: An Exploratory Sequential Mixed-Methods Comparative Study. Journal of Personalized Medicine. 2022; 12(7):1112. https://doi.org/10.3390/jpm12071112

Chicago/Turabian Style

Forrest, Laura E., Rowan Forbes Shepherd, Erin Tutty, Angela Pearce, Ian Campbell, Lisa Devereux, Alison H. Trainer, Paul A. James, and Mary-Anne Young. 2022. "The Clinical and Psychosocial Outcomes for Women Who Received Unexpected Clinically Actionable Germline Information Identified through Research: An Exploratory Sequential Mixed-Methods Comparative Study" Journal of Personalized Medicine 12, no. 7: 1112. https://doi.org/10.3390/jpm12071112

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop