Next Article in Journal
Gene-Metabolite Interaction in the One Carbon Metabolism Pathway: Predictors of Colorectal Cancer in Multi-Ethnic Families
Previous Article in Journal
Physician-Reported Benefits and Barriers to Clinical Implementation of Genomic Medicine: A Multi-Site IGNITE-Network Survey
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Early Outcome Data Assessing Utility of a Post-Test Genomic Counseling Framework for the Scalable Delivery of Precision Health

1
Division of Human Genetics, Ohio State University Wexner Medical Center, Columbus, OH 43221, USA
2
Genomic Medicine Institute, Geisinger, Danville, PA 17821, USA
3
Coriell Institute for Medical Research, 403 Haddon Avenue, Camden, NJ 08103, USA
4
School of Communication, Ohio State University, Columbus, OH 43214, USA
5
Department of Biomedical Informatics, Center for Biostatistics, The Ohio State University, Columbus, OH 43221, USA
6
Department of Health Behavior & Health Education, University of Michigan School of Public Health, Ann Arbor, MI 48109, USA
*
Author to whom correspondence should be addressed.
J. Pers. Med. 2018, 8(3), 25; https://doi.org/10.3390/jpm8030025
Submission received: 7 May 2018 / Revised: 2 July 2018 / Accepted: 13 July 2018 / Published: 25 July 2018

Abstract

:
Information on patients’ preferences is essential to guide the development of more efficient genomic counseling service delivery models. We examined patient preferences in the context of use of a post-test genomic counseling framework on patients (n = 44) with chronic disease receiving online test reports for eight different diseases and one drug-response result. We also explored patients’ disease risk awareness, recall of test report information, and confidence in knowing what to do with their test results. Prior to the post-test genomic counseling session, all participants viewed at least one test report; 81.6% of available test reports were reviewed in total. Participants requested more phone (36) than in-person counseling sessions (8), and phone sessions were shorter (mean 29.1 min; range 12–75 min) than in-person sessions (mean 52.8 min; range 23–85 min). A total of 182 test reports were discussed over the course of 44 counseling sessions (mean 4.13, range 1–9). Thirty-six (81.8%) participants requested assessment for additional medical/family history concerns. In exploring patient experiences of disease risk awareness and recall, no significant differences were identified in comparison to those of participants (n = 199) that had received in-person post-test genomic counseling in a parent study randomized controlled trial (RCT). In summary, a novel post-test genomic counseling framework allowed for a tailored approach to counseling based on the participants’ predetermined choices.

1. Introduction

Novel approaches for effective genomic counseling are needed to meet increasing demand and to provide better efficiency. The continued integration of precision health into clinical practice will require re-evaluation of conventional approaches as more patients seek genomic counseling and testing. The conventional practice of in-person genetic counseling includes both pre-test and post-test components [1], an approach that is labor and cost-intensive [2] and impractical on a large scale [3,4]. Conventional practice is also provider driven, in that the counselors speak more than the clients [5], even though many clients may prefer to learn about their genetic risks in different ways, at different rates, and when it is convenient to them [6]. Use of alternative modes of service delivery to include genetic counseling such as by telephone or telegenetics (interactive video and a secure high-speed connection) [7,8,9,10,11] are well accepted by patients and decrease the amount of provider time [1,3,12]. These alternative modes are equivalent with respect to educating and supporting patients, attending to psychosocial issues, facilitating decision-making, and improving quality of life outcomes when compared to conventional genetic counseling [10,13]. Utilization of technology may also help facilitate patient access to services that are limited due to geographical or financial barriers [1]. Limitations to the use of telephone or telegenetics include the inability to fully assess non-verbal behaviors/cues, the need for additional support and/or access to technology (e.g., internet), the potential for dropped or interrupted communication, and the need for additional support for targeted patient populations (e.g., minority women or rural populations) [14].
As previously reported, we developed a novel post-test genomic counseling framework [15] based on qualitative research [16] and key health behavior theories [17]. Following receipt of genomic-based results, the counseling framework provides the patient with an opportunity to set the counseling agenda by selecting the specific test results they wish to discuss, specifying questions for discussion, and indicating their preference for communication modality (telephone, telegenetics or in-person). The counselor uses these patient preferences to tailor the genomic counseling session and to personalize result communication and risk reduction recommendations. Tailored visual aids and result summary reports divide areas of risk (genetic variant, family history, lifestyle) for each disease to facilitate the discussion of multiple disease risks and drug-response findings. Post-test genomic counseling session summary reports are actively routed through the electronic health record and patient facing portal to both the patient and their healthcare provider team to encourage review and follow-up (e.g., screening, preventive health behaviors).
In the demonstration project evaluated here, we focus on the application of this novel genomic counseling framework [15] on patients with chronic disease who have received online genomic-based test reports. This demonstration project had several aims. First, we examined participant preferences for: (1) communication modality (telephone, telegenetics or in-person genomic counseling), (2) the number of test reports viewed prior to counseling and requested for discussion by the participant, (3) the number of requests for counseling on additional personal medical/family history concerns beyond that included on the test reports, and (4) consultation time with the genetic counselor. Second, we explored whether use of this novel genomic counseling framework was associated with disease risk awareness, recall of test report information, and confidence in knowing what to do with test results. We compared these patient experiences to those of patients with chronic disease that had received in-person post-test genomic counseling in a parent study randomized controlled trial (RCT) [18].

2. Materials and Methods

Participants in the Coriell Personalized Medicine Collaborative (CPMC) received reports for 19 complex disease and seven drug-gene pairs through a secure web portal [19]. CPMC participants were administered surveys that collected demographic, medical and family histories, lifestyle, and medication information which were used to produce personalized risk reports. These reports present risk information as relative risk for each disease. Individual risk was based on genetic variant, family history, medical history, and health behavior risk factors and was presented in both graphical and numeric format (Figure 1). The pharmacogenomics reports provided information pertaining to predicted drug response and a corresponding interpretation. The CPMC web portal also provided text and multimedia educational materials and tools that enabled study participants to learn more about basic genetics concepts, complex disease genetics and concepts, pharmacogenomics, family history risk, relative risk and health condition-specific summaries detailing disease etiology, risk factors, treatment and available preventative or risk reducing actions. Results from primary outcomes of various trials related to the CPMC have been previously reported [16,18,20,21,22,23].
Study Design and Participants: Participants in this study were also part of a distinct ancillary study (The Ohio State University-Coriell Personalized Medicine Collaborative (OSU-CPMC) comprised of adult patients diagnosed with either hypertension or congestive heart failure. All OSU-CPMC participants received an initial batch of results pertaining to eight health conditions (coronary artery disease, type 1 diabetes, type 2 diabetes, hemochromatosis, melanoma, age-related macular degeneration, prostate cancer, and lupus) and 1 drug response report (CYP2C19 and Clopidogrel) through an online portal. Once OSU-CPMC participants completed all the required study activities, they received additional disease risk and pharmacogenomics reports on a monthly basis until all available study reports were delivered online. Results from a RCT assessing in-person post-test genomic counseling on 199 OSU-CMPC participants were published previously [18]. The in-person sessions for the RCT counselees lasted between 60–90 min [18].
Demonstration Project (DP): To assess the utility of a pre-established genomic counseling framework [15], an additional cohort of 55 patients with chronic disease was recruited to the OSU-CPMC study. Institutional review board approval was obtained (#2014H0358), and patients were recruited between November 2014 and February 2015. Adult patients (n = 61) diagnosed with either congestive heart failure or hypertension were identified as study eligible by OSU Family Medicine physicians (n = 5). Eligible patients were contacted by a trained study recruiter to participate in one of three offered group education/consent sessions. In these sessions, they were administered a PowerPoint educational presentation that covered background information on DNA, genes, and single nucleotide polymorphisms (SNPs); the genetic basis of common, complex disease and pharmacogenomics; and logistics including access to the online OSU-CPMC web portal and the composition of the online test reports. Study recruiters also explained that from qualitative research conducted on OSU-CPMC study participants [16], investigators had developed and planned to test a new genomic counseling framework as part of a demonstration project (DP).
In all, 55 (90.1%) of 61 eligible patients were recruited to the DP and completed baseline surveys (Figure 2). DP participants then received email notice of the availability for online viewing of their nine initial test reports. Participants had the option to choose whether to view each test report. When viewing a test report, participants were initially directed to an OSU-CPMC webpage containing written and video-based educational material describing the specific condition, the role of each risk factor, and approaches to prevention and treatment. Participants could also choose not to view these educational materials and proceed directly to their individual test reports.
Demonstration project participants subsequently received an email notification reminding them of the availability of free genomic counseling. This email provided a secure link to a Qualtrics survey, which was designed to elicit participant preferences for the post-test genomic counseling session (Supplementary Figure S1). The survey provided: (1) options for counseling modality (telephone, telegenetics or in-person); (2) a checklist for which of nine test reports the participant had viewed, which reports they would like to discuss, and the option to ask specific questions; and (3) an option to list additional medical/family history concerns beyond those associated with the test reports. Completion of the Qualtrics survey allowed investigators to reach out by phone/email to schedule the genomic counseling appointment.
One of two licensed genetic counselors provided genomic counseling to DP participants. These were the same genetic counselors (A.C.S., K.S.) that provided in-person counseling for the RCT [18]. Specifically, the genetic counselor utilized the participant preferences from the Qualtrics survey to tailor the genomic counseling session, personalize result communication, and risk reduction recommendations following a semi-scripted template (Supplementary Figure S2). To facilitate learning and discussion of multiple disease risks, the genomic counseling framework utilized personalized visual aids and result summary reports to divide and highlight areas of risk (genetic variant, family history, behavior/lifestyle) for each disease [15]. For phone counselees, these documents were secure-emailed 24 h prior to the genomic counseling session. In addition to discussing any test reports requested by the participant, if there was any increased risk (genetic variant, family history, behavior/lifestyle) noted in a test report, the genetic counselor discussed these risks. If the counselee had asked on the Qualtrics survey for assessment of additional personal medical/family history concerns beyond that afforded by the nine test reports, this was provided. After the post-test genomic counseling session, summary reports were actively routed to the DP participant by secure email, and their healthcare provider team through the electronic medical record (EMR), to encourage additional review and follow-up.

2.1. Patient Preferences, Patient Experience Measures and Statistical Analyses

2.1.1. Patient Preferences

Patient preferences were assessed by reviewing: (1) choice for communication modality (telephone, telegenetics or in-person counseling); (2) the number of test reports viewed, and requested for discussion; (3) the number of additional questions on personal medical/family history concerns beyond those associated with the test reports; and (4) consultation time with the genetic counselor.

2.1.2. Patient Experience Measures

Demonstration project participants completed pre- and post-test genomic counseling surveys to measure patient experiences, including recall and accuracy of their actual disease risk identified in the test report(s) they had requested for discussion. Additionally, we evaluated personal (1) disease risk awareness, (2) risk accuracy and recall of test report information, and (3) perceived confidence in knowing what to do with test results, and compared these patient experiences to those of RCT counselees that had received in-person genomic counseling and had completed the same patient experience measures [18].
Accuracy of actual risk for test reports requested for discussion: To assess whether or not a specific test report requested for discussion by the participant influenced accurate understanding of their risk, we first reviewed the number of test reports requested and then assessed risk accuracy by comparing their response on the follow-up survey for each disease (“Do you have an increased risk for any of the following conditions due to family history, genetic variant, non-genetic risk factors?”). For this binary variable (discussed vs. not discussed), we required at least 10 participants per group to perform analyses. For each disease, the follow-up risk answer was modeled with ordinal logistic regression with co-variates of gender, age, and education, and a main effect of discussed/not-discussed. Baseline data were not available for these questions. Logistic regression was used to model follow-up accuracy of risk including the co-variates baseline correctness, gender, age, education, and a main effect for discussed/not-discussed. The latter analyses were performed within risk type (variant, family history, or lifestyle/behavior). Comparison-wise raw p = 0.05 was considered the significance threshold herein.

2.2. RCT Comparisons

2.2.1. Disease risk awareness

A participant’s causal attribution of risk for each disease was assessed for each risk factor at baseline and follow-up (e.g., “How much do you think having a genetic risk variant determines whether or not a person will develop each of the following conditions?”) [24]. Five point Likert scales were used and ratings were combined across all eight diseases (e.g., macular degeneration) to generate composite scores of the overall importance a participant placed on genetic variants, family history, and lifestyle/behavior for disease risk. Cronbach’s alphas were >0.80 at baseline, respectively, for these composite items. A linear model was fit to the composite follow-up scores with co-variates for baseline composite score, gender, age, education level, and a main effect for group.
We assessed each participant’s personal awareness of risk due to family history (“Do you have an increased risk for any of the following conditions due to your family history?”) and lifestyle/behavior factors (“Do you have an increased risk for any of the following conditions due to lifestyle/behavior factors (for example, smoking, poor diet, high body mass index (BMI)?”) at baseline and at follow-up for each disease. Personal awareness of risk based on the addition of genetic variant risk for each disease was then also assessed only at follow-up (“Do you have an increased risk for any of the following conditions due to a genetic risk variant?”). Response options for these questions included: yes, no, not applicable, don’t know, do not want to answer. We compared these responses to the actual risk for each disease based on what was indicated in the test report (Supplementary Figure S1). A logistic regression model was employed to test the accuracy of risk (correct/incorrect) within each disease and risk type (lifestyle/behavior/family history), with co-variates for baseline correctness, gender, age, education level, and a main effect for group. As baseline data for the genetic variant did not exist for this dataset, it was not included as a co-variable in the analyses. To compare confidence in personal awareness of risk (know vs. do not know if at increased risk, for the genetic variant only), a 2 × 2 Fisher’s exact test (FET) was employed for comparison between DP participants and RCT counselees.

2.2.2. Perceived Risk and Recall of Risk

To assess each participant’s perceived risk of developing a particular disease, and their recall of this risk, we used a single question at follow-up only (“What do you think is your chance of developing each of the following diseases in your lifetime?” [25]). Using a 5-point Likert scale (1, certain not to happen; 5, certain to happen) for each disease, the participant’s answer was modeled with ordinal logistic regression with co-variates of gender, age, and education, and a main effect of group.

2.2.3. Confidence in knowing what to do with test results

We assessed confidence in use of multiple test results with their level of agreement with the statement: “I know what to do with my results”. To compare confidence in use of these results, the follow-up answer (1–5) was modeled with ordinal logistic regression with co-variates of gender, age, and education, and a main effect of group.

3. Results

Socio-demographics: Table 1 depicts enrollment numbers and socio-demographic information. Of 55 DP participants enrolled, two were subsequently removed from study because they failed to complete required baseline surveys. All 53 DP participants received an initial batch of nine test reports, with 44 (83%) subsequently completing the pre-session Qualtrics survey. Forty-two participants (95.5%) completed the study follow up surveys.
Patient Preferences: Of 44 DP participants, 36 (81.8%) chose to have telephone counseling (mean, 29.1 min; range 12–75 min); the remainder (18.2%) chose in-person counseling (mean, 52.8 min; range 23–85 min). No participants chose the telegenetics option. In comparison, as noted for RCT counselees, the conventional in-person counseling sessions lasted between 60–90 min. Based on analytic data obtained from the the web portal before post-test genomic counseling, all 44 DP participants viewed at least one test report, with 81.6% of the available online test reports reviewed in total; this was slightly more than RCT counselees (75.4%; Table 2).
In all, 22 (50%) DP participants requested discussion of one or more test reports (total number of report requests, 75; mean, 1.70; range 0–9; Table 3). There were a total of 182 test reports discussed over the course of the 44 post-test genomic counseling sessions (mean 4.13, range 1–9). To assess any differences between DP and RCT participants and the viewing of test reports, we found a significantly higher percentage of DP participants viewed test reports for two diseases (Type 2 diabetes p = 0.019; melanoma, p = 0.047) but did not find an association between the number of test reports viewed and the number of topics discussed in the post-test genomic counseling session (p = 0.928). We also assessed the number of participant requests pre-session, for additional questions on personal medical/family history concerns beyond those associated with the test reports. Thirty-six (81.8%) DP participants requested additional topics for discussion in the post-test counseling session. For 18, there were concerns regarding both cancer and heart disease risks; 11 for cancer and three heart disease risk only; and three were “not sure”. Besides these specific concerns, there were 12 additional disease topics discussed in the post-test genomic counseling sessions (Table 3).

3.1. Patient Experience Measures

Accuracy of risk for test reports requested for discussion: For participants who asked to discuss a particular disease test report before counseling, we sought to determine if genomic counseling altered their personal awareness of risk for these diseases compared to those who did not ask to discuss a particular disease. We found that across risk type analyses, there were suggestive associations for two diseases (Table 4a: AMD (age-related macular degeneration), p = 0.166; SLE (systemic lupus erythematosus), p = 0.082). That is, there was a trend for participants that felt they were at a higher risk for AMD or SLE to ask to discuss these diseases in counseling. We evaluated the accuracy of their risk perceptions for family history, genetic variant and non-genetic risk for any diseases the participant asked to discuss in the post-test genomic counseling session. We found significant associations for the DM2 (type 2 diabetes) genetic variant (FET p = 0.035); that is, if DM2 was discussed, the participant’s accuracy of their perceived DM2 risk due to the genetic variant tended to be correct. Notably, for the SLE genetic variant, the association was in the opposite direction than expected (p = 0.036), in that if it was discussed, the participant was less often correct (Table 4b). Given that the relative risk afforded to the genetic variant for DM2 and SLE was quite similar (RR 1.2–1.3), while the relative risk for family history was more significant and complex for SLE (RR 4.0/11.0) than for DM2 (RR 1.3), we looked at the individual pedigrees, and assessment of family history risk for both diseases, pre- and post-test genomic counseling. We found that for the 13 participants who were accurate in their recall of SLE genetic variant risk, six did not provide a family history of SLE or other autoimmune disease upon entering the study; thus, family history risk was not included in their SLE test report. In the counseling session, more in-depth discussion of SLE family history risk was performed for these 13 participants, and family history risk was subsequently modified for five participants. We compared this assessment to the seven participants who were “incorrect” for recall of their SLE variant, and the five participants who were “incorrect” for recall of their DM2 genetic variant; for these 12 participants, disease risk afforded by family history was not modified by counseling. This suggests that participant’s recall of their SLE genetic variant risk may have been complicated by the family history risk, especially as history of 11 separate autoimmune diseases (e.g., lupus, rheumatoid arthritis, Sjogren’s, vitiligo, multiple sclerosis, celiac disease, type 1 diabetes, autoimmune hyperthyroidism aka Grave’s disease, Crohn’s disease, ulcerative colitis and psoriasis) are considered within the SLE family history risk assessment.

3.2. RCT Comparisons

In exploring patient experiences of disease risk awareness and recall, no significant differences (p > 0.05) were identified in comparison to RCT counselees. The majority of DP participants (92.3%) receiving post-test genomic counseling with the new framework expressed confidence (i.e., agree, strongly agree) in knowing what to do with their test results, which was comparable to that of RCT counselees who received in-person genomic counseling (p > 0.05; Table 5).

4. Discussion

Information on patients’ preferences, risk perceptions, and informational needs are essential to the provision of personalized, yet efficient, genomic counseling. In this demonstration project, we examined patient preferences in the context of use of a novel post-test genomic counseling framework on patients with chronic disease receiving multiple online genomic-based test reports. As a result of assessing patient preferences (communication modality preference, reports viewed/requested for discussion, requests for additional medical concerns, and consultation time), we found the new framework in practice to be more patient-driven and more flexible than conventional in-person approaches. The post-test genomic counseling framework allowed for a tailored approach to counseling based on the participants’ predetermined choices. Participants read more test reports and their risk awareness, recall of test results and confidence in knowing what to do with their test results was comparable to that of RCT participants.
Our post-test genomic counseling framework was designed with a participant-driven focus in mind, with an additional intent to increase efficiency. This includes the use of online contracting (establishing the counseling agenda) pre-session with question prompts. This approach allowed the counselor to highlight and focus on what the patient wanted to discuss, versus a comprehensive review of all test results (even those for which there was no increased risk). This approach also allowed for opportunity to provide counseling and clarification of risks, as well as assess and discuss additional disease risk influences per the patients’ requests. We did not complete full 3–4 generation pedigrees during the session unless it became necessary to answer specific questions posed by the counselee. We did have baseline family history information provided pre-session by the participant through use of the OSU-CPMC web portal, and participants also had the opportunity to visit an online cancer and heart disease family history risk assessment tool [26]. This approach resulted in focused and shorter counseling sessions. The new post-test genomic counseling framework also allows for inclusion and discussion of multiple disease risk and pharmacogenomics results, as well as assessment of additional personal and family history risk concerns per the patient’s request. Much of the extra discussion centered on personal or family cancer or heart disease risks, which at times bridged into Mendelian disease risks, illustrating how this approach allows for counseling for both rare and common disease risk influences, and applicability for any type of genetic/genomic result.
A number of studies (e.g., OSU-CPMC; My46; PGen) have now explored the provision of genomic results through interactive web-based information systems for use in the research and clinical settings [19,27,28,29,30]. In general, these models of service delivery allow convenient, adaptable and scalable access to results in a confidential and secure manner. Furthermore, it allows patients or research participants to set and modify result return preferences, process results in their own time, and access self-directed education and learning tools [19,27,28,29]. Because of the numerous and varied genomic variants provided with different levels of risk (e.g., few are high risk, many are moderate or low risk) [31,32], allowing patients to access their results prior to genomic counseling gives them the opportunity to evaluate their results, formulate questions, and identify areas of interest or concern for discussion with their genetic counselor. Information technologies (e.g., laptops, hand held devices), and innovative health communication approaches, such as including increased access to on-demand information and communication with health professionals (e.g., phone, text, email or chat), engaging in social networking, and tailoring of information to the specific needs or characteristics of individuals or groups of users can also help facilitate delivery of genomic results [33]. Use of e-technologies could also provide an opportunity for education and support in a more participatory and less healthcare provider time-intensive fashion [33,34,35].
An increase in usage of conventional genetic testing and direct-to-consumer (DTC) services has created tremendous demand and genetic counselors struggle to be more efficient without sacrificing quality of care. Importantly, between 2016 and 2018, almost half of practicing genetic counselors reported an increase in patient volume [36]. The number of genetic counselors available to help patients parse through genetic test results, likewise, is not keeping pace with the demand [36,37]. Currently there are only about 4000 certified genetic counselors in the U.S. [36]. The most common model of service delivery remains in-person, although in 2018, 62% of genetic counselors reported using more than one delivery model (e.g., telephone; telegenetics) to interact with their patients [36]. The post-test genomic counseling framework was designed to be more flexible for use in a variety of genomic counseling settings and alternative methods of service delivery. The framework could also be further developed and expanded upon to incorporate any type of potentially actionable genomic test result (e.g., exome sequencing), to include rare Mendelian in addition to common multifactorial disease (e.g., polygenic risk scores). An effective post-test genomic counseling framework in the clinical setting would have significant public health implications for modifying conventional genetic counseling, with the potential to increase efficiency and ultimately reduce costs and resource limitations.
Further integration of genetic and genomic counseling services within the genomic results delivery process is essential [38,39]. Development of service delivery frameworks that are more participant-driven is also timely given the rise of large, population-wide efforts of genomic sequencing to include the National Institutes of Health All of USSM Research Program [38], the Geisinger MyCode Community Health Initiative, and other initiatives [40,41]. The ultimate goal is to provide more accurate predictions of risk for multiple diseases and medical indications, while utilizing and expanding upon online and interactive service delivery platforms so that individuals can take a more personalized, preventive and participatory approach to their health [38].
Limitations: The between study comparisons are confounded by known and potentially unknown variables such as differences in distribution of disease diagnoses as well as limited sample size. Our participants were for the most part Caucasian and well educated. Some data was self-reported (e.g., family history). Use of only two genetic counselors for the post-test genomic counseling sessions and the use of a study-specific web portal may reduce the generalizability of study findings.

Supplementary Materials

The following are available online at https://www.mdpi.com/2075-4426/8/3/25/s1. Figure S1: Qualtrics Survey; Figure S2: Post-Test Genomic Counseling Semi-Scripted Template.

Author Contributions

Conceptualization, A.C.S., T.S., L.S., J.P.M., A.E.T., J.S.R. and K.S.; Data curation, A.C.S., T.S., J.P.M. and K.S.; Formal analysis, A.C.S., T.S., L.S., S.H., J.P.M., A.E.T., J.S.R. and K.S.; Funding acquisition, K.S.; Investigation, A.C.S., T.S., L.S., S.H., J.P.M., A.E.T., J.S.R. and K.S.; Methodology, T.S., L.S., S.H., J.P.M., J.S.R. and K.S.; Writing—original draft, A.C.S., T.S. and K.S.; Writing—review & editing, A.C.S., T.S., L.S., S.H., J.P.M., A.E.T., J.S.R. and K.S.

Funding

Research was supported by the National Human Genome Research Institute of the National Institutes of Health under Award Number R21HG006575. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. This work was also supported in part by the Ohio State University Comprehensive Cancer Center. The Coriell Personalized Medicine Collaborative was funded by the William G. Rohrer Foundation, the RNR Foundation and a grant from the endowment of the Coriell Institute for Medical Research.

Acknowledgments

We thank Jordan Gilbert for her kind assistance.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Trepanier, A.M.; Allain, D.C. Models of service delivery for cancer genetic risk assessment and counseling. J. Genet. Couns. 2014, 23, 239–253. [Google Scholar] [CrossRef] [PubMed]
  2. Williams, L.; Jones, W.; Elwyn, G.; Edwards, A. Interactive patient decision aids for women facing genetic testing for familial breast cancer: A systematic web and literature review. J. Eval. Clin. Pract. 2008, 14, 70–74. [Google Scholar] [CrossRef] [PubMed]
  3. Cohen, S.A.; Huziak, R.C.; Gustafson, S.; Grubs, R.E. Analysis of advantages, limitations, and barriers of genetic counseling service delivery models. J. Genet. Couns. 2016, 25, 1010–1018. [Google Scholar] [CrossRef] [PubMed]
  4. Hughes, K.S. Genetic testing: What problem are we trying to solve? J. Clin. Oncol. 2017, 35, 3789–3791. [Google Scholar] [CrossRef] [PubMed]
  5. Meiser, B.; Irle, J.; Lobb, E.; Barlow-Stewart, K. Assessment of the content and process of genetic counseling: A critical review of empirical studies. J. Genet. Couns. 2008, 17, 434–451. [Google Scholar] [CrossRef] [PubMed]
  6. Townsend, A.; Adam, S.; Birch, P.H.; Lohn, Z.; Rousseau, F.; Friedman, J.M. I want to know what’s in Pandora’s box: Comparing stakeholder perspectives on incidental findings in clinical whole genomic sequencing. Am. J. Med. Genet. Part A 2012, 158, 2519–2525. [Google Scholar] [CrossRef] [PubMed]
  7. Cohen, S.A.; Marvin, M.L.; Riley, B.D.; Vig, H.S.; Rousseau, J.A.; Gustafson, S.L. Identification of genetic counseling service delivery models in practice: A report from the NSGC service delivery model task force. J. Genet. Couns. 2013, 22, 411–421. [Google Scholar] [CrossRef] [PubMed]
  8. Meropol, N.J.; Daly, M.B.; Vig, H.S.; Manion, F.J.; Manne, S.L.; Mazar, C.; Murphy, C.; Solarino, N.; Zubarev, V. Delivery of internet-based cancer genetic counselling services to patients’ homes: A feasibility study. J. Telemed. Telecare 2011, 17, 36–40. [Google Scholar] [CrossRef] [PubMed]
  9. Voils, C.I.; Venne, V.L.; Weidenbacher, H.; Sperber, N.; Datta, S. Comparison of telephone and televideo modes for delivery of genetic counseling: A randomized trial. J. Genet. Couns. 2018, 27, 339–348. [Google Scholar] [CrossRef] [PubMed]
  10. Kinney, A.Y.; Steffen, L.E.; Brumbach, B.H.; Kohlmann, W.; Du, R.; Lee, J.H.; Gammon, A.; Butler, K.; Buys, S.S.; Stroup, A.M.; et al. Randomized noninferiority trial of telephone delivery of BRCA1/2 genetic counseling compared with in-person counseling: 1-year follow-up. J. Clin. Oncol. 2016, 34, 2914–2924. [Google Scholar] [CrossRef] [PubMed]
  11. Christensen, K.D.; Uhlmann, W.R.; Roberts, J.S.; Linnenbringer, E.; Whitehouse, P.J.; Royal, C.D.M.; Obisesan, T.O.; Cupples, L.A.; Butson, M.B.; Fasaye, G.A.; et al. A randomized controlled trial of disclosing genetic risk information for alzheimer disease via telephone. Genet. Med. 2018, 20, 132–141. [Google Scholar] [CrossRef] [PubMed]
  12. Sie, A.S.; Spruijt, L.; van Zelst-Stams, W.A.; Mensenkamp, A.R.; Ligtenberg, M.J.; Brunner, H.G.; Prins, J.B.; Hoogerbrugge, N. High satisfaction and low distress in breast cancer patients one year after BRCA-mutation testing without prior face-to-face genetic counseling. J. Genet. Couns. 2016, 25, 504–514. [Google Scholar] [CrossRef] [PubMed]
  13. Schwartz, M.D.; Valdimarsdottir, H.B.; Peshkin, B.N.; Mandelblatt, J.; Nusbaum, R.; Huang, A.T.; Chang, Y.; Graves, K.; Isaacs, C.; Wood, M.; et al. Randomized noninferiority trial of telephone versus in-person genetic counseling for hereditary breast and ovarian cancer. J. Clin. Oncol. 2014, 32, 618–626. [Google Scholar] [CrossRef] [PubMed]
  14. Peshkin, B.N.; Kelly, S.; Nusbaum, R.H.; Similuk, M.; DeMarco, T.A.; Hooker, G.W.; Valdimarsdottir, H.B.; Forman, A.D.; Joines, J.R.; Davis, C.; et al. Patient perceptions of telephone vs. In-person BRCA1/BRCA2 genetic counseling. J. Genet. Couns. 2016, 25, 472–482. [Google Scholar] [CrossRef] [PubMed]
  15. Schmidlen, T.; Sturm, A.C.; Hovick, S.; Scheinfeldt, L.; Scott Roberts, J.; Morr, L.; McElroy, J.; Toland, A.E.; Christman, M.; O’Daniel, J.M.; et al. Operationalizing the reciprocal engagement model of genetic counseling practice: A framework for the scalable delivery of genomic counseling and testing. J. Genet. Couns. 2018. [Google Scholar] [CrossRef] [PubMed]
  16. Sweet, K.; Hovick, S.; Sturm, A.C.; Schmidlen, T.; Gordon, E.; Bernhardt, B.; Wawak, L.; Wernke, K.; McElroy, J.; Scheinfeldt, L.; et al. Counselees’ perspectives of genomic counseling following online receipt of multiple actionable complex disease and pharmacogenomic results: A qualitative research study. J. Genet. Couns. 2017, 26, 738–751. [Google Scholar] [CrossRef] [PubMed]
  17. Rimal, R.N.; Real, K. Perceived risk and efficacy beliefs as motivators of change: Use of the risk perception attitude (RPA) framework to understand health behaviors. Hum. Commun. Res. 2003, 29, 370–399. [Google Scholar] [CrossRef]
  18. Sweet, K.; Sturm, A.C.; Schmidlen, T.; McElroy, J.; Scheinfeldt, L.; Manickam, K.; Gordon, E.S.; Hovick, S.; Scott Roberts, J.; Toland, A.E.; et al. Outcomes of a randomized controlled trial of genomic counseling for patients receiving personalized and actionable complex disease reports. J. Genet. Couns. 2017, 26, 980–998. [Google Scholar] [CrossRef] [PubMed]
  19. Keller, M.A.; Gordon, E.S.; Stack, C.B.; Gharani, N.; Sill, C.J.; Schmidlen, T.J.; Mintzer, J.; Pallies, J.; Gerry, N.P.; Christman, M.F. Coriell Personalized Medicine Collaborative®: A prospective study of the utility of personalized medicine. Personal. Med. 2010, 7, 301–317. [Google Scholar] [CrossRef] [PubMed]
  20. Schmidlen, T.S.L.; Zhaoyang, R.; Kasper, R.; Sweet, K.; Gordon, E.S.; Keller, M.; Stack, C.; Gharani, N.; Daly, M.B.; Jarvis, J.; et al. Genetic knowledge among participants in the coriell personalized medicine collaborative. J. Genet. Couns. 2016, 2, 385–394. [Google Scholar] [CrossRef] [PubMed]
  21. Schmidlen, T.J.; Wawak, L.; Kasper, R.; Garcia-Espana, J.F.; Christman, M.F.; Gordon, E.S. Personalized genomic results: Analysis of informational needs. J. Genet. Couns. 2014, 23, 578–587. [Google Scholar] [CrossRef] [PubMed]
  22. Gordon, E.S.; Griffin, G.; Wawak, L.; Pang, H.; Gollust, S.E.; Bernhardt, B.A. “It’s not like judgment day”: Public understanding of and reactions to personalized genomic risk information. J. Genet. Couns. 2012, 21, 423–432. [Google Scholar] [CrossRef] [PubMed]
  23. Mukherjee, C.; Sweet, K.M.; Luzum, J.A.; Abdel-Rasoul, M.; Christman, M.F.; Kitzmiller, J.P. Clinical pharmacogenomics: Patient perspectives of pharmacogenomic testing and the incidence of actionable test results in a chronic disease cohort. Pers. Med. 2017, 14, 383–388. [Google Scholar] [CrossRef] [PubMed]
  24. O’Neill, S.C.; McBride, C.M.; Alford, S.H.; Kaphingst, K.A. Preferences for genetic and behavioral health information: The impact of risk factors and disease attributions. Ann. Behav. Med. 2010, 40, 127–137. [Google Scholar] [CrossRef] [PubMed]
  25. McBride, C.M.; Alford, S.H.; Reid, R.J.; Larson, E.B.; Baxevanis, A.D.; Brody, L.C. Characteristics of users of online personalized genomic risk assessments: Implications for physician-patient interactions. Genet. Med. 2009, 11, 582–587. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Sweet, K.; Sturm, A.C.; Rettig, A.; McElroy, J.; Agnese, D. Clinically relevant lessons from family healthlink: A cancer and coronary heart disease familial risk assessment tool. Genet. Med. 2015, 17, 493–500. [Google Scholar] [CrossRef] [PubMed]
  27. Tabor, H.K.; Jamal, S.M.; Yu, J.H.; Crouch, J.M.; Shankar, A.G.; Dent, K.M.; Anderson, N.; Miller, D.A.; Futral, B.T.; Bamshad, M.J. My46: A web-based tool for self-guided management of genomic test results in research and clinical settings. Genet. Med. 2017, 19, 467–475. [Google Scholar] [CrossRef] [PubMed]
  28. Sweet, K.; Gordon, E.S.; Sturm, A.C.; Schmidlen, T.J.; Manickam, K.; Toland, A.E.; Keller, M.A.; Stack, C.B.; García-España, J.F.; Bellafante, M.; et al. Design and implementation of a randomized controlled trial of genomic counseling for patients with chronic disease. J. Pers. Med. 2014, 4, 1–19. [Google Scholar] [CrossRef] [PubMed]
  29. Roberts, J.S.; Gornick, M.C.; Carere, D.A.; Uhlmann, W.R.; Ruffin, M.T.; Green, R.C. Direct-to-consumer genetic testing: User motivations, decision making, and perceived utility of results. Public Health Genom. 2017, 20, 36–45. [Google Scholar] [CrossRef] [PubMed]
  30. Lewis, K.L.; Umstead, K.L.; Johnston, J.J.; Miller, I.M.; Thompson, L.J.; Fishler, K.P.; Biesecker, L.G.; Biesecker, B.B. Outcomes of counseling after education about carrier results: A randomized controlled trial. Am. J. Hum. Genet. 2018, 102, 540–546. [Google Scholar] [CrossRef] [PubMed]
  31. Foster, M.W.; Mulvihill, J.J.; Sharp, R.R. Evaluating the utility of personal genomic information. Genet. Med. 2009, 11, 570–574. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Tabor, H.K.; Auer, P.L.; Jamal, S.M.; Chong, J.X.; Yu, J.H.; Gordon, A.S.; Graubert, T.A.; O’Donnell, C.J.; Rich, S.S.; Nickerson, D.A.; et al. Pathogenic variants for Mendelian and complex traits in exomes of 6,517 European and African Americans: Implications for the return of incidental results. Am. J. Hum. Genet. 2014, 95, 183–193. [Google Scholar] [CrossRef] [PubMed]
  33. Murray, E.; Burns, J.; See, T.S.; Lai, R.; Nazareth, I. Interactive health communication applications for people with chronic disease. Cochrane Database Syst. Rev. 2005, 4, CD004274. [Google Scholar] [CrossRef] [PubMed]
  34. Mishra, S.R.; Neupane, D.; Briffa, T.G.; Kallestrup, P. Mhealth plus community health worker interventions: The future research agenda. Lancet Diabetes Endocrinol. 2016, 4, 387–388. [Google Scholar] [CrossRef]
  35. Chow, C.K.; Redfern, J.; Hillis, G.S.; Thakkar, J.; Santo, K.; Hackett, M.L.; Jan, S.; Graves, N.; de Keizer, L.; Barry, T.; et al. Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: A randomized clinical trial. Jama 2015, 314, 1255–1263. [Google Scholar] [CrossRef] [PubMed]
  36. Force NSoGCDT. 2018 Professional Status Survey Executive Summary. Available online: https://www.nsgc.org/p/do/sd/sid=7525 (accessed on 7 February 2018).
  37. Pan, V.; Yashar, B.M.; Pothast, R.; Wicklund, C. Expanding the genetic counseling workforce: Program directors’ views on increasing the size of genetic counseling graduate programs. Genet. Med. 2016, 18, 842–849. [Google Scholar] [CrossRef] [PubMed]
  38. Collins, F.S.; Varmus, H. A new initiative on precision medicine. N. Engl. J. Med. 2015, 372, 793–795. [Google Scholar] [CrossRef] [PubMed]
  39. Lewis, K.L.; Hooker, G.W.; Connors, P.D.; Hyams, T.C.; Wright, M.F.; Caldwell, S.; Biesecker, L.G.; Biesecker, B.B. Participant use and communication of findings from exome sequencing: A mixed-methods study. Genet. Med. 2016, 18, 577–583. [Google Scholar] [CrossRef] [PubMed]
  40. Manolio, T.A.; Abramowicz, M.; Al-Mulla, F.; Anderson, W.; Balling, R.; Berger, A.C.; Bleyl, S.; Chakravarti, A.; Chantratita, W.; Chisholm, R.L.; et al. Global implementation of genomic medicine: We are not alone. Sci. Transl. Med. 2015, 7, 290ps13. [Google Scholar] [CrossRef] [PubMed]
  41. Carey, D.J.; Fetterolf, S.N.; Davis, F.D.; Faucett, W.A.; Kirchner, H.L.; Mirshahi, U.; Murray, M.F.; Smelser, D.T.; Gerhard, G.S.; Ledbetter, D.H. The Geisinger Mycode community health initiative: An electronic health record-linked biobank for precision medicine research. Genet. Med. 2016, 18, 906–913. [Google Scholar] [PubMed]
Figure 1. Sample CPMC (Coriell Personalized Medicine Collaborative) coronary artery disease report. Legend: Solid discs represent the participant’s relative risk and vertical cylinders depict the range of relative risk (RR) values possible for the risk variable. On-line risk reports are organized using a tabbed approach, with separate tabs for disease condition information, risk results, limitations, methods or review educational material. To ensure readability, the CPMC test report design was informed by multiple rounds of pilot testing conducted by allowing individuals with no scientific background to review report drafts and provide feedback.
Figure 1. Sample CPMC (Coriell Personalized Medicine Collaborative) coronary artery disease report. Legend: Solid discs represent the participant’s relative risk and vertical cylinders depict the range of relative risk (RR) values possible for the risk variable. On-line risk reports are organized using a tabbed approach, with separate tabs for disease condition information, risk results, limitations, methods or review educational material. To ensure readability, the CPMC test report design was informed by multiple rounds of pilot testing conducted by allowing individuals with no scientific background to review report drafts and provide feedback.
Jpm 08 00025 g001
Figure 2. Study schematic. Legend: AMD: age-related macular degeneration; CAD: coronary artery disease; DM1: type 1 diabetes; DM2: type 2 diabetes; HH: hemochromatosis; SLE: systemic lupus erythematosus; MEL: melanoma; PRO: prostate cancer.
Figure 2. Study schematic. Legend: AMD: age-related macular degeneration; CAD: coronary artery disease; DM1: type 1 diabetes; DM2: type 2 diabetes; HH: hemochromatosis; SLE: systemic lupus erythematosus; MEL: melanoma; PRO: prostate cancer.
Jpm 08 00025 g002
Table 1. Demographic information.
Table 1. Demographic information.
Demographic CategorySubject CategoryDP (n = 53)RCT (n = 75)p-ValueTest
Mean Age (sd) 58.89 (10.55)57.73 (13.58)0.59t-test
Education (mode (%)) Graduate Degree (47%)Graduate Degree (33%)0.273Fisher’s Exact Test
Race (Caucasian)Yes47671Fisher’s Exact Test
No68
GenderFemale31320.106Fisher’s Exact Test
Male2243
Income<$25k250.3406Fisher’s Exact Test
$25–50k412
$50–75k1519
$75–100k1119
>$100k2119
Did not want to answer02
DiagnosisHTN5142<0.05Fisher’s Exact Test
HF233
Heath Care OccupationYes890.609Fisher’s Exact Test
No4566
Legend: DP: demonstration project participants; HTN: hypertension; HF: heart failure; RCT: randomized controlled trial counselees; sd: standard deviation.
Table 2. Number of test risk reports viewed post-test genomic counseling.
Table 2. Number of test risk reports viewed post-test genomic counseling.
DiseaseDP (n = 44)RCT (n = 75)
AMD38 (86.4%)70 (93.4%)
CAD37 (84.1%)63 (84.2%)
DM137 (84.1%)53 (71.1%)
DM240 (91.0%)54 (72.4%)
HH34 (77.3%)58 (77.6%)
SLE36 (81.8%)53 (71.1%)
MEL38 (86.4%)52 (69.7%)
PRO30 (68.2%)44 (59.2%)
Plavix33 (75%)53 (71.1%)
TOTAL323 (81.6%)509 (75.4%)
Table 3. Number of test reports and additional concerns requested for discussion.
Table 3. Number of test reports and additional concerns requested for discussion.
ParticipantNumber of Test Reports Viewed Pre-Session by ParticipantNumber of Test Reports Requested for DiscussionTest Reports Discussed in Post-Test Counseling Cancer and/or Heart Disease ConcernsAdditional Specific Concerns
1AllDM1, SLEDM1; DM2; SLE Cancer-
2AMD, DM1, DM2, CAD, HH, MEL, PRO, SLEAMD, CAD, DM2, SLEAMD, CAD, DM2, SLEBoth -
3AllNone checkedDM2Both Family history Lynch syndrome
4AllNone checkedAMD, CAD, DM2, MEL, SLECancer Family history antiphospholipid antibody syndrome
5AMD, DM1, DM2, CAD, MEL, PRO, SLE, PlavixNone checkedAMD, CAD, SLE--
6AMD, DM1, DM2, CAD, MEL, PRO, SLE, PlavixNone checkedAMD, CAD, DM2, SLE--
7AMD, DM2, CADCADCAD, DM2, SLEBoth BRCA1 mutation carrier
8AMD, DM1, DM2, CAD, HH, MEL, SLEAMD, DM1, DM2, CAD, HH, MEL, SLE, PlavixAMD, DM1, DM2, CAD, HH, MEL, SLE, PlavixBoth -
9AllNone checkedAMD, DM2, SLECancer -
10AllAMD, DM2, CAD, MELAMD, DM2, CAD, MEL, SLEBoth-
11AMD, DM1, DM2, CAD, HH, MEL, SLE, PlavixNone checkedCAD, DM2, MEL--
12AllAMD, DM1, DM2, SLEAMD, DM1, DM2, MEL, SLEBoth -
13AllAMD, DM1, DM2, CAD, MEL, SLEAMD, DM1, DM2, CAD, MEL, SLEBoth -
14AllNone checkedAMD, CAD, DM2, SLEBoth -
15DM1, DM2, HH, MEL, SLENone checkedAMD, CAD, DM2, SLEBoth Family history thrombophilia
16AMD, SLE, PlavixAMD, SLE, PlavixAMD, DM2, MEL, SLE, PlavixNot sureFamily history congestive heart failure
17AllNone checkedAMD, CAD, DM2, SLEHeart disease -
18AllDM1, DM2, CAD, MELAMD, CAD, DM1, DM2, MEL, SLEBoth Personal history of colon polyps
19AMD, DM1, DM2, CAD, HH, MEL, PRO, SLEPlavixCAD, DM2, PRO, SLE, PlavixCancer-
20AllPRO; PlavixCAD, DM2, PRO, SLE, PlavixCancer-
21AllAMD, DM1, DM2, CAD, HH, MEL, PRO, SLE, PlavixAMD, DM1, DM2, CAD, HH, MEL, PRO, SLE, PlavixNot surePersonal history of cholesterol and hypertension
22AllCADAMD, CAD, DM2Heart disease -
23CADAMD, PROAMD, CAD, DM2, PRO--
24AMD, DM1, DM2, CAD, MEL, PRO SLE, PlavixDM1, DM2DM1, DM2-Personal history of cholesterol and hypertension
25DM2None checkedAMD, CAD, DM2, SLEHeart disease -
26AllNone checkedDM2, SLE PROBoth Family history Tetralogy of Fallot
27AMD, DM1, DM2, CAD, HH, MEL, SLE, PlavixDM2, CAD, SLECAD, DM2, MEL, SLEBoth Family history intestinal malrotation
28AMD, DM1, DM2, CADNone checkedCAD, DM2--
29DM2None checkedAMD, CAD, DM2, MEL, SLECancer -
30AllCAD, PlavixAMD, CAD, DM2, Plavix--
31AllNone checkedAMD, DM2, PROCancer-
32CAD, DM1, DM2, HH, MEL, Plavix DM1, CAD, PlavixCAD, DM1, DM2, SLE, PlavixBoth Family history kidney failure
33AMD, CAD, MEL, PRO, Plavix None checkedAMD, CADBoth Family history depression
34AllAMD, CAD, MEL, PRO, PlavixAMD, CAD, DM2, MEL, PRO, SLE, PlavixBoth -
35AMD, DM2, CAD, HH, MEL, PRO, SLE, PlavixNone checkedAMD, CAD, DM2Cancer -
36AllNone checkedAMD, CAD, DM2Cancer-
37AllAMD, DM2, MEL, PROAMD, CAD, DM2, MEL, PRO, SLEBoth -
38AllNone checkedCAD, MEL, SLE--
39AMD, DM1, DM2, CAD, HH, MEL, SLE, PlavixNone checkedAMD, CAD, DM2, SLEBoth -
40All AMD, CAD, PRO, SLE AMD, CAD, DM2, PRO, SLENot sure-
41AllNone checkedAMD, CAD, DM2, SLECancer-
42DM2, MELPlavixAMD, CAD, DM2, SLE, PlavixCancer-
43AllNone checkedCAD, DM2, MEL, PRO, SLE--
44AllNone checkedCAD, DM2Both -
TOTAL323751823612
Table 4. (a) Personal awareness of risk for any test reports the participant requested for discussion. (b) Accuracy of risk perception.
Table 4. (a) Personal awareness of risk for any test reports the participant requested for discussion. (b) Accuracy of risk perception.
(a)
DiseaseEstimateStd Errorz-Valuep-Value
AMD−0.9620.694−1.3870.166
SLE−1.2720.731−1.740.082
(b)
Disease risk factorEstimateStd Errorz-Valuep-Value
AMD variant1.3461.2691.0610.289
CAD family history5.3496.9740.7670.443
CAD variant1.701.3341.2740.202
DM2 family history−164.0240−0.00070.999
DM2 variant1.8630.8852.1050.035
SLE family history−161.2212−0.00070.999
SLE variant−3.3591.598−2.1020.036
Table 5. Confidence in knowing what to do with test results.
Table 5. Confidence in knowing what to do with test results.
SDDNASADNWAA%D%
DP111306092.3%0.05%
RCT0093514183.1%0%
DP Comparison to RCT
EstimateStd. Errorz-Valuep-Value2.5% CI97.5% CI
0.2360.4430.5320.595−0.6341.114
A: agree; CI: confidence interval; D: disagree; DNWA: did not want to answer; DP: demonstration project participants; N: neutral; RCT: RCT counselees; SA: strongly agree; SD: strongly disagree.

Share and Cite

MDPI and ACS Style

Sturm, A.C.; Schmidlen, T.; Scheinfeldt, L.; Hovick, S.; McElroy, J.P.; Toland, A.E.; Roberts, J.S.; Sweet, K. Early Outcome Data Assessing Utility of a Post-Test Genomic Counseling Framework for the Scalable Delivery of Precision Health. J. Pers. Med. 2018, 8, 25. https://doi.org/10.3390/jpm8030025

AMA Style

Sturm AC, Schmidlen T, Scheinfeldt L, Hovick S, McElroy JP, Toland AE, Roberts JS, Sweet K. Early Outcome Data Assessing Utility of a Post-Test Genomic Counseling Framework for the Scalable Delivery of Precision Health. Journal of Personalized Medicine. 2018; 8(3):25. https://doi.org/10.3390/jpm8030025

Chicago/Turabian Style

Sturm, Amy C., Tara Schmidlen, Laura Scheinfeldt, Shelly Hovick, Joseph P. McElroy, Amanda E. Toland, J. Scott Roberts, and Kevin Sweet. 2018. "Early Outcome Data Assessing Utility of a Post-Test Genomic Counseling Framework for the Scalable Delivery of Precision Health" Journal of Personalized Medicine 8, no. 3: 25. https://doi.org/10.3390/jpm8030025

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop