From Information Overload to Actionable Insights: Digital Solutions for Interpreting Cancer Variants from Genomic Testing

Abstract

Given the increase in genomic testing in routine clinical use, there is a growing need for digital technology solutions to assist pathologists, oncologists, and researchers in translating variant calls into actionable knowledge to personalize patient management plans. In this article, we discuss the challenges facing molecular geneticists and medical oncologists in working with test results from next-generation sequencing for somatic oncology, and propose key considerations for implementing a decision support software to aid the interpretation of clinically important variants. In addition, we review results from an example decision support software, NAVIFY Mutation Profiler. NAVIFY Mutation Profiler is a cloud-based software that provides curation, annotation, interpretation, and reporting of somatic variants identified by next-generation sequencing. The software reports a tiered classification based on consensus recommendations from AMP, ASCO, CAP, and ACMG. Studies with NAVIFY Mutation Profiler demonstrated that the software provided timely updates and accurate curation, as well as interpretation of variant combinations, demonstrating that decision support tools can help advance implementation of precision oncology.

1. Introduction

Precision medicine has transformed cancer care, with next-generation sequencing (NGS) enabling the simultaneous analysis of multiple genomic alterations with therapeutic implications [1]. Across community oncology practices in the United States, there has been an increasing uptake of NGS testing to cover multiple biomarkers at once [2,3]. A similar trend is also expected in Europe, where 17% of molecular diagnostic labs had an NGS machine in 2016 and another 21% planned to acquire an NGS instrument in the next 5 years [4]. Furthermore, in August 2020, the European Society for Medical Oncology (ESMO) issued the first recommendations for using NGS in advanced cancers [5], supporting routine use of NGS tests.

However, with a growing number of actionable targets, NGS panels must cover more content, which becomes even more challenging to analyze and interpret. Half of the oncologists in the United States find NGS results sometimes or often difficult to interpret [6], and 31% of researchers in Europe view a lack of knowledge, training, and exposure to routine analyses and interpretation as the most critical bottleneck in NGS testing [7]. Data interpretation is a challenge not only for oncologists, but also laboratories and pathologists. Substantial time and effort are needed for post-sequencing steps, including filtering and interpreting variants, and writing lab reports [8].

Laboratories struggle to provide consistent variant interpretation, and will classify variants differently due to the use of different databases [9]. Studies comparing precision oncology databases, such as Clinical Interpretation of Variants In Cancer (CIViC) [10], Oncology Knowledge Base (OncoKB) [11], and The Jackson Laboratory’s Clinical Knowledge Database (JAX-CKB) [12], have found little overlap in variant coverage [13,14]. Therefore, laboratories need to pull data from multiple sources to achieve a comprehensive understanding of each variant. The synthesis of multiple lines of evidence requires effort to collect, curate, and weigh different evidence items. This process is also variable between laboratories. When 20 molecular diagnostics experts were asked to tier 51 solid tumor variants, there was only 58% agreement [15]. The disagreements were potentially due to the lack of experience with the guidelines, not being accustomed to emphasizing actionability over pathobiologic effects, and differences in synthesizing evidence.

Expanding medical knowledge, evolving guidelines, fragmented data sources, and effort required to synthesize evidence all pose challenges to NGS variant interpretation. We argue that this is where decision support software can help. There is growing evidence that decision support in cancer care can improve compliance with guidelines, increase time efficiency, and enhance quality of care [16]. Given the limits in human cognitive capacity, decision support helps distill knowledge from data, and should be integrated into the clinical workflow to realize value, such as improved quality of care and reduction in medical errors [17]. The challenges in interpreting clinically significant variants from oncology NGS testing pose a unique opportunity for decision support software to streamline the interpretation and reporting process for laboratories, providing clear reports to oncologists and, ultimately, helping patients receive the best treatment.

2. Considerations for Implementing Decision Support Software

To integrate decision support software into a laboratory workflow for supporting NGS oncology data interpretation, Song and Hussain previously outlined 34 parameters to evaluate available annotation solutions [18]. We propose to further simplify the considerations into three key areas: accuracy of the content, approach to curation, and usability (see

Table 1. Consideration for implementing decision support software.

Key Areas	Description
Accuracy of content	Correctness and reliability of the scientific and clinical information
Approach to curation	Method used to organize and maintain evidence data
Usability	Intuitiveness and user-friendliness of the software

).

Accuracy of the content refers to whether the information presented is clinically and scientifically correct. Equally important is checking whether critical information may be absent, essentially a “false negative” in terms of reporting clinically relevant data about a variant. A critical aspect to accuracy is time and geography. In light of changing guidelines by different medical societies and different drug approval timelines around the world, the content needs to be up-to-date and reflective of local medical guidelines. Depending on the scope of the reporting workflow, the software may need to include matched therapies (on-label or off-label) and clinical trials, as well as interpretation of co-occurring variants, which is especially relevant in the context of resistance mutations.

Closely tied to accuracy of content is the approach to curation. The content itself may be scientifically accurate, but the interpretation and representation of the evidence can easily become subjective without guidance on how to prioritize evidence. Therefore, the approach to curation refers to how existing evidence is interpreted; specifically, which database sources are pulled in and what variant tiering or prioritization scheme is used. Professional societies have published guidelines on interpreting somatic oncology variants, including a joint consensus recommendation from the Association for Molecular Pathology (AMP), American Society of Clinical Oncology (ASCO), College of American Pathologists (CAP), and American College of Medical Genetics (ACMG) [19], as well as the European Society for Medical Oncology Scale for Clinical Actionability of molecular Targets (ESCAT) [20]. The AMP/ASCO/CAP/ACMG recommendation provides a tiered classification system for prioritizing variants based on clinical significance and actionability. Some institutions may adopt these guidelines with or without modification [21], or choose to utilize a local guideline [22,23,24,25]. However, there is some degree of interoperability between the different classification systems [26].

Usability refers to how intuitive and easy-to-use the software workflow is, which can reduce errors. The software should be used as intended and fit the needs of the laboratory. Therefore, this can cover a wide range of requirements, such as setup and maintenance, flexibility, report readability, data management, data security, and lab system integration. First, there is upfront investment to establish an interpretation and reporting workflow, and continued investment required to properly maintain and update the system. While there are publicly available resources to explore cancer variant data, such as cBioPortal [27], gaps remain in using these to support clinicians in molecular tumor boards [28] or dedicated bioinformatics and IT teams are needed to setup computational tools such as Variant Interpretation for Cancer [29]. Second, laboratories may need to have the flexibility to support different NGS platforms and assays, to perform a combination of automated and manual curation, and to modify the report format for different end-users. Third, once a substantial number of cases have been analyzed, the ability to quickly retrieve a case or track the prevalence of a mutation may be important. Overall data management and connectivity to the laboratory’s workflow are critical.

3. Recent Results on NAVIFY Mutation Profiler

To illustrate these considerations, we will review study results from an example cloud-based software called NAVIFY Mutation Profiler. Briefly, NAVIFY Mutation Profiler is a CE-IVD software that can simplify the process of interpreting and reporting variants from NGS somatic oncology test results. A Variant Call File (VCF), generated from the sequencing steps, can be uploaded onto NAVIFY Mutation Profiler to create a case and be analyzed by the software. It utilizes a Roche knowledgebase that is richly curated and up to date based on publicly available evidence, drawing from multiple databases for variant annotation, including Catalogue Of Somatic Mutations In Cancer (COSMIC), The Cancer Genome Atlas (TCGA), CIViC, biomedical literature, clinical trial results, and medical guidelines. The curation also tracks drug approvals and recommendations by region, such as whether a drug has been approved in Europe by the European Medicines Agency (EMA) versus SwissMedic in Switzerland or National Institute for Health and Care Excellence (NICE) in the UK. For the approach to tiering and definitions of levels of evidence, NAVIFY Mutation Profiler uses the AMP/ASCO/CAP/ACMG guidelines [19].

Several results have been presented on the accuracy of NAVIFY Mutation Profiler compared to manual curation across NGS panels of different sizes. There was 100% agreement with manual variant reporting across 70 solid tumor variants from a 15-gene NGS panel [30], and 100% agreement in classifying Tier I and II for 41 different colorectal cancer variants identified with a 50-gene panel [31]. When 81 solid tumor variants were distributed to six molecular genetic experts for manual tiering, the tiers agreed 94% of the time with the automated tiering in NAVIFY Mutation Profiler; the agreement for Tier I variants, the most clinically significant variants, was 99.6% [32].

In a retrospective study of 37 cases of non-small cell lung cancer with known treatment regimens and plasma sequencing using a 200-kilobase panel, NAVIFY Mutation Profiler correctly listed therapy options for clinically relevant variants in EGFR and ALK, and did not list targeted therapies for cases treated with chemotherapy [33]. NAVIFY Mutation Profiler also remained up to date in terms of releasing knowledgebase updates in line with approvals by different drug agencies for various EGFR and ALK tyrosine kinase inhibitors over the course of the study [33]. Furthermore, compared to evidence levels in OncoKB, there was 81% agreement in tiers assigned by NAVIFY Mutation Profiler, which revealed additional evidence on variants matching inclusion criteria for clinical trials and variant-variant interactions. Variant combinations are clinically relevant, as observed in the NCI-MATCH trial where 38% of patients with actionable mutations were excluded from treatment due to co-occurring resistance mutations [34]. Variant combinations are often missed in manual curation [31] or other software tools [35]. Compared to other databases, NAVIFY Mutation Profiler provided more annotation on resistance mutations [33].

Using the same 51 variants from Sirohi et al. [15], NAVIFY Mutation Profiler had 74.5% agreement with the consensus tiering of the 20 experts, with disagreements due to emerging biomedical evidence and potentially updated guidelines used in NAVIFY Mutation Profiler [36]. This highlights the need for continual updates based on new scientific information. The knowledgebase behind NAVIFY Mutation Profiler, called Ephesus [37], is regularly updated and has more content on actionable mutations than other knowledgebases. When over 60,000 clinical samples across 17 cancer types from the American Association for Cancer Research (AACR) Genomics Evidence Neoplasia Information Exchange (GENIE) project were queried against Ephesus, the percentage of patients with an interpretation exceeded that of Cancer Genome Interpreter (CGI), CIViC, and ClinVar [36]. Similarly, using NAVIFY Mutation Profiler to analyze 3810 solid tumor cases from TCGA across 10 cancer types, there was more actionable content than OncoKB [38]. Molecular matches to clinical trials also require regular updates and can be difficult to keep up manually. In a clinical trial matching exercise performed on 103 patients with breast, ovarian, lung, or colorectal cancer and molecularly profiled using the Oncomine Comprehensive Assay v3, potentially 62% of the cases matched to clinical trials open in Canada using NAVIFY Mutation Profiler [39]. In 3920 TCGA and cBioPortal cancer samples (across lung, colorectal, breast, prostate, stomach, bladder, head and neck, and melanoma), 76% matched to a clinical trial [38,40].

The Sirohi et al. study also showed that agreement in tiering between the 20 molecular diagnostics experts could increase from 58% to 84% after sharing classifications and evidence with one another [15]. The ability to share variant classifications is possible in NAVIFY Mutation Profiler, where laboratories can opt into sharing aggregated tier classifications with a network of laboratories. Sharing classifications for variants could improve consistency in interpretation between laboratories, and further enhance decision support software by collating multiple sources of information from the public domain as well as a peer network.

While it is clear that decision support tools can help automate some of the variant interpretation, studies have shown that the agreement in annotating actionability can vary widely between different commercial and public tools for variant annotation [41,42]. Perakis et al. compared the performance of three commercial clinical decision support tools, NAVIFY Mutation Profiler, QIAGEN Clinical Insight Interpret (QCI-I), and CureMatch Bionov, and found that each platform had a different strategy for tier classification and defining actionability [43]. Differences could be explained by the use of the AMP/ASCO/CAP/ACMG tiers and strict adherence to drug labels and guidelines in NAVIFY Mutation Profiler. These results emphasize the importance of the approach to curation in any decision support software for variant interpretation. In fact, other studies comparing other tools also found discrepancies. One study compared Oncomine Knowledgebase Reporter, QCI-I, and IBM Watson for Genomics, and found around 60% overlap in annotation of variants with therapeutic information [41]. Another study assessed annotations for pathogenicity and clinical actionability of three tools available to the Veterans Affairs National Precision Oncology Program [42]. When comparing N-of-One, Watson for Genomics, and OncoKB, the study found agreement in annotating pathogenicity ranged from 30% to 76%. Clearly, not every decision support tool is the same in the interpretation content it provides. Laboratories need to understand the content behind decision support tools.

In terms of usability, one potential metric to assess is the time required from NGS result upload to the final report. Results on time savings in the interpretation and reporting workflow using NAVIFY Mutation Profiler have been previously presented. One laboratory found that it took, on average, 60 min to prepare a report with manual curation, and only 15 min to prepare a report using NAVIFY Mutation Profiler, equivalent to a 75% time savings [31]. Another study found that it took 20 min to prepare a report with QCI-I and only seven minutes to prepare a report with NAVIFY Mutation Profiler, a 65% time savings that could save four hours of hands-on technologist time per week for a workload of 20 reports [35]. In these two studies, cases were manually created and uploaded to NAVIFY Mutation Profiler. Here is an opportunity to further streamline those steps if the laboratory has a high testing volume. This would not only further save time, but also minimize the risk of error with manual entry. This is enabled with an application programming interface (API) and scripts to automate the case creation and file upload. API usage in NAVIFY Mutation Profiler has been demonstrated on over 4000 solid tumor cases and 745 cases with hematologic malignancies [38,44]. Once the scripts were set up, the upload process was highly efficient, and 99.7% of the cases were successfully uploaded and analyzed on the first attempt. This enhanced connectivity to a laboratory’s existing analysis and data infrastructure can support scalability, traceability, and reproducibility for interpretation and reporting oncology NGS results.

While decision support software such as NAVIFY Mutation Profiler is a useful tool for automated variant interpretation, manual curation is still necessary for the annotation of variants. Public information from multiple databases is automatically pulled using available systems and resources to assist in annotation. However, due to the inconsistencies of classifications of some variants across databases such as ClinVar, CIViC, and COSMIC, manual review, interpretation, and validation by scientists is still a required step. Manual curation is also necessary to remove irrelevant or incorrect information pulled in from the automated process [18]. The function of the automated annotation of the software is to facilitate the manual curation process rather than to replace it.

4. Conclusions

Frequent but irregular updates to medical guidelines, drug approvals, clinical trial results, and clinical research findings pose challenges to clinicians in making the most informed decisions for their patients. Decision support software can address the challenges in interpretation and reporting posed by growing amounts of complex and evolving data from NGS results in somatic oncology testing.

A decision support software such as NAVIFY Mutation Profiler can help bring together multiple sources of information, including public databases, guidelines, publications, a laboratory’s internal knowledge, and peer laboratory information on variant classifications. Such a tool is also able to support a variety of NGS assays, which may be important if a laboratory is running different NGS panels, various bioinformatics workflows, or possibly different sequencing platforms, and needs to harmonize the results into a common report format, yet have flexibility to modify content and report sections as needed. As an example, we reviewed assessments on accuracy and usability of NAVIFY Mutation Profiler. Some key results showed that the software had accurate and efficient curation, offered the interpretation of variant combinations, and provided timely updates and tiering by regional approvals.

Whether building a homebrew solution or evaluating external options, one needs to understand and assess the accuracy of content, approach to curation, and usability. Decision support solutions, appropriately implemented, will help the field realize the full potential of precision oncology.