Performance Characteristics of Oncomine Focus Assay for Theranostic Analysis of Solid Tumors, A (21-Months) Real-Life Study

Next generation sequencing analysis is crucial for therapeutic decision in various solid tumor contexts. The sequencing method must remain accurate and robust throughout the instrument lifespan allowing the biological validation of patients’ results. This study aims to evaluate the long-term sequencing performances of the Oncomine Focus assay kit allowing theranostic DNA and RNA variants detection on the Ion S5XL instrument. We evaluated the sequencing performances of 73 consecutive chips over a 21-month period and detailed the sequencing data obtained from both quality controls and clinical samples. The metrics describing sequencing quality remained stable throughout the study. We showed that an average of 11 × 106 (±0.3 × 106) reads were obtained using a 520 chip leading to an average of 6.0 × 105 (±2.6 × 105) mapped reads per sample. Of 400 consecutive samples, 95.8 ± 16% of amplicons reached the depth threshold of 500X. Slight modifications of the bioinformatics workflow improved DNA analytical sensitivity and allowed the systematic detection of expected SNV, indel, CNV, and RNA alterations in quality controls samples. The low inter-run variability of DNA and RNA—even at low variant allelic fraction, amplification factor, or reads counts—indicated that our method was adapted to clinical practice. The analysis of 429 clinical DNA samples indicated that the modified bioinformatics workflow allowed detection of 353 DNA variants and 88 gene amplifications. RNA analysis of 55 clinical samples revealed 7 alterations. This is the first study showing the long-term robustness of the Oncomine Focus assay in clinical routine practice.


Introduction
Theranostic analysis of solid tumors gains continuous complexity requiring the search for point mutations, short insertions or deletions, copy number variations, or gene fusions in always larger panels of genes [1,2]. Next generation sequencing (NGS) allows to keep up with the search for a growing number of genetic alterations in limited amounts of biologic material with a temporality compatible with clinical needs. NGS may allow reporting results to clinicians in a minimum of five days from the acquiring of formalin-fixed, paraffinembedded (FFPE) samples. Implementation of a new NGS method follows well-defined procedures that are usually defined by academic societies or by state medical agencies [3,4]. Following the new NGS method set up, identification of factors that contribute to alter data quality over time is of critical importance. Indeed, several environmental, material, or human factors may impact NGS long-term performances (i.e., analytical sensitivity), possibly affecting the choice of clinical treatments [2,5]. Indeed, the initial validation does not ensure that the method will be robust enough to produce accurate results throughout the instrument lifespan.
Two major NGS methods are commonly used in diagnostic laboratories. Although both methods are based on sequencing by synthesis, Thermo Fisher and Illumina platforms use different principles for sequencing. Thermo Fisher Ion Torrent TM technology directly converts nucleotide sequences into digital information using a semiconductor chip measuring variation of pH [6]. Illumina "bridge amplification" technology is based on incorporation and detection of fluorescent nucleotides in DNA fragments immobilized on a glass slide [7]. A recent study reported that both technologies achieved comparable NGS performances (i.e., mean read coverage, mean coverage uniformity, and variant allele frequency) and are suitable for diagnosis purpose [8].
Our laboratory implemented the Oncomine TM Focus Assay (OFA) on an Ion S5XL (ThermoFisher, Les Ulis, France). This assay is dedicated to the identification of all types of theranostic variations in FFPE tissues and enables the concurrent analysis of DNA and RNA to detect variations in 52 genes in solid tumors [9,10]. To our knowledge, no study has been dedicated to investigate the evolution of the performances of OFA over a long period of time.
The aim of this study was to evaluate retrospectively the evolution of the performances of the OFA during a 21-month period in order to verify whether it is suitable for hospital routine practice.
Anonymized FFPE patient samples or quality control (QC) samples originating from 73 consecutive sequencing runs were used for quality metrics assessment. DNA alterations were analyzed from 420 patient samples originating from various tumors including 277 lung, 64 colon, 34 skin, and 45 other tissues (18 pancreas, 16 bile ducts, 5 breast, 3 thyroids, and 3 spleens) over a 21-month period. RNA fusions and exon skipping were analyzed from 55 tissue samples (45 lung, 8 bile duct, 1 pancreas, and 1 thyroid) over a 12-month period.

DNA and RNA Extraction
DNA and RNA were isolated using the Maxwell

Library Preparation and Sequencing
For RNA assay, prior to library preparation, cDNA were synthetized using SuperScript VILO cDNA synthesis kit (catalog no. 11756050, Thermo Fisher). Library preparations were carried out using Oncomine Focus Assay, Chef-Ready Library (Thermo Fisher) using an Ion Chef Instrument (Thermo Fisher) following the manufacturer's instructions using a total of 10ng of DNA or 2ng of RNA.

Bioinformatic Pipeline
For each run, quality metrics were assessed including chip loading density, number of total reads, percentage of clonality, percentage of adapter dimer, percentage of low quality, read length, and alignment of the reads to the hg19 human reference genome (Torrent server, version 5.12, Thermo Fisher). The "coverage Analysis" plugging was applied to assess the quality of sample sequencing. Each sample must reach strict validation criteria, i.e., yield a minimum of 400,000 reads, 98% of the amplicons with a minimal sequencing depth of 500X, 90% of the reads located within the target region boundaries, 80% of the amplicons being read from end-to-end, and 90% of the amplicons being read without strand bias.
For DNA variant annotation, the Ion Reporter software was carried out (version 5.10). The following default parameters were modified from the "Oncomine Focus w2.4-DNA-Single Sample" workflow: variants were reported with allele view, complex variants were allowed, down-sampled to coverage was set to 5000 reads, and the variants were generated at a minimum VAF of 0.03. The hotspot file was also modified: some hotspot mutations were added to report minor variants within BRAF, KRAS, and NRAS genes (Appendix A). The minimal value for VAF detection was decreased to 0.02 for positions of theranostic interest within BRAF, EGFR, KRAS, and NRAS genes. CNV were reported by the pipeline if the MAPD (median absolute pairwise difference) was below 0.4 and if the amplification factor was above 4.
For RNA variant annotation, the "Oncomine Focus-520-w2.4-Fusions-Single Sample" workflow was used with default parameters. The validation criteria for RNA samples were as follows: each sample must generate at least 20,000 reads and have a minimum mean read length of 50 pb. Additionally, at least three of the five RNA internal controls (TBP, LRP1, ITGB7, MYC, and HMBS) must be called. Finally, RNA alterations were reported only if a minimum number of reads was reached: 20 for targeted fusions, 250 for non-targeted fusions, and 120 for exon skipping.
For the inter-run variability comparison study, BAM files generated by the MiSeq instrument were analyzed using a tailored Sophia Genetics pipeline (Sophia Genetics, Lausanne, Switzerland). Variant calls were confirmed using the Integrative Genomics Viewer tool (Broad Institute, Cambridge, MA, USA) when necessary.

Sequencing Performances
The quality metrics of the OFA DNA panel were assessed using 73 consecutive runs on 520 chips including QC and clinical samples. An average of 11 × 10 6 (±0.3 × 10 6 ; CV = 3.4%) reads were initially generated per run. The polyclonal ion sphere particles (ISP), primer dimers, low quality reads, and test quality sequences were then filtered out (5.8 × 10 6 ± 0.8 × 10 6 reads). Hence, 49 ± 7% (CV = 13%) of the amplicons presented the required quality for further bioinformatics analysis (Figure 1a). Among an average number of sequenced bases of 77 × 10 6 (±2.5 × 10 6 ), 72 × 10 6 (±2.4 × 10 6 ) bases were sequenced with a quality score of 20 (Q20). The mean sequencing depth for a given on-target base was 2246X ± 939X. All the metrics, independent of DNA quality, were measured at each run during the 21-month period and did not reveal any loss of performance over time, as assessed by the low coefficients of variation.   For each sample, an average of 6.0 × 10 5 (±2.6 × 10 5 ; CV = 44%) reads were mapped to hg19 genome reference and 98.2 ± 7.1% of the reads were aligned over a target region. The mean read length was 114 bp (± 4 bp). The uniformity of base coverage, defined as the percentage of on-target bases covered by at least 20% of mean coverage depth, was 98.5 ± 8.4%.

Long Term Inter-Run Variability for SNVs and Indels
Inter-run variability for SNVs and indels analysis was performed using both QC materials and patient samples. To assess the inter-run performance of the OFA using S5XL instrument, different QC materials were analyzed (HD300: n = 42 runs; HD200: n = 35 runs). All expected mutations in the reference material were called at expected VAFs. Coefficient of variation of measured VAF ranged from 5.6% (p.His1047Arg, expected VAF = 17.5%) to 37.7% (p.Thr790Met, expected VAF = 1%) ( Table 1; Figure 2 and Figure S1). The follow-up of the VAF QC material by Levey-Jennings charts assessed the low inter-run variability for VAF ranging from 1% to 11%. Furthermore, no drift of VAF values was observed. Indeed, during the 21-month period, for the lowest expected VAF (EGFR p.Thr790Met), only one value (2.3%) overpassed the limit of 3SD ( Figure 2). Table 1. Inter-run variability of VAF obtained with HD200 and HD300 QC materials on S5XL and MiSeq instruments.

Oncomine Focus Assay
Tumor Hot Spot   These data were also compared to a similar inter-run variability study performed using the THS on the Miseq instrument (HD300: n = 42 runs; HD200: n = 43 runs). The comparison of the VAF obtained for expected variants indicated a strong correlation between both methods (Pearson's r = 0.990) ( Table 1).
To further evaluate inter-run variability, clinical samples from colorectal and lung tissues were analyzed within six different runs. Indeed, PIK3CA p.His1047Arg, EGFR p.Leu747_Ala750del, AKT1 p.Gly17Leu, BRAF p.Val600Glu, and KRAS p.Gly12Val mutations previously detected using the THS assay were called at each run using the OFA tween both methods (Pearson's r = 0.990) ( Table 1).
To further evaluate inter-run variability, clinical samples from colorectal and lu tissues were analyzed within six different runs. Indeed, PIK3CA p.His1047Arg, EG p.Leu747_Ala750del, AKT1 p.Gly17Leu, BRAF p.Val600Glu, and KRAS p.Gly12Val mu tions previously detected using the THS assay were called at each run using the O (Figure 3). The VAF coefficients of variation ranged from 6.7% (BRAF p.Val600Glu) to 3 % (KRAS p.Glu12Val). and (c) lung tissues. Expected VAF were determined using THS assay (n = 1); mean VAF were termined using OFA (n = 6).

Performances of CNV Detection
To assess inter-run variability, 20 QC samples were analyzed in different runs. The MAPD scores ranged from 0.272 to 0.356, assessing a low read coverage noise (Table 2). Furthermore, the inter-run variability of MAPD was low (mean: 0.305 ± 0.032), indicating the long-term performance stability of CNV determination. MET and MYCN gene amplifications were systematically reported close to expected levels. In addition, the variation of inter-run CNV quantification remained low as assessed by the low coefficient of variation for each CNV level. No drift of the measured CNV value was observed, indicating that the performance of the method remained stable over time. This study was extended to three clinical samples, which were sequenced twice. KRAS, EGFR, FGFR2, CCND1, CDK4, and ERBB2 genes amplifications were reported in the two runs with equivalent amplification rates.
Furthermore, we confirmed CNV detection of ERBB2 gene by studying protein expression level using immunochemistry assay in four lung tumors samples (Table 3; Figure 5). Nuclei were stained in blue (hematoxylin), cytoplasm of positives cells for ERBB2 were stained in brown, while negative cells remained unstained. A large view of bone tissue section showing negative staining for ERBB2 is shown in Figure S2.

Performances of CNV Detection
To assess inter-run variability, 20 QC samples were analyzed in different runs. The MAPD scores ranged from 0.272 to 0.356, assessing a low read coverage noise (Table 2). Furthermore, the inter-run variability of MAPD was low (mean: 0.305 ± 0.032), indicating the long-term performance stability of CNV determination. MET and MYCN gene amplifications were systematically reported close to expected levels. In addition, the variation of inter-run CNV quantification remained low as assessed by the low coefficient of variation for each CNV level. No drift of the measured CNV value was observed, indicating that the performance of the method remained stable over time.
This study was extended to three clinical samples, which were sequenced twice. KRAS, EGFR, FGFR2, CCND1, CDK4, and ERBB2 genes amplifications were reported in the two runs with equivalent amplification rates. Furthermore, we confirmed CNV detection of ERBB2 gene by studying protein expression level using immunochemistry assay in four lung tumors samples (Table 3; Figure  5). Nuclei were stained in blue (hematoxylin), cytoplasm of positives cells for ERBB2 were stained in brown, while negative cells remained unstained. A large view of bone tissue section showing negative staining for ERBB2 is shown in Figure S2.

Gene-Fusion and Exon-Skipping Detection
RNA samples from HD784 and Seracare QC materials were sequenced in 29 runs, respectively. Quality criteria were satisfactory for each sample, allowing furthe analysis. Expected RNA fusions and exon skipping variations were systematicall tected. Fusion reads represented 1.6 to 17.8% of total reads counts ( Table 4). The inte CV of the percentage of total reads counts ranged from 7.9 to 38.4%. No drift in the d tion of both fusions and exon skipping was observed, indicating that the performan the method remained stable over time.

Gene-Fusion and Exon-Skipping Detection
RNA samples from HD784 and Seracare QC materials were sequenced in 29 and 7 runs, respectively. Quality criteria were satisfactory for each sample, allowing further data analysis. Expected RNA fusions and exon skipping variations were systematically detected. Fusion reads represented 1.6 to 17.8% of total reads counts ( Table 4). The interrun CV of the percentage of total reads counts ranged from 7.9 to 38.4%. No drift in the detection of both fusions and exon skipping was observed, indicating that the performance of the method remained stable over time. To complete the validation of MET exon 14 skipping detection, we analyzed RNA material from two patients with MET alterations previously detected at DNA level with the THS assay (MET c.3082 +1G>C and MET c.2888-2A>C). These mutations were detected at the RNA level with the OFA in two independent experimentsIn order to validate ALK and ROS detection, two gene fusions determined by OFA [(EML4(6)-ALK(20) (reads counts: 2024, total mapped fusion reads: 49,617) and CD74(6)-ROS1(34) (reads counts: 1027, total mapped fusion reads: 31,980)] were confirmed by immunochemistry (Figure 7). Nuclei were stained in blue (hematoxylin), cytoplasm of positives cells for ALK and ROS1 were stained in brown, while negative cells remained unstained. Negative ROS1 staining of non-tumoral lung tissue is shown in Figure S3.

Discussion
Theranostic analysis of tumors becomes continuously more complex requiring the study of ever more larger panels of genes in search of point mutations, short insertions and deletions, CNV, or gene fusions. In addition, the development of less invasive sampling techniques such as needle biopsy lead to the decrease of available material. Despite these limitations, genetic analysis must remain compatible with a rapid determination of therapeutic strategy. NGS is the method of choice to fulfill these challenges, assuming that optimal analytical performances are maintained over time.
The aim of this study was to determine whether the analytical performances of the Oncomine Focus Assay remain suitable for clinical practice over a 21-month period. For this purpose, we evaluated the sequencing results obtained from both QC and clinical samples along 73 consecutive runs on an Ion S5 XL sequencer. During this period, we observed no technical failure and the inter-runs variability of critical parameters was adapted to clinical practice.
We first observed that the template generation was reproducible over time. Indeed, the coefficient of variation of the chips loading was 3.4% (mean: 11 × 10 6 amplicons). Subsequent bioinformatic processing indicated that 49% (CV = 13%) of these amplicons had a sufficient quality for further processing. Additionally, we showed that 6.0 × 10 5 (CV = 44%) reads per sample were mapped to hg19 reference genome. These CV reflected also the variation in DNA quality of samples. However, our results were consistent with other studies using the same technique. Indeed, Bartlett et al. showed that intra-laboratory variability of mapped reads generation may range from 0.18 × 10 6 to 1.6 × 10 6 (10). In our study, the quality of mapped reads within both QC and clinical samples was maintained over time, assessed by the low variability of percentage of end-to-end reads (97.3%; CV =

Discussion
Theranostic analysis of tumors becomes continuously more complex requiring the study of ever more larger panels of genes in search of point mutations, short insertions and deletions, CNV, or gene fusions. In addition, the development of less invasive sampling techniques such as needle biopsy lead to the decrease of available material. Despite these limitations, genetic analysis must remain compatible with a rapid determination of therapeutic strategy. NGS is the method of choice to fulfill these challenges, assuming that optimal analytical performances are maintained over time.
The aim of this study was to determine whether the analytical performances of the Oncomine Focus Assay remain suitable for clinical practice over a 21-month period. For this purpose, we evaluated the sequencing results obtained from both QC and clinical samples along 73 consecutive runs on an Ion S5 XL sequencer. During this period, we observed no technical failure and the inter-runs variability of critical parameters was adapted to clinical practice.
We first observed that the template generation was reproducible over time. Indeed, the coefficient of variation of the chips loading was 3.4% (mean: 11 × 10 6 amplicons). Subsequent bioinformatic processing indicated that 49% (CV = 13%) of these amplicons had a sufficient quality for further processing. Additionally, we showed that 6.0 × 10 5 (CV = 44%) reads per sample were mapped to hg19 reference genome. These CV reflected also the variation in DNA quality of samples. However, our results were consistent with other studies using the same technique. Indeed, Bartlett et al. showed that intra-laboratory variability of mapped reads generation may range from 0.18 × 10 6 to 1.6 × 10 6 (10). In our study, the quality of mapped reads within both QC and clinical samples was maintained over time, assessed by the low variability of percentage of end-to-end reads (97.3%; CV = 5.6%) and reads without strand bias (98.4%; CV 2.0%).
The main point of variation, influenced by DNA quantity, was the percentage of amplicons reaching 500X per sample [2]. In our routine practice, although a depth below 500X may remain informative when mutations are at a high allelic frequency, we consider that the depth should reach 500X to validate a negative result. We showed that, for a given sample, 95.8% (CV = 16.7%) of the amplicons reached a depth of 500X. Moreover, all amplicons were individually covered with at least 845X ( Figure 1). These results are comparable with those of Williams et al.'s study reporting 89.7% of samples with an average amplicon coverage above 500X [9]. In our analysis, all hotspot regions were sequenced with at least 1574X. Of note, the amplicons sequenced with a lower depth targeted only CNV positions (Figure 1c; Appendix B). We successfully sequenced 98% of clinical samples, a proportion comparable to the 95% described by Williams et al. [9]. These data suggest (i) the tissular heterogeneity of samples did not affect the quality of the sequencing and (ii) the performance of sequencing did not decline over time. Considering that numerous samples were tissue biopsies, we show that the OFA is adapted to this limited amount of material.
The call of the mutations with the Ion Reporter software could fail because of inadequacy of the bioinformatic pipeline. Therefore, we modify some parameters of the workflow. Clinical purpose led us to use samples with a very low acid nucleic concentration, potentially causing limited sequencing depth. In order to improve robustness of our analysis, we modified the hotspot file (Appendix A). We set the detection threshold down to 3% of AF for SNV and indel, and down to 2% for driver variations in BRAF, ERBB2, EGFR, KRAS, and NRAS genes. In addition, we increased the down sampled coverage up to 5000 reads (the minimum reads randomly considered), allowing to improve the inter-run variability. Finally, we and others observed incorrect nomenclature calls [9]. To attribute the mutation position to the correct variant annotation, the "allele view" and the "allow complex" parameters were selected.
In order to validate our modifications of the bioinformatic pipeline, we analyzed the inter-run variability of two QC materials. We observed similar inter-run variabilities between the OFA and THS kits ( Table 1). As expected, the variability was higher for low VAFs. However, the low coefficients of variation allowed the detection of mutations down to 1% VAF at each run. Furthermore, mutations from 3 to 5% expected VAF were systematically called (Table 1; Figures 2 and S1). This long-term follow-up of VAF values highlighted that there were no loss of performance and no drift over time. This low variability was appropriate for clinical sample analysis. An inter-laboratory study involving six teams and analyzing the same QC materials obtained similar VAF for most mutations. However, we observed differences only for mutations with VAF < 5%. Five of the six laboratories did not report the EGFR p.Leu858Arg SNP with a VAF of 4% and none reported the EGFR p.Thr790Met and p.Glu746_Ala750del variations at VAF 1% and 2%, respectively [10]. This increased sensibility may be due to our optimization of the bioinformatic pipeline.
Using FFPE clinical samples, the observed inter-run variability was higher ( Figure 3). These results, which were expected, were possibly due to lower DNA quality. Nevertheless, this larger variability remained sufficient to systematically report each expected mutation.
In our routine practice, we include a QC sample at each library preparation for quality purpose. Considering clinical results interpretation, to qualify a variant as both authentic and relevant, we take into consideration the number of reads at the corresponding genomic position, the tumor cellularity, and the clinical context.
In the case of patient follow-up or for investigation of therapeutic resistance mechanism, the communication to clinicians of variants with VAF below the threshold of 5% may have a significant impact on prognosis [13,14]. During the 21-month period of DNA analysis, we detected 353 mutations among tissue samples. The increased sensitivity of the bioinformatic analysis allowed the detection of 10 mutations with VAF below 3%. The mutation with the lowest VAF (KRAS p.Gly12Asp; 2%) was detected in a colon sample. When such low VAF mutation are detected, the biologist has to consider also detailed metrics and genomic alignment visualization to appreciate its theranostic relevance [13][14][15].
Considering gene amplification, the CNV value is interpreted according to the MAPD score. The analysis of inter-run variability using QC material revealed that both expected gene amplifications were detected and associated MAPD scores were below the threshold of 0.4, giving confidence to these results. The coefficients of variation were notably low for MET and MYCN amplification (5.2% and 3.7%, respectively), attesting the inter-run stability over time. In four clinical samples, ERBB2 gene amplifications were confirmed by immunocytochemistry (Table 3; Figure 5). This limited comparison suggests that, using OFA, the CNV represented more a confidence level of positivity than an absolute value [16]. In order to avoid false positive, we choose to report only CNV with a "5% confidence lower limit" above 6 [17]. Among the 88 amplifications called by the pipeline, we reported 79 (91%) results to the clinicians.
The scarcity of patients presenting RNA alteration imposed a validation protocol using two different QC materials in order to increase the diversity of fusion transcripts. The expected fusion transcripts were systematically detected throughout the 12-month period. Despite a high heterogeneity of determined inter-run variability CV, fusions reads counts were always above the manufacturer recommended thresholds (i.e., 20 for targeted fusions, 250 for non-targeted fusions, and 120 for exon skipping). Like others, we chose to determine the ratio of fusion reads counts to total mapped reads counts (Table 4) [10]. This ratio, similar to VAF, was useful to interpret clinical samples with low total RNA quantity. In our QC study, a fusion transcript could be reported with a percentage as low as 1% of total mapped reads. In contrast, in samples with a high amount of total RNA mapped reads, a ratio below 10% may indicate a false positive result.
Regarding HD784 QC material, we obtained similar fusion reads and fusion reads ratios to those reported by Bartlett et al.'s study [10]. In our clinical samples study, we reported seven fusions or exon skipping variants; among them, two were validated by immunochemistry (Figure 7).
Our results demonstrated that the sensitivity and the accuracy of OFA were suitable for molecular analysis of low quality and/or limited amounts of nucleic acids obtained from FFPE tumor samples in clinical routine practice. Slight modifications of the bioinformatic pipeline allowed to improve detection of low VAF variants, which may be useful to early detect genetic alterations involved in resistance mechanisms. This long-term study conducted in real-life conditions demonstrated that the performances of the OFA remained stable over time and ensured the reliability of results with theranostic impact.
Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/diagnostics13050937/s1, Figure S1: Levey-Jennings plots of VAF obtained for five additional representative variants from HD200 and HD300 QC materials on S5XL instrument over a 21-months follow-up period; Figure S2: Immuno-cytochemistry analysis of ERBB2 protein in a non-tumoral bone tissue highlighting negative staining; Figure S3: Immuno-cytochemistry analysis of ROS1 protein in a non-tumoral lung tissue highlighting negative staining. Informed Consent Statement: Patient consent was waived since this non-interventional study did not require any additional biological or clinical exploration nor any modification of standard clinical practices.

Data Availability Statement:
Restrictions apply to the availability of these data. Data were obtained from Assistance Publique-Hôpitaux de Paris and are available from the authors with the permission of Assistance Publique-Hôpitaux de Paris.

Acknowledgments:
The authors thank C. Elie, Z. Achach and N. Araci for expert technical assistance.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
Genomic position of the added clinically relevant hotspot mutations.