Next Article in Journal
Thermal Liquid Biopsy (TLB) Focused on Benign and Premalignant Pancreatic Cyst Diagnosis
Previous Article in Journal
Biomarkers for Lifetime Caries-Free Status
Article

Personalized beyond Precision: Designing Unbiased Gold Standards to Improve Single-Subject Studies of Personal Genome Dynamics from Gene Products

1
Center for Biomedical Informatics & Biostatistics of the University of Arizona Health Sciences, The University of Arizona, 1230 N. Cherry Avenue, Tucson, AZ 85721, USA
2
Graduate Interdisciplinary Program in Statistics of the University of Arizona, The University of Arizona, 617 N. Santa Rita Avenue, P.O. Box 210089, Tucson, AZ 85721, USA
3
Department of Mathematics, The University of Arizona, 617 N. Santa Rita Avenue, P.O. Box 210089, Tucson, AZ 85721, USA
4
Department of Medicine, College of Medicine Tucson, 1501 N. Campbell Avenue, P.O. Box 245017, Tucson, AZ 85724, USA
5
Arizona Cancer Center, 1501 N. Campbell Avenue, P.O. Box 245017, Tucson, AZ 85724, USA
*
Author to whom correspondence should be addressed.
J. Pers. Med. 2021, 11(1), 24; https://doi.org/10.3390/jpm11010024
Received: 10 November 2020 / Revised: 19 December 2020 / Accepted: 25 December 2020 / Published: 31 December 2020
(This article belongs to the Section Omics/Informatics)
Background: Developing patient-centric baseline standards that enable the detection of clinically significant outlier gene products on a genome-scale remains an unaddressed challenge required for advancing personalized medicine beyond the small pools of subjects implied by “precision medicine”. This manuscript proposes a novel approach for reference standard development to evaluate the accuracy of single-subject analyses of transcriptomes and offers extensions into proteomes and metabolomes. In evaluation frameworks for which the distributional assumptions of statistical testing imperfectly model genome dynamics of gene products, artefacts and biases are confounded with authentic signals. Model confirmation biases escalate when studies use the same analytical methods in the discovery sets and reference standards. In such studies, replicated biases are confounded with measures of accuracy. We hypothesized that developing method-agnostic reference standards would reduce such replication biases. We propose to evaluate discovery methods with a reference standard derived from a consensus of analytical methods distinct from the discovery one to minimize statistical artefact biases. Our methods involve thresholding effect-size and expression-level filtering of results to improve consensus between analytical methods. We developed and released an R package “referenceNof1” to facilitate the construction of robust reference standards. Results: Since RNA-Seq data analysis methods often rely on binomial and negative binomial assumptions to non-parametric analyses, the differences create statistical noise and make the reference standards method dependent. In our experimental design, the accuracy of 30 distinct combinations of fold changes (FC) and expression counts (hereinafter “expression”) were determined for five types of RNA analyses in two different datasets. This design was applied to two distinct datasets: Breast cancer cell lines and a yeast study with isogenic biological replicates in two experimental conditions. Furthermore, the reference standard (RS) comprised all RNA analytical methods with the exception of the method testing accuracy. To mitigate biases towards a specific analytical method, the pairwise Jaccard Concordance Index between observed results of distinct analytical methods were calculated for optimization. Optimization through thresholding effect-size and expression-level reduced the greatest discordances between distinct methods’ analytical results and resulted in a 65% increase in concordance. Conclusions: We have demonstrated that comparing accuracies of different single-subject analysis methods for clinical optimization in transcriptomics requires a new evaluation framework. Reliable and robust reference standards, independent of the evaluated method, can be obtained under a limited number of parameter combinations: Fold change (FC) ranges thresholds, expression level cutoffs, and exclusion of the tested method from the RS development process. When applying anticonservative reference standard frameworks (e.g., using the same method for RS development and prediction), most of the concordant signal between prediction and Gold Standard (GS) cannot be confirmed by other methods, which we conclude as biased results. Statistical tests to determine DEGs from a single-subject study generate many biased results requiring subsequent filtering to increase reliability. Conventional single-subject studies pertain to one or a few patient’s measures over time and require a substantial conceptual framework extension to address the numerous measures in genome-wide analyses of gene products. The proposed referenceNof1 framework addresses some of the inherent challenges for improving transcriptome scale single-subject analyses by providing a robust approach to constructing reference standards. View Full-Text
Keywords: single-subject studies; personalized medicine; precision medicine; reference standards; gold standards; biomarkers; open-source single-subject studies; personalized medicine; precision medicine; reference standards; gold standards; biomarkers; open-source
Show Figures

Graphical abstract

MDPI and ACS Style

Zaim, S.R.; Kenost, C.; Zhang, H.H.; Lussier, Y.A. Personalized beyond Precision: Designing Unbiased Gold Standards to Improve Single-Subject Studies of Personal Genome Dynamics from Gene Products. J. Pers. Med. 2021, 11, 24. https://doi.org/10.3390/jpm11010024

AMA Style

Zaim SR, Kenost C, Zhang HH, Lussier YA. Personalized beyond Precision: Designing Unbiased Gold Standards to Improve Single-Subject Studies of Personal Genome Dynamics from Gene Products. Journal of Personalized Medicine. 2021; 11(1):24. https://doi.org/10.3390/jpm11010024

Chicago/Turabian Style

Zaim, Samir R., Colleen Kenost, Hao H. Zhang, and Yves A. Lussier 2021. "Personalized beyond Precision: Designing Unbiased Gold Standards to Improve Single-Subject Studies of Personal Genome Dynamics from Gene Products" Journal of Personalized Medicine 11, no. 1: 24. https://doi.org/10.3390/jpm11010024

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop