A Comparative Evaluation of Four Bioinformatic Tools for Identifying HIV-1 pol Drug Resistance Mutations Using Illumina MiSeq Data
Simple Summary
Abstract
1. Introduction
2. Materials and Methods
2.1. Sequencing Dataset
2.2. Bioinformatics Pipelines
2.2.1. Exatype NGS HIV
2.2.2. Stanford University HIVdb-NGS
2.2.3. Quasitools (HyDRA)
2.2.4. iLunaR
2.3. Drug Resistance Mutation Interpretation and Consensus Definition
2.4. Statistical Analysis
3. Results
3.1. Concordance and Sanger Comparison
3.2. Cohen’s Kappa, Positive Percent Agreement, and Negative Percent Agreement
3.3. Discrepancy Analysis
4. Discussion
4.1. Quasitools (HyDRA)
4.2. Exatype NGS HIV
4.3. Stanford University HIVdb-NGS
4.4. Implications for the High-Sensitivity Era of Next-Generation Sequencing
4.5. Limitations of the Study
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- UNAIDS. Understanding Measures of Progress Towards the 95–95–95 HIV Testing, Treatment and Viral Suppression Targets; UNAIDS: Geneva, Switzerland, 2020; Available online: https://www.unaids.org/sites/default/files/media_asset/progress-towards-95-95-95_en.pdf (accessed on 18 August 2025).
- Alidjinou, E.K.; Deldalle, J.; Hallaert, C.; Robineau, O.; Ajana, F.; Choisy, P.; Hober, D.; Bocket, L. RNA and DNA Sanger sequencing versus next-generation sequencing for RAM detection in HIV-1. J. Antimicrob. Chemother. 2017, 72, 2823–2830. [Google Scholar] [CrossRef]
- Li, J.Z.; Paredes, R.; Ribaudo, H.J.; Svarovskaia, E.S.; Metzner, K.J.; Kozal, M.J.; Hullsiek, K.H.; Balduin, M.; Jakobsen, M.R.; Geretti, A.M.; et al. Low-frequency HIV-1 drug resistance mutations and risk of NNRTI-based antiretroviral treatment failure: A systematic review and pooled analysis. JAMA 2011, 305, 1327–1335. [Google Scholar] [CrossRef] [PubMed]
- Teo, C.H.Y.; Norhisham, N.H.B.; Lee, O.F.; Png, S.; Chai, C.N.; Yan, G.; Tang, J.W.; Lee, C.K. Towards Next-Generation Sequencing for HIV-1 Drug Resistance Testing in a Clinical Setting. Viruses 2022, 14, 2208. [Google Scholar] [CrossRef] [PubMed]
- Wright, I.A.; Travers, S.A. RAMICS: Trainable, high-speed and biologically relevant alignment of high-throughput sequencing reads to coding DNA. Nucleic Acids Res. 2014, 42, e106. [Google Scholar] [CrossRef]
- Lee, C.K.; Lee, H.K.; Huan, P.T.; Chiu, L.L.; Loh, T.P.; Koay, E.S. A point mutation in the thiopurine S-methyltransferase gene that led to exon 5 deletion in the transcribed mRNA. Clin. Chem. Lab. Med. 2016, 54, e301–e303. [Google Scholar] [CrossRef]
- Lee, C.K.; Chua, C.W.; Chiu, L.; Koay, E.S. Clinical use of targeted high-throughput whole-genome sequencing for a dengue virus variant. Clin. Chem. Lab. Med. 2017, 55, e209–e212. [Google Scholar] [CrossRef]
- Lee, C.K.; Huan, P.T.; Chai, C.N.; Ng, L.J.; Koay, E.S.; Lee, O.F.; Tan, M.; Loh, T.P. Novel thiopurine S-methyltransferase (TPMT) variant identified in Malay individuals. Clin. Chem. Lab. Med. 2024, 62, e247–e250. [Google Scholar] [CrossRef]
- Bolger, A.M.; Lohse, M.; Usadel, B. Trimmomatic: A flexible trimmer for Illumina sequence data. Bioinformatics 2014, 30, 2114–2120. [Google Scholar] [CrossRef]
- Li, D.; Liu, C.M.; Luo, R.; Sadakane, K.; Lam, T.W. MEGAHIT: An ultra-fast single-node solution for large and complex metagenomics assembly via succinct de Bruijn graph. Bioinformatics 2015, 31, 1674–1676. [Google Scholar] [CrossRef] [PubMed]
- Pearson, W.R. Finding Protein and Nucleotide Similarities with FASTA. Curr. Protoc. Bioinform. 2016, 53, 3.9.1–3.9.25. [Google Scholar] [CrossRef]
- Li, H.; Durbin, R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics 2009, 25, 1754–1760. [Google Scholar] [CrossRef]
- Li, H.; Handsaker, B.; Wysoker, A.; Fennell, T.; Ruan, J.; Homer, N.; Marth, G.; Abecasis, G.; Durbin, R. The Sequence Alignment/Map format and SAMtools. Bioinformatics 2009, 25, 2078–2079. [Google Scholar] [CrossRef]
- Quinlan, A.R.; Hall, I.M. BEDTools: A flexible suite of utilities for comparing genomic features. Bioinformatics 2010, 26, 841–842. [Google Scholar] [CrossRef]
- Wensing, A.M.; Calvez, V.; Ceccherini-Silberstein, F.; Charpentier, C.; Günthard, H.F.; Jacobsen, D.M.; Paredes, R.; Shafer, R.W.; Richman, D.D. 2025 update of the drug resistance mutations in HIV-1. Top. Antivir. Med. 2025, 33, 457–473. [Google Scholar]
- García-Lerma, J.G.; Nidtha, S.; Blumoff, K.; Weinstock, H.; Heneine, W. Increased ability for selection of zidovudine resistance in a distinct class of wild-type HIV-1 from drug-naive persons. Proc. Natl. Acad. Sci. USA 2001, 98, 13907–13912. [Google Scholar] [CrossRef] [PubMed]
- Saravanan, S.; Madhavan, V.; Kantor, R.; Sivamalar, S.; Gomathi, S.; Solomon, S.S.; Kumarasamy, N.; Smith, D.M.; Schooley, R.T.; Solomon, S.; et al. Unusual insertion and deletion at codon 67 and 69 of HIV type 1 subtype C reverse transcriptase among first-line highly active antiretroviral treatment-failing South Indian patients: Association with other resistance mutations. AIDS Res. Hum. Retroviruses 2012, 28, 1763–1765. [Google Scholar] [CrossRef] [PubMed]
- Goethals, O.; Clayton, R.; Van Ginderen, M.; Vereycken, I.; Wagemans, E.; Geluykens, P.; Dockx, K.; Strijbos, R.; Smits, V.; Vos, A.; et al. Resistance mutations in human immunodeficiency virus type 1 integrase selected with elvitegravir confer reduced susceptibility to a wide range of integrase inhibitors. J. Virol. 2008, 82, 10366–10374. [Google Scholar] [CrossRef] [PubMed]
- García-Lerma, J.G.; MacInnes, H.; Bennett, D.; Weinstock, H.; Heneine, W. Transmitted human immunodeficiency virus type 1 carrying the D67N or K219Q/E mutation evolves rapidly to zidovudine resistance in vitro and shows a high replicative fitness in the presence of zidovudine. J. Virol. 2004, 78, 7545–7552. [Google Scholar] [CrossRef]
- Paredes, R.; Lalama, C.M.; Ribaudo, H.J.; Schackman, B.R.; Shikuma, C.; Giguel, F.; Meyer, W.A.; Johnson, V.A.; Fiscus, S.A.; D’Aquila, R.T.; et al. Pre-existing minority drug-resistant HIV-1 variants, adherence, and risk of antiretroviral treatment failure. J. Infect. Dis. 2010, 201, 662–671. [Google Scholar] [CrossRef]
- Li, J.Z.; Paredes, R.; Ribaudo, H.J.; Kozal, M.J.; Svarovskaia, E.S.; Johnson, J.A.; Geretti, A.M.; Metzner, K.J.; Jakobsen, M.R.; Hullsiek, K.H.; et al. Impact of minority nonnucleoside reverse transcriptase inhibitor resistance mutations on resistance genotype after virologic failure. J. Infect. Dis. 2013, 207, 893–897. [Google Scholar] [CrossRef]
- Ji, H.; Enns, E.; Brumme, C.J.; Parkin, N.; Howison, M.; Lee, E.R.; Capina, R.; Marinier, E.; Avila-Rios, S.; Sandstrom, P.; et al. Bioinformatic data processing pipelines in support of next-generation sequencing-based HIV drug resistance testing: The Winnipeg Consensus. J. Int. AIDS Soc. 2018, 21, e25193. [Google Scholar] [CrossRef] [PubMed]

| Parameter | Mean | 95% Confidence Interval |
|---|---|---|
| Raw Read Count | 83,764 | 75,635–91,893 |
| Quality Score (% of bases ≥ Q30) | 96.37% | 95.92–96.82% |
| Sample | PI DRM | NRTI DRM | NNRTI DRM | INSTI DRM | Remarks |
|---|---|---|---|---|---|
| S1 | L33F | M41L, L74I, M184V, L210W, T215F, K219N | K103N, V108I, N348I | - | Concordant |
| S2 | - | A62V, K65R, V75I, M184V | K103N, Y318F, N348I | - | Concordant |
| S3 | - | - | V106I, Y181C, G190A | - | Concordant |
| S4 | - | - | V179D | - | Concordant |
| S5 | - | A62V, K65R, D67H, S68G, V75I, K219E | L100I, Y181I, G190A, H221Y | - | Concordant |
| S6 | - | M184I | K103N, H221Y, Y318F | E157Q | Discordant (Missed H221Y by Quasitools) |
| S7 | - | T215S | V106I | - | Discordant (Missed T215S by Quasitools) |
| S8 | - | M41L, D67N, K70R, M184V, L210W, T215Y | A98G, K101E, Y181YC, G190A | - | Concordant |
| S9 | - | M184V, M184I | K103N, Y188L, H221Y, P225H | - | Discordant (Missed H221Y by Quasitools) |
| S11 | - | M184I, K219E | E138K, E138Q | - | Concordant |
| S12 | M46I | A62V, K65R, D67H, S68G, V75I, K219E | L100I, Y181I, G190A, H221Y | - | Concordant |
| S13 | - | M184V | K103R, V106M, V179D | - | Concordant |
| S17 | N88D | - | - | - | Concordant |
| S18 | - | K70Q | K103N, V106A, H221Y, P225H | - | Concordant |
| S20 | - | D67N, K70E, M184I, M184V | K103N, V179E, P225H | - | Concordant |
| S21 | - | D67N, K70R, M184V, K219E, K219N | K103N, V106I, V108I, L234I, Y318F | - | Concordant |
| S22 | L10F | Y115F, M184V | V106I, Y181C, Y188L, H221Y | R263K | Discordant (Missed L10F by Exatype) |
| S23 | - | M41L, K65R, S68N, L74I, M184V | K101H, K103N, G190A, P225H | - | Concordant |
| S24 | - | L74I, M184V | K103N, V179E, P225H | - | Concordant |
| S25 | - | M41L, D67N, K70R, M184V, K219Q | Y181C, H221Y, N348I | - | Discordant (D67N misreported as D67E by Stanford) |
| S26 | - | - | K103N, G190A | E92G | Discordant (Missed E92G by Exatype) |
| S27 | - | - | K103N | - | Concordant |
| S28 | - | K219E | Y181C, G190A, H221Y | N155H | Concordant |
| S29 | - | M184V | - | - | Concordant |
| S31 | - | - | - | T97A | Concordant |
| S33 | - | - | V179D | E157Q | Concordant |
| S34 | - | L74I, M184V | K103N, V108I, P225H, M230L | - | Discordant (Missed M230L by Stanford) |
| S35 | - | K70E, M184V | K101P, E138K | - | Concordant |
| S36 | - | M184V | - | T97A, Y143R, N155H | Concordant |
| S38 | - | M184V | Y181C | - | Discordant (Missed Y181C by Stanford) |
| S39 | - | K70R, L74I, L74V, M184V, K219E | V108I, Y181C, M230L | - | Concordant |
| S40 | - | K65R, S68G, V75I, M184V | K101E, V106I, V108I, V179D, Y181C, G190A, H221Y | - | Concordant |
| S41 | - | M41L, K70Q, M184V | K103N | - | Concordant |
| S42 | - | M184V | V106I, E138Q, H221Y, F227C | - | Concordant |
| S43 | - | T69Tdel | K103N, Y181C, G190A | - | Discordant (T69Tdel misreported as T69G by Quasitools) |
| Pipeline | TP | TN | FP | FN | PPA (%) [95% CI] | NPA (%) [95% CI] | Cohen’s Kappa [95% CI] |
|---|---|---|---|---|---|---|---|
| iLunaR | 35 | 50 | 0 | 0 | 100.00 [90.00, 100.00] | 100.00 [92.90, 100.00] | 1.000 [1.000, 1.000] |
| Exatype | 33 | 50 | 0 | 2 | 94.30 [80.80, 99.30] | 100.00 [92.90, 100.00] | 0.951 [0.884, 1.000] |
| Stanford | 32 | 50 | 0 | 3 | 91.40 [76.90, 98.20] | 100.00 [92.90, 100.00] | 0.926 [0.846, 1.000] |
| Quasitools | 31 | 50 | 0 | 4 | 88.60 [73.30, 96.80] | 100.00 [92.90, 100.00] | 0.901 [0.807, 0.995] |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Lee, O.F.; Lee, C.K. A Comparative Evaluation of Four Bioinformatic Tools for Identifying HIV-1 pol Drug Resistance Mutations Using Illumina MiSeq Data. Biology 2026, 15, 438. https://doi.org/10.3390/biology15050438
Lee OF, Lee CK. A Comparative Evaluation of Four Bioinformatic Tools for Identifying HIV-1 pol Drug Resistance Mutations Using Illumina MiSeq Data. Biology. 2026; 15(5):438. https://doi.org/10.3390/biology15050438
Chicago/Turabian StyleLee, Ogestelli Fabia, and Chun Kiat Lee. 2026. "A Comparative Evaluation of Four Bioinformatic Tools for Identifying HIV-1 pol Drug Resistance Mutations Using Illumina MiSeq Data" Biology 15, no. 5: 438. https://doi.org/10.3390/biology15050438
APA StyleLee, O. F., & Lee, C. K. (2026). A Comparative Evaluation of Four Bioinformatic Tools for Identifying HIV-1 pol Drug Resistance Mutations Using Illumina MiSeq Data. Biology, 15(5), 438. https://doi.org/10.3390/biology15050438

