Next Article in Journal
A New Comestible Formulation of Parasiticide Fungi to Reduce the Risk of Soil-Transmitted Helminth Infections in a Canine Shelter
Previous Article in Journal
Re-Emergence of Circulation of Seasonal Influenza during COVID-19 Pandemic in Russia and Receptor Specificity of New and Dominant Clade 3C.2a1b.2a.2 A(H3N2) Viruses in 2021–2022
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Clinical Evaluation of Three Commercial RT-PCR Kits for Routine COVID-19 Diagnosis

by
Rifat Tasnim Juthi
,
Saiful Arefeen Sazed
,
Md Fahad Zamil
and
Mohammad Shafiul Alam
*
International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), Dhaka 1212, Bangladesh
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Pathogens 2022, 11(11), 1389; https://doi.org/10.3390/pathogens11111389
Submission received: 24 October 2022 / Revised: 18 November 2022 / Accepted: 19 November 2022 / Published: 21 November 2022

Abstract

:
Amongst the multiple ways to diagnose coronavirus disease-2019 (COVID-19), reverse transcription polymerase chain reaction (RT-PCR) remains the reference gold standard, providing fast and accurate results. This study evaluated and compared the performance of three commercially available COVID-19 RT-PCR kits-Aridia® COVID-19 Real-Time PCR Test (CTK Biotech, Inc., Poway, CA, USA), Novel Coronavirus (2019-nCoV) Nucleic Acid Detection Kit (Sansure Biotech Inc., Changsha, China) and AllplexTM 2019-nCoV assay (Seegene Inc., Seoul, Republic of Korea) for the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A total of 326 clinically suspected patients were enrolled for the study, and among them, 209 were diagnosed as positive and 117 as negative when tested with the reference method, US CDC 2019-Novel Coronavirus (2019-nCoV) Real Time RT-PCR Diagnostic Panel. The Aridia® kit showed total agreement with the reference test, with a sensitivity of 100% (95% CI: 98.25% to 100.0%) and a specificity of 100% (96.90% to 100.00%). The AllplexTM kit also showed 100% specificity (95% CI: 96.90% to 100.00%), but a lower sensitivity (98.09%, 95% CI: 95.17% to 99.48%). Among the three kits, the Novel Coronavirus (2019-nCoV) Nucleic Acid Detection Kit showed the worst performance, with a sensitivity of 98.6% (95% CI: 95.9% to 99.7%) and a specificity of 95.73, 95% (CI: 90.31% to 98.60%). While all these kits conform to the requirement for routine molecular diagnosis with high performances, the Aridia® COVID-19 Real-Time PCR Test showed the best performance among the three kits.

1. Introduction

COVID-19 has already caused more than 6.5 million deaths and over 625 million cases all over the world (accessed on 21 October 2022) [1]. For fast diagnosis of the disease, there are numerous rapid tests commercially available in the market [2]. Among other diagnostic methods reverse transcription polymerase chain reaction (RT-PCR), Enzyme-linked Immuno Sorbent Assay (ELISA), radiographic analysis (X-Ray, CT-scan), next-generation sequencing (NGS), and whole genome sequencing (WGS) are currently available [3,4,5,6]. Of these, the most common and accurate method to diagnose COVID-19 is quantitative (real-time) RT-PCR analysis of viral RNA [7]. The most used samples for this analysis are nasal, nasopharyngeal, oropharyngeal or naso-oropharyngeal derived, collected with cotton swabs and then transferred into viral transport medium (VTM) [8]. Viral RNA is then extracted from the VTM and subjected to RT-PCR analysis that provides the SARS-CoV-2 status of the sample.
There are more than 350 commercially available RT-PCR kits that offer highly sensitive and more specific platforms for the diagnosis of COVID-19 [9]. Several commercially available RT-PCR kits for COVID-19 have obtained authorization for emergency use by various authorities such as World Health Organization (WHO) and the United States food and drug administration (US-FDA). Up to recently, twenty-nine kits have obtained US-FDA Emergency Use Authorization (EUA) [9]. Multiple other kits are also available commercially, but a thorough investigation of their performance is warranted so that the most accurate result possible can be obtained at the patient level.
Coronavirus RNA contains several gene targets for molecular detection and, by the amplification of any one or more target genes by RT-PCR, the presence or absence of the virus that causes COVID-19 can be determined. Some of the most commonly amplified genes include the nucleocapsid (N) gene, envelope (E) gene, RNA-dependent RNA polymerase (RdRP), open reading frame 1ab (ORF1ab), N1 and N2 regions of N gene, etc. [10]. Different manufacturers choose different sets of genes for their RT-PCR kits [11]. The different RT-PCR kits generally contain primers, probes, reverse transcriptase enzymes, PCR buffers, RNase-free water, etc. These components come in different formats and containers, and the user typically has to mix them to prepare a master mix for PCR analysis [4]. Some manufacturers offer more user-friendly kits that include a single tube master mix, and the analysts just have to add the RNA template, which minimizes sample handling errors and setup time [12].
In this study, we evaluated and compared the diagnostic efficacy of these three commercially available RT-PCR kits-Aridia® COVID-19 Real-Time PCR Test (CTK Biotech, Inc., Poway, CA, USA), Novel Coronavirus (2019-nCoV) Nucleic Acid Detection Kit (Sansure Biotech Inc., Changsha, China) and AllplexTM 2019-nCoV assay (Seegene Inc., Seoul, Republic of Korea) for the purpose of detecting SARS-CoV-2, because these kits are widely used in the diagnostic laboratories of many countries of the world, including ours.

2. Materials and Methods

2.1. Study Population and Ethical Consideration

The participants of this study were from Dhaka city, Bangladesh, and were symptomatic at the time of sample collection. Patients coming to the diagnostic facility of the International Center for Diarrhoeal Disease Research Bangladesh (icddr,b) in Dhaka, Bangladesh for a COVID-19 RT-PCR test between 3rd February and 8th August of 2021 were enrolled in the study when most of the circulating variants were beta variant followed by the delta variant. The study was conducted with 326 patients showing fever, cough, sore throats and any other COVID-19 symptoms selected and enrolled for testing by healthcare professionals during that time. This study was approved by the institutional Ethical Review Committee (ERC) of icddr,b. Written informed consent was obtained from the adult participants. For children below 11 years, written informed consent was obtained from the parent or legal guardian. In the case of children between 11 and 17 years of age, verbal assent was also obtained in addition to written consent from the parent or legal guardian.

2.2. Specimen Collection

During the specimen collection session, nasopharyngeal as well as oropharyngeal swabs were collected from the enrolled participants in 3 mL of VTM by expert healthcare professionals to perform the gold standard RT-PCR assay.

2.3. COVID-19 Diagnosis by RT-PCR

RT-PCR amplification of RNA extracted from naso-oropharyngeal swabs collected in VTM from prospective patients was carried out in the Molecular Diagnostic Laboratory, which is an ISO 15189 and ISO 15190 accredited facility.
The naso-oropharyngeal samples were extracted using MagMAX™ Viral/Pathogen II nucleic acid isolation kit with a King Fisher Flex high-throughput automated extraction system (Thermo Fisher Scientific, Waltham, MA, USA).
For the reference assay, N1 and N2 gene targets with RNaseP as human internal control were amplified using primer probes suggested by CDC (Center for Disease Control). In brief, TaqPath™ 1-Step RT-qPCR mastermix (Thermo Fisher Scientific, Waltham, MA, USA) was used in ABI 7500 Fast DX instrument (Thermo Fisher Scientific, Waltham, MA, USA) in a 20 μL reaction mix containing 5 μL of the template. The thermocycler conditions consist of a step of reverse transcription (10 min/55 °C), an initial denaturation (1 min/95 °C), and subsequent 45 cycles of denaturation (10 s/95 °C) and annealing/elongation (30 s/55 °C) [7,13]. This assay was considered as a reference RT-PCR assay comprising N1 and N2 CDC primers for COVID-19.

2.4. Aridia® COVID-19 Real-Time PCR Test

CTK’s Aridia® COVID-19 Real-Time PCR Test packaging contains a COVID-19 PCR mix, COVID-19 positive and negative controls and PCR-grade water is also provided to reconstitute the lyophilized components. The COVID-19 PCR mix contains all real-time PCR components including DNA polymerase, reverse transcriptase, primers, probes, and dNTPs. For the RT-PCR, 15 µL of PCR mix and 5 µL of RNA sample template, positive control and negative control were mixed to target the ORF1ab, N, E, and a housekeeping gene RNase P. PCR was performed according to the manufacturer’s instruction. A Bio-Rad CFX Opus96 Real-Time PCR System was used for the amplification and the thermocycler was programmed as followed: reverse transcription (10min, 50 °C, one cycle), (2 min, 95 °C, one cycle), denaturation (5 s, 95 °C, 45 cycles) and finally, annealing/extension (20 s, 60 °C, 45 cycles). At the extension step, fluorescence data are collected through HEX, JOE or VIC (ORF1ab), FAM (N gene), Cy5 (E gene), and ROX (RNase P) channels by Bio-Rad CFX Maestro Software Version 2.2. When N, ORF1ab and E genes showed cycle threshold (Ct) < 40, samples were considered positive and when ≥40, samples were considered negative for COVID-19.

2.5. AllplexTM 2019-nCoV Assay (Seegene Inc.)

Components of Seegene’s AllplexTM 2019-nCoV assay kit (ref.: RP10252W, Lot: RP5022A01) are slightly different from Aridia®. This kit contains 2019-nCoV MOM, real-time one-step enzyme, 5X real-time one-step buffer and RNase-free water. For preparing the master mix, 2 µL of real-time one-step enzyme and 5 µL of each of the other three components are mixed and 8 µL of RNA template is added for RT-PCR. For amplification, the same instrument was used as the Aridia® test kit; the thermocycler protocol was as follows: reverse transcription (20 min, 50 °C, one cycle), initial denaturation (15 min, 95 °C, one cycle), denaturation (15 s, 94 °C, 45 cycles) and finally, annealing (30 s, 58 °C, 45 cycles). Four fluorophores: Quasar 670, FAM, Cal Red 610 and HEX are selected for the analytes: N gene, E gene, RdRp gene and internal control respectively; the same software recorded the fluorescence data. The Ct value of internal control must be <40, otherwise, the test will be considered invalid and that particular sample should be retested for clarification. If the Ct value is <40 for three other genes, or only for the N gene, the sample is considered COVID-19 positive [14].

2.6. Novel Coronavirus (2019-nCoV) Nucleic Acid Detection Kit (Sansure Biotech)

Sansure Biotech’s Novel Coronavirus (2019-nCoV) Nucleic Acid Detection Kit (Ref: S3102E, Lot: 2022001-2) for 24 tests contains 2019-nCoV-PCR Mix, 2019-nCoV-PCR-enzyme mix, 2019-nCoV-PCR-positive control and 2019-nCoV-PCR-negative control. As per the manufacturer’s manual, the volume of the reaction mix is 50 µL (30 µL PCR-master mix and 20 µL RNA template) but in this study, maintaining the recommended proportion, 15 µL reaction mix was prepared (9 µL PCR-master mix and 6 µL RNA template) [15]. Cycle parameters were as follows: reverse transcription (50 °C, 30 min, one cycle), cDNA denaturation (95 °C, 1 min, one cycle), denaturation (95 °C, 15 s, 45 cycles), annealing, extension and fluorescence collection (60 °C, 30 s, 45 cycles) and finally device cooling (25 °C, 10 s, one cycle). FAM(ORF1ab), ROX(N) and CY5(IC) channels were selected for signal receiving. Just like the AllplexTM kit, the internal control gene’s Ct value must be <40 to confirm the presence of RNA, otherwise, the samples were retested. To consider a sample COVID-19 positive, the Ct value of the N gene and/or ORF1ab gene must be <40 [15].

2.7. Lower Limit of Detection (LLOD), Linearity and Assay Reproducibility

To detect the analytical sensitivity of these three kits, the NGS sequence verified SARS-CoV-2 Plasmid Controls with 200,000 copies/µL (Research Use Only) was diluted sequentially to obtain a set of 100,000 copies/µL to 0.01 copies/µL. For each kit, the RT-PCR was performed in triplicates using similar conditions for each one to detect the LLOD. A standard curve was also generated for each kit and the assay was repeated to verify its linearity and reproducibility. The variation was determined in terms of the Coefficient of Variation, CV (the ratio of the standard deviation, SD to the mean) for intra and inter-assay [16].

2.8. Data Analysis

For statistical analysis and graphical representation of the data, GraphPad Prism 8.0.2 (GraphPad Software, Inc., San Diego, CA, USA) was used and the diagnostic sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive and negative likelihood ratio with respective confidence interval were calculated. MedCalc statistical software (V.20.013) (MedCalc Software, Ostend, Belgium) was also used for building Bland–Altman plots. The Cohen’s kappa coefficient was estimated to determine the performance of the kits using SPSS statistics software (IBM, version 20.0.0.2, Armonk, NY, USA) [7].

3. Results

3.1. RT-qPCR Results

Among the 326 prospective COVID-19 patients, 209 (64.10%) were diagnosed positive and the remaining 117 (35.89%) were negative by the reference RT-PCR method. The demographic data can be found in our previous rapid test study [7].

3.2. Performance of Aridia® COVID-19 Real-Time PCR Test

The Aridia® COVID-19 kit gave similar results to the reference method. Thus, the sensitivity and specificity of the Aridia® COVID-19 kit were 100% in both cases (95% CI: 98.3% to 100.0% for sensitivity and 96.9% to 100.0% for specificity), compared to the reference method. The performance has been described in Table 1.

3.3. Performance of AllplexTM 2019-nCoV Assay

The AllplexTM 2019-nCoV assay kit could detect 205 out of 209 positive samples. For the 117 negative samples, AllplexTM successfully detected all of them as negative. Comparing this result to that of reference RT-PCR, the sensitivity and specificity of the AllplexTM kit were 98.1% (95% CI: 95.2% to 99.5%) and 100% (95% CI: 96.9% to 100%) respectively. These have been described in brief in Table 1.

3.4. Performance of Novel Coronavirus (2019-nCoV) Nucleic Acid Detection Kit, Sansure Biotech

Among the 209 positive samples, Sansure Kit could detect 206 samples. On the other hand, 112 out of 117 negative samples were detected as negative by the Sansure kit. The kit falsely amplified the target gene of five negative samples. These false positive and false negative values resulted in the reduced specificity of the kit (95.7%, 95% CI: 90.3% to 98.6%). The sensitivity (98.6%; 95% CI: 95.9% to 99.7%) was similar to the AllplexTM kit, and lower than the Aridia® COVID-19 kit (Table 1).

3.5. Sensitivity in Detection of N Gene

The N gene was a common target of all three kits but their sensitivity of detection was different. The distribution of Ct values of the N gene is summarized in Table 2 and illustrated in Figure 1.
The Ct values of the N gene were slightly higher for the AllplexTM kits than the Aridia® kits but for the Sansure kits, they were mostly lower for the corresponding samples (p < 0.001). These have been shown in Figure 2 with a line graph indicating the Ct values of each sample by the terminal of each line.
The difference in Ct values has been also portrayed through the Bland–Altman plot in Figure 3a,b for the N gene of different kits. The analysis shows a significant mean difference of Ct values for AllplexTM-Aridia® is 2.7 and 1.8 for Aridia®-Sansure (p < 0.001).

3.6. Sensitivity in Detection of E Gene

E gene was a common target of the Aridia® and AllplexTM kits but their sensitivity of detection was different. The E gene of six positive samples could not be amplified by AllplexTM kits. Ct values were significantly different from each other (p < 0.0001). The distribution of Ct values of the E gene is summarized in Table 3 and illustrated in Figure 4.

3.7. Sensitivity in Detection of ORF1ab Gene

The ORF1ab gene was a common target of the Aridia® and Sansure kits but their sensitivity of detection was different. The ORF1ab gene of one sample could not be amplified by the Sansure kit. The distribution of Ct values of the ORF1ab gene is summarized in Table 4 and illustrated in Figure 5.
Additionally, the ORF1ab gene of two negative samples was falsely amplified by the Sansure kit with Ct > 35 which was shown in red triangles in Figure 5.

3.8. LLOD, Linearity, and Reproducibility

All three real-time RT-PCR kits evaluated in this study could detect as low as 1 copy/µL. Some of these kits could even detect even lower copy numbers but, due to lack of reproducibility, we have found the lower limit of detection as 1 copy/µL. The correlation coefficients (R2) were ≥0.994 and efficiency was around 100% in all cases. The intra-assay CV of Ct values for Aridia®, AllplexTM, and Sansure ranged from 0.17% to 0.97%, 0.26% to 0.54% and 0.31% to 2.82% respectively over six different concentrations. In the case of the Sansure kit, increased variations were observed in low copy numbers. Assay reproducibility was assessed through inter-assay Ct value variation for the same concentration level in two different independent triplicate runs. The CV for inter-assay fluctuated from 0.8% to 1.28%, 0.02% to 0.63% and 0.01% to 1.17% for Aridia®, AllplexTM, and Sansure respectively. The CV values point out the excellent reproducibility of all three assay methods. The LLOD, linearity, repeatability and reproducibility have been summarized in Table 5, Table 6 and Table 7 and portrayed in Figure 6, Figure 7 and Figure 8.

4. Discussion

A number of studies have been done so far to assess the performance of commercially available COVID-19 RT-PCR kits, but there are many kits being used for regular diagnosis that warrant further evaluation as well. Of the kits we evaluated in this study, CTK’s Aridia® Kit was the most convenient compared to the two other kits, as it was an all-in-one master mix in ambient-stable lyophilized form, easy to prepare with the provided nuclease-free water and controls for each assay. The lyophilized format, also eases shipment and storage at room temperature. On the other hand, AllplexTM kit and Sansure kits contain several vials and the contents are in liquid form, which require maintenance of the cold chain (−20 °C to 4 °C) and if the cold chain is interrupted because of logistic issues, the performance of the kits will be compromised.
Although several studies have already evaluated the AllplexTM kit and Sansure Kit [15,17,18,19], none has been undertaken so far for the Aridia® kit. Our study evaluated and compared the performance of the Aridia® kit with already established and available commercial kits that are also highly used in this region, AllplexTM kit and Sansure kit. The Aridia® kit result showed the best performance among the three kits, with a sensitivity and specificity of 100%. The AllplexTM kit’s specificity was also 100%, but it showed higher Ct values for the detection of the N gene (p < 0.001) than the other two kits. The performance of the Sansure Kit in this study fell slightly behind, as it failed to detect three positive and five negative samples correctly.
Compared to our study, a recent study evaluated three PCR kits, namely, GeneFinderTM (OSANG Healthcare Co., Seongnam, Korea), Sansure Biotech (Sansure Biotech Inc., Changsha, China), and TaqPathTM (Thermo Fisher, Waltham, MA, USA) on 354 randomly selected samples. They showed that the Sansure Biotech assay had better diagnostic performance than the GeneFinderTM and TaqPathTM PCR kits [20]. The Sansure kit was evaluated against the FDA EUA 2019-nCoV CDC kit (IDT, Coralville, IA) and showed 95.3% of sensitivity with a limit of detection as low as 1000 copies/mL in South America [15]. In another study, the Sansure kit had a low sensitivity of 83.3% although a specificity of 100% in a very limited number of samples in a study conducted in Guangxi, China [21]. For routine COVID-19 real-time RT-PCR tests, the Sansure kit had a 90% of positive percent agreement in 86 symptomatic patients in Malaysia [22]. The Sansure kit performs well in samples Ct < 30 with a sensitivity of 94.6% even for extraction-free SARS-CoV-2 assay but the overall sensitivity falls to 69.6% in a sample size of 94 patients with 69 positives. Compared to that, the AllplexTM had a sensitivity of 98.7% in a study with 115 positive samples when NGS was used as a reference in Malaysia [19]. Another study assessed the performance of the AllplexTM in more than 1000 samples including extracted RNA, nasopharyngeal swabs, and saline gargles using the whole genome sequencing process as standard. This kit could easily detect several mutations with 100% sensitivity [18]. The performance can drastically fall to 74% of sensitivity for the extraction-free assay method [23].
In our study, we found the LLOD for all kits to be the same, 1 copy/µL and hence, a comparative analysis could not be drawn among these kits for this parameter. A notable major limitation was the multiple freeze-thaw of the samples. The reference RT-PCR assay was performed within 24 h of sample collection, with the maintenance of the cold chain. Due to limitations of resources and logistics, these kits were evaluated later, with archived extracted RNA that has been thawed multiple times. Another drawback of the study is that we did not have any choice in sample collection, especially with higher Ct values or patients in critical hospital settings for better comparison of the kits as we used the diagnostic facilities only. Additionally, these are commercial kits and hence, they only mentioned their gene target but did not disclose the sequences. Their varied performance can be due to different gene targets than the reference one. The primer can also play a role in better detection but we could not analyze the performance in terms of primer sequences. Nevertheless, these kits performed very well in terms of sensitivity and specificity. These kits are only used for diagnostic purposes, and other studies are required to evaluate these kits’ ability to detect all the variants of concerns and point mutations that are emerging worldwide.

5. Conclusions

Many countries are already using rapid tests for the diagnosis of COVID-19 in clinical settings which require further confirmation for negative tests in highly symptomatic patients. There is no better alternative to RT-PCR tests, and they are still performed in hospitals extensively. Therefore, good-quality kits are required so that no patient remains misdiagnosed. From the detailed investigation in our study, we conclude that the Aridia® kit showed the best performance of all the three kits evaluated, followed by the other two commercially available RT-PCR kits included in this study, and thus, can be used for the routine molecular diagnosis of COVID-19.

Author Contributions

Conceptualization, R.T.J. and S.A.S.; methodology, S.A.S. and R.T.J.; formal analysis, S.A.S.; investigation, M.F.Z.; data curation, M.F.Z.; writing—original draft preparation, R.T.J. and S.A.S.; writing—review and editing, M.S.A.; supervision, M.S.A.; project administration, M.S.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research study was funded by CTK Biotech Inc., Poway, CA, USA, grant number GR-02066. The APC will be provided by the PI of the study. The donor has no influence over the study design, sample collection, data analysis and manuscript writing.

Institutional Review Board Statement

The study was approved by the institutional ethical review committee (ERC) of the International Center for Diarrhoeal Disease Research, Bangladesh (icddr,b). The institutional review board (IRB) approved the study with protocol no PR-21034.

Informed Consent Statement

A written informed consent was obtained from the adult participants. For children below 11 years, written informed consent was obtained from the parent or legal guardian. In the case of children between 11 and 17 years of age, verbal assent was also obtained in addition to written consent from the parent or legal guardian.

Data Availability Statement

Not applicable.

Acknowledgments

icddr,b acknowledges with gratitude the commitment of CTK Biotech Inc to its research efforts. icddr,b is also grateful to the Governments of Bangladesh, Canada, Sweden and the UK for providing core/unrestricted support.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. World Health Organization (WHO). Coronavirus COVID-19 Dashboard. Available online: https://covid19.who.int (accessed on 8 December 2021).
  2. Randriamahazo, T.R.; Andrianarivelo, A.M.; Rakotoarivo, A.T.; Raheritiana, T.M.; Rakotovao, L.A.; Randriamanantany, Z.A.; Alson, A.O.R.; Rasamindrakotroka, A. Evaluation of antigen-based rapid detection test for the diagnosis of SARS CoV-2 in low-income countries. J. Virol. Methods 2021, 300, 114409. [Google Scholar] [CrossRef] [PubMed]
  3. Sharma, A.; Balda, S.; Apreja, M.; Kataria, K.; Capalash, N.; Sharma, P. COVID-19 Diagnosis: Current and Future Techniques. Int. J. Biol. Macromol. 2021, 193, 1835–1844. [Google Scholar] [CrossRef] [PubMed]
  4. Yüce, M.; Filiztekin, E.; Özkaya, K.G. COVID-19 diagnosis—A review of current methods. Biosens. Bioelectron. 2020, 172, 112752. [Google Scholar] [CrossRef] [PubMed]
  5. Apostolopoulos, I.D.; Mpesiana, T.A. Covid-19: Automatic detection from X-ray images utilizing transfer learning with convolutional neural networks. Phys. Eng. Sci. Med. 2020, 43, 635–640. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Verma, J.; Mishra, A.S. COVID-19 infection: Disease detection and mobile technology. PeerJ 2020, 8, e10345. [Google Scholar] [CrossRef]
  7. Sazed, S.A.; Kibria, M.G.; Hossain, M.S.; Zamil, F.; Adhikary, P.C.; Hossain, M.E.; Ahmed, D.; Haque, R.; Alam, M.S. Clinical Evaluation of a New Antigen-Based COVID-19 Rapid Diagnostic Test from Symptomatic Patients. Diagnostics 2021, 11, 2300. [Google Scholar] [CrossRef] [PubMed]
  8. Torretta, S.; Zuccotti, G.; Cristofaro, V.; Ettori, J.; Solimeno, L.; Battilocchi, L.; D’Onghia, A.; Bonsembiante, A.; Pignataro, L.; Marchisio, P.; et al. Diagnosis of SARS-CoV-2 by RT-PCR using different sample sources: Review of the literature. Ear Nose Throat J. 2021, 100 (Suppl. 2), 131S–138S. [Google Scholar] [CrossRef]
  9. Garg, A.; Ghoshal, U.; Patel, S.S.; Singh, D.V.; Arya, A.K.; Vasanth, S.; Pandey, A.; Srivastava, N. Evaluation of seven commercial RT-PCR kits for COVID-19 testing in pooled clinical specimens. J. Med. Virol. 2020, 93, 2281–2286. [Google Scholar] [CrossRef]
  10. Hu, B.; Guo, H.; Zhou, P.; Shi, Z.L. Characteristics of SARS-CoV-2 and COVID-19. Nat. Rev. Microbiol. 2021, 19, 141–154. [Google Scholar] [CrossRef]
  11. van Kasteren, P.B.; van Der Veer, B.; van den Brink, S.; Wijsman, L.; de Jonge, J.; van den Brandt, A.; Molenkamp, R.; Reusken, C.B.; Meijer, A. Comparison of seven commercial RT-PCR diagnostic kits for COVID-19. J. Clin. Virol. 2020, 128, 104412. [Google Scholar] [CrossRef] [PubMed]
  12. Sisay, A.; Abera, A.; Dufera, B.; Endrias, T.; Tasew, G.; Tesfaye, A.; Hartnack, S.; Beyene, D.; Desta, A.F. Diagnostic accuracy of three commercially available one step RT-PCR assays for the detection of SARS-CoV-2 in resource limited settings. PLoS ONE 2022, 17, e0262178. [Google Scholar] [CrossRef]
  13. Kudo, E.; Israelow, B.; Vogels, C.B.; Lu, P.; Wyllie, A.L.; Tokuyama, M.; Venkataraman, A.; Brackney, D.E.; Ott, I.M.; Petrone, M.E.; et al. Detection of SARS-CoV-2 RNA by multiplex RT-qPCR. PLoS Biol. 2020, 18, e3000867. [Google Scholar] [CrossRef]
  14. Umunnakwe, C.N.; Makatini, Z.N.; Maphanga, M.; Mdunyelwa, A.; Mlambo, K.M.; Manyaka, P.; Nijhuis, M.; Wensing, A.; Tempelman, H.A. Evaluation of a commercial SARS-CoV-2 multiplex PCR genotyping assay for variant identification in resource-scarce settings. PLoS ONE 2022, 17, e0269071. [Google Scholar] [CrossRef]
  15. Freire-Paspuel, B. and Garcia-Bereguiain, M.A. Clinical Performance and Analytical Sensitivity of Three SARS-CoV-2 Nucleic Acid Diagnostic Tests. Am. J. Trop. Med. Hyg. 2021, 104, 1516–1518. [Google Scholar] [CrossRef]
  16. Sazed, S.; Kibria, M.; Alam, M. An Optimized Real-Time qPCR Method for the Effective Detection of Human Malaria Infections. Diagnostics 2021, 11, 736. [Google Scholar] [CrossRef]
  17. Čretnik, T.Ž.; Retelj, M.; Janežič, S.; Mahnič, A.; Tesovnik, T.; Šket, R.; Bizjan, B.J.; Jeverica, S.; Ravnik, M.; Rak, M.; et al. Accuracy of Allplex SARS-CoV-2 assay amplification curve analysis for the detection of SARS-CoV-2 variant Alpha. Future Microbiol. 2022, 17, 1125–1131. [Google Scholar] [CrossRef] [PubMed]
  18. Caza, M.; Hogan, C.A.; Jassem, A.; Prystajecky, N.; Hadzic, A.; Wilmer, A. Evaluation of the clinical and analytical performance of the Seegene allplex™ SARS-CoV-2 variants I assay for the detection of variants of concern (VOC) and variants of interests (VOI). J. Clin. Virol. 2021, 144, 104996. [Google Scholar] [CrossRef] [PubMed]
  19. Fu, J.Y.; Chong, Y.M.; Sam, I.C.; Chan, Y.F. SARS-CoV-2 multiplex RT-PCR to detect variants of concern (VOCs) in Malaysia, between January to May 2021. J. Virol. Methods 2022, 301, 114462. [Google Scholar] [CrossRef]
  20. Banko, A.; Petrovic, G.; Miljanovic, D.; Loncar, A.; Vukcevic, M.; Despot, D.; Cirkovic, A. Comparison and Sensitivity Evaluation of Three Different Commercial Real-Time Quantitative PCR Kits for SARS-CoV-2 Detection. Viruses 2021, 13, 1321. [Google Scholar] [CrossRef]
  21. Lu, Y.; Li, L.; Ren, S.; Liu, X.; Zhang, L.; Li, W.; Yu, H. Comparison of the diagnostic efficacy between two PCR test kits for SARS-CoV-2 nucleic acid detection. J. Clin. Lab. Anal. 2020, 34, e23554. [Google Scholar] [CrossRef]
  22. Wen, D.; Yang, S.; Li, G.; Xuan, Q.; Guo, W.; Wu, W. Sample-to-Answer and Routine Real-Time RT-PCR: A Comparison of Different Platforms for SARS-CoV-2 Detection. J. Mol. Diagn. 2021, 23, 665–670. [Google Scholar] [CrossRef]
  23. Morecchiato, F.; Coppi, M.; Baccani, I.; Maggini, N.; Ciccone, N.; Antonelli, A.; Rossolini, G.M. Evaluation of extraction-free RT-PCR methods for faster and cheaper detection of SARS-CoV-2 using two commercial systems. Int. J. Infect. Dis. 2021, 112, 264–268. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Distribution of Ct values (N gene). The green symbols in Aridia® Kit depict the samples which were positive for Aridia® only. The green symbols in Sansure Kit depict three falsely amplified N gene on negative samples.
Figure 1. Distribution of Ct values (N gene). The green symbols in Aridia® Kit depict the samples which were positive for Aridia® only. The green symbols in Sansure Kit depict three falsely amplified N gene on negative samples.
Pathogens 11 01389 g001
Figure 2. Line Graph comparing the Ct values of N gene for each sample, generated by different kits.
Figure 2. Line Graph comparing the Ct values of N gene for each sample, generated by different kits.
Pathogens 11 01389 g002
Figure 3. (a) Bland–Altman Plot of Ct values of N gene for AlplexTM-Aridia® kits. The X-axis contains Ct values of Aridia® kit and the Y axis contains the difference of Ct (AllplexTM kit)-Ct (Aridia® kit); (b) Bland–Altman Plot of Ct values of N gene for Aridia®-Sansure kits. The X-axis contains Ct values of Aridia® kit and the Y axis contains the difference of Ct (Aridia® kit)-Ct (Sansure kit).
Figure 3. (a) Bland–Altman Plot of Ct values of N gene for AlplexTM-Aridia® kits. The X-axis contains Ct values of Aridia® kit and the Y axis contains the difference of Ct (AllplexTM kit)-Ct (Aridia® kit); (b) Bland–Altman Plot of Ct values of N gene for Aridia®-Sansure kits. The X-axis contains Ct values of Aridia® kit and the Y axis contains the difference of Ct (Aridia® kit)-Ct (Sansure kit).
Pathogens 11 01389 g003aPathogens 11 01389 g003b
Figure 4. Distribution of Ct values (E gene). The red symbols in Aridia® Kit depict the E gene of samples that were positive for Aridia® kit only.
Figure 4. Distribution of Ct values (E gene). The red symbols in Aridia® Kit depict the E gene of samples that were positive for Aridia® kit only.
Pathogens 11 01389 g004
Figure 5. Distribution of Ct values (ORF1ab gene). The red circles for the Aridia® plot depict the ORF1ab gene of samples that were positive for the Aridia® kit only. Two red triangles for Sansure kit represent the false positive result.
Figure 5. Distribution of Ct values (ORF1ab gene). The red circles for the Aridia® plot depict the ORF1ab gene of samples that were positive for the Aridia® kit only. Two red triangles for Sansure kit represent the false positive result.
Pathogens 11 01389 g005
Figure 6. Analytical performance and LOD of the Aridia® real-time RT-PCR. Plasmid control was diluted from 100,000 to 1 copy/µL. Amplification curves are represented by different colors for each different viral load. The standard curve has been made by a linear regression curve fit analysis (R2 = 0.999, E = 104.8%).
Figure 6. Analytical performance and LOD of the Aridia® real-time RT-PCR. Plasmid control was diluted from 100,000 to 1 copy/µL. Amplification curves are represented by different colors for each different viral load. The standard curve has been made by a linear regression curve fit analysis (R2 = 0.999, E = 104.8%).
Pathogens 11 01389 g006
Figure 7. Analytical performance and LOD of the AllplexTM real-time RT-PCR. Plasmid control was diluted from 100,000 to 01 copy/µL. Amplification curves are shown by different colors for each different viral load. The standard curve has been made by a linear regression curve fit analysis (R2 = 0.999, E = 100.2%).
Figure 7. Analytical performance and LOD of the AllplexTM real-time RT-PCR. Plasmid control was diluted from 100,000 to 01 copy/µL. Amplification curves are shown by different colors for each different viral load. The standard curve has been made by a linear regression curve fit analysis (R2 = 0.999, E = 100.2%).
Pathogens 11 01389 g007
Figure 8. Analytical performance and LOD of the Sansure real-time RT-PCR. Plasmid control was diluted from 100,000 to 01 copy/µL. Amplification curves are portrayed by different colors for each different viral load. The standard curve has been made by a linear regression curve fit analysis (R2 = 0.994, E = 102.4%).
Figure 8. Analytical performance and LOD of the Sansure real-time RT-PCR. Plasmid control was diluted from 100,000 to 01 copy/µL. Amplification curves are portrayed by different colors for each different viral load. The standard curve has been made by a linear regression curve fit analysis (R2 = 0.994, E = 102.4%).
Pathogens 11 01389 g008
Table 1. Performance analysis of Aridia®, AllplexTM and Sansure COVID-19 RT-PCR kits with CDC RT-PCR as reference.
Table 1. Performance analysis of Aridia®, AllplexTM and Sansure COVID-19 RT-PCR kits with CDC RT-PCR as reference.
ParameterAridia®AllplexTMSansure
Sensitivity100 (95% CI: 98.3% to 100%)98.1% (95% CI:
95.2% to 99.5% )
98.6% (95% CI:
95.9% to 99.7%)
Specificity100.00% (95% CI:
96.9% to 100%)
100% (95% CI:
96.9% to 100%)
95.7% (95% CI:
90.3% to 98.6%)
AUC (Area Under Curve)1 (95% CI:
0.99 to 1.0)
0.99 (95% CI:
0.97 to 0.99)
0.971 (95% CI:
0.95 to 0.99)
Positive Predictive Value100 %100.00%97.6% (95% CI:
94.6% to 98.9%)
Negative Predictive Value100 %99.90% (95% CI:
99.7% to 99.9%)
99.9% (95% CI:
99.8% to 99.9%)
Accuracy100.00% (98.9% to 100%)99.90% (95% CI:
98.7% to 100%)
95.87% (95% CI:
93.1% to 97.8%)
κ1.00.970.95
Table 2. Distribution of Ct values of N gene in different ranges.
Table 2. Distribution of Ct values of N gene in different ranges.
Ct ValuesAridia®AllplexTMSansure
<25128 (61.2%)89 (43.6%)147 (70.3%)
25–3044 (21.2%)59 (28.9%)42 (20.1%)
30–3533 (15.8%)40 (19.6%)17 (8.1%)
>354 (1.9%)16 (7.8%)0 (0%)
Total samples209204 **206
** 204 samples were false negative and AllplexTM kit could not amplify N gene of one sample.
Table 3. Distribution of Ct values of E gene in different ranges.
Table 3. Distribution of Ct values of E gene in different ranges.
Ct ValuesAridia® Kit, n (%)AllplexTM Kit, n (%)
<25136 (65.1%)117 (57.6%)
25–3039 (18.7%)53 (26.1%)
30–3531 (14.8%)28 (13.8%)
>3503 (1.4%)05 (2.5%)
Total samples209203 ***
*** E gene was unamplified in six samples.
Table 4. Distribution of Ct values of ORF1ab gene in different ranges.
Table 4. Distribution of Ct values of ORF1ab gene in different ranges.
Ct ValuesAridia® Kit, n (%)Sansure Kit, n (%)
<25157 (75.1%)137 (65.9%)
25–3038 (18.2%)47 (22.6%)
30–3514 (6.7%)20 (9.6%)
>3500 (0%)04 (1.9%)
Total samples209208 ****
**** ORF1ab gene was unamplified in one sample and amplified in two false positive samples with Ct value > 35.
Table 5. Repeatability and reproducibility of real-time RT-PCR assay for Aridia® kit.
Table 5. Repeatability and reproducibility of real-time RT-PCR assay for Aridia® kit.
Copies/µLIntra Assay Variation of Ct ValuesInter Assay Variation of Ct Values
Replicate 1Replicate 2Replicate 3MeanSDCV (%)Assay 1Assay 2MeanSDCV (%)
100,00022.5322.3822.722.540.160.7122.5422.7922.670.180.80
10,00025.8825.9925.9125.930.060.2225.9325.4625.700.331.28
100029.2329.1729.0629.150.090.3029.1528.5728.860.411.44
10032.3132.4632.232.320.130.4032.3231.9032.110.300.92
1036.0535.5235.435.660.350.9735.6635.2135.430.320.89
138.6538.5438.5338.570.070.1738.5739.0438.810.330.85
Table 6. Repeatability and reproducibility of real-time RT-PCR assay for AllplexTM kit.
Table 6. Repeatability and reproducibility of real-time RT-PCR assay for AllplexTM kit.
Copies/µLIntra Assay Variation of Ct ValuesInter Assay Variation of Ct Values
Replicate 1Replicate 2Replicate 3MeanSDCV (%)Assay 1Assay 2MeanSDCV (%)
100,00023.2223.1323.2623.200.070.2923.2023.1823.190.020.07
10,00026.6326.3526.5526.510.140.5426.5126.4926.500.020.06
100030.130.0629.8830.010.120.3930.0130.0130.010.000.02
10033.5633.3533.3333.410.130.3833.4133.3433.380.050.16
1036.5236.7636.6636.650.120.3336.6536.5936.620.040.10
139.6639.5539.7639.660.110.2639.6639.3039.480.250.63
Table 7. Repeatability and reproducibility of real-time RT-PCR assay for Sansure kit.
Table 7. Repeatability and reproducibility of real-time RT-PCR assay for Sansure kit.
Copies/µLIntra ASSAY Variation of Ct ValuesInter Assay Variation of Ct Values
Replicate 1Replicate 2Replicate 3MeanSDCV (%)Assay 1Assay 2MeanSDCV (%)
100,00020.8620.7620.7420.790.060.3120.7920.6820.740.070.35
10,00024.123.952424.020.080.3224.0224.0024.010.010.04
100027.5227.3627.2827.390.120.4527.3927.2427.310.110.39
10030.8730.4730.3530.560.270.8930.5630.5330.550.020.07
1034.432.8733.5633.610.772.2833.6133.6233.610.000.01
136.58--38.0737.331.052.8237.3336.7137.020.431.17
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Juthi, R.T.; Sazed, S.A.; Zamil, M.F.; Alam, M.S. Clinical Evaluation of Three Commercial RT-PCR Kits for Routine COVID-19 Diagnosis. Pathogens 2022, 11, 1389. https://doi.org/10.3390/pathogens11111389

AMA Style

Juthi RT, Sazed SA, Zamil MF, Alam MS. Clinical Evaluation of Three Commercial RT-PCR Kits for Routine COVID-19 Diagnosis. Pathogens. 2022; 11(11):1389. https://doi.org/10.3390/pathogens11111389

Chicago/Turabian Style

Juthi, Rifat Tasnim, Saiful Arefeen Sazed, Md Fahad Zamil, and Mohammad Shafiul Alam. 2022. "Clinical Evaluation of Three Commercial RT-PCR Kits for Routine COVID-19 Diagnosis" Pathogens 11, no. 11: 1389. https://doi.org/10.3390/pathogens11111389

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop