Diagnostic Performance of a Rapid Antigen Test Compared with the Reverse Transcription Polymerase Chain Reaction for SARS-CoV-2 Detection in Asymptomatic Individuals Referring to a Drive-in Testing Facility

Abstract

Quick and reliable identification of severe acute respiratory syndrome coronavirus SARS-CoV-2 in the population is required to manage the COVID-19 pandemic. This is a prospective observational study of diagnostic accuracy. Paired swab samples from 317 asymptomatic individuals referring to a drive-in testing facility were tested in parallel by means of the rapid antigen test developed by Jiangsu Bioperfectus Technologies and routine nucleic acid detection. Overall specificity was 100% and sensitivity was 49% but reached 87% at higher viral loads (Ct < 25). In this study, the antigen detection test showed high specificity and good sensitivity in asymptomatic individuals carrying higher viral loads. The assay performance worsened with lower viral loads, making it useful when a rapidly deployable test is essential and to assess a potential risk of immediate transmission in the community, but not recommended for testing asymptomatic individuals.

1. Introduction

Severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), the causal agent of coronavirus disease 2019 (COVID-19), is having a profound impact on global health with long-lasting effects on the economy worldwide. Global control of the pandemic will require a pervasive rollout of effective vaccines. However, an efficient contact-tracing protocol and mass community testing will remain essential until full coverage is obtained. Therefore, there is an urgent need for accurate and rapid test methods to quickly identify large numbers of infected individuals and asymptomatic carriers to prevent virus transmission and assure timely isolation procedures.

A large proportion of SARS-CoV-2-infected individuals with high viral loads might be asymptomatic [1,2]. Nucleic acid amplification tests (NAAT) such as the reverse transcription polymerase chain reaction (RT-PCR) technology are the routine for virus detection. RT-PCR is highly sensitive and specific [3,4]. However, RT-PCR has disadvantages, including the necessity of professional laboratory expertise, costly reagents, and centralized equipment. Therefore, alternatives have been developed, including antigen tests that detect viral proteins in respiratory samples [5]. Antigen tests are more cost-effective and generally optimized for point-of-care (POC) use. The advantages of antigen tests, such as relatively low cost and short turnaround time, can contribute to prompt identification of contagious individuals with high viral loads. According to the Centers for Disease Control and Prevention (CDC), RT-PCR testing should be considered after negative antigen test results in symptomatic individuals and after positive antigen test results in asymptomatic individuals [6]. Although antigen tests might not be as accurate as RT-PCR testing, they are more accessible in terms of availability and ease of use and can be used to scale-up testing in the community [7,8].

Rapid antigen diagnostic tests (Ag-RDT), also called antigen point-of-care tests (Ag-POCT) or lateral flow devices (LFD), are a viable alternative to NAAT for detecting infected individuals in the population [9]. There is a large number of Ag-RDTs available on the market, with different sensitivities [10,11,12,13,14]. Hence, independent assessment is key to informed decisions on their proper use for mass testing.

The main objective of this study was to evaluate the performance of the Bioperfectus Technologies Novel Corona Virus (SARS-CoV-2) Ag Rapid Test Kit produced by Jiangsu Bioperfectus Technologies for the direct detection of SARS-CoV-2 antigens in comparison to the gold-standard RT-PCR method. Paired specimens were nasopharyngeal swabs prospectively collected from asymptomatic or pauci-symptomatic individuals in an area of intense virus circulation. The results show an overall sensitivity of 49%, with specificity of 100% and increased sensitivity of 87% below 25 Ct.

2. Materials and Methods

2.1. Study Design

This was an observational and cross-sectional study on swab samples collected prospectively from 317 individuals who referred for a SARS-CoV-2 molecular assay at a drive-in testing center in Trieste, Italy, between February and May 2021. Two deep nasopharyngeal swab specimens were collected from the same individual for RT-PCR and Ag-RDT. One swab was analyzed by RT-PCR and, once the specimen was confirmed positive, the other swab from the same donor was tested with the antigen test within 24 h from collection, with specimens preserved at 4 °C. The swabs for RT-PCR were maintained in virus transport media (VTM), while the swabs for AG-RDT were not resuspended in a medium during storage and were resuspended in a lysis buffer for RDT at the time of testing (<24 h from collection). Study design required at least 200 PCR specimens and 100 negative PCR specimens to be evaluated: 70% of the specimens were selected to have cycle threshold (Ct) values < 30, and 30% of the specimens had Ct values ≥ 30 on the comparator PCR assay according to the WHO recommendations for emergency use listing (EUL) [15].

2.2. Reagents

The index Ag-RDT was the Novel Corona Virus (SARS-CoV-2) Ag Rapid Test Kit manufactured by Jiangsu Bioperfectus Technologies Co., Ltd. (reference number SC30107W, batch number 20201101) performed according to the instructions. Briefly, the swab was resuspended vigorously in a tube with the supplied extraction buffer, closed with a dropper cap, and mixed by flicking. Then, the tube was inverted, and 2–3 drops were deposited in the reservoir of a test cassette. The assay was visually assessed after 15 min of incubation at room temperature and recorded with a photo.

The comparator RT-PCR test varied depending on the source available at the time of sampling. The following kits were used during this study: GSD NovaPrime^®® SARS-CoV-2 (COVID-19) (Eurofins Tecna s.r.l.); NLM Leaflet ITA SARS-CoV-2 Real Time (AA1571/96S and /960S); NeoPlex™ COVID-19 (Eurospital); SARS-CoV-2 ELITe MGB^®®Kit (ELITechGroup); Bosphore novel coronavirus (2019-nCoV) detection kit. Each test has its own targets in the SARS-CoV-2 genome and internal controls. Positive and negative samples were assigned based on the RT-PCR results and referred to the participants. The Ct values were averaged for each sample to obtain a single value used for stratification of data.

Quantification of SARS-CoV-2 genomes was performed using synthetic viral RNA as previously described, with a procedure and reagents freely available through the ICGEB [16,17,18]. Calculated logarithmic dilutions of viral RNA from a laboratory isolate (SARS-CoV-2 ICGEB-FVG_5) were plotted against the average Ct values of the Liferiver RT-PCR method from two independent replicates in triplicate repeats to obtain a nonlinear regression standard curve to interpolate the unknown values using GraphPad Prism version 9.0.

2.3. Statistical Analysis

The confusion matrix (contingency 2 × 2 table) method for data analysis was used to derive parameters such as sensitivity, specificity, negative and positive predictive values, precision, and accuracy of the index assay compared with the reference. The Cohen’s kappa coefficient was used to measure the interrater reliability. The data were analyzed with R [19]. To compare the assay with the reference, the caret package (https://CRAN.R-project.org/package=caret - accessed on 1 September 2021) was used to compute the main confusion matrix statistics. The irr package (https://CRAN.R-project.org/package=irr - accessed on 1 September 2021) was used to compute the value of Cohen’s kappa, a parameter commonly used to measure interrater reliability [20], while the vcd package (https://CRAN.R-project.org/package=vcd- accessed on 1 September 2021) was employed to obtain its 95% confidence interval. For data refining, assay sensitivity and accuracy were calculated considering only the reference positive cases in function of their RT-PCR results. The results were finally visualized using the “ggplot2 (https://CRAN.R-project.org/package=ggplot2- accessed on 1 September 2021) and gridExtra (https://CRAN.R-project.org/package=gridExtra- accessed on 1 September 2021) packages.

3. Results

Sensitivity and Specificity of Ag-RDT

A total of 317 samples were collected between February and May 2021 in a drive-in SARS-CoV-2 testing facility located in Trieste, Italy. The sample included 120 negative RT-PCR samples collected in parallel with the positive ones each week across the study. According to study design, 67% of the positive RT-PCR samples were selected with a Ct < 30, while the remaining ones had Ct ≥ 30. Individuals referring to the drive-in for testing were asymptomatic or pauci-symptomatic subjects who needed a COVID-19 test for various reasons, including travel and exit from quarantine. Circulation of SARS-CoV-2 in the area during the period of the study showed a shift towards the B.117 variant (alpha variant), typical of the third epidemic wave that hit Italy between February and April 2021 [21]. Indeed, the major variants of concern (VOC: alpha, beta, gamma, delta) and variants of interest (VOI: epsilon, zeta, theta, iota, kappa, lambda) are covered by the test and could be detected with the limits of detection (LOD) comparable with the original sequence (40 pg/mL). The rapid antigen tests should also remain unaffected based on the preliminary analysis of the nucleocapsid of the omicron VOC as well. The nucleocapsid of omicron carries mutations G204R and R203K that are already present in the previous variants, so it is unlikely to cause problems, as well as P13L and 31–33del (deletion E31, R32, S33) that are outside of the binding sites of the antibodies used in the test according to in silico simulations. However, verification of this is underway.

Overall clinical sensitivity of the Ag-RDT was 49%, with a specificity of 100%, positive predictive value (PPV) of 100%, and negative predictive value (NPV) of 55%. The overall accuracy of the test was 68%, with the 95% confidence interval between 35% and 50% (95% confidence interval (CI), 35–50%) and the p-value of 4.2 × 10⁻²³ (McNemar’s test). Stratification of the Ag-RDT data on the RT-PCR Ct values showed that sensitivity spiked to 87% at Ct < 25 (N = 91, accuracy of 87% with 95% CI = 78–93%, p-value = 0.0015) while decreasing rapidly at Ct ≥ 25 (N = 106, sensitivity of 17%, accuracy of 17% with 95% CI = 10–26%, p-value = 18 × 10⁻²⁰). The validity of the Ag-RDT for Ct < 25 is visually evident in Figure 1 and

Table 1. Stratification of data on Ct values.

	Sensitivity	Specificity	PPV	NPV	N	Precision	Accuracy (95% CI)	p (McNemar’s Test)	Cohen’s Kappa (95% CI)
Aggregate	49%	100%	100%	55%	317	100%	68% (63–74%)	4.20 × 10−23	42% (35–50%)
CT < 25	87%	NA	NA	NA	91	100%	87% (78–93%)	0.0015	NA
CT ≥ 25	17%	NA	NA	NA	106	100%	17% (10–26%)	1.80 × 10−20	NA

, which summarizes the data.

According to the manufacturer’s specification, the clinical sensitivity of the Ag-RDT compared to RT-PCR was 97.06%, with a specificity of 99.15%, from studies conducted on symptomatic individuals [22]. These values meet the WHO criteria on the use of SARS-CoV-2 Ag-RDTs (≥80% sensitivity and ≥97% specificity compared to an approved NAAT [9]. These assays are to be considered for use only in the areas where NAAT is unavailable or where the health system may be overburdened, leading to prolonged NAAT turnaround times (>48−72 h). The manufacturer’s declared LOD was 100 plaque-forming units/mL, which was confirmed in an independent assessment and calculated to correspond to 1.5 × 10⁸ genomes/mL, well above the LOD of the RT-PCR assays used in this study that ranged between 104 to 2.6 × 10² genomes/mL according to the manufacturer’s specifications. These data confirm that Ag-RDTs are most likely to perform well in patients with high viral loads, which usually appear 1–3 days before symptom onset and early symptomatic phases of the illness (first 5–7 days).

4. Discussion

The study presented here, which includes a significant proportion of low viral load samples, provides further support for the correct use of Ag-RDTs in early symptomatic individuals and in the scenarios depicted by WHO [9] that include (i) quick evaluation of suspected outbreaks in remote settings and in closed or semi-closed groups including schools, care homes, cruise ships, prisons, workplaces, etc.; (ii) monitoring of trends in disease incidence and actuation of isolation procedures where there is widespread community transmission. Ag-RDT can also be applied in the following specific situations: (i) when testing is required for large gatherings of people, such as concerts or other events; (ii) to allow to return to work in the countries adopting the requirement of a green pass (e.g., Italy [21]); (iii) for travel. In these scenarios, the validity of the Ag-RDT results should not exceed 48 h. In fact, the chances of isolating infectious SARS-CoV-2 and transmitting the virus are drastically reduced at low viral loads with Ct < 25 [23,24,25,26].

To conclude, the results clearly show that Ag-RDT is highly specific, but sensitivity is acceptable only at Ct < 25 with higher viral loads. The test is therefore applicable in situations where POCT is essential or where a short-term evaluation of infectivity is required. Ag-RDT remains not recommended for testing individuals in a drive-in setting.