Evaluation of an Antigen Detection Rapid Diagnostic Test for Detection of SARS-CoV-2 in Clinical Samples

Abstract

Antigen detection rapid diagnostic tests have been developed for first-line large-scale screening given their rapidity, simplicity, and accuracy. This study evaluates the diagnostic performance of an antigen detection rapid diagnostic test (BLOK BioScience, London, UK) detecting SARS-CoV-2 nucleocapsid protein. Serially diluted SARS-CoV-2 isolate and 110 NPS from COVID-19 patients were tested to determine the test’s sensitivity, and other viral isolates and 20 NPS from non-infected individuals were, for specificity, also tested. Ten clinical samples from COVID-19 patients with SARS-CoV-2 variants, including alpha, beta, gamma, delta, and eta variants, were collected to evaluate the test’s potential application in detecting emerging variants. Overall sensitivity was 92%, and stratifying into viral loads yielded 100% for Ct < 25 samples including SARS-CoV-2 variants, but 11.11% for Ct ≥ 30 samples. The analytical sensitivity of log₁₀ TCID₅₀/mL 2.0 was identified for SARS-CoV-2. Ninety-seven percent specificity with only SARS-CoV cross-reactivity lead to the Youden index of 0.89. The rapid diagnostic test has a high sensitivity for detecting SARS-CoV-2 in high viral load samples, possibly including emerging SARS-CoV-2 variants, but reduced sensitivity in low viral load samples suggests its optimized usage as a complementary testing method to other tests, including RT-PCR or a point-of-care test for large-scale screening, particularly for pandemic areas or airport border infection control.

1. Introduction

Coronaviruses, first reported as a cause of avian respiratory disease in 1931 [1] and first isolated from humans in the 1960s [2,3], are medium-sized viruses with a lipid envelope and a non-segmented, single-stranded, positive-sense RNA genome that is the largest in known RNA viruses [4,5]. Under the order of Nidovirales and the family Coronaviridae, betacoronaviruses include the species Severe acute respiratory syndrome-related coronavirus that consists of the prototype severe acute respiratory syndrome coronavirus (SARS-CoV) and phylogenetically related viruses, including severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [6,7]. Initially reported as a cluster of pneumonia of unknown aetiology in December 2019, SARS-CoV-2 infections in humans were documented across all seven continents by 22 December 2020, leading to the coronavirus disease 2019 (COVID-19) pandemic with more than 245 million confirmed cases and 4.9 million deaths by 1 November 2021 that requires active interventions to prevent the continuation of spreading [8,9,10].

Given its high transmissibility with a propensity to spread in asymptomatic or pre-symptomatic stages due to the long incubation period reaching above 14 days [11,12,13], and the peak viral load at symptom onset [14,15,16,17], the early detection of cases of SARS-CoV-2 infection is essential to prevent further spread from infected patients, as well as permanent physical damage and long-term complications due to delayed treatment [18]. Different diagnostic modalities have been developed for various applications, in which nucleic acid amplification by reverse transcription-polymerase chain reaction (RT-PCR) is the current gold standard [19], which is highly sensitive but time-consuming and requires specific laboratory skills and instruments such as thermocyclers. For rapid diagnosis, antigen detection methods, including commercial rapid diagnostic tests (RDT), were developed to generate results within 30 min and without any particular equipment and biosafety facilities, enabling their applications as point-of-care tests, although with varying accuracy for diagnosis [20,21]. This study evaluates the diagnostic performance of the Diagnostic Kit for SARS-CoV-2 Ag (BLOK BioScience, London, UK) test cassettes with results detected by Immunofluorescence Analyzer FIA-680 (DIAsia, Shenzhen, China) by testing cultured isolates and clinical samples, including nasopharyngeal swabs (NPS).

2. Materials and Methods

2.1. Clinical Specimens

Clinical samples were collected as NPS with cotton swabs that were immediately put in 2.5 mL of viral transport medium [22] and immediately sent to the Clinical Microbiology Laboratory, Queen Mary Hospital of Hong Kong, for viral investigation. Samples including 110 NPS from COVID-19 patients, and 20 from non-infected individuals as controls, were collected in the Queen Mary Hospital, whose results were confirmed by the clinical laboratory using RT-PCR with LightMix Modular E-gene kit (TIB Molbiol, Berlin, Germany) [23] and verified by RdRp/Hel [24]. Ten clinical specimens of either NPS or deep-throat saliva with SARS-CoV-2 variants were collected with its lineage confirmed by sequencing. Leftover samples after confirmation were collected for RDT testing. The present study was approved by the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster (UW13-372) and was performed in accordance with the guidelines and regulations.

2.2. Test Evaluation

The lateral flow fluorescence immunochromatographic RDT (BLOK BioScience) was optimized for testing nasopharyngeal and oropharyngeal secretions collected by sterile flocked swabs, with the suggested procedures involving direct elution of swabs into 500 μL of extraction buffer, in which 100 μL of samples in extraction buffer were added to the sample port of the test cassette for fluorescent signals to be detected by an immunofluorescence analyzer after 10 min. The manufacturer’s protocol was unfeasible in this study, given that all NPS were transferred to the laboratory after immersion into the viral transport medium. Thus the actual operation of RDT requires estimation of the maximum quantity of samples to be absorbed by and eluted from the provided swabs, which was approximated to be 100 μL of samples added to 500 μL of extraction buffer. To prevent wastage of NPS samples, as only 100 μL was added to the test cassette, it was deduced that the addition of 17 μL of NPS liquid samples to 83 μL of extraction buffer was sufficient, which formed the revised protocol for liquid samples in this study. Then, 100 μL of extracted sample was added to the sample port of the test cassette, followed by 10 min of waiting, and finally insertion of the test cassette into the immunofluorescence analyzer for measurement of fluorescent signals. The addition of NPS samples and waiting were performed in a biosafety cabinet II for safety reasons. RDT (BLOK BioScience) gives semi-quantitative results showing both concentrations in ng/mL and interpretations including −, +, ++, +++ and ++++.

2.3. Virus Isolates

To determine the lower limit of detection of the RDT (BLOK BioScience), serial 10-fold dilution and subsequent 2-fold dilution were performed on SARS-CoV-2 cultured isolates from Vero E6 cells. Viral titres of each diluted sample were deduced from the median tissue culture infectious dose (TCID₅₀) method and RT-PCR method in terms of Ct value and number of target RdRp/Hel gene copies per mL, and these methods were performed in parallel with the RDT evaluation of each diluted viral isolate. Other than the 20 clinical NPS from non-infected individuals, specificity was examined in terms of cross-reactivity with other viruses by adding cultured isolates onto the test cassettes, including SARS-CoV, MERS-CoV, HCoV-229E, HCoV-NL63, HCoV-OC43, influenza A virus, influenza B virus, respiratory syncytial virus, adenovirus, human metapneumovirus, and parainfluenza virus types 1, 2, 3 and 4. The cultured isolates of SARS-CoV2, SARS-CoV and MERS-CoV were prepared in a biosafety level 3 laboratory.

3. Results

A SARS-CoV-2 cultured isolate of titre 6.5 log₁₀ TCID₅₀/mL was serially diluted in 10-fold to test the detection limit of the RDT [22], with further 2-fold dilutions performed in quadruplicate, from the endpoint above, to determine the limit of detection more accurately. The median tissue culture infectious dose (TCID₅₀), copy numbers of RdRp/Hel gene, and Ct values of each diluted isolate were used to compare RDT and RT-PCR detection. At 95% detection probability derived from probit analysis, the detection limit of the RDT was 2.0 log₁₀ TCID₅₀/mL and equivalent to 5.2 log₁₀ RdRp/Hel RNA copies/mL with the Ct value of around 31.9.

Among the 110 NPS samples from COVID-19 patients, in which 56% were male and the median age was 54, the mean Ct value was 21.1 (interquartile range (IQR) = 24.2 − 19.2 = 5.0), which was low and indicated a generally high viral load. A total of 101 samples were positive, in which semi-quantitative results generated from the immunofluorescence analyzer revealed the number of samples with ++++, +++, ++, + and − results to be 79, 5, 11, 6 and 9 respectively, with mean Ct values of 19.9 (standard deviation (SD) = 2.4), 24.6 (SD = 0.2), 24.0 (SD = 1.8), 28.1 (SD = 2.2), and 33.2 (SD = 3.1). With reference to the RT-PCR results, all clinical samples with a Ct value lower than 21.2 showed ++++ results in the RDT, and positive results could be up to Ct value of 30.9. As for the false-negative results, the sample with the highest viral load that yielded negative RDT results had Ct value of 28.9, and the minimum viral load of the RT-PCR positive sample tested with RDT had Ct value of 37.8. Figure 1 revealed a clear difference between the Ct values of samples that are RDT positive and negative, with negative results clustering above Ct value of 30, as compared to positive results in which samples being interpreted as ++++ were below a 25 Ct value and most positive samples had Ct values lower than 30, despite a slight overlap in error bars and the presence of an outlier. Pearson’s correlation was used to identify any relationships between RDT positivity and RT-PCR Ct values, which demonstrated a statistically significant strong correlation between the two (r = −0.717, p-value < 0.05).

For specificity, none of the 20 clinical NPS from non-infected individuals were positive. Fourteen cultured isolates of related viruses were tested, including common respiratory viruses and all human coronaviruses except unculturable HCoV-HKU1, in which only SARS-CoV showed cross-reactivity. SARS-CoV viral isolate [22] was serially diluted to identify the detection limit at 4.4 log₁₀ TCID₅₀/mL.

Based on all NPS samples, the test performance characteristics were evaluated, with a high, overall sensitivity of 92% (95% CI 85–96%) and combined specificity of 97% (95% CI 85–100%), leading to the Youden index being 0.89. For better analysis of data, samples were stratified into four groups by RT-PCR Ct values, revealing distinct differences in RDT sensitivity at different viral loads (

Table 1. Test performance characteristics of RDT.

Parameters	Results [95% Confidence Interval (CI)]
Overall Sensitivity	92% (85–96%)
Sensitivity in Ct < 20 samples	100% (91–100%)
Sensitivity in 20 ≤ Ct < 25 samples	100% (94–100%)
Sensitivity in 25 ≤ Ct < 30 samples	88% (47–100%)
Sensitivity in Ct ≥ 30 samples	11% (0–48%)
Specificity	97% (85–100%)
Positive predictive value (PPV)	99% (94–100%)
Negative predictive value (NPV)	79% (66–87%)
Youden index	0.89

and Figure 2). RDT could accurately detect SARS-CoV-2 in samples with a high viral load with a Ct value below 30.

For SARS-CoV-2 variants, nine clinical specimens with alpha, beta, gamma, delta and eta variants showed positive RDT results, with Ct values ranging from 15.0 to 25.0, while an alpha variant sample with a Ct value of 35.0 was negative (

Table 2. Results of RDT in clinical samples containing SARS-CoV-2 variants.

Variant	Sample Type	Ct Value	RDT Result
Alpha	Deep throat saliva	18.1	++++
Alpha	Deep throat saliva	19.9	++++
Alpha	Deep throat saliva	21.0	++++
Alpha	Deep throat saliva	35.0	–
Beta	Deep throat saliva	21.0	++++
Gamma	Deep throat saliva	15.0	++++
Delta	Deep throat saliva	16.3	++++
Delta	Deep throat saliva	18.1	++++
Delta	NPS	21.3	+++
Eta	NPS	25.0	+++

).

4. Discussion

The analytical sensitivity is important for the evaluation of RDT, given that the minimum viral load required for reliable detection of a positive result is clinically relevant, with each 10-fold rise in the detection limit of SARS-CoV-2 leading to an additional 13% false-negativity rate [25]. In this study, PPV of RDT (BLOK BioScience) was high at 99%, and NPV was lower due to false-negative results, yet their calculations depend on disease prevalence, which was 76% in the tested samples. Hence, the PPV and NPV would be altered with a reduced disease prevalence. For example, disease prevalence of 1% and 10% would yield 24% PPV (95% CI 4–69%) and 100% NPV (95% CI 100–100%), and 78% PPV (95% CI 33–96%) and 99% NPV (95% CI 98–99%), respectively. Thus, the true estimate of PPV and NPV depends on epidemiological conditions influencing the pre-test probability. The Youden index, deduced from overall sensitivity and specificity, of 0.89 is appropriate as a rapid diagnostic test.

RDT (BLOK BioScience) can detect SARS-CoV but at a higher detection limit than SARS-CoV-2. This would be expected since SARS-CoV-2 shares approximately 80% genomic homology with SARS-CoV [26]. The combined specificity of RDT (BLOK BioScience) being 97% (95% CI 85–100%) was in fact only contributed by the cross-reactivity with SARS-CoV, but no other cultured isolates and clinical samples.

The difference in Ct values of samples yielding positive and negative results were shown in Figure 1, which is compatible with the estimated detection limit of SARS-CoV-2 at a Ct value of 31.9, correlating with a 5.2 log₁₀ RNA copy number. Interpretation as ++++ generally represents samples with low Ct values, allowing RDT as a quick estimate of the viral load. The stratified sensitivity and stacked bar chart (Figure 2) further exemplified the accuracy of RDT to detect samples with a high viral load, as 100% positivity was detected at Ct < 25, which was supported by the strong correlation between RDT positivity and RT-PCR Ct values from the Pearson’s correlation coefficient. This enables its application in clinical settings to accurately identify patients with a high viral load, which are postulated to be linked to increased infectivity and disease severity [27,28]. The highly conserved nucleocapsid also provides a stable antigen against the emerging SARS-CoV-2 variants, as demonstrated by the strong positive results of ++++ and +++ in clinical specimens with alpha, beta, gamma, delta, and eta variants (Table 2). The negative result of a sample with an alpha variant and a high Ct value of 35.0 was compatible with the results of the non-variant SARS-CoV-2 NPS and detection limit of 31.9 and did not demonstrate any reduced sensitivity for SARS-CoV-2 variants despite a limited sample size. Deep-throat saliva samples were utilized for some samples with SARS-CoV-2 variants that generated positive results, revealing its potential as a sample type for RDT (BLOK BioScience), although deep-throat saliva samples may yield poorer sensitivity than NPS [29]. Its ability to detect SARS-CoV-2 variants thus allows the test to be used for airport border infection control to screen incoming visitors with a high viral load.

A recent meta-analysis has revealed the pooled sensitivity and specificity of antigen detection RDT as 68% and 99% [30], as compared to the overall sensitivity and specificity of 92% and 97% in this study. Previous reviews of different studies, evaluating various antigen detection RDT, have shown similar results, where samples with Ct values less than 25 could be identified accurately but not at high Ct values [21,31], although no studies on the diagnostic performance of RDT from BLOK BioScience have been published at the time of writing. An equivalently high sensitivity of 100% for high viral load samples with reduction to 72% at Ct > 25.1 was reported in a study evaluating another fluorescence-based antigen detection RDT (Bioeasy Biotechnology Co., Shenzhen, China) [32], while another similar RDT (SD Biosensor, Suwon, South Korea) documented 100% sensitivity for Ct < 18 samples that was reduced to 94% for 18 ≤ Ct < 25, 42% for 25 ≤ Ct < 35 and 21% for Ct ≥ 35 with detection limits of 500 TCID₅₀/mL (2.7 log₁₀ TCID₅₀/mL) [33]. This study revealed similar detection patterns as the two RDTs using fluorescence readers, with comparable sensitivity as RDT from Bioeasy Biotechnology Co., given that their Ct values were not further stratified at a Ct value of 30, and higher sensitivity for Ct < 25 and lower detection limits than RDT from SD Biosensor. This study also demonstrated successful detection of SARS-CoV-2 variants in both NPS and deep-throat saliva samples. Among all 110 clinical NPS from COVID-19 patients, 101 samples were tested positive by RDT (BLOK BioScience), contributing to the high sensitivity of 92% (95% CI 85–96%). However, a limitation of the study would be the small sample size with a relatively high viral load among all samples leading to a mean Ct value below 25, and sampling bias due to the recruitment of only hospitalized COVID-19 patients that may differ from a community setting, leading to a potential overestimation of sensitivity. Thus, stratification of sensitivity based on Ct values was necessary. Further evaluation of its performance in low viral load samples is required, although this study demonstrated its high sensitivity at high viral loads. Moreover, the small sample size of negative samples may be insufficient to confirm the specificity, which may require further investigations. Another possible limitation of the study is the non-adherence to suggested procedures of immersing respiratory swabs into extraction buffer directly; such deviation of testing liquid samples eluted from cotton swabs may lead to the underestimation of sensitivity, as diluted clinical samples contain viruses of lower concentrations. Yet, the dilution effect by the immersion of cotton swabs into 2.5 mL transport medium (<0.5 log₁₀ dilution) is unlikely to significantly affect the sensitivity, and this study demonstrated high sensitivity and specificity despite modifications to suit liquid samples for convenience and the best use of limited resources, suggesting its potential application to liquid samples in the future.

With its ease-of-use, rapidity, simple procedures not requiring any special facilities or expertise, cost-effectiveness, and ability to detect SARS-CoV-2 variants at a high viral load, the evaluated RDT (BLOK BioScience) could serve as a point-of-care test that enables on-site results interpretation and immediate action, including isolation and contact tracing, in contrast to RT-PCR diagnosis that requires several hours-to-days before results are released. This facilitates decentralizing and increasing the efficiency of COVID-19 testing, yet false-negative results at low viral loads necessitate confirmatory testing of negative samples by RT-PCR, such that RDT (BLOK BioScience) is a complementary method to identify high viral load patients. Minimal hands-on work, with a short turnaround time of around 11 min per sample, allows simultaneous testing of multiple samples to the throughput of up to approximately 60 samples per hour by a single operator. The flexibility of RDT (BLOK BioScience) is also contributed by the stability of all ready-to-use materials under room temperature that allows long-term storage, the disposability of all materials that omits steps of autoclaving or disinfection before reuse, and the lack of requirements of sophisticated hardware. For the tested RDT, the machine-generated results minimize person-to-person variability, and semi-quantitative results offered additional information, shedding light on viral concentration, as compared to subjective endpoints such as colorimetric changes. RDT with a reader generally has higher sensitivity than non-reader RDT [34,35].

The overall sensitivity and specificity of the evaluated tests exceeded the WHO threshold for antigen detection RDT that is an 80% and 97% specificity recommendation [20], yet it is important to note that the sample population may not represent the general population, as exemplified by the high viral load, and the testing of SARS-CoV viral isolate for specificity that may not be present in the community. To further evaluate its potential diagnostic applications, expansion in sample size, especially in testing other specimen types such as deep-throat saliva and other variant lineages, could be performed. Application in the community could be evaluated by collecting samples from COVID-19 testing centers rather than hospitals.

Ultimately, rapid diagnostic tests, including the evaluated lateral flow fluorescence immunochromatographic assay, could be applied for large-scale and decentralized screening or diagnosis of SARS-CoV-2, including SARS-CoV-2 variants, especially to identify patients of a high viral load. Low viral load cases may be missed by the RDT, and thus it is recommended for the test to be complementary to molecular assays such as RT-PCR, with RDT-negative samples with a high pre-test probability, especially symptomatic patients, sent for RT-PCR, while RDT-positive cases would be immediately isolated and treated as confirmed cases, allowing early identification and prevention of transmission, especially for pandemic areas or airport border infection control.