Currently, the revelation of SARS-CoV-2 RNA through real-time reverse transcription-polymerase chain reaction (RT-PCR) from nasopharyngeal swab samples is considered the standard gold method for the diagnosis of SARS-CoV-2 infection. The targeted genes may include a combination of N
, and orf1b
]. However, even if the RT-PCR method is highly sensitive, positive SARS-CoV-2 samples do not always allow definitive conclusions about whether the subject is contagious or not [3
]. This can be partly assessed by the cycle threshold (Ct) value, which has been correlated to the amount of viral RNA in the sample, informing on the viral load and infectivity [4
In addition, in most laboratories, the analysis throughput of RT-PCR testing is limited because it requires a high workload, skillful operators, expensive instrumentation, and crucial biosafety measures [5
]. In contrast, antigen detection (AD) tests are more rapid, less laborious, less expensive, and require only a comparatively short training period. Currently, there are approximately 100 different AD tests on the market [7
]. Head-to-head comparisons of AD test performance for SARS-CoV-2 detection are scarce. The majority of the studies only included few patients, precluding a wide characterization of the “real-life” performance of these devices [3
]. Recently, a Cochrane study found an average sensitivity of only 56.2% (95% confidence interval (CI): 29.5–79.8%) for rapid AD (RAD) tests [11
]. Independent reporting of automated AD test performance remains limited [18
This study aimed to compare the clinical performance of 4 RAD tests ((i) the Biotical SARS-CoV-2 Ag card, (ii) the Panbio™ COVID-19 Ag Rapid Test Device, (iii) the Coronavirus Ag Rapid Test Cassette, and (iv) the Roche SARS-CoV-2 Rapid Antigen Test)) and one automated AD test (VITROS Immunodiagnostic Products SARS-CoV-2 Antigen test), to the results of the RT-PCR.
This study compared and analyzed the clinical performance of 5 AD tests, including the automated assay from Ortho Clinical Diagnostics. The RAD tests were most effective to identify RT-PCR positive symptomatic patients or asymptomatic subjects with higher viral loads (i.e., Ct values ≤25). The sensitivity of these RAD tests for samples with a Ct values ≤25 was 93.1% for the Biotical and the Panbio assays, while it was 96.6% for the Healgen and the Roche assays. Only the Biotical assay missed an asymptomatic patient with a Ct value of 23.4. For Ct values comprised between 25 and 30, all RAD tests were not able to detect three asymptomatic subjects with Ct values of 25.5, 26.3, and 29.3, respectively. Sensitivities decreased proportionally with higher Ct values. The best sensitivity was observed with the Healgen assay (i.e., 88.8% for samples with Ct values ≤33). The VITROS automated assay outclassed all RAD tests with a sensitivity and a specificity of 100% for Ct values up to 33 (Table 2
). The fact that the VITROS assay presented better performance compared to RAD tests is certainly related to an increased limit of quantification made possible by the chemiluminescent technology used.
In the literature, we identified at least four studies that evaluated the Panbio COVID-19 Ag rapid test [8
]. In a population of 341 patients, Fenollar et al. found a specificity of 94.9% and a good sensitivity but only for samples with Ct values <25 (i.e., 94.2 to 100%) [12
]. In a smaller cohort of 51 RT-PCR positive patients, Lanser et al. found a sensitivity of 85.8% in Ct values <25 [3
]. Albert et al. confirmed a high specificity (i.e., 100%) with an overall sensitivity of 79.6% in a population of 54 RT-PCR positive patients. For Ct values <25, the sensitivity increased above 95% [8
]. The same conclusion was found by Mak et al., using only 35 specimens [17
]. The lower sensitivity observed with higher Ct values is in line with our evaluation (Table 2
). Krüttgen et al. evaluated the Roche SARS-CoV-2 Rapid Antigen Test on a population of 75 patients with a positive RT-PCR and 75 with a negative RT-PCR and found a specificity of 96% and a 100% specificity in samples with Ct values lower than 25 [13
]. Using the same assay, Salvagno et al. found a sensitivity ranging from 97 to 100% for specimens with Ct values <25, but lower sensitivities for higher Ct values (i.e., 50–81% for Ct values of 25 to <30) [23
]. These results are also consistent with the data obtained in this study. Of note, this study is the first to evaluate the Biotical and Healgen RAD tests.
Our study is also the first to evaluate the new automated AD test provided by Ortho Clinical Diagnostics. Compared to RAD tests, the sensitivity of the test was 100% for Ct values up to 33, and the specificity of the test was 100% (Table 2
). The assay only missed three patient samples for Ct values up to 35 (i.e., at Ct values of 33.2, 34.1, and 34.5) (Supplemental Table S2
). Interestingly, we observed an excellent correlation between the antigen signal and Ct values obtained by RT-PCR (Figure 2
). Hirotsu et al. evaluated another automated AG test, namely the LUMIPULSE SARS-CoV-2 Ag kit (Fujirebio, Tokyo, Japan), performed on a LUMIPULSE G600II chemiluminescent assay [29
]. Out of 58 RT-PCR positive patients, 26 (44.8%) were negative on the LUMIPULSE. The specificity of the test ranged from 97.3% to 99.6% [29
]. They also observed a weaker correlation with viral load (i.e., R2
= 0.77) compared to our evaluation with the VITROS automated assay (i.e., R2
= 0.94), which in our hands, showed excellent clinical performance.
In the scientific literature, there is an ongoing debate regarding the Ct value corresponding to the threshold of infectivity (i.e., patient considered as contagious) [26
]. Bullard et al. showed that SARS-CoV-2 Vero cell infectivity was only observed for RT-PCR Ct value <24 in samples obtained less than eight days since symptom onset [4
]. Albert et al. also found that SARS-CoV-2 could not be cultured from positive RT-PCR samples with Ct value >25. The same observation was made for samples positive using RT-PCR but negative using the Panbio RAD test (n
= 11) [8
]. However, in a larger cohort, Singanayagam et al. reported that a Ct value of 25 was still correlated to 80% of positive cultures and that only 8% of samples had a positive culture for Ct value >35 [28
]. Jaafar et al. also showed that up to 70% of cultures were positive for a Ct value of 25, and only 3% of cultures were positive for a Ct value >35 [25
]. La Scola et al. found that patients with Ct value >33–34 are not contagious because of the low number of positive cultures [27
]. This is consistent with the Centers for Disease Control and Prevention (CDC) recommendations, which propose a Ct value of 33 as a surrogate of contagiousness [24
]. Based on these studies and on CDC recommendations, we calculate the performance of the five AG tests at the different proposed cut-offs for contagiousness (Table 2
, Supplemental Tables S1–S3
The main advantages of RAD tests are their rapidity, and their ease of interpretation, and the limited technical skill/infrastructure requirements. Moreover, the use of RAD tests in mass screening programs could decrease the burden on laboratories that have been overwhelmed during the last COVID-19 pandemics [9
]. Nevertheless, it is important to keep in mind that even with a high viral load (i.e., Ct values <25), a RAD test can result in false-negative estimates in both symptomatic patients and asymptomatic subjects [12
]. Mass community screening would, therefore, require the use of more sensitive techniques.
The performance of the VITROS automated assay outclassed the ones of RAD tests. Based on the categorization proposed by Kost and considering a Ct value of 33 [31
], the VITROS automated assay can be placed in the “Tier 3” category, meaning that the assay has high sensitivity and specificity (i.e., 100%), leading to high NPV and PPV whatever the prevalence of the disease (Supplemental Table S3
In most laboratories using RT-PCR for diagnosis, the VITROS automated assay could be used as a routine high-throughput test in a clinical setting with an approximative cadence of 150 samples per hour. More importantly, according to the CDC cut-off, the VITROS automated assay may replace the RT-PCR for the detection of positive cases thanks to its excellent performance at the Ct value of 33. However, compared to RAD tests that can be used in any facility, an automated test obviously requires specialized equipment.
These analytical and clinical performance have to be interpreted together with the prevalence of the disease. While some studies have discussed the importance of the disease prevalence for serological testing [31
], few have discussed the importance of the disease prevalence in the performance of AD tests. Simulations showed that, at low disease prevalence, e.g., 2%, the PPV was low (24.6% to 64.5%) while the NPV was high (99.5% to 99.8%). At high disease prevalence, e.g., 25%, the PPV increased from 84.2% to 96.7% while the NPV decreased from 92.4% to 96.2%. Consequently, RAD tests are not appropriate for mass community screening since they will lead to a high rate of false-positive and negative results. On the opposite, the VITROS assay showed remarkable performance with PPV and NPV of 100% both.
While this study confirms that RAD tests have limited performance and should only be used in facilities where access to RT-PCR or automated AD is not possible within an appropriate timeframe, further studies are needed to confirm our data on a larger population. In addition, a global consensus is also needed to define the Ct value that could be used as a surrogate of infectivity and contagiousness [2