Comparison of Four Streptococcus pneumoniae Urinary Antigen Tests Using Automated Readers

Streptococcus pneumoniae urinary antigen tests (UATs) may be interpreted using automatic readers to potentially automate sample incubation and provide standardized results reading. Here, we evaluated four UATs the BinaxNOW S. pneumoniae Antigen Card (Abbott, Chicago, IL, USA), ImmuView S. pneumoniae and Legionella (SSI Diagnostica, Hillerød, Denmark), STANDARD F S. pneumoniae Ag FIA (SD Biosensor, Gyeonggi, South Korea), and Sofia S. pneumoniae FIA (Quidel Corporation, San Diego, CA, USA) with their respective benchtop readers for their ability to detect S. pneumoniae urinary antigen. We found that these assays had a sensitivity of 76.9–86.5%, and specificity of 84.2–89.7%, with no significant difference found among the four UATs. The assays had a high level of agreement with each other, with 84.5% of samples testing consistently across all four assays. The automatically and visually read test results from the two immunochromatographic assays, BinaxNOW and ImmuView, were compared and showed excellent agreement between the two types of reading. Immunofluorescent-based assays, Sofia and STANDARD F, had significantly less time to detect compared to the two immunochromatographic assays due to having less assay setup procedures and shorter sample incubation times. In conclusion, the four UATs performed similarly in the detection of S. pneumoniae urinary antigen, and readers can bring increased flexibility to running UATs in the clinical routine.


Introduction
Streptococcus pneumoniae is a leading cause of bacterial infections such as pneumonia, otitis media, sepsis, and meningitis [1], responsible for invasive infections in 10 to 100 cases per 100,000 people in Europe and United States per year [2]. Traditional culture-based diagnostic methods for blood, cerebrospinal fluid (CSF), or respiratory tract samples remain standard methods for diagnosis, but the procedures have a lengthy time-to-detection (TTD) and are labor-intensive. Since early detection of S. pneumoniae is crucial in patients with invasive disease, there is a need for reliable, rapid, and user-friendly diagnostic assays.
Previous studies of immunochromatography-based S. pneumoniae urine antigen tests (UATs) have reported sensitivities of 62-73% and specificities of 86-100% [3][4][5][6][7]. Immunofluorescent-based UATs have shown higher sensitivity compared to immunochromatographybased UATs [5,6]. While automated readers are required for the interpretation of immunofluorescence-based UATs, they are not necessary for interpretation of immunochromatographybased S. pneumoniae UATs but may be used. We have recently showed a high agreement between automatically and visually read BinaxNOW S. pneumoniae UAT [8], and another study has demonstrated similar results also for the ImmuView S. pneumoniae and L. pneumophila UAT [9]. The use of readers could potentially automate sample incubation and results interpretation.
Here, we compare the automatic interpretation of two immunochromatographic UATs, the BinaxNOW S. pneumoniae Antigen Card (BinaxNOW) and ImmuView S. pneumoniae and L. pneumophila (ImmuView), and two fluorescent based UATs, Sofia S. pneumoniae Fluorescent Immunoassay (FIA) (Sofia) and STANDARD F S. pneumoniae Ag FIA (STAN-DARD F). The present study compares the performance of four UATs in conjunction with their respective readers. Samples were tested simultaneously with the four UATs. For the immunochromatography-based UATs, the results obtained from the readers were compared to visually interpreted results. In addition, we examined the performance of antigen detection after refreezing, and compared time to result and user friendliness between the UATs.

Samples
Urine samples from adult patients ≥18 years old were obtained from the clinical routine at the Department of Clinical Microbiology at Karolinska University Hospital Huddinge (Stockholm, Sweden) and the Department of Laboratory Medicine at Örebro University Hospital (Örebro, Sweden). Inclusion criteria were urine samples from bacteremic patients with simultaneously collected blood culture and/or respiratory tract culture ±2 days of urine sample collection. Blood and respiratory culture results were obtained from a Laboratory Information System. Blood and respiratory tract samples were cultured according to standard clinical protocols. Briefly, after Gram stain was performed, positive blood culture bottles with diplococci were subsequently sub-cultured on blood, chocolate, and crystal violet blood (in-house production, Karolinska University Hospital Huddinge, Stockholm, Sweden) agar plates. Respiratory samples were cultured on relevant agar plates including crystal violet blood agar plates. Identification of S. pneumoniae was performed according to optocin susceptibility. Urine samples from patients with blood and/or respiratory sample cultures positive for S. pneumoniae were used as positive cases Urine samples from patients negative for S. pneumoniae in blood and respiratory tract cultures were classified as negative controls. All samples were anonymized after culture results had been retrieved. Samples were stored at −80 • C and were thawed in room temperature prior to testing. Each patient sample was tested with all four UATs (BinaxNOW, ImmuView, Sofia, STANDARD F) and immediately re-frozen. In order to analyze the effect of long-term storage after re-freezing, the urine samples were frozen at −80 • C for another 4 months before re-testing with the four antigen tests.

Urinary Antigen Tests (UATs)
UATs from the four manufacturers were used together with their respective readers (Table 1). For the BinaxNOW, Sofia, and STANDARD F, samples were incubated and read by the readers on "Walk Away" mode. UAT setup procedures are briefly described as follows. BinaxNOW setup included dipping the included swab into the sample, inserting it into the sample card, adding reagent buffer, and then inserting the sample card into the reader for incubation and interpretation as per manufacturer's instructions. For Sofia and STANDARD F, the sample was introduced to the sample well on the cassette by a dropper, and inserted into their respective reader for incubation and interpretation as per manufacturers' instructions. For ImmuView, three drops of sample were added to the included polypropylene tube followed by addition of two drops of reagent buffer and gently mixed. The UAT strip was then inserted into the tube, and incubation was timed manually before the UAT was inserted into the reader for interpretation. Sample incubation time was 15 min for BinaxNOW and ImmuView, and 10 min for Sofia and STANDARD F. Samples were run simultaneously with each of the four UATs, and the results provided by the readers were noted. The readers indicated if the test was valid, and if the sample result was positive or negative for S. pneumoniae. If a result obtained for a sample was invalid or the reader displayed an error message, the sample was re-analyzed using a new UAT. The STANDARD F200 Analyzer also indicated the cut-off index (COI) value along with the sample result. In addition, the test results were visually interpreted by two independent researchers for BinaxNOW and ImmuView after automated reading was performed. The appearance of a correctly colored band that was visible to the naked eye after 15 min was considered as a positive result. If the visual interpretations of the UAT differed between researchers, the result that was consistent with automated reading was used. The time to result was registered by measuring the time from the start of "Walk Away" mode (except for ImmuView) to when the sample result was presented (printed out or displayed) by the reader.

Statistical Analysis
Sensitivities and specificities of the UATs were determined, and the confidence intervals (CI) were computed using the Wilson-Brown method. The significance of differences between tests were computed using Cochran's Q test (χ 2 (3)) using Microsoft Excel version 16.40 [10]. Differences in time to result between UATs were analyzed by calculating mean time from three measurements for each UAT and performing one-way ANOVA followed by Bonferroni's multiple comparisons test. Statistical calculations were performed using Prism 8 (GraphPad Software LLC, San Diego, CA, USA). A p-value of ≤0.05 was considered as statistically significant. A CI of 95% was used for statistical precision.

Sample Characteristics
A total of 110 samples were analyzed, 52 were positive cases and 58 negative controls ( Figure 1, Tables S1 and S2).

Agreement of Results between UATs
Excellent agreement between the four UATs was observed. In positive group samples, there was 82.7% concordance, where 73.1% of samples tested positive and 9.6% tested negative in all four UATs. For the negative group samples, the concordance rate was 86.2%. In total, 77.6% of samples tested negative, and 8.6% tested positive in all four assays. Overall, the four assays had discordant results in 17/110 (15.5%) samples ( Figure 2).

Agreement of Results between UATs
Excellent agreement between the four UATs was observed. In positive group samples, there was 82.7% concordance, where 73.1% of samples tested positive and 9.6% tested negative in all four UATs. For the negative group samples, the concordance rate was 86.2%. In total, 77.6% of samples tested negative, and 8.6% tested positive in all four assays. Overall, the four assays had discordant results in 17/110 (15.5%) samples ( Figure  2).

Agreement between Visual and Automatic Reading of UATs
Visual readings of the BinaxNOW UAT by the two researchers were 100% in agreement with each other, while that for the ImmuView UAT was mostly in agreement apart from one sample (P19 from Table 3 and Table S1), where one researcher interpreted the pneumococci result as negative while the other interpreted it as positive. There was a very good agreement between visual and automatic reading for BinaxNOW. Only two samples (P35 from Table S1, and N17 from Table S2) that were interpreted as negative visually by both researchers was positive by the DIGIVAL reader. Similarly, very good agreement between visual and automatic reading of ImmuView was also observed. Only one sample (P5 from Table S1) that was interpreted as positive visually by both researchers but negative by the ImmuView reader.

Agreement between Visual and Automatic Reading of UATs
Visual readings of the BinaxNOW UAT by the two researchers were 100% in agreement with each other, while that for the ImmuView UAT was mostly in agreement apart from one sample (P19 from Table 3 and Table S1), where one researcher interpreted the pneumococci result as negative while the other interpreted it as positive. There was a very good agreement between visual and automatic reading for BinaxNOW. Only two samples (P35 from Table S1, and N17 from Table S2) that were interpreted as negative visually by both researchers was positive by the DIGIVAL reader. Similarly, very good agreement between visual and automatic reading of ImmuView was also observed. Only one sample (P5 from Table S1) that was interpreted as positive visually by both researchers but negative by the ImmuView reader.

Assay Performance after Long-Time Storage
For performance analysis after long-time freezing, 37 samples (positive cases, n = 20; negative controls, n = 17) were randomly selected for re-testing with BinaxNOW, Immu-View, and STANDARD F after four months of storage at −80 • C. For 32 (86.5%) samples, the UATs showed the same result after storage, while for five (13.5%) samples, the results changed ( Figure 3 and Table S3).

Assay Performance after Long-Time Storage
For performance analysis after long-time freezing, 37 samples (positive cases, n = 20; negative controls, n = 17) were randomly selected for re-testing with BinaxNOW, Immu-View, and STANDARD F after four months of storage at −80 °C. For 32 (86.5%) samples, the UATs showed the same result after storage, while for five (13.5%) samples, the results changed ( Figure 3 and Table S3).

Time to Result and User Friendliness
The mean time to result for BinaxNOW, ImmuView, Sofia, and STANDARD F were 17.0, 17.3, 11.7, and 11.3 min, respectively, with significant differences between immunochromatographic-based and immunofluorescent-based assays that could attributed to more steps required to set up a test, thus, more hands-on time (Figure 4).

Time to Result and User Friendliness
The mean time to result for BinaxNOW, ImmuView, Sofia, and STANDARD F were 17.0, 17.3, 11.7, and 11.3 min, respectively, with significant differences between immunochromatographic-based and immunofluorescent-based assays that could attributed to more steps required to set up a test, thus, more hands-on time (Figure 4).

Assay Performance after Long-Time Storage
For performance analysis after long-time freezing, 37 samples (positive cases, n = 20; negative controls, n = 17) were randomly selected for re-testing with BinaxNOW, Immu-View, and STANDARD F after four months of storage at −80 °C. For 32 (86.5%) samples, the UATs showed the same result after storage, while for five (13.5%) samples, the results changed ( Figure 3 and Table S3).

Time to Result and User Friendliness
The mean time to result for BinaxNOW, ImmuView, Sofia, and STANDARD F were 17.0, 17.3, 11.7, and 11.3 min, respectively, with significant differences between immunochromatographic-based and immunofluorescent-based assays that could attributed to more steps required to set up a test, thus, more hands-on time (Figure 4). Observations with regard to the UAT usage, barcode scanning, and result presentation were noted for assessing user friendliness. While the UATs of the BinaxNOW, Sofia, and STANDARD F were enclosed, thus, reducing the chance of samples coming directly into contact with the readers, the ImmuView UAT was exposed, which led to frequent cleaning of the drawer that came into contact with the sample. The readers for BinaxNOW, Sofia, and STANDARD F were equipped with barcode scanners, either fixed on either a stand or at the side of the reader, which facilitated hands-free scanning (BinaxNOW and Sofia), or as a handheld barcode scanner with a button for activating scanning (STANDARD F). Lastly, the readers for BinaxNOW, Sofia, and STANDARD F had the option to print results, while results from the ImmuView reader is read on the display only. The BinaxNOW reader can be connected to a small external printer, while Sofia and STANDARD F readers had built-in printers.

Discussion
The present study evaluated the performance of four S. pneumoniae urinary antigen detection assays using automatic readers for interpretation of test results. The UATs demonstrated similar sensitivities and specificities of 76.9-86.5% and 84.2-89.7%, respectively. The test results showed high concordance across different UATs, as well as a high level of agreement between automatically and visually read test results. Furthermore, our investigations showed that a majority of samples retained the same result after re-freezing and long storage. Lastly, the four UATs were generally easy to use, providing features such as barcode scanning, automated incubation, and results printing. However, immunochromatographicbased assays have more setup procedures compared to immunofluorescent-based assays and, hence, longer hands-on time.
Our findings of S. pneumoniae UAT performance are within the range of sensitivities and specificities from previous studies [3][4][5][6][7]11]. Additionally, the agreement between automatically and visually interpreted results for BinaxNOW and ImmuView was excellent. However, there were no difference in sensitivity yields between immunochromatographybased and immunofluorescent-based assays, unlike what was previously reported [5,6]. Accordingly, we have previously reported that the automatically read Sofia showed similar performance rates, as the visually read BinaxNOW and ImmuView on frozen samples from bacteremic patients as well as on consecutive non-frozen urine samples [8]. The reason for discrepant study results may be due to differences in sample collection, storage, and/or testing, which must be sufficiently clarified throughout the study protocol in order to make results comparable across studies.
All four UATs failed to detect S. pneumoniae antigen in 5/52 urine samples among the pneumococcal cases, and one or more of the UATs were unable to detect pneumococcal antigen in another nine samples similar to what was found previously [12]. Most likely, low levels of pneumococcal antigen in urine is the reason for negative test results.
In 10.3-15.5% of negative control samples, false positive results were obtained, mainly from patients with either Gram-negative infections including patients with blood cultures positive for B. fragilis (n = 1), E. coli (n = 2), and P. mirabilis (n = 1) or respiratory tract cultures positive for H. influenzae (n = 2), K. pneumoniae (n = 1), and M. catarrhalis (n = 1). In only two cases, false positive results were obtained from patients with negative blood and respiratory tract cultures. Prior studies have also reported false positive results on samples from patients infected by Gram-negative bacteria or alpha-streptococci [13][14][15]. Preheating of the urine may help to increase the specificity of urinary antigen assays [6,15] but was not performed for this study as the majority of manufacturers did not recommend it.
An important factor for assay reliability is the rate of invalid test results. In the present study, only 1/440 (0.2%) of all UATs performed showed an invalid result, as it was invalid for one Sofia UAT. We previously reported a higher number of invalid test results for S. pneumoniae urine antigen assays, in particular for the ImmuView [3] and Sofia UATs [8], which could be due to presence of interfering substances in samples or high sample viscosities.
To our knowledge, this is the first time the four S. pneumoniae UATs were evaluated simultaneously with their respective readers, including the first evaluation of STANDARD F performance for S. pneumoniae antigen detection. Only the STANDARD F reader presented COI values as a way to quantify the concentration of pneumococcal antigen in urine samples in addition to the dichotomous test results (positive or negative). The importance of COI values in clinical routine has not been studied as we know; however, as the other readers did not provide quantifiable results, we were unable to compare result signal intensity across the four assays. Automated readers could bring convenience to the workflow, as they provide onscreen prompts on assay setup procedures. The BinaxNOW, Sofia, and STANDARD F readers also provide the alternatives to run in either "Walk Away" or "Read Now" mode. This allows the flexibility to choose whether to automatically incubate and read single samples, or to manually incubate multiple samples and read them in quick succession with the reader. Therefore, although automatic reading did not enhance the sensitivity of the assays in the present material, readers may facilitate flexibility and time-efficiency in the workflow.
We previously reported various aspects of user friendliness of the BinaxNOW, Immu-View, Sofia, and STANDARD F Legionella UATs that are also applicable to this study [16].
Regarding the time to result of the assays, the immunochromatographic-based assays had longer time to result as compared to the immunofluorescent-based assays due to more assay setup procedures as well as longer incubation time. The longer hands-on time and time to result for immunochromatographic-based assays could perhaps be viewed as a trade-off to increased flexibility, since these assays do not require readers to obtain results. The option to interpret assay results visually could be more ideal for low-resource clinical settings.
A limitation of our study is that we used frozen urine samples for evaluation of test performances. The pneumococcal cell wall polysaccharide antigen is considered to be temperature stable [8,17,18]. A previous study evaluating the performance of BinaxNOW UAT showed similar test results on both fresh and frozen unconcentrated urine samples where the frozen samples were only thawed once [19]. We do not expect that storage at −80 • C would have a significant impact on sample results if the sample was thawed once; however, our results with re-frozen samples show that subsequent freeze-thaw cycles have the potential to change results in a small subset of samples.
Another limitation is that no additional patient information other than blood and respiratory tract culture results were obtained from the Laboratory Information System. This made it difficult interpreting false positive and false negative results obtained, and we were not able to conclude how long S. pneumoniae urinary antigen detection lasts after an infection.
In conclusion, the present study compares for the first time four S. pneumoniae UATs with automatic interpretation using their respective readers on defined clinical samples. The four assays performed similarly and are largely in agreement for the detection of S. pneumoniae urinary antigen. The use of a reader, availability of barcode scanning, and automated sample incubation brings increased flexibility, ease of use, and standardization in performing UATs in the clinical routine, thus improving patient diagnosis through increased efficiency and decreased probability of errors.
Supplementary Materials: The following are available online at https://www.mdpi.com/article/ 10.3390/microorganisms9040827/s1, Table S1: Streptococcus pneumoniae UAT results for positive group patient urine samples, Table S2: Streptococcus pneumoniae UAT results for negative group patient urine samples., Table S3: Results of Streptococcus pneumoniae UATs after re-freezing and four months of storage.  Institutional Review Board Statement: Ethical review and approval were waived for this study, due to the use of leftover samples that were anonymized. The test results were not sent to the clinicians and the patient charts were not analyzed.
Informed Consent Statement: Written informed consent has been obtained from the patient(s) were not necessary as described above.

Data Availability Statement:
The data presented in this study are openly available upon request.