Next Article in Journal
Sustainable and Ecofriendly Approach of Managing Soil Born Bacterium Ralstonia solanacearum (Smith) Using Dried Powder of Conyza canadensis
Previous Article in Journal
Transcriptional Alteration of Gene Biomarkers in Hemocytes of Wild Ostrea edulis with Molecular Evidence of Infections with Bonamia spp. and/or Marteilia refringens Parasites
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Pathogens
Volume 9
Issue 5
10.3390/pathogens9050326
pathogens-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Validation of a Laboratory-Developed Triplex Molecular Assay for Simultaneous Detection of Gastrointestinal Adenovirus and Rotavirus in Stool Specimens

by
Rachel R. Higgins
1,*,
Adriana Peci
1,
Mark Cardona
1 and
Jonathan B. Gubbay
1,2,3
1
Public Health Ontario Laboratory, 661 University Avenue, Toronto, ON M5G 1M1, Canada
2
Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 2V8, Canada
3
The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada
*
Author to whom correspondence should be addressed.
Pathogens 2020, 9(5), 326; https://doi.org/10.3390/pathogens9050326
Submission received: 14 March 2020 / Revised: 20 April 2020 / Accepted: 23 April 2020 / Published: 27 April 2020
Browse Figures
Versions Notes

Abstract

:
Enteric viral pathogens causing gastroenteritis include adenovirus and rotavirus, among others. Rotavirus is the leading cause of severe diarrhea in infants and young children worldwide. Among the adenoviruses known to cause gastroenteritis are those of species F (serotypes 40, 41). Here, we describe the development and validation of a laboratory-developed gastrointestinal triplex rRT-PCR (triplex) assay that targets adenovirus and rotavirus. Stool specimens were tested from patients across Ontario. Specimens were previously tested for adenovirus and/or rotavirus by electron microscopy (EM) or immunochromatographic test (ICT). Triplex sensitivity, specificity, positive and negative predictive values compared to Seegene assay (a commercial assay used here as the standard reference method) were 100%, 97.8%, 86.0%, 100% for adenovirus, and 99.1%, 98.4%, 96.3%. 99.6% for rotavirus, respectively. The triplex assay had a 95.2% and 97.3% overall percent agreements (OPAs) when compared to EM for adenovirus or rotavirus detection, respectively, and an OPA of 90.9% when compared to rotavirus ICT for rotavirus detection. Triplex assay exhibited similar performance to the Seegene assay for both adenovirus and rotavirus and detected more adenovirus and rotavirus than traditional testing methods. The high performance along with lower cost and reduced turnaround time makes the triplex assay a desirable testing method for a clinical microbiology laboratory.

1. Introduction

Infectious diarrhea is the second most common infectious condition worldwide, after acute respiratory infections, affecting humans of all age groups, but mostly the young, the elderly, the immunocompromised patients and people in enclosed communities such as hospitals, nursing homes, military bases and cruise ships [1,2,3]. Known enteric viral pathogens, as causative agents of gastroenteritis, include adenovirus, rotavirus, norovirus, sapovirus and astrovirus [4,5,6]. Several bacterial pathogens, such as Campylobacter jejuni, enterotoxigenic Escherichia Coli, Shigella species and Vibrio cholera, are also reported to cause gastroenteritis, particularly in developing countries and refugee camps [7]. Viral gastroenteritis is reported to be more common than bacterial gastroenteritis; however, due to lack of molecular testing in developing countries, it is difficult to quantify it properly [7].
Among viruses, norovirus is the most common cause of acute gastroenteritis outbreaks in humans [8,9]. Rotavirus was the leading pathogen causing diarrhea in infants and young children in the United States, before the introduction of vaccine in 2006. Rotavirus infection in adults and the elderly is less likely; however, when it happens, a prolonged and severe infection can occur, particularly among immunocompromised patients [10].
Since the introduction of publicly funded rotavirus vaccine in Ontario, in August 2011, the prevalence of rotavirus has been reduced [6,11]. Stool specimens showed a significant decrease in rotavirus percent positivity from 14.4% to a 36-month period before implementation of vaccine to 6.1% during a 46-month period after rotavirus vaccine implementation [11].
Of the adenoviruses reported to cause gastroenteritis are those of species F (serotypes 40 and 41) [12,13]. Adenovirus infection can be transmitted via aerosolized droplets or via fecal oral spread, or direct conjunctival inoculation including contact with tap water, and environmental surfaces [13]. Symptoms most frequently reported include watery diarrhea, vomiting, and low temperature [13]. While adenovirus is less common than rotavirus, the duration of diarrhea is longer [12]. Adenoviruses can be infectious even after several weeks in moisture free environments and the virus is resistant to many disinfectants because it is a non-enveloped DNA virus [14]. Viral persistence of adenovirus in the gastrointestinal tract and shedding for a long period are common and frequently reported to not be associated with clinical symptoms [15].
The great burden of viral gastroenteritis on health care due to the related illness and hospitalization, particularly among younger and older age groups, highlights the need for fast, sensitive and reliable diagnostic assays to guide infection control measures. The existing methods for detecting enteric viruses at Public Health Ontario (PHO) laboratory rely on conventional and labor-intensive electron microscopy (EM), which has low sensitivity, is subjective, and it requires highly skilled and experienced personnel. Imunochromatographic test (ICT) is another method used at PHO laboratory, which is rapid and simple to use but it lacks sensitivity and specificity [16]. At PHO laboratory virus culture is used to detect enteroviruses only during the summer-fall months.
Real Time Polymerase Chain Reaction (RT-PCR) assays have been successfully implemented for the detection of adenovirus [17] and/or rotavirus [18]. The simultaneous detection of several viruses can be accelerated and simplified by the use of a multiplex RT-PCR assay. Commercial methods for detection of both viruses are available but are expensive and not amenable to multiplexing of additional viruses [19].
The objective of this study was to develop an in-house reverse transcription RT-PCR (rRT-PCR) assay (triplex) that targets two diarrheal pathogens, adenovirus and rotavirus, along with an MS2 internal control. The triplex assay was developed to replace the two conventional assay methods (EM and rotavirus ICT) and complement the norovirus rRT-PCR assay in use at PHO laboratory that targets norovirus GI and GII, the two most common pathogenic genotypes in humans. In addition, the triplex assay will be essential when virus titer is low and high sensitivity is desired, as in the case of water contamination. Here, we describe the development and validation of the triplex assay and its evaluation against Seegene, EM and rotavirus ICT. Knowledge gained from this study can be used to expand this laboratory-developed multiplex assay to simultaneously detect additional viral pathogens causing gastroenteritis.

2. Materials and Methods

This study utilized clinical stool specimens submitted to PHO laboratory for routine gastrointestinal virus testing, including adenovirus and rotavirus, from patients across Ontario. Of these, a convenience subset (n = 685), consisting of available positive and negative specimens for rotavirus and/or adenovirus was used for this study (Figure 1). These specimens were submitted to PHO laboratory over a 4 year period, between February 2009 and February 2013. For the purpose of this validation, positive specimens for adenovirus or rotavirus refer to those previously confirmed positive by EM and/or ICT tests.
A panel of controls as illustrated in Table 1, comprised of positive bacterial specimens, viral specimens and bacterial culture isolates was used to determine assay specificity.

2.1. Testing Methods

All specimens were previously tested for adenovirus or rotavirus by EM (EM, Philips CM10 or Field Electronic and Ion Company (FEI) Morgagni 268, Eindhoven, The Netherlands) or for rotavirus only by rotavirus ICT, with commercial rotavirus antigen detection kits (RIDAQUICK Rotavirus (Phoenix Airmid Biomedical Corp., Oakville, ON, Canada), or Rotascreen (Microgen Bioproducts Ltd., Camberley, UK)).
The triplex assay developed for the detection of rotavirus/adenovirus/MS2 and evaluated here, is an in-house laboratory-developed multiplex rRT-PCR viral diarrheal assay. This triplex assay consists of rRT-PCR on the ABI 7500 fast instrument (Applied Biosystems, CA, USA). For the purpose of this validation, a group of specimens included in this study was also tested for rotavirus and adenovirus using the Seegene, (Seeplex Diarrhea-V ACE Seegene, Seoul, Korea) commercial assay [20]. Triplex assay results were evaluated against Seegene assay, used here as the reference standard method and against EM and rotavirus ICT, two additional “non-standard” reference methods.

2.2. RNA and DNA Extraction

All specimens used in this study were stored at −70 °C prior to thawing and testing. Stool specimens were homogenized (20% weight/volume) in sterile water, centrifuged at 3000× g for 20 min at room temperature and 250 uL of each clarified supernatant was spiked with the MS2 control (Roche, Mannheime, Germany) at 1 × 104 copies per specimen and subjected to automated nucleic acid extraction. Total nucleic acid was eluted in a final volume of 110 uL, of which 5 uL was used for PCR amplification. Nucleic acid was extracted directly from diluted stool specimens using (NucliSENS easyMag, bioMérieux, France) as recommended by the manufacturer [21]. Bacterial cultures grown in appropriate culture media at PHO laboratory were diluted to 0.5 McFarland turbidity standard in phosphate buffer saline (Gibco PBS, Grand Island, NY, USA) prior to automated extraction on the NucliSENS easyMAG.
The adequacy of nucleic acid extraction was confirmed by RT-PCR amplification of human GAPDH gene and/or Bacterioidis fragilis (B. fragilis) strain VPI 2553 (ATCC 25285). GAPDH was used in addition to B. fragilis in a subset of infant’s specimens (n = 140) since B. fragilis may not be present in breast-fed infants’ stool due to maternal protection [22,23]. The remaining specimens (n = 545) were tested by B. fragilis only.

2.3. PCR Amplification

Amplification and detection of nucleic acid using the triplex rRT-PCR gastrointestinal virus assay was carried out using the ABI7500 RT-PCR instrument. The rRT-PCR amplification was performed using Qiagen one step RT-PCR kit (Qiagen, Frederick, MD, USA). Primers and probes used in the triplex assay are shown in Table 2 [17,18,24,25].
Primers and probes were chosen from the most conserved regions for both viruses; 103 bp in the hexon gene region for adenovirus and non-structural protein 3 (NSP3) gene was used for rotavirus. These together with the use of degenerate primers and probes allowed detection of all serotypes including A, B, C, D, E, F, G serotypes for adenovirus and A, B, C, D, E, F, G, H, I, J for rotavirus [10,12].
The rRT-PCR reaction conditions were as follows: Five microliters (uL) of extracted DNA or RNA template was used in the triplex assay in a final volume of 25 uL per reaction. The final concentrations of primers and probes in the reaction were 300 nM and 100 nM for primers and dual-labelled probes, respectively. MS2 primer and probe concentrations were 100 nM and 50 nM, respectively. The assay was performed in 25 uL reaction volume containing 1× Qiagen one step RT-PCR kit. Cycling conditions were as follows: One cycle of reverse transcription of RNA at 48 °C for 30 min, one cycle of activation/denaturation step at 95 °C for 15 min followed by 40 cycles of amplification, each with 15 s denaturation at 95 °C and 60 s amplification/extension at 60 °C. Analysis of amplification curves utilized threshold 3 to 15 with manual adjustments of base line. Amplification of specimens for the Seegene assay was performed on Bio-Rad thermocyler (Bio-Rad Laboratories, Hercules, CA, USA) according to the manufacturer’s product insert. Since rotavirus is an RNA virus and adenovirus a DNA virus, several components of reaction conditions were tested and optimized including primer/probe concentrations, PCR cycling conditions, annealing temperature etc. Singleplexes of adenovirus and rotavirus were initially developed and performance was compared to duplex assay (adenovirus/MS2 and rotavirus/MS2) and later to triplex assay where all three targets were included (adenovirus/rotavirus/MS2). rRT-PCR inhibitory effects in the triplex assay were assessed by co-amplification of an internal MS2 control included in the primer-probe mix. In the event of PCR failure, the nucleic acid was re-extracted from stool specimens and triplex PCR assay was repeated. The MS2 was used as an internal control to eliminate false negatives results due to assay failure. MS2 was calculated such that at the applied concentration (10−4 ug/mL), mean CT value of 22.5, standard deviation ±1.034, and range 22–24 were obtained. The assay developed is qualitative, and amplification of MS2 in the absence of target pathogens was consistently maintained. In the presence of target pathogen, the CT value for MS2 increased based on the viral titer. The higher the concentration of target viruses, the higher the CT value for MS2.

2.4. Accuracy

Clinical specimens previously tested for adenovirus and/or rotavirus were used for this study. Controls included during test validation were excluded from the performance evaluation analyses. Results of triplex rotavirus/adenovirus were compared with Seegene, the standard reference method in two-way performance comparison analysis. Performance indicators such as sensitivity, specificity, positive predicted value (PPV) and negative predictive value (NPV) were calculated with 95% confidence intervals. Results of triplex assay testing for both adenovirus and rotavirus were further compared to EM, and for rotavirus only, to rotavirus ICT, two non-standard reference methods. Positive percent agreement (PPA), negative percent agreement (NPA) and overall percent agreement (OPA) were reported when compared with the non-reference standard methods, as recommended by FDA guidelines for evaluating diagnostic tests [26]. Chi-square was also employed to compare results between two methods (triplex versus Seegene; triplex versus EM; and triplex versus ICT). Three-way comparisons were also performed on specimens tested for adenovirus and rotavirus by triplex, EM and Seegene; and for rotavirus only by triplex, rotavirus ICT, and Seegene. In both comparisons, Seegene was used as the standard reference method. Three-way comparisons were used to increase the accuracy of this validation and to reduce the effect of discrepant results between methods [26].

2.5. Analytical Sensitivity: Limit of Detection

The 95% limit of detection (LOD) in copies per reaction for each virus in the multiplex rRT-PCR gastrointestinal assay was determined using commercially produced plasmid DNA (Seegene, South Korea) and/or quantified tissue culture fluid (TCF, Zeptometrix Corporation, NY, USA). TCF for each viral target was serially diluted 10-fold to 10−12 of the starting material in PCR grade water and tested in the triplex assay. The respective 95% LODs were assessed by testing a minimum of four replicates of each of the 12 dilution steps. The LOD indicates the lowest concentration of genome target at which a pathogen will be detected with a probability of 95%. The LODs and 95% confidence interval for each target was determined using probit regression (SPSS software regression package version 18.0, SPSS, Chicago, IL, USA).

2.6. Precision: Repeatability (Intra Assay Precision) and Reproducibility (Inter Assay Precision)

Intra and inter assay precision was assessed by using data generated from the LOD calculation and from additional testing of specimens on three different days. Concordance for intra and inter-assay reproducibility were calculated.

2.7. Turnaround Time

Turnaround time (TAT) was calculated for the triplex assay based on five runs, each run of 96 specimens completed by one laboratory staff member. TAT reflected the average time to process 96 specimens, including hands-on time, assay time and resulting/reporting times (pre-analytical, analytical and post-analytical components).

2.8. Analytical Specificity

The absence of cross reactivity with other unrelated pathogens was assessed by a panel of 32 control specimens comprised of primary specimens as well as bacteria and culture isolates of known pathogens. Interference was assessed to show that the presence of other pathogens did not interfere with virus detection.

2.9. Ethics Statement

This study was considered exempt from Public Health Ontario’s Research Ethics Board review as it involved de-identified stool specimens that were tested as part of a routine clinical virology services provided by PHO laboratory.

3. Results

This study employed a convenience selection of previously tested specimens for gastrointestinal viruses. Specimens were submitted as part of routine clinical testing by clinicians across Ontario, Canada to PHO laboratory. Among 685 unique specimens included in this study 349 (50.9%) and 39 (5.7%) were positive for rotavirus and adenovirus, respectively by traditional testing methods (EM and/or ICT) (Table 1). The median age of patients for which age was reported (n = 583) was 1 year old with a range of 2 months to 97 years. Of 562 patient’s specimens that had gender reported 42.3% (238/562) were female and 57.6% (324/562) were male.

4. Accuracy

4.1. Performance Characteristics of Triplex Assay Compared to Seegene PCR Assay

Due to limited resources, only 369 unique specimens from samples included in this study were tested by both triplex and Seegene assays for detection of adenovirus and rotavirus as illustrated in (Figure 1 and Table 3A). For this comparison a performance below 85% was considered as ‘below average’, at 86–90% as ‘average’ and above 90% as ‘above average’. Sensitivity, specificity, PPV and NPV for adenovirus were 100%, 97.8%, 86% and 100%, respectively and for rotavirus were 99.1%, 98.4%, 96.3% and 99.6%, respectively (Table 4A).

4.2. Performance Characteristics of Triplex Assay Against EM or Rotavirus ICT (Two Non-Standard Reference Testing Methods)

Five hundred and seventy-eight specimens had both triplex assay and EM results for adenovirus and rotavirus (Figure 1 and Table 3B). Three hundred and twenty specimens were tested by triplex assay and rotavirus ICT for detection of rotavirus (Table 3C). Performance indicators of triplex for the comparison with these non-standard reference testing methods (PPA, NPA and OPA) are reported in Table 4B for each paired comparison. Triplex assay had an OPA of 95.6% when compared to EM for both adenovirus and rotavirus and an OPA of 88.1% when compared to rotavirus ICT for rotavirus detection. Performance indicators of the triplex assay testing for adenovirus and rotavirus using Seegene as the standard reference method are reported in Table 4A.

4.3. Three-Way Comparison of Triplex Assay Against EM and/or ICT

To resolve discordant results, the gastrointestinal triplex assay is evaluated against EM and Seegene assay for adenovirus and rotavirus or against ICT and Seegene assay for rotavirus.

4.4. Adenovirus (Triplex-EM-Seegene)

A total of 334 specimens were tested by triplex assay, EM and Seegene assay for the detection of adenovirus and rotavirus (Figure 1 and Table 5A,B).
Among 40 positive adenovirus specimens by Seegene, 32 (80%) were confirmed by both triplex and EM. Subsequently, among 294 negative adenovirus specimens by Seegene, 286 (97.2%) were also confirmed by both EM and triplex. The overall percent agreement between triplex, EM and Seegene for adenovirus detection was (318/334) 95.2% (Table 5A).

4.5. Rotavirus (Triplex-EM-Seegene)

For rotavirus testing, percent agreement among positive rotavirus specimens by Seegene was 97% (97/100); percent agreement among negative rotavirus specimens by Seegene was 97.4% (228/234); and the overall percent agreement between triplex, EM and Seegene for detection of rotavirus was 97.3% (325/334) (Table 5B).

4.6. Rotavirus (Triplex-ICT-Seegene)

A total of 144 specimens were tested by triplex assay, rotavirus ICT and Seegene assay for rotavirus detection (Figure 1 and Table 5C). Percent agreement among positive rotavirus specimen by Seegene was 88.5% (46/52); percent agreement among negative rotavirus specimens by Seegene was 92.4% (85/92); and the overall agreement between triplex, ICT and Seegene for detection of rotavirus was (90.9%) 131/144.

4.7. Analytical Sensitivity: Limit of Detection

Figure 2 depicts partial results (replicates not shown) of a typical LOD experiment using 10-fold serial dilutions of quantified TCF. As seen in Figure 2, adenovirus was detected at 10−6.5 TCID50/mL (Figure 2a) and rotavirus was detected at 10−5.6 TCID50/mL (Figure 2b). Using plasmid DNA, the 95% limit of detection in copies/reaction for adenovirus in the triplex assay was 0.45 copies or 30.8 genome equivalent and for rotavirus 4.5 copies or 10.2 genome equivalent. The estimated 95% LODs were 30.8 genome equivalent GEq/reaction for adenovirus and 10.2 GEq/reaction for rotavirus (results not shown).

4.8. Analytical Specificity

There was no cross-reaction with unrelated gastrointestinal bacteria and viruses in the analytical specificity experiments (Table 1).

4.9. Interference

Triplex primers and probes had no impact on assay sensitivity or performance for any individual target. The MS2, GAPDH and/or B. fragilis controls were detected in all clinical specimens. The presence of co-detection indicated the lack of interference when other pathogens were present in the specimen. Specifically, 12 (1.8%) specimens had rotavirus and adenovirus co-detected in the triplex assay.

4.10. Precision and Turnaround Time

Intra and inter assay precision were both 100%. Average TATs for processing 96 specimens in one run were as follows: 9–10 h for the Seegene assay, 4–5 h for the triplex assay and 3 days for EM.

5. Discussion

We describe the development and evaluation of a triplex molecular assay targeting adenovirus and rotavirus with MS2 included as an internal control. This is a laboratory developed multiplex rRT-PCR assay that simultaneously detects two gastroenteritis viral pathogens from clinical stool specimens. The incorporation of the MS2 internal control assessed for possible inhibition and prevents subsequent false negative reporting. Performance of triplex assay for rotavirus and adenovirus detection was compared with one standard and two non-standard reference testing methods. Three-way comparisons were also performed to enhance this validation and to help interpret discrepant results between methods [26].
Compared to Seegene, the triplex assay demonstrated above average sensitivity, specificity, and NPV for both adenovirus and rotavirus detections. Triplex assay performance was characterized by an above average PPV (96.3%) for rotavirus detection, but an average PPV for adenovirus detection (at 86%) (Table 4A). In comparison to Seegene assay, the triplex assay detected seven (16%) additional positive specimens for adenovirus and four (3.7%) additional specimens positive for rotavirus but missed one rotavirus positive specimen detected by the Seegene assay (Table 3A) (p-values < 0.001). The discrepancies between the two molecular methods could be explained by low viral loads found in these specimens, which plays a critical role in the assay sensitivity and test performance when comparing different methods [27]. Average sensitivities (89.3% and 86.2%) and above average specificities (99.9% and 97.2%) were reported in another study in which two commercial multiplex real time reverse transcription PCR assays were compared with a conventional RT-PCR assay, targeting adenovirus, rotavirus in addition to norovirus, human bocavirus and human paraechovirus [28]. Co-detection of both viruses was found among 12/685 (1.8%) of specimens tested by the triplex assay compared to 4/392 (0.7%) specimens tested by the Seegene assay. The four specimens detected in the Seegene assay were among the twelve co-detections identified by the triplex assay. This also suggests that the triplex assay had higher sensitivity for co-detecting co-pathogens compared to the Seegene assay (Table 3A and Table 4A), though false positivity is also a possibility.
When compared to EM, for detection of adenovirus and rotavirus, all PPA, NPA and OPA were above average (above 90%) (Table 4B). The triplex assay detected 23 (53%) additional adenovirus specimens and 20 (5%) additional rotavirus specimens compared to EM (Table 3B) (p-value < 0.001). These results are very similar to the observations of Logan at al., who reported 100% and 186% increase in adenovirus and rotavirus detection by rRT-PCR compared to EM [29]. Low sensitivity of EM compared to RT-PCR was reported previously, however, positive results by EM offer better correlation with disease in rotavirus infection [30]. PCR results cannot prove disease causality, due to the known presence of the virus shedding in asymptomatic individuals. Two and five stool specimens were positive by EM for adenovirus and rotavirus, respectively but failed to produce amplicons for adenovirus or rotavirus in the triplex assay (Table 3B). This has been reported before and it was suggested that specimens may contain virus particles that are morphologically similar to adenovirus or rotavirus but with little nucleotide sequence homology [31]. This could also be due to loss of virus particles caused by repeat freeze-thaw cycles. These specimens were likely tested by EM closer to disease onset and after prolonged incubation were tested retrospectively by the triplex assay.
In comparison to the rotavirus ICT method, the performance of the triplex assay for rotavirus detection was above average; PPA of (90.6%) and an average NPA of (85.6%) (Table 4B). The triplex assay detected rotavirus in 23 additional specimens compared to rotavirus ICT (Table 3C) (p-value < 0.001). The lower NPA was a result of 15 specimens in which rotavirus was detected by the rotavirus ICT test, but not by the triplex assay. This could be due to false positive results by the rotavirus ICT test, which are commonly reported for the rotavirus ICT test [32,33]. It could also be due to repeat freeze-thaw of specimens used for this validation study, which could cause genome degradation and lower detection. In fact, three of these specimens were confirmed to be rotavirus-positive by EM and one was also confirmed by the Seegene assay, suggesting genome degradation.
When a new testing method is compared to a non-reference standard method, the results may be biased and the information provided for the new testing method may not be accurate [26]. Comparing triplex assay with EM performance for detection of both adenovirus and rotavirus, demonstrated above average OPAs of 95.6% for each comparison, respectively. Triplex assay performance for rotavirus detection compared to rotavirus ICT, shows an average OPA (88.1%). A three-way comparison using a standard-reference method enhances this validation and provide a better way of resolving discrepant results [26].
In summary, the triplex assay demonstrated similar performance to the Seegene assay for both adenovirus and rotavirus targets and performed better than traditional testing methods (EM and rotavirus ICT) used at PHO laboratory. The average PPA of the triplex assay detected for adenovirus in comparison to the Seegene assay could be due to mismatches with the viral templates and differences in oligonucleotide primers and probes used by these assay systems [33]. Possible explanations include higher sensitivity of the Seegene assay for adenovirus detection, or alternatively false positive results by Seegene. In addition to the high performance of the triplex assay, the high through-put, the lower cost compared to commercial assays and EM and the reduced TAT makes it a more desirable testing method. The findings of this study have been extended to guide the planned development of a multiplex platform to test stool specimens for the simultaneous detection of additional gastro-intestinal targets. Despite the current trend towards multiplex assays, single, duplex and triplex assays will remain useful in a research or public health settings when the investigation of a single pathogen is required and high sensitivity assay is needed.
Limitations: This study has some limitations. First, due to high sensitivity, molecular assays (triplex or Seegene assays) may detect more viruses than traditional methods. The clinical relevance of these findings may not be assured as some of these findings may be from asymptomatic patients. Specimens submitted to PHO laboratory for testing are usually collected from patients with clinical symptoms present; however, compliance with PHO laboratory testing guidelines cannot be assured. As well, patient’s specimens were not pre-treated with intercalating dye, such as ethidium monoazide (EMA) or propidium monoazide (PMA), prior to rRT-PCR to enables differentiation of infectious from non-infectious particles [34]. Second, the results of this study are based on a convenience set of specimens and not randomly selected specimens. In this study, the number of positive specimens for rotavirus or adenovirus was slightly higher than that reported in other studies [28,29,30]. This could also affect the performance indicators reported in this study. Last, specimens used in this study were retrospectively tested which may have affected the detection rate; however, since retrospective testing was performed for both triplex and Seegene assays the freeze-thaw effect should have affected both testing methods in a similar manner.
In conclusion, the laboratory–developed triplex assay provides a sensitive, specific, and efficient platform for the simultaneous detection of adenovirus and rotavirus in stool specimens among patients with gastroenteritis. Triplex assay detected more adenovirus and rotavirus than traditional methods. This assay can be effectively implemented in a clinical microbiology setting within and outside reference laboratories. The high performance and through-put, the lower cost than a commercial assay, the reduced TAT in addition to the potential for customizing the number of diarrheal viral targets, makes it a more desirable testing method. Pre-treatment of specimens with EMA or PMA (prior to rRT-PCR) in future studies, could eliminate some of the uncertainties about false positives and false negatives results and add more relevance to clinically usefulness. As such, it may lead to a lower burden of disease for hospital acquired gastrointestinal infections and a monetary saving for the institution for not keeping non-infectious patients in isolation.

Author Contributions

R.R.H. Conceptualization, Methodology, Data analyses, Writing, Review & Editing; A.P. Data analyses, Writing, Review & Editing; M.C. Methodology; J.B.G. Methodology, Review & Editing. All authors have approved the submitted version and agree to be personally accountable for the author’s own contribution and for ensuring that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved and documented in the literature. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

We thank Steve Masney, Eddie King-Chong and the team at Virus Detection and Bacteriology Department for providing adenovirus and rotavirus specimens and controls. We thank Zahra Abbas and Adam Tenn for their help with triplex assay testing.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Lopman, B.A.; Reacher, M.H.; Van Duijnhoven, Y.; Hanon, F.X.; Brown, D.; Koopmans, M. Viral gastroenteritis outbreaks in Europe, 1995–2000. Emerg. Infect. Dis. 2003, 9, 90–96. [Google Scholar] [CrossRef] [PubMed]
  2. Koopmans, M. Outbreaks of viral gastroenteritis: What’s new in 2004? Curr. Opin. Infect. Dis. 2005, 18, 295–299. [Google Scholar] [CrossRef] [PubMed]
  3. Krones, E.; Hogenauer, C. Diarrhea in the immunocompromised patient. Gastroenterol. Clin. North. Am. 2012, 41, 677–701. [Google Scholar] [CrossRef] [PubMed]
  4. Blanton, L.H.; Adams, S.M.; Berard, R.S.; Wei, G.; Bulens, S.N.; Widdowson, M.A.; Glass, R.I.; Monroe, S.S. Molecular and epidemiologic trends of caliciviruses associated with outbreaks of acute gastroenteritis in the United States, 2000–2004. J. Infect. Dis. 2006, 193, 413–421. [Google Scholar] [CrossRef] [Green Version]
  5. Svraka, S.; Duizer, E.; Venema, H.; de Bruin, E.; van der Veer, B.; Dorrestejn, B.; Keoopmans, M. Etiological role of viruses in outbreaks of acute gastroenteritis in the Netherlands from 1994 through 2005. J. Clin. Microbiol. 2007, 45, 1389–1394. [Google Scholar] [CrossRef] [Green Version]
  6. Isabel, S.; Higgins, R.R.; Peci, A.; Isabel, M.R.; Deeks, S.L.; Gubbay, J. Rotavirus genotypes circulating in Ontario, Canada, before and after implementation of the rotavirus immunization program. Vaccine 2018, 36, 2033–2040. [Google Scholar] [CrossRef]
  7. Girard, M.P.; Steele, D.; Chaignat, C.L.; Kieny, M.P. A review of vaccine research and development: Human enteric infections. Vaccine 2006, 24, 2732–2750. [Google Scholar] [CrossRef]
  8. Glass, R.I.; Noel, J.; Ando, T.; Fankhauser, R.; Belliot, G.; Mounts, A.; Parashar, U.D.; Breese, J.S.; Monroe, S.S. The epidemiology of enteric caliciviruses from humans: A reassessment using new diagnostics. J. Infect. Dis. 2000, 181, 254–261. [Google Scholar] [CrossRef]
  9. CDC. Rotavirus. Available online: https://www.cdc.gov/rotavirus/index.html (accessed on 4 June 2019).
  10. Pereira, L.A.; Ferreira, C.E.; Turchetto, G.D.; Nogueira, M.B.; Vidal, L.R.; Cruz, C.R.; Debur, M.C.; de Almeida, S.M.; Raboni, S.M. Molecular characterization of rotavirus genotypes in immunosuppressed and non-immunosuppressed pediatric patients. J. Pediatr. 2013, 89, 278–285. [Google Scholar] [CrossRef] [Green Version]
  11. Wilson, S.E.; Rosella, L.C.; Wang, J.; Renaud, A.; Le Saux, N.; Crowcroft, N.S.; Desai, S.; Harris, T.; Bolotin, S.; Gubbay, J.; et al. Equity and impact: Ontario’s infant rotavirus immunization program five years following implementation. A population based cohort study. Vaccine 2019, 37, 2408–2414. [Google Scholar] [CrossRef]
  12. Uhnoo, I.; Wadell, G.; Svensson, L.; Johansson, M.E. Importance of enteric Adenoviruses 40 and 41 in acute gastroenteritis in infants and young children. J. Clin Microbiol. 1984, 20, 365–372. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Baerjee, A.; De, P.; Manna, B.; Chawla-Sarkar, M. Molecular characterization of enteric adenovirus genotypes 40 and 41 identified in children with acute gastroenteritis in Kolkata, India during 2013–2014. J. Med. Virol. 2017, 89, 606–614. [Google Scholar] [CrossRef] [PubMed]
  14. Matthes-Martin, S.; Boztug, H.; Lion, T. Diagnosis and treatment of adenovirus infection in immunocompromised patients. Expert Rev. Anti Infect. Ther. 2013, 11, 1017–1028. [Google Scholar] [CrossRef] [PubMed]
  15. Lion, T.; Kosulin, K.; Landlinger, C.; Rauch, M.; Preuner, S.; Jugovic, D.; Pötschger, U.; Lawitschka, A.; Peters, C.; Fritsch, G.; et al. Monitoring of adenovirus load in stool by real-time PCR permits early detection of impending invasive infection in patients after allogeneic stem cell transplantation. Leukemia 2010, 24, 706–714. [Google Scholar] [CrossRef]
  16. Ibrahim, S.B.; El-Bialy, A.A.; Mohammed, M.S.; El-Sheikh, A.O.; Elhewala, A.; Bahgat, S. Detection of Rotavirus in children with acute gastroenteritis in Zagazig University Hospitals in Egypt. Electron. Physician. 2015, 7, 1227–1233. [Google Scholar] [CrossRef]
  17. Damen, M.; Minnaar, R.; Glasius, P.; vanderHam, A.; Koen, G.; Wertheim, P.; Beld, M. Real-time PCR with an internal control for detection of all known human adenovirus serotypes. J. Clin. Microbiol. 2008, 46, 3997–4003. [Google Scholar] [CrossRef] [Green Version]
  18. Zeng, S.Q.; Halkosalo, A.; Salminen, M.; Szakal, E.D.; Puustinen, L.; Vesikari, T. One-step quantitative RT-PCR for the detection of rotavirus in acute gastroenteritis. J. Virol. Meth. 2008, 153, 238–240. [Google Scholar] [CrossRef]
  19. Bennett, S.; Gunson, R. The development of a multiplex real-time RT-PCR for the detection of adenovirus, astrovirus, rotavirus and sapovirus from stool samples. J. Virol. Meth. 2017, 242, 30–34. [Google Scholar] [CrossRef]
  20. Higgins, R.R.; Beniprashad, M.; Cardona, M.; Masney, S.; Low, D.E.; Gubbay, J.B. Evaluation and Verification of the Seeplex Diarrhea-V ACE Assay for Simultaneous Detection of Adenovirus, Rotavirus and Norovirus Genogroups I and II in Clinical Stool Specimens. J. Clin. Microb. 2011, 49, 3154–3162. [Google Scholar] [CrossRef] [Green Version]
  21. Garcia, L.; Isenberg, H. Clinical Microbiology Procedures Handbook, 2nd ed.; ASM Press: Santa Monica, CA, USA, 2007. [Google Scholar]
  22. Barber, R.D.; Harmer, D.W.; Coleman, R.A.; Clark, B.J. GAPDH as a housekeeping gene: Analysis of GAPDH mRNA expression in a panel of 72 human tissues. Physiol. Genomics. 2005, 21, 389–395. [Google Scholar] [CrossRef] [Green Version]
  23. Wexler, H.M. Bacteroides: The good, the bad, and the nitty-gritty. Clin. Microbiol. Rev. 2007, 20, 593–621. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Layton, A.; McKay, L.; Williams, D.; Garret, V.; Gentry, R.; Sayler, G. Development of Bacteroides 16S rRNA Gene TaqMan-Based Real-Time PCR Assays for Estimation of Total, Human, and Bovine Fecal Pollution in Water. Appl. Environ. Microbiol. 2006, 72, 4214–4224. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Dreier, J.; Stomer, M.; Kleesiek, K. Use of bacteriphage MS2 as an internal control in viral reverse transcription-PCR assays. J. Clin. Mirobiol. 2005, 43, 4551–4557. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. FDA. Guidance for Industry and FDA Staff. Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests. 2007. Available online: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/statistical-guidance-reporting-results-studies-evaluating-diagnostic-tests-guidance-industry-and-fda (accessed on 19 September 2018).
  27. Soltan, M.A.; Tsai, Y.L.; Lee, P.A.; Tsai, C.F.; Chang, H.G.; Wang, H.T.; Wilkes, R.P. Comparison of electron microscopy, ELISA, real time RT-PCR and insulated isothermal RT-PCR for the detection of Rotavirus group A (RVA) in feces of different animal species. J. Vir. Meth. 2016, 235, 99–104. [Google Scholar] [CrossRef] [Green Version]
  28. Chen, D.; Sun, Y.; Zhao, L.; Qian, Y.; Zhu, R.; Liu, L.; Jia, L.; Dong, H.; Deng, J.; Deng, L. Development of Two Multiplex Real-Time PCR Assays for the Rapid Detection of RNA and DNA Viruses Associated with Gastroenteritis in Pediatric Patients. Pediat. Therapeut. 2014, 4, 208. [Google Scholar] [CrossRef] [Green Version]
  29. Logan, C.; O’Leary, J.J.; O’Sullivan, N. Real-time reverse transcription-PCR for detection of rotavirus and adenovirus as causative agents of acute viral gastroenteritis in children. J. Clin. Microbiol. 2006, 44, 3189–3195. [Google Scholar] [CrossRef] [Green Version]
  30. Philips, G.; Lopman, B.; Tam, C.C.; Iturriza-Gomara, M.; Brown, D.; Gray, J. Diagnosing rotavirus A associated IID: Using ELISA to identify a cut-off for real time RT-PCR. J. Clin. Vir. 2009, 44, 242–245. [Google Scholar] [CrossRef] [Green Version]
  31. Shen, X.X.; Qiu, F.Z.; Li, G.X.; Zhao, M.C.; Wang, J.; Chen, C.; Zhao, L.; Qi, J.J.; Liu, H.; Zhang, Y.; et al. A case control study on the prevalence of enterovirus in children samples and its association with diarrhea. Arch. Vir. 2018. [Google Scholar] [CrossRef]
  32. Tian, Y.; Chughtai, A.A.; Gao, Z.; Yan, H.; Chen, Y.; Liu, B.; Huo, D.; Jia, L.; Wang, Q.; MacIntyre, C.R. Prevalence and genotypes of group A rotavirus among outpatient children under five years old with diarrhea in Beijing, China, 2011–2016. BMC. Infect. Dis. 2018, 18, 497. [Google Scholar] [CrossRef] [Green Version]
  33. Chhabra, P.; Gregoricus, N.; Weinberg, G.A.; Halasa, N.; Chappell, J.; Hassan, F.; Selvarangan, R.; Mijatovic-Rustempasic, S.; Ward, M.L.; Bowen, M.; et al. Comparison of three multiplex gastrointestinal platforms for the detection of gastroenteritis viruses. J. Clin. Virol. 2017, 95, 66–71. [Google Scholar] [CrossRef] [Green Version]
  34. Can, V.D.; Kasuga, I.; Furumaj, H.; Katayama, H. Viability RT-qPCR Combined with Sodium Deoxycholate Pre-treatment for Selective Quantification of Infectious Viruses in Drinking Water Samples. Food. Environ. Virol. 2019, 11, 40–51. [Google Scholar] [CrossRef] [PubMed]
Pathogens 09 00326 g001 550
Figure 1. The distribution of specimens used for performance evaluation of triplex Adenovirus/Rotavirus assay against Seegene, Electron Microscopy and Imunochromatographic test.
Figure 1. The distribution of specimens used for performance evaluation of triplex Adenovirus/Rotavirus assay against Seegene, Electron Microscopy and Imunochromatographic test.
Pathogens 09 00326 g001
Pathogens 09 00326 g002 550
Figure 2. Limit of detection for the triplex assay for adenovirus (a) Adenovirus TCID50 = 10−6.5 and rotavirus (b) Rotavirus TCID50 = 10−5.6. Footnote: TCID50, Tissue Culture Infectivity Dose 50 is the median infectious dose in which 50% of the cells are infected with the virus.
Figure 2. Limit of detection for the triplex assay for adenovirus (a) Adenovirus TCID50 = 10−6.5 and rotavirus (b) Rotavirus TCID50 = 10−5.6. Footnote: TCID50, Tissue Culture Infectivity Dose 50 is the median infectious dose in which 50% of the cells are infected with the virus.
Pathogens 09 00326 g002
Table
Table 1. Distribution of viral and bacterial pathogens used for validation of gastrointestinal triplex Assay.
Table 1. Distribution of viral and bacterial pathogens used for validation of gastrointestinal triplex Assay.
TypePathogenNumber of SpecimensSource
Patients specimensAdenovirus39Stool
Rotavirus349Stool
Negative specimens297Stool
Control specimensSamonella Paratyphi A2Stool
Samonella Typhi1Stool
Shigella Dysenteriae Type 25Stool
Escherichia coli - 0:157:H71Culture
Samonella enteritidis1Culture
Shigella flexneri1Culture
Shigella sonnei1Culture
Vibrio cholera1Culture
Camplyobacter jejuni1Culture
Influenza B2NP swabs
Influenza A/ H1N1 5NP swabs
Influenza A/ H3N2 1NP swabs
Picornavirus2NP swabs
Aermonas hydrophila1Culture
Neisseria meningitidis1Culture
Streptococcus pyogenes2Culture
Streptococcus pneumoniae2Culture
Staphylococcus epidermidis1Culture
Total716
Footnote: Distribution of 685 and 32 stool and control specimens, respectively used for the development and validation of the triplex assay. Stool specimens were submitted for testing from February 2009 to February 2013.
Table
Table 2. Primers and probes used for target amplification in the gastrointestinal virus triplex assay.
Table 2. Primers and probes used for target amplification in the gastrointestinal virus triplex assay.
TargetPrimers and Probe SequencesAmp Size (BP)NA PositionPCR MethodReferences
Adenovirus
(hexon region)		FO: CAG GAC GCC TCG GRG TAY CTS AG
RE: GGA GCC ACV GTG GGR TT
PR: FAM-CCG GGT CTG GTG CAG TTT GCC CGC-BHQ		10317649–17752 *rRT-PCRDamen, M. et al. 2008 [17]
Rotavirus
(NSP3 gene)		FO: CCA TCT WCA CRT RAC CCT CTA TGA G
RE: GGT CAC ATA ACG CCC CTA TAG C
PR: CY5-AGT TAA AAG CTA ACA CTG TCA AA-BHQ		86963–1049rRT-PCRZeng, S. et al. 2008 [18]
B. fragilis
(16S rRNA gene)		FO: GAG AGG AAG GTC CC
RE: CGC TAC TTG GCT GG
PR: FAM-CCA TTG ACC AAT ATT CCT CAC TGC TGC CT-BHQ		129296–425RT-PCRLayton, A. et al. 2006 [24]
MS2
(MS2-TM2JOE)		FO: GGC TGC TCG CGG ATA CC
RE: TGA GGG AAT GTG GGA ACC G
PR: JOE-ACC TCG GGT TTC CGT CTT GCT CGT-BHQ1		2013166–3367rRT-PCRDrier et al., 2005 [25]
Footnote: R, A/G; Y, C/T; V, A or C or G; W, A/T. FO, forward; RE, reverse; PR, probe; rRT-PCR, reverse real time reverse transcription PCR; Amp, amplicon; NA, nucleic acid. Base pair position for the adenovirus amplicon varies in different serotypes; * The position shown are for hexon gene of human adenovirus F41 strain (DQ#315364).
Table
Table 3. Results obtained for adenovirus and rotavirus in the gastrointestinal triplex assay compared to Seegene assay, EM and ICT testing methods.
Table 3. Results obtained for adenovirus and rotavirus in the gastrointestinal triplex assay compared to Seegene assay, EM and ICT testing methods.
A-Triplex Compared to Seegene
Seegene result for Adenovirus Seegene result for Rotavirus
TriplexDetectedNegativeTotalTriplexDetectedNegativeTotal
Adenovirus43750Rotavirus1064110
Negative0319319Negative1258259
Total43326369Total107262369
B-Triplex Compared to EM
EM result for Adenovirus EM result for Rotavirus
TriplexDetectedNegativeTotalTriplexDetectedNegativeTotal
Adenovirus372360Rotavirus27820298
Negative2516518Negative5275280
Total39539578Total283295578
C-Triplex Compared to ICT
ICT result for Rotavirus
TriplexDetectedNegativeTotal
Rotavirus14523168
Negative15137152
Total160160320
Footnote: Distribution of testing results for adenovirus and rotavirus of specimens tested by two testing methods (triplex and Seegene), (triplex and EM), (triplex and ICT). PHO laboratory test: Seegene assay and EM detects adenovirus and rotavirus; ICT detects rotavirus.
Table
Table 4. Measuring the accuracy of gastrointestinal triplex assay for identification of adenovirus and/or rotavirus against Seegene assay, EM and ICT testing methods.
Table 4. Measuring the accuracy of gastrointestinal triplex assay for identification of adenovirus and/or rotavirus against Seegene assay, EM and ICT testing methods.
A-Performance of Triplex Assay Against a Reference Standard Method
Seegene
AdenovirusRotavirus
Triplex AccuracyFormulaEstimate % (95% CI)Estimate % (95% CI)
SensitivityTP/(TP + FN) × 100100 (91.7–100)99.1 (94.9–99.8)
SpecificityTN/(FN + TN) × 10097.8 (95.6–99.1)98.4 (96.1–99.6)
PPVTP/(TP + FN) × 10086 (74.7–92.7)96.3 (90.9–98.6)
NPVTN/(FN + TN) × 10010099.6 (97.3–99.9)
B-Performance of Triplex Assay Against Two Non-Reference Standard Methods
EMICT
AdenovirusRotavirusRotavirus
Triplex AccuracyFormulaEstimate % (95% CI)Estimate % (95% CI)Estimate % (95% CI)
PPAa/(a + c) × 10094.8 (81.3–99.1)98.2 (95.6–99.3)90.6 (84.7–94.4)
NPAd/(b + d) × 10095.7 (93.5–97.2)93.2 (89.5–95.7)85.6 (79.0–90.4)
OPA(a + d)/(a + b + c + d) × 10095.6 (93.5–97.1)95.6 (93.5–97.1)88.1 (83.9–91.3)
Footnote: Sensitivity, Specificity, Positive Predictive Value and Negative Predictive Value were calculated when triplex was compared to a standard reference testing method. Positive Percent Agreement, Negative Percent Agreement, Overall Percent Agreement were calculated when triplex was compared to a non-standard reference testing method. TP-True positive, FN-false negative, TN-true negative, FN-false negative a-number of positive results concordant between both methods; b and c number of discordant positive/negative results between two methods, d- number of concordant negative results between two methods. For this comparison a performance below 85% was considered as ‘below average’, at 86%–90% as ‘average’ and above 90% as ‘above average’.
Table
Table 5. A three-way comparison of results obtained for adenovirus and rotavirus in the gastrointestinal triplex assay compared to EM (A and B) or ICT (C) using Seegene assay as the standard reference method.
Table 5. A three-way comparison of results obtained for adenovirus and rotavirus in the gastrointestinal triplex assay compared to EM (A and B) or ICT (C) using Seegene assay as the standard reference method.
A-Adenovirus Results
New TestNon-Reference Standard MethodTotalReference Standard Method
Seegene
TriplexEM+−
++32320
+−1587
−+101
−−2860286
Total33440294
B-Rotavirus Results
New TestNon-reference Standard MethodTotalReference Standard method
Seegene
TriplexEM+−
++97970
+−734
−+202
−−2280228
Total334100234
C-Rotavirus Results
New TestNon-Reference Standard MethodTotalReference Standard Method
TriplexICTSeegene
+−
++46460
+−752
−+615
−−85085
Total1445292
Footnote: In both analyses Seegene assay is the reference standard method. Results shown are per specimen based and are compared side to side. The sign under each test method indicates confirmed positive (+) or negative (−) results by each method. For this comparison a performance below 85% was considered as ‘below average’, at 86%–90% as ‘average’ and above 90% as ‘above average’.

Share and Cite

MDPI and ACS Style

Higgins, R.R.; Peci, A.; Cardona, M.; Gubbay, J.B. Validation of a Laboratory-Developed Triplex Molecular Assay for Simultaneous Detection of Gastrointestinal Adenovirus and Rotavirus in Stool Specimens. Pathogens 2020, 9, 326. https://doi.org/10.3390/pathogens9050326

AMA Style

Higgins RR, Peci A, Cardona M, Gubbay JB. Validation of a Laboratory-Developed Triplex Molecular Assay for Simultaneous Detection of Gastrointestinal Adenovirus and Rotavirus in Stool Specimens. Pathogens. 2020; 9(5):326. https://doi.org/10.3390/pathogens9050326

Chicago/Turabian Style

Higgins, Rachel R., Adriana Peci, Mark Cardona, and Jonathan B. Gubbay. 2020. "Validation of a Laboratory-Developed Triplex Molecular Assay for Simultaneous Detection of Gastrointestinal Adenovirus and Rotavirus in Stool Specimens" Pathogens 9, no. 5: 326. https://doi.org/10.3390/pathogens9050326

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop