Skip to Content
MDPIMDPI

Advanced

MicroorganismsMicroorganisms
PDF
  • Article
  • Open Access

26 January 2026

14 Pages

Full Factorial Comparison of the Diagnostic Performance of Three Nucleic Acid Extraction Kits and Three PRRSV RT-qPCR Assays Using Swine Oral Fluids of Known Status

,
,
,
,
,
,
,
,
and
1
Department of Veterinary Diagnostic and Production Animal Medicine, College of Veterinary Medicine, Iowa State University, Ames, IA 50011, USA
2
Department of Statistics, College of Liberal Arts and Sciences, Iowa State University, Ames, IA 50011, USA
3
Tetracore Inc., Rockville, MD 20850, USA
*
Author to whom correspondence should be addressed.
Microorganisms2026, 14(2), 282;https://doi.org/10.3390/microorganisms14020282
This article belongs to the Special Issue Viral Infection on Swine: Pathogenesis, Diagnosis and Control
Version Notes
Order Reprints

Abstract

Porcine reproductive and respiratory syndrome (PRRS) is one of the costliest diseases in swine production, causing >$1.2 billion USD in annual losses in the United States. Oral fluids are widely used for PRRS virus (PRRSV) surveillance, accounting for 42% of nearly 480,000 PRRSV RT-qPCR cases submitted to six Midwestern U.S. laboratories between 2020 and 2025. Despite this reliance, few studies have applied appropriate methodological approaches to compare the performance of commercial extraction and PRRSV RT-qPCR protocols for oral fluid specimens. In this study, we evaluated nine extraction-amplification protocols for PRRSV RNA detection, based on three commercial extraction kits and three commercial RT-qPCR assays. For each protocol, performance was evaluated using 314 oral fluid samples of known status (215 positive, 99 negative), collected under controlled conditions from 72 pigs assigned to five groups inoculated with contemporary PRRSV isolates and from one negative control group. The Cq values were normalized as efficiency-standardized Cqs (ECqs) and then analyzed by receiver operating characteristic (ROC) analysis. The mean amplification efficiencies ranged from 67 to 92%, repeatability from 0.98 to 0.99, and overall reproducibility was 0.91. The ROC AUCs ranged from 0.916 to 0.986, with significant pairwise differences (p < 0.05). At optimal ECq cutoffs, sensitivities ranged from 83 to 98.1% with 100% specificity. Normalization enabled objective protocol comparisons and statistically valid diagnostic cutoffs and supports future improvements in PRRSV diagnostics.

1. Introduction

Porcine reproductive and respiratory syndrome virus (PRRSV) remains the costliest disease in modern swine production. In the United States, for example, industry-wide losses have been estimated at over USD 1.2 billion annually [1]. In Europe, farm-level losses range from EUR 255 per sow in endemic settings [2] to over EUR 650,000 annually in severely affected wean-to-finish operations [3].
Given its substantial economic impact, accurate and timely detection of PRRSV in commercial herds is essential to its management and control. Thus, between 2020 and 2025, 479,743 diagnostic cases (the majority containing multiple samples for testing) were submitted to six Midwestern US veterinary diagnostic laboratories for PRRSV RT-qPCR testing [4]. The variety of specimens submitted for testing included blood-derived samples, environmental samples, oral fluid, processing fluid, semen, swabs (nasal, blood, oral), tissues, tongue tip fluids, and others. Among these, oral fluid samples were included in 42% of all submissions (199,179 cases). This reflects both the routine use of oral fluids in herd-level PRRSV surveillance and its integration into laboratory workflows and testing protocols.
In the laboratory, PRRSV detection has followed historical technological developments. Initially, PRRSV detection was based on isolation in either primary cells (macrophages) or continuous cell lines [5]. The introduction of PRRSV reverse transcription polymerase chain reaction (RT-PCR) technology precluded the need to isolate the virus from the clinical specimens [6,7,8,9]. By the early 2000s, real-time RT-PCR (RT-qPCR) assays provided a more efficient approach to PRRSV RNA detection [10,11]. After 2004, the availability of commercial PRRSV RT-qPCR assays facilitated standardized testing across laboratories [12].
A number of PRRSV commercial RT-qPCR assays are currently available and the test is used extensively. For example, the Iowa State University veterinary diagnostic laboratory performed approximately 1,400,000 PRRSV RT-qPCR tests between 2020 and 2025 [4]. Despite its widespread use, comparisons of diagnostic performance among commercial PRRSV RT-qPCRs, particularly for swine oral fluids, are limited. Moreover, the existing studies often lack appropriate methodological approaches for evaluating extraction and amplification workflows, thereby limiting the ability to validate and standardize diagnostic cutoffs. Therefore, the objective of this study was to apply a full-factorial design and the appropriate analytical methodology to evaluate the diagnostic performance of commercially available extraction and amplification protocols, using oral fluid samples from pigs inoculated with contemporary PRRSV isolates.

2. Materials and Methods

2.1. Experimental Design

Commercially available nucleic acid extraction kits (n = 3) and PRRSV RT-qPCR assays (n = 3) were evaluated in a 3 × 3 full factorial design (Table 1), using 99 PRRSV-negative and 215 PRRSV-positive pen-based oral fluid samples from five groups of pigs, each inoculated with a different contemporary PRRSV isolate, plus a negative control group [13]. Estimates of diagnostic sensitivity and specificity were determined by receiver operating characteristic (ROC) analysis.
Table 1. Full factorial design—3 nucleic acid extraction kits and 3 PRRSV RT-qPCR assays.

2.2. PRRSV Isolates

All PRRSV-2 isolates used in this study were recovered from routine submissions to the Iowa State University (ISU) Veterinary Diagnostic Laboratory (Ames, IA, USA). Virus isolation, propagation, and titration were performed in ZMAC cells [14] to a concentration of 1 × 106 TCID50 per ml, as previously described [15]. The pairwise nucleotide identity (%) between isolates, based on complete genome alignment, was determined by using the blastn suite-2sequences tool in NCBI BLAST® (https://blast.ncbi.nlm.nih.gov/, accessed on 14 August 2025) [16] (Table 2).
Table 2. Whole genome similarity matrix for five PRRSV-2 isolates included in the study and VR-2332.

2.3. Oral Fluid Samples from Pigs of Known PRRSV Infection Status

Pen-based oral fluid samples (n = 314) of known PRRSV status were derived from a study approved by the ISU Office of Research Ethics (IACUC-21-124; IBC-21-124) and conducted in biosafety level 2 (BSL-2) animal research facilities. Full details of this study are provided elsewhere [13].
Pigs (n = 72) in the study were serum tested for the PRRSV RNA and antibody 8 days prior to study enrollment. Upon arrival at 3 weeks of age, pigs were ear-tagged, re-tested for PRRSV infection, blocked by weight, and randomly assigned to one of 6 groups, i.e., 5 groups inoculated with PRRSV and a negative control group, using a random number generator (Excel Microsoft), and allowed to acclimate for 7 days. Groups to be inoculated with PRRSV were housed in separate rooms. Each room contained one large pen, which was divided into two smaller pens (inoculated pen and contact pen), using metal gating that allowed nose-to-nose interactions between pigs in adjacent pens. Initially, the inoculated pen was populated with 8 pigs and the contact pen remained vacant. Pigs were placed in the contact pen on day post-inoculation (DPI) 2.
Prior to inoculation, pigs were trained for oral fluid sampling, beginning 6 days before inoculation (designated as DPI −6). Training consisted of placing a 50 cm (19.7″) × 1.3 cm (1/2″) diameter, 3-strand, twisted 100% cotton rope (Skydog Rigging Supply, Lake in the Hills, IL, USA) on the pen floor for ~15 min and then hanging the rope from a bracket attached to the side of the pen gating. Thereafter, pen-based oral fluid samples were collected twice daily from DPI −5 to DPI 28. At the end of each study day, samples were aliquoted (10 aliquots per sample), labeled appropriately (including a unique random number), and stored at −80 °C until testing.
On DPI 0, pigs were inoculated with PRRSV isolates at a concentration of 1 × 106 TCID50/mL, which were administered intramuscularly (2 mL/pig) and intranasally (2 mL/naris). The five virus-inoculated groups received the following isolates: Group 1, isolate I; Group 2, isolate II; Group 3, isolate III; Group 4, isolate IV; and Group 5, isolate V. Pigs in the negative control group were mock-inoculated with medium (Roswell Park Memorial Institute (RPMI) 1640 medium, Gibco, Thermo Fisher Scientific, Inc., Waltham, MA, USA) by the same routes and volume. On DPI 2, PRRSV-naive pigs (n = 4) were placed in the contact pen in each room of PRRSV-inoculated pigs.

2.4. PRRSV RNA Extraction Procedures

As listed in Table 1, 3 commercial extraction kits were evaluated. Each extraction kit was evaluated independently, i.e., one complete sample set was processed using a single extraction kit before proceeding to the next. Prior to extraction, a full set of oral fluid samples (previously unthawed) was thawed overnight at 4 °C [17]. All nucleic acid extractions were performed on the KingFisher™ Flex automated extraction instrument (Thermo Fisher Scientific, Inc.), following the manufacturer’s recommended procedures for oral fluid samples. Every nucleic acid extraction plate included 86 test samples, 4 PRRSV-specific reference standards, and 2 negative extraction controls (nuclease free water). Technical comparisons of plate setup and consumables can be found in Table 3.
Table 3. Comparison of technical attributes among nucleic acid (NA) extraction kits.
Procedure for Extraction Kit A: Mix oral fluid (300 μL) with lysis/binding solution [lysis/binding solution concentrate (450 μL), carrier RNA (2 μL), Xeno™ Internal Positive Control RNA (Thermo Fisher Scientific, Inc.) (2 μL), and VetAlert™ inhibition control (IC) RNA Control (Tetracore, Inc.) (6 μL)]. Combine prepared sample (600 μL), magnetic bead mix (20 μL), and 100% isopropanol (350 μL) in a deep-well plate (KingFisher™ 96 deep-well plate, v-bottom, polypropylene, Thermo Fisher Scientific, Inc.). Prior to loading plate into extraction instrument, pre-fill two deep-well plates with Wash Solution 1 (300 μL per well, two washes per sample) and two deep-well plates with Wash Solution 2 (400 μL per well, two washes per sample). Elute purified nucleic acids into MagMAX™ elution buffer (90 μL).
Procedure for Extraction Kit B: Mix oral fluid (300 μL) with buffer VXL (150 μL). Combine prepared sample (300 μL) with buffer ACB mixture [Buffer ACB (400 μL), MagAttract Suspension G (25 μL), carrier RNA (1 μL), Xeno™ Internal Positive Control RNA (Thermo Fisher Scientific, Inc.) (2 μL), and VetAlert™ inhibition control (IC) RNA Control (Tetracore, Inc.) (6 μL)]. Prior to loading plate into extraction instrument, pre-fill one deep-well plate with Buffer AW1 (700 μL per well), one deep-well plate with Buffer AW2 (700 μL per well), and one deep-well plate with ethanol 100% (750 μL per well). Elute nucleic acids into Buffer AVE (100 μL).
Procedure for Extraction Kit C: Mix oral fluid sample (250 μL) with stabilization solution (125 μL), lysis solution (350 μL), 2 μL of Xeno™ Internal Positive Control RNA (Thermo Fisher Scientific, Inc.), and 6 μL of the VetAlert™ IC RNA Control (Tetracore, Inc.). Prior to loading plate into extraction instrument, pre-fill one deep-well plate for Wash 1 [50% ethanol (100 μL) and wash buffer (900 μL)], and one deep-well plate with 50% ethanol (450 μL). Elute nucleic acids into elution buffer (50 μL).

2.5. RT-qPCR Reference Standards

To normalize RT-qPCR results across plates and to serve as positive extraction controls, PRRSV-specific reference standards were included on each plate. Reference standards were generated by reconstituting a lyophilized 10-dose PRRSV modified live virus (MLV) vaccine (Ingelvac® PRRSV MLV, Boehringer Ingelheim Vetmedica, Inc., Duluth, GA, USA) with 20 mL of PRRSV-free oral fluid. This solution was serially diluted ten-fold in PRRSV-free oral fluids and the 2 dilutions that yielded quantification cycle (Cq) values of ~27 (1 × 10−3) and ~30 (1 × 10−4) were selected to provide a reference range for normalization across plates. For testing, each dilution (in duplicate) was included on each plate.

2.6. PRRSV RNA Amplification

For PRRSV RNA amplification, each fresh sample extract was divided into three aliquots, which served as templates for the three RT-qPCR assays. The use of three thermocyclers enabled simultaneous testing of these aliquots. Procedure for Assay a: Combine RT-qPCR mixture [10X PRRSV 3.0 Primer Probe Mix (2 μL) (Thermo Fisher Scientific, Inc.), 2X PRRSV 3.0 Master Mix (10 μL) (Thermo Fisher Scientific, Inc.)], with template [8 μL of sample extract, positive amplification control (PAC; VetMAX™ PRRSV 3.0 Controls, Thermo Fisher Scientific, Inc.), or negative amplification control (NAC; nuclease free water)] for a final reaction volume of 20 μL.
Procedure for Assay b: Combine RT-qPCR mixture [virotype® Mix + IC(JOE)-RNA (15 μL), virotype® PRRSV PRO Primers/Probes (2 μL)] with template [8 μL of sample extract, PAC (virotype® PRRSV PRO Positive Control, Indical Bioscience), or NAC] for a final reaction volume of 25 μL.
Procedure for Assay c: Combine RT-qPCR mixture [EZ-PRRSV™ MPX 4.0 (17.25 μL), enzyme blend (0.75 μL)] with template [7 μL of sample extract, NAC, or PAC (North American PRRSV PC (3.5 μL), and European PRRSV PC (3.5 μL) (EZ-PRRSV™ MPX 4.0 Control Set, Tetracore, Inc.)] for a final reaction volume of 25 μL.
Each RT-qPCR plate (MicroAmp® Fast Optical 96-Well Reaction Plate, Thermo Fisher Scientific, Inc.) included one assay-specific positive amplification control and one negative amplification control (nuclease free water), extracts from 2 negative extraction controls, and extracts from 4 PRRSV-specific reference standards. Plates were sealed using clear heat-sealing film (Clear Seal Diamond Heat Sealing Film, Thermo Fisher Scientific, Inc.) with a manual heat sealer (Thermo Scientific™ ALPS 50 V-Manual Heat Sealer, Thermo Scientific™). RT-qPCRs were run on the Applied Biosystems™ 7500 fast thermal cycler (Thermo Fisher Scientific, Inc.) and results were evaluated and reported as raw Cqs with the Design and Analysis Software (DA2 Software v2.8.0; Thermo Fisher Scientific, Inc.). Cycling program details and other technical features can be found in Table 4.
Table 4. Comparison of technical attributes among PRRSV RT-qPCR assays.

2.7. Normalization of PRRSV RT-qPCR Cq Results

Raw Cq values from PRRSV RT-qPCRs were re-expressed as “Efficiency standardized Cqs (ECqs),” using Equation (1) [18,19].
Efficiency standardized Cq (ECq) = E−∆Cq = E−(Sample Cq − Mean Cq of reference standards)
In Equation (1), E denotes amplification efficiency and ΔCq represents the difference between a sample’s Cq and the mean Cq of the reference standards run on the same plate. Amplification efficiency can be expressed either as a ratio (number of target amplicons at the end of a PCR cycle, divided by the number at the beginning) or as a percentage. An efficiency of 2 (or 100%) would represent doubling at each cycle, which may be interpreted as perfect amplification.
To normalize the results, E values (expressed as ratios) for each reference standard were estimated from the raw fluorescence data, using web-based software LinRegPCR (https://www.gear-genomics.com/rdml-tools/linregpcr.html, accessed on 14 August 2025) [20]. In Equation (1), E was replaced with the arithmetic mean of the 4 reference standards E values. E estimates > 2 (i.e., 100%) were truncated at 2, as recommended elsewhere [18]. For samples with undetermined Cq values, i.e., a sample with no amplification or no detectable fluorescence, the final Cq value was defaulted to 40 (Assay a or b) or 45 (Assay c), depending on the assay.
ECq values are interpreted as the fold change in target concentration between a sample and the reference standard. For example, a sample with a Cq of 28 run on a plate with a reference standard mean E of 92% (E = 1.92) and mean Cq of 30 would have an ECq of 3.69 (ECq = E−ΔCq = 1.92−(28–30) = 3.69): that is, the concentration of the target in the sample is 3.69 times the concentration of the target in the reference standard.

2.8. Analyses

The analysis of each protocol included testing results from all negative samples (n = 99), i.e., negative control group (n = 58; DPI −5 to 28), contact pigs prior to placement (n = 16; DPI −5 to 2), and PRRSV-inoculated groups prior to exposure (n = 25; DPI −5 to −1). Positive samples (n = 215) were defined as those from the pens of the PRRSV-inoculated groups after exposure (n = 115; DPI 1 through 28) and from contact pens (n = 100; DPI 4 through 28).
ECqs were cube-root-transformed and then analyzed using RStudio v4.2.2 [21]. Receiver operating characteristic (ROC) analyses (pROC package v1.18.5) [22] were performed to estimate the area under the curve (AUC) and the diagnostic specificities and sensitivities of each protocol over a range of ECq cutoffs. Estimates were generated using cluster bootstrapping with 1000 iterations, in which pens were treated as the resampling unit. This approach preserves the within-pen correlation structure of the data by including all of the results associated with a pen in each bootstrap sample. Pairwise comparisons of the AUC values were conducted using DeLong’s test for correlated ROC curves [23]: a nonparametric method for assessing statistically significant differences.
Repeatability (precision within a protocol) and reproducibility (precision across all protocols) coefficients were estimated by using the ECq responses from the reference standards. To estimate repeatability, a random-effects model was fitted for each protocol, using lme4 (v1.1-34.1) [24] as given in Equation (2):
ECq ijk =   Dilution i   +   Plate j   +   ϵ ijk
The repeatability coefficient of each protocol was calculated with Equation (3) as the proportion of total ECq variance explained by consistent differences between the dilutions of reference standards relative to plate-to-plate and measurement variability.
Repeatability   coefficient = σ Dilution 2 σ Dilution 2   + σ Plate 2   + σ ϵ 2
The reproducibility coefficient was estimated from a random-effects model (lme4 package v1.1-34.1) [24], based on Equation (4):
ECq ijkl =   Dilution i   +   Protocol l   +   Dilution   ×   Protocol il   +   Plat e jl   +   ϵ ijkl
where Dilutioni is the dilution effect, Protocoll is the protocol effect, (Dilution × Protocol)il is the interaction between dilution level i and protocol l, Platejl is the effect of plate j within protocol l, and ϵijkl is the measurement error. The reproducibility coefficient was estimated to address the consistency of ECq responses across all protocol combinations, using Equation (5).
Reproducibility   coefficient = σ Dilution 2 σ Dilution 2   + σ Protocol   2 + σ Dilution   ×   Protocol 2   + σ Plate 2   + σ ϵ 2

3. Results

As shown in Table 5, mean amplification efficiencies (E) estimated from the reference standards tested on every plate varied between 1.67 (67%) and 1.92 (92%), with most protocols exceeding 1.70 (70%). Protocol-specific repeatability coefficients were between 0.98 and 0.99. The overall reproducibility coefficient across all protocols was 0.91, based on ECq testing results from 215 positive and 99 negative oral fluid samples (n = 314).
Table 5. Performance metrics for PRRSV RT-qPCR protocols.
Estimated AUC values for the nine PRRSV RT-qPCR protocols ranged from 0.916 (95% CI: 0.882, 0.959; protocol 6) to 0.986 (0.974, 1.000; protocol 8).
Pairwise comparisons of AUC values demonstrated significant differences (p < 0.05) between several protocol pairs (Table 6). Significant differences were observed in multiple pairwise comparisons, indicating heterogeneity in overall diagnostic discrimination across the evaluated extraction and RT-qPCR workflows. Although all protocols demonstrated high diagnostic accuracy, pairwise AUC testing detected measurable differences in ROC-based performance metrics.
Table 6. p-values from pairwise AUC a comparisons for 9 PRRSV RT-qPCR protocols.
As shown in Table 7, test cutoffs based on Youden’s index produced diagnostic sensitivities ranging from 83% (protocol 6) to 98.1% (protocols 2 and 8) but diagnostic specificity was uniformly 100%. For comparison, diagnostic sensitivities and specificities were estimated for arbitrary cutoffs of ECq = 0.1 and ECq = 0.2. As shown in Table 7, use of these arbitrary cutoffs affected diagnostic sensitivity, diagnostic specificity, or both, but generally did not improve upon the overall test performance achieved using the Youden cutoff.
Table 7. Estimated diagnostic sensitivity (Dx Se) and specificity (Dx Sp) by protocol and cutoff a.
Table 8 provides the number of PRRSV RT-qPCR-positive samples and the mean ECq values by protocol over time based on a 0.2 ECq cutoff. This longitudinal analysis of positive detection across DPIs demonstrated a consistent early detection (DPI 1–3) in inoculated pens across all protocols. Positivity peaked for most protocols around DPI 10–12, with declining detection thereafter. Similarly, contact pens began showing positivity 2 days post-placement (DPI 4–6). In general, the mean ECq values in positive samples showed greater variability at later DPIs, which was consistent with the declining viral loads.
Table 8. Number of PRRSV RT-qPCR-positive samples by protocol over time, based on a 0.2 ECq cutoff for each protocol.

4. Discussion

According to the World Organization for Animal Health (WOAH), diagnostic assay assessments should be grounded in two distinct performance domains: analytical performance, i.e., the assay’s technical capacity to detect the target analyte, and diagnostic performance, i.e., the probability that the assay will correctly differentiate positive and negative animals [25].
Analytical performance parameters include precision (repeatability/reproducibility), analytical sensitivity (limit of detection), and analytical specificity. In contrast, diagnostic performance is the probability that a test will correctly identify a diagnostic specimen as originating from a positive (diagnostic sensitivity) or negative (diagnostic specificity) individual. Estimates of analytical performance are useful for evaluating and understanding an assay’s reliability in the laboratory, but diagnostic performance, i.e., diagnostic sensitivity, and specificity are an assay’s “primary performance indicators” [25] and are fundamental to the interpretation of field results. Although a variety of approaches are used in test evaluation, e.g., kappa coefficient [26], latent variable models [27], Bayesian models [27], the WOAH Terrestrial Manual states that estimates of diagnostic sensitivity and specificity are ideally based on testing samples from animals of known infection status and an evaluation of the results using receiver operating characteristic (ROC) curve analysis [25].
ROC analysis was first developed in World War II as a method to measure how well radar operators differentiated friendly aircraft from hostile aircraft [28], but the approach has been applied to a wide variety of disciplines, including data mining, diagnostic medicine, machine learning, preventive medicine, and psychophysics [29]. ROC analysis is grounded in two principles: (1) the diagnostic outcome is intended to discriminate between two mutually exclusive states, e.g., positive vs. negative, whose distributions overlap over a specific range [30], and (2) sensitivity and specificity must be evaluated across a continuum of cutoff values [31]. As shown in Table 9, ROC analysis has previously been used to estimate the diagnostic performance of PCR assays for viruses, e.g., Getah virus [32], norovirus [33], PRRSV [34], Aujeszky’s disease virus [35], rotavirus A [36], and bacteria, e.g., Clostridium difficile [37]. The advantage ROC brings to the analysis of test performance is that it assesses assay accuracy across the full range of decision thresholds (cutoffs), thereby providing an overview of a test’s ability to distinguish between positive and negative samples [29]. When expressed graphically, ROC summarizes the relationship between the rate of true positive vs. false positive results as a single metric: the area under the curve (AUC) [29].
Table 9. Examples of ROC-derived qPCR cutoffs and their respective performance metrics.
The disadvantage of ROC is that it requires samples of known infection status: a problem addressed in this study by collecting samples from animals following inoculation with PRRS viruses under experimental conditions [13]. An additional requirement of ROC is that all sampling results have a numeric result. Thus, “right censored” results, e.g., test results reported as “Cq ≥ 40”, violate the ROC requirement for binormal data distribution [30,38,39]. In this study, PRRSV RT-qPCR results on oral fluid specimens from animals of known PRRSV status were re-expressed as efficiency-standardized quantification cycles (ECqs), i.e., Cq values were normalized as a function of amplification efficiency, using matrix- and pathogen-specific reference standards derived from a commercial PRRSV modified live vaccine [18,19,35]. This approach meets the key criteria for the reference standards used for relative quantification and normalization of qPCR results, i.e., purity, amplification equivalence, matrix compatibility, and concurrent amplification, and is practical for service laboratories [18]. ECqs yielded valid numerical values for all samples, including those that would otherwise be censored, thereby restoring the continuous distribution required for ROC analysis. To fully explore assay performance, we evaluated the effect of ECq cutoffs on diagnostic sensitivity and specificity (Youden’s index, 0.1, 0.2). Ultimately, cutoff selection may be guided by the impact of false positive results on commercial swine production systems.
There are relatively few publications that report PCR diagnostic performance based on ROC analyses. However, among these publications (Table 9), diagnostic sensitivity and specificity varied markedly across pathogens and specimen types. These differences underscore the fact that a robust, statistically valid approach to evaluating RT-qPCR diagnostic performance is needed to provide reliable comparisons.
PRRSV surveillance based on periodic sampling (serum, tissue exudates, udder wipes, or oral fluids) and RT-qPCR testing has become the norm on swine farms in many parts of the world. This study evaluated the diagnostic performance of PRRSV RT-qPCR using oral fluid samples from pigs inoculated with contemporary PRRSV isolates, including lineage 1 representatives that have been associated with high mortality, transmissibility, and substantial production losses. Although these do not represent the exhaustive diversity of PRRSV-2 circulating in the United States, the use of contemporary isolates is particularly relevant because the ongoing emergence of new PRRSV variants compels diagnostic companies to continually update and validate their assays. Accordingly, evaluating extraction-amplification protocols against contemporary PRRSV isolates ensures that surveillance tools remain effective in the face of viral genetic diversity.
Across the nine extraction-amplification workflows, diagnostic performance was consistently high, but measurable differences were observed. In the longitudinal analysis of the PRRSV RT-qPCR response, inoculated pens showed a minimum detection rate of 85.7% on DPI 1–3, with five of the nine protocols achieving 100% detection during this period. Similarly, all protocols reached 100% detection in contact pens at two days post-placement (DPI 4–6). For comparison, studies of PRRSV RNA detection in oral fluids have reported a 70% detection rate on DPI 2 [40]. A transient reduction in the detection rate and mean ECq was observed across protocols on DPI 13–15, which was followed by a rebound in detection on DPI 16–18. Similar dynamics have been reported in experimental PRRSV infections, where viremia initially declines but subsequently reappears: a phenomenon referred to as “virus rebound” [41,42]. Thus, the temporary drop in detection observed here reflects the biological variability of PRRSV infection, rather than a limitation of the diagnostic protocols themselves. Further, between DPI 22–28, the number of positive samples remained relatively high, but the mean ECq values began to decline, indicating lower viral loads among detected positives. Thus, differences in detection performance during late-stage infections may be attributed to the immunological resolution of the infection and declining concentrations of the virus in the samples.

5. Conclusions

In the present study, normalization of Cq values to ECqs and subsequent ROC analysis identified thresholds that optimized diagnostic sensitivity and specificity. Across the nine protocols, ECq values demonstrated high within- and between-protocol precision, with the repeatability and reproducibility coefficients approaching “1”. Analyses of test results by ROC and pairwise comparisons of AUC values revealed statistically significant differences among protocols. Taken together, these findings demonstrated that normalization (ECqs) and ROC-based cutoff selection avoided reliance on arbitrary Cq thresholds and provided precise estimates of diagnostic sensitivity and specificity. We recognize, however, that the present work was limited to experimental samples, and future studies incorporating field-derived specimens will be essential to confirm applicability under diverse diagnostic conditions. Overall, this approach offers a practical framework for assessing molecular assay performance and strengthening surveillance strategies in routine diagnostics.

Author Contributions

Conceptualization, B.A.-L., L.G.G.-L., and J.J.Z.; methodology, B.A.-L.; validation, B.A.-L., L.G.G.-L., and J.J.Z.; formal analysis, D.Z.; investigation, B.A.-L., G.R., J.Z., B.M.-R., G.T., and J.J.Z.; resources, G.R., J.Z., R.R., and K.-J.Y.; data curation, B.A.-L.; writing—original draft preparation, B.A.-L. and J.J.Z.; writing—review and editing, B.A.-L., G.R., J.Z., B.M.-R., G.T., D.Z., K.-J.Y., L.G.G.-L., and J.J.Z.; visualization, B.A.-L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Iowa State University Veterinary Diagnostic Laboratory.

Institutional Review Board Statement

The animal study protocol was approved by the Iowa State University Institutional Animal Care and Use Committee (IACUC-21-124, 3 August 2021) and the Institutional Biosafety Committee (IBC-21-054, 10 August 2021), and was conducted in accordance with institutional guidelines for the care and use of animals in BSL-2 facilities.

Data Availability Statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Acknowledgments

Tetracore’s extraction and RT-qPCR reagents were kindly provided by Tetracore, Inc.

Conflicts of Interest

Author Rolf Rauh was employed by the company Tetracore, Inc. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. Osemeke, O.; Silva, G.S.; Corzo, C.A.; Kikuti, M.; Vadnais, S.; Yue, X.; Linhares, D.; Holtkamp, D. Economic impact of productivity losses attributable to porcine reproductive and respiratory syndrome virus in United States pork production, 2016–2020. Prev. Vet. Med. 2025, 244, 106627. [Google Scholar]
  2. Renken, C.; Nathues, C.; Swam, H.; Fiebig, K.; Weiss, C.; Eddicks, M.; Ritzmann, M.; Nathues, H. Application of an economic calculator to determine the cost of porcine reproductive and respiratory syndrome at farm-level in 21 pig herds in Germany. Porc. Health Manag. 2021, 7, 3. [Google Scholar]
  3. Nathues, H.; Alarcon, P.; Rushton, J.; Jolie, R.; Fiebig, K.; Jimenez, M.; Geurts, V.; Nathues, C. Cost of porcine reproductive and respiratory syndrome virus at individual farm level—An economic disease model. Prev. Vet. Med. 2017, 142, 16–29. [Google Scholar]
  4. Swine Disease Reporting System Dashboards. Available online: https://fieldepi.org/SDRS/ (accessed on 5 August 2025).
  5. Wensvoort, G.; Terpstra, C.; Pol, J.M.A.; ter Laak, E.A.; Bloemraad, M.; de Kluyver, E.P.; Kragten, C.; van Buiten, L.; den Besten, A.; Wagenaar, F.; et al. Mystery swine disease in the Netherlands: The isolation of Lelystad virus. Vet. Q. 1991, 13, 121–130. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  6. Christopher-Hennings, J.; Nelson, E.A.; Nelson, J.K.; Hines, R.J.; Swenson, S.L.; Hill, H.T.; Zimmerman, J.J.; Katz, J.B.; Yaeger, M.J.; Chase, C.C. Detection of porcine reproductive and respiratory syndrome virus in boar semen by PCR. J. Clin. Microbiol. 1995, 33, 1730–1734. [Google Scholar] [CrossRef] [Scilit]
  7. Mardassi, H.; Wilson, L.; Mounir, S.; Dea, S. Detection of porcine reproductive and respiratory syndrome virus and efficient differentiation between Canadian and European strains by reverse transcription and PCR amplification. J. Clin. Microbiol. 1994, 32, 2197–2203. [Google Scholar] [CrossRef] [Scilit]
  8. Suárez, P.; Zardoya, R.; Prieto, C.; Solana, A.; Tabarés, E.; Bautista, J.M.; Castro, J.M. Direct detection of the porcine reproductive and respiratory syndrome (PRRS) virus by reverse polymerase chain reaction (RT-PCR). Arch. Virol. 1994, 135, 89–99. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  9. Van Woensel, P.; Van Der Wouw, J.; Visser, N. Detection of porcine reproductive respiratory syndrome virus by the polymerase chain reaction. J. Virol. Methods 1994, 47, 273–278. [Google Scholar] [CrossRef] [Scilit]
  10. Egli, C.; Thür, B.; Liu, L.; Hofmann, M.A. Quantitative TaqMan® RT-PCR for the detection and differentiation of European and North American strains of porcine reproductive and respiratory syndrome virus. J. Virol. Methods 2001, 98, 63–75. [Google Scholar] [CrossRef] [Scilit]
  11. Spagnuolo-Weaver, M.; Walker, I.W.; Campbell, S.T.; McNeilly, F.; Adair, B.M.; Allan, G.M. Rapid detection of porcine reproductive and respiratory syndrome viral nucleic acid in blood using a fluorimeter-based PCR method. Vet. Microbiol. 2000, 76, 15–23. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  12. Wasilk, A.; Callahan, J.D.; Christopher-Hennings, J.; Gay, T.A.; Fang, Y.; Dammen, M.; Reos, M.E.; Torremorell, M.; Polson, D.; Mellencamp, M.; et al. Detection of U.S., Lelystad, and European-like porcine reproductive and respiratory syndrome viruses and relative quantitation in boar semen and serum samples by real-time PCR. J. Clin. Microbiol. 2004, 42, 4453–4461. [Google Scholar] [CrossRef] [Scilit]
  13. Rawal, G.; Almeida, M.N.; Gauger, P.C.; Zimmerman, J.J.; Ye, F.; Rademacher, C.J.; Armenta Leyva, B.; Munguia-Ramirez, B.; Tarasiuk, G.; Schumacher, L.L.; et al. In vivo and in vitro characterization of the recently emergent PRRSV 1-4-4 L1C variant (L1C.5) in comparison with other PRRSV-2 Lineage 1 isolates. Viruses 2023, 15, 2233. [Google Scholar] [CrossRef] [Scilit]
  14. Zuckermann, F.A. A Study on the Feasibility of Using Porcine Alveolar Macrophage Cell Line to Produce a PRRS Modified Live Virus Vaccine—NPB #05-200. Pork Checkoff Research Report 05-200. National Pork Board. 2007. Available online: https://porkcheckoff.org/wp-content/uploads/2021/02/05-200-ZUCKERMAN-UofILL.pdf (accessed on 14 August 2025).
  15. Yim-im, W.; Huang, H.; Park, J.; Wang, C.; Calzada, G.; Gauger, P.; Harmon, K.; Main, R.; Zhang, J. Comparison of ZMAC and MARC-145 cell lines for improving porcine reproductive and respiratory syndrome virus isolation from clinical samples. J. Clin. Microbiol. 2021, 59, e01757-20. [Google Scholar] [CrossRef] [Scilit]
  16. Zhang, Z.; Schwartz, S.; Wagner, L.; Miller, W. A greedy algorithm for aligning DNA sequences. J. Comput. Biol. 2000, 7, 203–214. [Google Scholar] [CrossRef] [Scilit]
  17. Weiser, A.C.; Poonsuk, K.; Bade, S.A.; Rotolo, M.; Gauger, P.C.; Harmon, K.M.; Gonzalez, W.M.; Wang, C.; Main, R.; Zimmerman, J.J. Effects of sample handling on the detection of porcine reproductive and respiratory syndrome virus in oral fluids by reverse-transcription real-time PCR. J. Vet. Diagn. Investig. 2018, 30, 807–812. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  18. Armenta-Leyva, B.; Munguía-Ramírez, B.; Cheng, T.-Y.; Ye, F.; Henao-Díaz, A.; Giménez-Lirola, L.G.; Zimmerman, J. Normalizing real-time PCR results in routine testing. J. Vet. Diagn. Investig. 2024, 36, 78–85. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  19. Pfaffl, M.W. A new mathematical model for relative quantification in real-time RT-PCR. Nucleic Acids Res. 2001, 29, e45. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  20. Untergasser, A.; Ruijter, J.M.; Benes, V.; van den Hoff, M.J.B. Web-based LinRegPCR: Application for the visualization and analysis of (RT)-qPCR amplification and melting data. BMC Bioinform. 2021, 22, 398. [Google Scholar] [CrossRef] [Scilit]
  21. RStudio Team. RStudio: Integrated Development Environment for R. RStudio: PBC, Boston, USA. 2020. Available online: http://www.rstudio.com (accessed on 18 June 2025).
  22. Robin, X.; Turck, N.; Hainard, A.; Tiberti, N.; Lisacek, F.; Sanchez, J.-C.; Müller, M. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform. 2011, 12, 77. [Google Scholar] [CrossRef] [Scilit]
  23. DeLong, E.R.; DeLong, D.M.; Clarke-Pearson, D.L. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 1988, 44, 837–845. [Google Scholar] [CrossRef] [Scilit]
  24. Bates, D.; Maechler, M.; Bolker, B.; Walker, S.; Christensen, R.H.B.; Singmann, H.; Dai, B.; Scheipl, F.; Grothendieck, G.; Green, P.; et al. Package ‘lme4’. Convergence 2015, 12, 2. [Google Scholar]
  25. Colling, A.; Gardner, I. Validation of diagnostic assays for infectious diseases of terrestrial animals. In Manual of Diagnostic Tests and Vaccines for Terrestrial Animals, 12th ed.; World Organisation for Animal Health (WOAH): Paris, France, 2023; pp. 1–27. [Google Scholar]
  26. Dhamnetiya, D.; Jha, R.P.; Shalini, S.; Bhattacharyya, K. How to analyze the diagnostic performance of a new test? Explained with illustrations. J. Lab. Physicians 2022, 14, 90–98. [Google Scholar]
  27. Fanshawe, T.R.; Nicholson, B.D.; Perera, R.; Oke, J.L. A review of methods for the analysis of diagnostic tests performed in sequence. Diagn. Progn. Res. 2024, 8, 8. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  28. Brown, C.D.; Davis, H.T. Receiver operating characteristics curves and related decision measures: A tutorial. Chemom. Intell. Lab. Syst. 2006, 80, 24–38. [Google Scholar] [CrossRef] [Scilit]
  29. Greiner, M.; Pfeiffer, D.; Smith, R.D. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev. Vet. Med. 2000, 45, 23–41. [Google Scholar] [PubMed]
  30. Hajian-Tilaki, K. Receiver Operating Characteristic (ROC) curve analysis for medical diagnostic test evaluation. Casp. J. Intern. Med. 2013, 4, 627–635. [Google Scholar]
  31. Zhou, X.H. Testing an underlying assumption on a ROC curve based on rating data. Med. Decis. Mak. 1995, 15, 276–282. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  32. Cao, X.; Qiu, X.; Shi, N.; Ha, Z.; Zhang, H.; Xie, Y.; Wang, P.; Zhu, X.; Zhao, W.; Zhao, G.; et al. Establishment of a reverse transcription real-time quantitative PCR method for Getah virus detection and its application for epidemiological investigation in Shandong, China. Front. Microbiol. 2022, 13, 1009610. [Google Scholar] [CrossRef] [Scilit]
  33. Phillips, G.; Lopman, B.; Tam, C.C.; Iturriza-Gomara, M.; Brown, D.; Gray, J. Diagnosing norovirus-associated infectious intestinal disease using viral load. BMC Infect. Dis. 2009, 9, 63. [Google Scholar] [CrossRef] [Scilit]
  34. Vilalta, C.; Sanhueza, J.M.; Schwartz, M.; Kikuti, M.; Torremorell, M.; Corzo, C.A. Assessing the litter level agreement of RT-PCR results for porcine reproductive and respiratory syndrome virus in testicles, tails and udder wipes diagnostic samples relative to serum from piglets. Prev. Vet. Med. 2021, 186, 105211. [Google Scholar] [CrossRef] [Scilit]
  35. Cheng, T.-Y.; Henao-Diaz, A.; Poonsuk, K.; Buckley, A.; van Geelen, A.; Lager, K.; Harmon, K.; Gauger, P.; Wang, C.; Ambagala, A.; et al. Pseudorabies (Aujeszky’s disease) virus DNA detection in swine nasal swab and oral fluid specimens using a gB-based real-time quantitative PCR. Prev. Vet. Med. 2021, 189, 105308. [Google Scholar] [CrossRef] [Scilit]
  36. Phillips, G.; Lopman, B.; Tam, C.C.; Iturriza-Gomara, M.; Brown, D.; Gray, J. Diagnosing rotavirus A associated IID: Using ELISA to identify a cut-off for real time RT-PCR. J. Clin. Virol. 2009, 44, 242–245. [Google Scholar] [CrossRef] [Scilit]
  37. Kamboj, M.; Brite, J.; McMillen, T.; Robilotti, E.; Herrera, A.; Sepkowitz, K.; Babady, N.E. Potential of real-time PCR threshold cycle (CT) to predict presence of free toxin and clinically relevant Clostridioides difficile infection in patients with cancer. J. Infect. 2018, 76, 369–375. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  38. Caraguel, C.G.B.; Stryhn, H.; Gagné, N.; Dohoo, I.R.; Hammell, K.L. Selection of a cutoff value for real-time polymerase chain reaction results to fit a diagnostic purpose: Analytical and epidemiologic approaches. J. Vet. Diagn. Investig. 2011, 23, 2–15. [Google Scholar]
  39. Hanley, J.A. The use of the ‘binormal’ model for parametric ROC analysis of quantitative diagnostic tests. Stat. Med. 1996, 15, 1575–1585. [Google Scholar] [CrossRef] [Scilit]
  40. Kittawornrat, A.; Prickett, J.; Chittick, W.; Wang, C.; Engle, M.; Johnson, J.; Patnayak, D.; Schwartz, T.; Whitney, D.; Olsen, C.; et al. Porcine reproductive and respiratory syndrome virus (PRRSV) in serum and oral fluid samples from individual boars: Will oral fluid replace serum for PRRSV surveillance? Virus Res. 2010, 154, 170–176. [Google Scholar] [CrossRef] [Scilit]
  41. Go, N.; Touzeau, S.; Islam, Z.; Belloc, C.; Doeschl-Wilson, A. Hot to prevent viremia rebound? Evidence frorm a PRRSv data-supported model of immune response. BMC Syst. Biol. 2019, 13, 15. [Google Scholar]
  42. Rowland, R.R.R.; Lunney, J.; Dekkers, J. Control of porcine reproductive and respiratory syndrome (PRRS) through genetic improvements in disease resistance and tolerance. Front. Genet. 2012, 3, 260. [Google Scholar] [CrossRef] [Scilit] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Molecular Docking and Structure–Activity Relationship Study of Polyphenols with Antibacterial and Antibiotic-Modulating Properties
Previous
Editorial for Special Issue “Biotechnology for Environmental Remediation”
Next