Inter-Laboratory Evaluation and Successful Implementation of MS2 Coliphage as a Surrogate to Establish Proficiency Using a BSL-3 Procedure

Abstract

The U.S. Environmental Protection Agency’s (EPA) Water Laboratory Alliance relies on the Centers for Disease Control and Prevention’s ultrafiltration-based Water Processing Procedure (WPP) for concentration of biosafety level 3 (BSL-3) agents from 10 L to 100 L of drinking water. The WPP requires comprehensive training and practice to maintain proficiency, resulting in a critical need for quality control (QC) criteria. The aim of this study was to develop criteria using male-specific (MS2) coliphage (BSL-2 agent) to minimize safety hazards associated with BSL-3 agents and to use the criteria to evaluate analytical proficiency during a demonstration exercise. EPA Method 1602 with EasyPhage was used during the study to develop QC criteria for 100-mL, and 40–100 L samples. The demonstration exercise indicated that the MS2 criteria would allow laboratories to demonstrate proficiency using the WPP with 40–100 L samples. In addition, the QC criteria developed for 100-mL samples has broad applicability at laboratories that are using MS2 for other types of analyses, such as assessment of water treatment devices. The development of MS2 QC criteria allows laboratories to develop and confirm ongoing proficiency using the WPP.

1. Introduction

During emergency responses to drinking water contamination events, it may be necessary to concentrate large volumes of water to detect biosafety level 3 (BSL-3) agents of concern, which can be present at very low levels in drinking water. The U.S. Environmental Protection Agency’s (EPA) Water Laboratory Alliance (WLA) currently relies on the Laboratory Response Network’s (LRN) Water Processing Protocol (WPP), which is a tangential flow, hollow fiber ultrafiltration (UF)-based procedure, for concentration and analysis of BSL-3 bioterrorism threat agents from large volumes (10-L to 100-L) of drinking water. The LRN’s WPP requires comprehensive training and ongoing practice for laboratories to achieve and maintain proficiency. In addition, non-laboratory groups such as hazmat (hazardous materials) responders may be required to collect and concentrate large-volume samples in an effort to increase the number of samples that can be shipped and analyzed during drinking water contamination incidents. As a result, it is critical that quality control (QC) criteria and method performance information be available, especially for laboratories and responders who may use a method for the first time during an emergency situation. Because BSL-3 agents are difficult to handle and present significant safety hazards, the EPA collaborated with the Centers for Disease Control and Prevention (CDC) from 2007 through 2009 to design and conduct a QC criteria development study using a viral surrogate. During the QC criteria study design process, male-specific (MS2) coliphage (American Type Culture Collection (ATCC^®) 15597-B1™, family Leviviridae, genus Levivirus, strain MS2 [1]) was evaluated as a potential non-BSL-3 virus [2,3] because it is readily available, has broad applicability (i.e., is in widespread use), and has been successfully concentrated using UF.

MS2 is commonly used as an indicator of fecal contamination and is used to evaluate the efficacy of size exclusion technologies for point-of-use and point-of-entry water treatment devices that are manufactured and deployed worldwide. Its broad applicability, such as its use in evaluation of the efficacy of systems to treat wastewater [4,5,6,7,8,9,10,11,12] and evaluation of ground-water sources [13] has resulted in MS2 testing becoming an integral component of multiple standards, including the National Sanitation Foundation’s (NSF) P248 protocol for testing Microbiological Water Purifiers for use in Emergency Military Operations [14] and the EPA/NSF Environmental Technology Verification Program (ETV) Protocol for equipment verification testing for inactivation of microbiological contaminants [15]. The World Health Organization (WHO) also recommends MS2 and somatic coliphage as surrogates for pathogenic viruses in testing and evaluating household water treatment devices [16]. In addition, multiple studies [17,18,19,20] have reported that MS2 can be successfully recovered from large-volume water samples following concentration using UF.

During the preliminary phases of the QC criteria study, analyses were conducted to evaluate MS2 spiking approaches, two analytical culture methods (Standard Methods (SM) 9224 F [21] and EPA Method 1602 [22]) and commercially-prepared plates with EasyPhage medium. Commercially-prepared EasyPhage plates incorporate a bacterial stain that provides contrast between blue plaques and a red bacterial host lawn, resulting in plaques that are much easier to identify and enumerate than those on the media specified in EPA Method 1602. Based on the preliminary results, a multi-laboratory study was conducted to generate QC criteria using Scientific Methods, Inc. (SMI)-prepared spikes and EPA Method 1602 using EasyPhage medium.

Subsequent to the 2009 QC criteria development study, the EPA collaborated with the CDC to conduct a demonstration exercise in 2012 that would increase the number of LRN laboratories deemed proficient by the LRN to respond to a water contamination event using LRN’s WPP. MS2 was included in the demonstration exercise as a viral surrogate due to the accessibility and practicality of its associated analytical detection method (real-time reverse transcription polymerase chain reaction (RT-PCR) by TaqMan^® assay (Chapel Hill, NC, USA)).

This paper provides results from the QC criteria development study and limited results from the demonstration exercise. It also provides an overview of MS2 QC criteria that laboratories could use to achieve and maintain proficiency with the UF procedure without the potential safety hazards and handling requirements associated with BSL-3 agents. In addition, the QC criteria provided for small volume (100-mL) analyses using EPA Method 1602 with EasyPhage has broad international applicability at laboratories that are using MS2 for other types of analyses, such as assessment of water treatment devices, as use of EasyPhage medium will simplify media preparation and plaque enumeration and allow for smaller laboratories with limited resources (e.g., water baths) to conduct these types of analyses.

2. Materials and Methods

A description of the materials and methods that were used during MS2 quantitative QC criteria development are included in this section for: (a) the analytical method using small-volume (100-mL) reference matrix and drinking water samples; and (b) large-volume (40–100 L) reference matrix and drinking water samples concentrated using UF (Section 2.1); and the use of MS2 to demonstrate proficiency as a BSL-3 viral surrogate during a demonstration exercise (Section 2.2).

2.1. MS2 QC Criteria Development

2.1.1. Ultrafiltration (UF)

The UF procedure utilized during the QC criteria development study used a hollow fiber ultrafilter (Fresenius Optiflux^® F200NR Dialysis Filter (Waltham, MA, USA)) to concentrate large volumes (40–100 L) of water. Each water sample was treated with 0.5 mL/L of a 10% w/v solution of sodium thiosulfate (if chlorine may have been present) and 10% sodium polyphosphate (NaPP) prior to concentration. The filter was pre-treated with a fetal bovine calf serum solution that was removed with a NaPP solution. The water sample was continually recirculated through the ultrafilter to remove the water and leave particulates (>30,000 dalton molecular weight) in the concentrated retentate. Analysts controlled the flow rate of the recirculation and removal of water by adjusting a flow regulator. After concentrating the water sample, the filter was eluted with a wash buffer containing Tween^® 80, NaPP, and Antifoam Y-30. The concentrated volume and wash solution were combined to produce the final retentate of ~300–500 mL.

2.1.2. Reference Matrix

Sterile phosphate-buffered saline (PBS) was selected as the reference matrix for which QC criteria would be developed and serves as a standardized matrix that is repeatable during future analyses. The QC criteria development study used 10X PBS (commercially available) that was diluted with reagent-grade water to a final concentration of 0.01X (i.e., 40 mL (10X PBS)/40 L reagent-grade water).

2.1.3. Preliminary QC Criteria Analyses: Selection of Spiking Approach and Analytical Method

UF concentrates target and non-target analytes, as well as potential inhibitory substances present in samples. As a result, the spiking approach and downstream analytical method had to be compatible with UF and the resulting retentates. During preliminary phases of the QC criteria development study, the spiking approach was confirmed and several analytical approaches were evaluated for both small-volume and large-volume samples, the latter being concentrated using UF. Key preliminary QC criteria study analyses are summarized in

Table 1. Summary of key preliminary QC criteria analyses.

Phase	n	Volume	Matrix	Spike 1 Level per Sample (PFU) 2	Large Volume Processing	Analytical Method
1A	3	100-mL	PBS	100 PFU	NA	SM 9224 F 3
	3	100-mL	DW 4	100 PFU
1B	3	40-L	PBS	1000 PFU	UF, followed by filtering and analyzing 100 mL of retentate per sample	SM 9224 F 3
	3	100-L	DW 4	1000 PFU
2	3	100-mL	PBS	100 PFU	NA	Method 1602 5
	3	100-mL	DW 4	100 PFU
	3	100-mL	PBS	100 PFU	NA	Method 1602 5, using EasyPhage
	3	100-mL	DW 4	100 PFU
	3	40-L	PBS	1000 PFU	UF, followed by analyzing 100 mL of retentate per sample	Method 1602 5
	3	100-L	DW 4	1000 PFU
	3	40-L	PBS	1000 PFU	UF, followed by analyzing 100 mL of retentate per sample	Method 1602 5, using EasyPhage
	3	100-L	DW 4	1000 PFU

Notes: ¹ Spiked with Scientific Methods, Inc. (SMI)-propagated and titered MS2 (ATCC^® 15597-B1™) suspensions; ² Plaque forming unit (PFU); ³ Samples were processed according to Standard Methods 9224 F: Detection of Coliphages-Membrane Filter Method (SM 9224 F); ⁴ Drinking water (DW); ⁵ Samples were processed according to the Single Agar Layer (SAL) procedure in EPA Method 1602, Section 12.

.

2.1.4. Spike Approach

Initial preliminary QC criteria study analyses were conducted using 100-mL PBS and 100-mL drinking water (DW) samples spiked with BioBalls manufactured by BTF, a bioMérieux company (ATCC^® 15597-B1™). Unfortunately, BTF discontinued production of MS2 in BioBall^® format due to manufacturing issues and variability of the product, resulting in the need for an alternate spike approach. Options considered included the use of either participant laboratory-prepared or referee-prepared MS2 spiking suspensions. Use of referee-prepared spiking suspensions was deemed more appropriate to minimize inter-laboratory variability, thus samples throughout the remainder of the QC criteria study were spiked with SMI-propagated and titered MS2 (ATCC^® 15597-B1™) suspensions.

2.1.5. Analytical Methods

Analytical methods were assessed in two phases and included the evaluation of membrane filtration (Phases 1A and 1B), and single agar layer (SAL) methods and media (Phase 2). During Phase 1A, 100-mL PBS and 100-mL DW samples were each spiked with 1 mL of ~100 plaque forming units (PFU)·mL⁻¹ MS2 spiking suspension prepared by SMI. Following spiking, these 100-mL samples were analyzed according to Standard Methods 9224 F: Detection of Coliphages-Membrane Filter Method (SM 9224 F). Phase 1A results indicated that SMI-prepared spikes and SM 9224 F were appropriate for evaluation with large-volume samples. During Phase 1B 40-L PBS and 100-L DW samples were analyzed, with 40-L of PBS (rather than 100-L) being utilized as a time- and cost-saving measure. Each large-volume sample was spiked with 1 mL of ~1000 PFU·mL⁻¹ MS2 spiking suspension prepared by SMI and concentrated using UF. Following concentration, 100 mL of the retentate from each sample was filtered and analyzed using SM 9224 F. While Phase 1A results indicated that SM 9224 F performed well for analyses of 100-mL samples, low recoveries (<5%) observed for large-volume (40-L PBS and 100-L DW) samples during Phase 1B indicated that SM 9224 F, as written, was not acceptable for the study. To address the low recoveries observed for the large-volume samples, retentates with final MgCl₂ concentrations of 0.1 M, 0.25 M, and 0.5 M were evaluated to potentially increase adherence of MS2 to the membrane filter. However, this approach did not mitigate the low recovery issue for large-volume samples. Based on these results, an alternate method was evaluated during Phase 2 preliminary analyses.

During Phase 2, small-volume samples and retentate from concentrated large-volume samples were processed according to the SAL procedure in EPA Method 1602: Male-specific (F+) and Somatic Coliphage in Water by Single Agar Layer (SAL) Procedure (EPA Method 1602, Section 12.0). However, because there was considerable concern regarding the time-consuming nature of the method and water bath requirements for tempering the molten agar for analyses, half of the samples were processed using commercially-prepared plates with EasyPhage medium by SAL as per Method 1602 to determine whether it could be used for QC criteria development. The potential advantages of EasyPhage medium included eliminating the need to prepare and temper large volumes of media in a water bath, as well as having plaques that were easier to identify and count due to the incorporation of a bacterial stain that provides contrast between blue plaques and the red bacterial host lawn. Prior to concentration and/or analysis during Phase 2, each 100-mL PBS and 100-mL DW sample was spiked with 1 mL of ~100 PFU·mL⁻¹ MS2 spiking suspension and each large-volume sample (40-L PBS and 100-L DW) was spiked with 1 mL of ~1000 PFU·mL⁻¹ MS2 spiking suspension prepared by SMI. Recoveries were improved for large-volume sample retentates analyzed using Method 1602, compared to SM 9224 F, regardless of media. Since higher recoveries were observed when EasyPhage was used, compared to EPA Method 1602 as written, EasyPhage medium was used for all subsequent QC criteria development analyses.

2.1.6. QC Criteria Development for the Analytical Method (without UF) and the UF Procedure

Recovery and precision QC criteria were developed for both the analytical method and the UF procedure. To develop the analytical method QC criteria, small-volume (100-mL) PBS and DW samples were processed using EPA Method 1602 with EasyPhage medium (without UF). These criteria will enable laboratories to confirm that analytical method performance is acceptable, independent of UF. To develop QC criteria for the UF procedure, large-volume (40-L PBS or 100-L DW) samples were concentrated using UF and MS2 detected using EPA Method 1602 with EasyPhage medium.

2.1.7. Laboratory Selection and Analyses

Twelve environmental/public health volunteer laboratories were recruited to participate in the study, six of which were members of the EPA WLA. Prior to generating data for development of QC criteria, the 12 participant laboratories were required to develop and demonstrate proficiency with UF and to practice the use of EPA Method 1602 with EasyPhage medium. To demonstrate proficiency with UF, laboratories spiked 40-L PBS and 100-L DW samples with Enterococcus faecalis 550 colony forming unit (CFU) BioBalls (ATCC^® 29212™) according to EPA Method 1600: Enterococci in Water by Membrane Filtration Using membrane-Enterococcus Indoxyl-β-d-Glucoside Agar (mEI) [23]. Laboratories were required to achieve E. faecalis recoveries of ≥50% to be considered proficient using UF. To practice using EPA Method 1602 and EasyPhage medium, laboratories analyzed unspiked and spiked 100-mL PBS and 100-mL DW samples. Laboratories were required to meet EPA Method 1602 QC criteria using EasyPhage media to be considered proficient. Once laboratories demonstrated proficiency with both UF and EPA Method 1602, study analyses were conducted.

For development of analytical method criteria, independent of UF, small-volume samples were analyzed (four, 100-mL PBS samples and four, 100-mL DW samples). Each sample was spiked with 1 mL of ~100 PFU·mL⁻¹ MS2 spiking suspension prepared by SMI and processed according to EPA Method 1602 using EasyPhage medium. Results from analyses of spiked, 100-mL PBS samples were used to generate initial precision and recovery (IPR) and ongoing precision and recovery (OPR) criteria for EPA Method 1602 with EasyPhage medium. Results from analyses of spiked, 100-mL DW samples were used to generate matrix spike (MS) and matrix spike/matrix spike duplicate (MS/MSD) criteria for EPA Method 1602 with EasyPhage medium.

For development of QC criteria for UF, large-volume samples (four, 40-L PBS samples and four, 100-L DW samples) were each spiked with 1 mL of SMI-prepared MS2 spiking suspension at ~1000 PFU·mL⁻¹ and concentrated using UF. Retentate was processed according to EPA Method 1602 using EasyPhage medium. Results from these analyses were used to generate IPR, OPR, MS, and MS/MSD criteria for large-volume samples concentrated by UF.

2.1.8. Calculations and Statistical Analyses

Two separate outlier tests, the Youden test for outlying laboratories and the Grubbs test for outlying results, were applied to the data used to determine QC criteria to assess whether the data were representative of the population of laboratories proficient in performing the method. Details on the application of these outlier tests are found in the American Society for Testing and Materials (ASTM) guidance D2777-98 [24]. Data identified as outliers were excluded from QC criteria calculations but not method performance summaries.

QC recovery and precision criteria were developed based on prediction intervals from all valid, spiked PBS reference matrix results for initial and ongoing performance, and all valid spiked drinking water sample results for MS and MS/MSD. When determining method performance, IPR criteria were applied to the analytical results of four replicate PBS samples; OPR criteria were applied to the analytical results of a single PBS sample. Recovery prediction limits were calculated based on within- and between-laboratory variance components [25]. When determining method performance in drinking water, recovery criteria were applied to the analytical results of a single MS sample. These variance components were calculated with PROC MIXED from SAS version 9 [26] by applying the maximum likelihood method of estimation to the recovery results. IPR criteria were calculated using an upper prediction limit for the relative standard deviation (RSD) of four replicate sample results (PBS) and MS/MSD precision criteria were calculated using an upper prediction limit for the relative percent difference (RPD) of two replicate sample results (drinking water). The exact formulas used to calculate the recovery and precision criteria were dependent on the distribution of the study sample results. All outlier tests were performed and all QC specifications were determined using untransformed results, because the distribution of individual recoveries did not depart from a normal distribution based on graphical and statistical assessments.

2.2. Demonstration Exercise

2.2.1. Laboratory Selection

Thirteen public health/environmental laboratories were identified based on previous participation in the QC criteria study or experience with LRN’s WPP to participate as volunteers in a demonstration exercise to evaluate laboratory proficiency with the LRN WPP, with nine of those laboratories being members of the EPA WLA.

2.2.2. Analyses

MS2 was used as a viral surrogate for laboratories to demonstrate the ability to correctly perform secondary concentration for viruses. MS2 (ATCC^® 15597-B1™) stocks were produced at CDC using trypticase soy broth, streptomycin/ampicillin, and host bacteria E. coli Famp (ATCC^® 700891™) according to ATCC guidance. Frozen, enumerated stocks used for spiking suspensions were diluted to a concentration of 100,000 PFU·mL⁻¹ by preparing 10-fold dilutions in PBS diluent consisting of 0.01 M PBS (Dulbecco’s Modification, pH 7.4), 0.01% (w/v) Tween^® 80, and 0.001% (w/v) Antifoam Y-30 emulsion. MS2 spiking suspensions were enumerated by SAL per EPA Method 1602, as written (without EasyPhage). MS2 RNA was extracted from water concentrates using a Qiagen Viral RNA Mini Kit, and RT-PCR analyses of MS2 spiking suspensions and water samples were performed using a TaqMan^® assay [18].

Prior to analyzing samples from the demonstration exercise, the participant laboratories had to analyze samples using the RT-PCR protocol to demonstrate the ability to successfully detect MS2. Each laboratory performed ribonucleic acid (RNA) extraction and limit of detection (LOD) with RT-PCR analyses using the MS2 positive control stock (100,000 PFU·mL⁻¹) provided by CDC. Following RNA extraction, 10-fold dilutions (1:10, 1:100, 1:1000) of the extracted RNA were made and assayed by RT-PCR. Laboratories were expected to achieve cycle threshold (Ct) values for the undiluted stock ranging from 23 to 27, along with producing a trend of increasing Ct values (roughly three Ct values for each 10-fold dilution) for the dilutions.

As part of the larger demonstration exercise, each participant laboratory received two blinded tubes that contained either 1 mL of MS2 spiking suspension (100,000 PFU·mL⁻¹) or 1 mL of deionized water (DI water). Each laboratory spiked 40-L samples (laboratory tap water) with the blinded tubes (one sample per tube) and concentrated using UF. A secondary concentration step was performed on a portion of the retentate using Centricon^® Plus-70 microconcentrators before being analyzed by RT-PCR. Since the demonstration exercise included other non-viral analytes, the secondary concentration step was performed only on a portion of the concentrated sample to allow for detection of the other analytes using the remainder of the concentrated sample. Based on testing at CDC, it was expected that 1:4 and 1:10 sample dilutions from spiked samples would result in average Ct values of 32–36 for a 1:4 dilution and 35–40 for a 1:10 dilution. These values could vary based on the quality of tap water tested by each laboratory and could also be lower, as assay performance improves with experience.

3. Results

3.1. MS2 QC Criteria Development

For the QC criteria development study, data review forms were used to ensure that the data packages submitted by each participant laboratory were complete and that study-specific and method-specific requirements were met. Data provided for each sample was reviewed to confirm that original forms were submitted, QC checks were performed and exhibited the appropriate response, method-specific holding times were met, media and reagents were used within expiration dates, and samples were spiked appropriately. Only valid results were included, as described below.

3.1.1. Results of Preliminary QC Criteria Analyses: Selection of Spiking Approach and Analytical Method

Mean recoveries for 100-mL PBS and DW samples spiked with SMI-prepared MS2 spikes and analyzed using SM 9224 F (Phase 1A) were 110% and 109%, respectively. Recovery results of large-volume water samples (40-L PBS or 100-L DW) spiked with SMI-prepared spikes and analyzed using SM 9224 F (Phase 1B) were <5%. Evaluation of three final concentrations of 0.1 M, 0.25 M, and 0.5 M MgCl2 in the retentates (large volume samples concentrated using UF) analyzed using SM 9224 F resulted in similar recoveries (<5%). To address the poor recoveries, analyses were conducted using EPA Method 1602 (Phase 2). Mean recoveries from 40-L PBS and 100-L DW samples improved to 26% and 29% for EPA Method 1602 as written, respectively. EPA Method 1602 with EasyPhage medium provided mean MS2 recoveries for 40-L PBS and 100-L DW samples of 38% and 48%, respectively (Phase 2).

3.1.2. Results of QC Criteria Development for the Analytical Method (without UF) and the UF Procedure

Laboratory-specific 100-mL PBS and 40-L PBS sample results are provided in

Table 2. Summary of MS2 recoveries for spiked, 100-mL and 40-L PBS samples analyzed using EPA Method 1602 and EasyPhage ¹.

Lab	100-mL PBS Samples (n = 4)				40-L PBS Samples (n = 4) 2
	Mean Recovery (%)	SD 3 (%)	RSD 4 (%)	Range of Recoveries (%)	Mean Recovery (%)	SD 3 (%)	RSD 4 (%)	Range of Recoveries (%)
1	47.1	9.19	19.5	35.6–56.2	36.9	7.47	20.3	27.4–44.5
2	49.1	2.78	5.66	45.3–52.0	53.1	8.92	16.8	43.1–64.4
3 5	59.5	15.0	25.2	47.2–80.3	-	-	-	-
4	78.3	2.97	3.79	74.4–81.0	37.4	6.79	18.2	31.2–46.5
5	45.4	3.99	8.78	39.7–48.8	37.9	3.09	8.14	35.5–42.5
6	96.3	7.09	7.37	87.6–102	38.7	5.66	14.6	32.2–45.1
7 6	44.1	11.5	26.0	34.7–60.3	55.5	3.51	6.33	53.1–58.0
8	54.8	5.92	10.8	48.4–60.3	37.2	2.42	6.50	35.5–38.9
9	96.1	18.3	19.0	84.3–123	28.5	10.5	36.7	22.7–44.1
10 7	-	-	-	-	-	-	-	-
11	51.5	4.12	8.00	48.2–56.7	43.0	6.33	14.7	35.2–50.5
12	102	8.15	8.02	92.5–111	48.3	5.13	10.6	41.4–53.6
13	52.5	7.28	13.9	46.3–62.8	26.2	8.51	32.5	17.9–34.9
Overall	64.7	9.25 8	14.9 9	34.7–123	40.0	6.97 8	19.9 9	17.9–64.4

Notes: ¹ All of the laboratories had UF experience, only two laboratories (3 and 7) had previous experience using EPA Method 1602, and none had experience with EasyPhage; ² Mean recovery includes two PBS samples (n = 2) for Laboratory 8 and three PBS samples (n = 3) for Laboratory 13; ³ Standard deviation; ⁴ Relative standard deviation; ⁵ Lab 3 utilized a laboratory-specific UF protocol; therefore, PBS data was invalidated; ⁶ Lab 7 only processed two 40-L samples; ⁷ Laboratory 10 did not analyze samples due to time constraints; ⁸ Pooled within-laboratory SD was determined by calculating the square root of the mean of the laboratory variances; ⁹ Pooled within-laboratory RSD was determined by calculating the square root of the mean of the squared laboratory RSDs.

; laboratory-specific 100-mL and 100-L DW results are provided in

Table 3. Summary of MS2 recoveries for spiked, 100-mL and 100-L DW samples analyzed using EPA Method 1602 and EasyPhage ¹.

Lab	100-mL DW Samples (n = 4)				100-L DW Samples (n = 4)
	Mean Recovery (%)	SD 2 (%)	RSD 3 (%)	Range of Recoveries (%)	Mean Recovery (%)	SD 2 (%)	RSD 3 (%)	Range of Recoveries (%)
1	45.4	4.87	10.7	41.2–52.5	9.52	3.69	38.8	6.63–14.9
2	27.6	4.32	15.6	21.7–32.1	5.64	2.68	47.5	2.42–8.90
3	58.3	12.2	20.9	47.2–74.6	45.3	6.09	13.4	39.9–54.0
4	77.3	12.4	16.0	71.1–95.9	33.1	13.9	41.9	16.3–46.4
5	50.4	6.00	11.9	43.0–57.0	32.0	10.7	33.3	21.0–44.4
6	95.2	10.8	11.3	85.9–110	49.8	6.83	13.7	46.2–60.0
7 4	58.2	8.48	14.6	51.1–70.3	41.2	3.66	8.89	38.6–43.8
8	55.3	11.5	20.8	39.3–66.7	41.5	7.62	18.4	34.6–51.5
9 5	134	6.80	5.09	125–140	31.9	10.1	31.8	24.7–39.0
10 6	-	-	-	-	-	-	-	-
11	54.5	3.73	6.84	49.1–57.6	38.2	3.92	10.3	33.0–41.4
12	106	7.28	6.86	100–116	17.1	19.5	114	0.00–34.8
13	46.5	3.66	7.87	43.0–50.4	28.0	15.5	55.5	16.6–49.8
Overall	67.4	8.31 7	13.4 8	21.7–140	30.6	10.4 7	47.3 8	0.00–60.0

Notes: ¹ All of the laboratories had UF experience, only two laboratories (3 and 7) had previous experience using EPA Method 1602, and none had experience with EasyPhage; ² Standard deviation; ³ Relative standard deviation; ⁴ Lab 7 only processed two 100-L DW samples; ⁵ Mean recovery includes two DW samples (n = 2) for Laboratory 9; ⁶ Laboratory 10 did not analyze samples due to time constraints; ⁷ Pooled within-laboratory SD was determined by calculating the square root of the mean of the laboratory variances; ⁸ Pooled within-laboratory RSD was determined by calculating the square root of the mean of the squared laboratory RSDs.

. While the overall mean recoveries for the 100-mL PBS and 100-mL DW samples were only 64.7% and 67.4%, respectively; it should be noted that four laboratories (4, 6, 9, and 12) achieved higher MS2 recoveries from 100-mL PBS (78.3%, 96.3%, 96.1%, 102%, respectively) and 100-mL DW (77.3%, 95.2%, 134%, 106%, respectively) samples compared to the other laboratories.

QC acceptance criteria for recovery and precision were developed for EPA Method 1602 (analytical method) with EasyPhage media and MS2 ATCC^® 15597-B1™ (SMI-prepared spikes), using all valid, spiked 100-mL sample results. QC acceptance criteria for recovery and precision using the LRN UF procedure were developed using all valid, spiked 40-L PBS and 100-L DW sample results. The calculated IPR (four, 100-mL spiked PBS samples or four, 40-L spiked PBS samples), OPR (one, 100-mL spiked PBS sample or one, 40-L spiked PBS sample), MS (one, 100-mL spiked DW sample or one, 100-L spiked DW sample), and MS/MSD (two, 100-mL spiked DW samples or two, 100-L spiked DW samples) QC acceptance criteria are provided in

Table 4. Calculated IPR, OPR, MS and MS/MSD criteria for analytical method (100-mL samples) and LRN UF procedure (40-L PBS and 100-L DW samples) based on 95% prediction interval.

Performance Test	Analytical Method	LRN UF Procedure
	Acceptance Criteria	Acceptance Criteria
PBS Samples	100-mL PBS Samples	40-L PBS Samples
IPR (4 PBS samples) Mean Percent Recovery	18%–105%	21%–100%
IPR (4 PBS Samples) Precision 1	26%	32%
OPR 2 (1 PBS sample)	16%–107%	18%–100%
DW Samples	100-mL DW Samples	100-L DW Samples
MS 3 (1 DW sample) Mean Percent Recovery	16%–114%	2%–100%
MS/MSD 4 (2 DW samples) Precision	39%	95%

Notes: ¹ Precision as maximum RSD; ² Ongoing precision and recovery (OPR) as percent recovery; ³ If an MS/MSD pair is run, each individual recovery, rather than the mean of the two recoveries, is compared to the percent recovery criterion; ⁴ Precision as maximum RPD of MS/MSD pair.

.

3.2. Demonstration Exercise MS2 Analyses

Summary Ct values generated during the LOD study prior to analyses of samples from the demonstration exercise are provided in

Table 5. Summary limit of detection (LOD) results.

Concentration	Mean Ct	Range of Ct Values
Undiluted	25	21–32
1:10	28	25–35
1:100	31	28–38
1:1000	35	31–44

. Results are based on data from 11 of the 13 laboratories. The remaining two laboratories were not able to conduct LOD analyses due to time constraints. The LOD analyses indicated that the participating laboratories achieved acceptable results for the RT-PCR procedure.

Summary results from the demonstration exercise for the 13 laboratories are provided in

Table 6. Summary of demonstration exercise results.

Concentration	Mean Ct	Range of Ct Values
Undiluted	31	25–41
1:4	30	27–40
1:10	31	28–35

. Each of the 13 laboratories successfully recovered MS2 from their spiked 40-L DW sample and detected MS2 RNA in replicate RT-PCR assays. Of particular note, 11 laboratories were able to detect MS2 in the undiluted sample and the 1:10 dilution, and all were able to detect MS2 in the 1:4 dilution.

4. Discussion

4.1. Preliminary QC Criteria Analyses: Selection of Spiking Approach and Analytical Method

The preliminary QC criteria study analyses described in Section 2.1.3 were conducted to confirm the spiking approach, and select an analytical method and media for detection of MS2 from small-volume (100-mL PBS and 100-mL DW) samples and large-volume (40-L PBS and 100-L DW) samples concentrated by UF. A number of options were considered for spiking (e.g., BioBalls and SMI-propagated spikes) and analytical methodology (e.g., SM 9224 F, EPA Method 1602 with EasyPhage medium). Based on results from the preliminary analyses presented in Section 3.1.1, subsequent QC criteria development analyses used MS2 spiking suspensions procured from SMI and EPA Method 1602 with EasyPhage medium. In addition to having better recoveries for large volume samples in comparison to SM 9224 F, EPA Method 1602 with EasyPhage medium has several advantages, including a significant reduction in media preparation, elimination of need to temper media in a water bath, and the incorporation of a bacterial stain that provides contrast between the blue MS2 plaques and red bacterial host lawn. It should be noted that subsequent to this study, EasyPhage medium was approved through the EPA Alternate Test Procedure (ATP) program for analysis of drinking water samples under the EPA Groundwater Rule.

4.2. QC Criteria Development Analyses for the Analytical Method (without UF) and the UF Procedure

4.2.1. Matrix Effects with UF

In contrast to 100-mL samples processed by the analytical method without UF, there are indications of matrix interferences for samples concentrated using UF. For example, results for 100-L DW samples exhibited increased variability in comparison to 40-L PBS samples, though this increase may simply be related to the larger sample volume for drinking water. Also, laboratories 1 and 2 reported mean MS2 recoveries of 9.52% and 5.64% from 100-L DW samples, but reported much higher recoveries (36.9% and 53.1%, respectively) from 40-L PBS samples. The lower recoveries and increased variability in 100-L DW samples compared to 40-L PBS may be due to a matrix effect and/or inhibitors concentrated in some retentates.

4.2.2. Laboratory Proficiency

While all of the laboratories had experience with UF, with the exception of laboratories 3 and 7 all others were using EPA Method 1602 for the first time, and none had experience with EasyPhage medium. Although each participating laboratory conducted analyses to become familiar with the UF procedure and EPA Method 1602 using EasyPhage medium prior to analyzing samples to collect data for development of QC criteria, observations indicate that some laboratories may not have been proficient with MS2 analyses.

During analysis of large-volume samples for development of UF QC criteria, laboratories also processed 100-mL PBS samples (OPRs) as QC checks to confirm that they were in control using EPA Method 1602 and EasyPhage medium. When comparing 100-mL PBS sample results generated during these analyses to results generated earlier (for development of analytical method criteria, independent of UF), results for some laboratories improved, an indication that laboratory proficiency was still being enhanced. This observation suggests the need to ensure that analytical method proficiency with EPA Method 1602 and EasyPhage is established prior to implementing MS2 for use as a QC check with the LRN UF procedure.

In addition, four laboratories (4, 6, 9, and 12) achieved higher MS2 recoveries (means ranging from 78.3% to 134%) from 100-mL PBS and 100-mL DW samples compared to the other laboratories. Of particular concern were those laboratories with sample recoveries in excess of 100%. It was suspected that MS2 recoveries may have been affected by seemingly minor nuances in procedural application that could have artificially increased recoveries. As a result, laboratories were asked whether each sample was individually spiked and combined with the bacterial host/medium mixture (individual processing) or if all the samples were prepared at the same time (batched processing). Batched processing would result in longer contact time between the host cells and MS2 prior to plating, which could lead to unexpectedly high recoveries (i.e., MS2 could have replicated prior to media solidification). Laboratories were also asked whether MS2 spike vials were vortexed before spiking, and the length of time between vortexing and spiking. Failure to vortex MS2 spikes according to the study instructions, or an increased length of time between vortexing and spiking, may have contributed to coliphage aggregates forming, or not being properly dispersed, resulting in higher or lower recoveries. While no processing differences were identified through discussions with the participating laboratories, it is possible that some differences in recoveries can be attributed to differences in how laboratories conducted analyses.

4.2.3. Warning Tool Criteria

QC criteria developed for implementation of the EPA microbiological methods are based on 95% prediction intervals (See Table 4). However, because not all the laboratories were familiar with EPA Method 1602 prior to this study and given the results discussed above, it is understood that they may not have been proficient despite practicing prior to conducting QC criteria development study analyses. As a result, the QC criteria that were developed using the 95% prediction interval may not be appropriately stringent. To ensure that laboratories have useful criteria for future implementation purposes, an additional set of QC criteria were developed based on 90% prediction intervals. These additional criteria are provided as a “warning tool”. While not required, it is recommended that laboratories meet the warning tool criteria (90% prediction interval), rather than the QC criteria (95% prediction interval), as failure to do so may indicate lack of proficiency.

The warning tool criteria for recovery and precision were developed for EPA Method 1602 (analytical method) with EasyPhage media and MS2 ATCC^® 15597-B1™ (SMI-prepared spikes), using all valid, spiked 100-mL results. Warning tool criteria for recovery and precision using the LRN UF procedure were developed using all valid, spiked 40-L PBS and 100-L DW results. The calculated IPR (four, 100-mL spiked PBS samples or four 40-L spiked PBS samples), OPR (one, 100-mL spiked PBS sample or one, 40-L spiked PBS sample), MS (one, 100-mL spiked DW sample or one, 100-L spiked DW sample), and MS/MSD (two, 100-mL spiked DW samples or two, 100-L spiked DW samples) warning tool criteria are provided in

Table 7. Warning tool–IPR, OPR, MS and MS/MSD criteria for analytical method (100-mL samples) and LRN UF procedure (40-L PBS and 100-L DW samples) based on 90% prediction interval.

Performance Test	Analytical Method	LRN UF Procedure
	Acceptance Criteria	Acceptance Criteria
PBS Samples	100-mL PBS Samples	40-L PBS Samples
IPR (4 PBS samples) Mean Percent Recovery	26%–100%	25%–100%
IPR (4 PBS samples) Precision 1	23%	28%
OPR 2 (1 PBS sample)	24%–100%	21%–100%
DW Samples	100-mL DW Samples	100-L DW Samples
MS 3 (1 DW sample) Mean Percent Recovery	25%–105%	7%–100%
MS/MSD 4 (2 DW samples) Precision	33%	79%

Notes: ¹ Precision as maximum RSD; ² Ongoing precision and recovery (OPR) as percent recovery; ³ If an MS/MSD pair is run, each individual recovery, rather than the mean of the two recoveries, is compared to the percent recovery criterion; ⁴ Precision as maximum RPD of MS/MSD pair.

.

4.2.4. Criteria Implementation

It should be noted that spiked, 100-mL PBS samples and spiked, 100-mL DW QC samples should always be processed along with large-volume samples concentrated by UF. In the event that a laboratory fails UF QC criteria, the data from the small volume analyses (without UF) can be used to determine whether the problem is associated with the UF concentration procedure or the MS2 analytical method. Implementation of these criteria, especially the warning tool, will allow laboratories to demonstrate proficiency for large-volume sample concentration using MS2. In addition, the criteria for EPA Method 1602 with EasyPhage could also have broad applicability at laboratories that are using MS2 for other types of analyses, such as assessment of water treatment devices, if these laboratories wish to implement use of EasyPhage media.

4.3. Demonstration Exercise

Of the 13 participant laboratories, all successfully recovered and detected MS2 using the LRN WPP followed by RT-PCR. Two laboratories were unable to detect MS2 from undiluted extracts due to inhibition from constituents in the concentrated water samples, but were able to successfully obtain Ct values from the 1:4 and/or 1:10 sample dilutions. These results illustrate the utility of testing dilutions to detect analytes in the presence of PCR inhibitors, especially when targeting RNA using RT-PCR. These data demonstrated successful use of MS2 to confirm proficiency with the LRN WPP procedure, which included concentration with UF and Centricon^® followed by detection using RT-PCR.

For molecular detection of MS2 in water, the theoretical method LOD from this study was ≤100,000 PFU in a 40-L sample. Hill et al. [27] reported a similar RT-PCR method LOD of 20,000 to 500,000 virions for noroviruses recovered from 50-L groundwater samples, using a UF-based procedure similar to LRN’s WPP.

5. Conclusions

Increased laboratory proficiency using the LRN UF protocol will enhance the WLA’s ability to respond to drinking water contamination events. The results of the QC criteria development study and demonstration exercise indicate that MS2 can be used to develop and maintain laboratory proficiency for the UF procedure and the overall LRN WPP for concentration of large-volume water samples. Although the demonstration exercise used an RT-PCR assay for detection of MS2, it is recommended that laboratories use EPA Method 1602 with EasyPhage to demonstrate proficiency with UF, as QC criteria would need to be developed for UF followed by RT-PCR to ensure the criteria are accurate measures of performance. Availability of QC criteria using MS2 allows laboratory analysts to assess proficiency with UF procedures without requiring BSL-3 agents and facilities. In addition, the QC criteria provided for small volume (100-mL) analyses using EPA Method 1602 with EasyPhage has broad applicability across a range of laboratories that are using MS2 for other types of analyses worldwide, such as assessment of water treatment devices, as use of EasyPhage medium will simplify media preparation and plaque enumeration, and allow for smaller laboratories with limited resources to conduct MS2 analyses.