Interlaboratory Performance of a Real-Time PCR Method for Detection of Ceratocystis platani, the Agent of Canker Stain of Platanus spp.

Ceratocystis platani (CP), an ascomycetous fungus, is the agent of canker stain, a lethal vascular disease of Platanus species. Ceratocystis platani has been listed as a quarantine pest (EPPO A2 list) due to extensive damage caused in Southern Europe and the Mediterranean region. As traditional diagnostic assays are ineffective, a Real-Time PCR detection method based on EvaGreen, SYBR Green, and Taqman assays was previously developed, validated in-house, and included in the official EPPO standard PM7/14 (2). Here, we describe the results of a test performance study performed by nine European laboratories for the purpose of an interlaboratory validation. Verification of the DNA extracted from biological samples guaranteed the high quality of preparations, and the stability and the homogeneity of the aliquots intended for the laboratories. All of the laboratories reproduced nearly identical standard curves with efficiencies close to 100%. Testing of blind-coded DNA extracted from wood samples revealed that all performance parameters—diagnostic sensitivity, diagnostic specificity, accuracy and reproducibility—were best fit in most cases both at the laboratory and at the assay level. The previously established limit of detection, 3 fg per PCR reaction, was also validated with similar excellent results. The high interlaboratory performance of this Real-Time PCR method confirms its value as a primary tool to safeguard C. platani-free countries by way of an accurate monitoring, and to investigate the resistance level of potentially canker stain-resistant Platanus genotypes.


Introduction
Canker stain is a destructive disease of the Platanus species, caused by the ascomycetous fungus Ceratocystis platani (J. M. Walter) Engelbr. and T. C. Harr., a host-specialized species within the Latin American Clade (LAC) of the genus Ceratocystis sensu stricto [1][2][3]. The pathogen colonizes the roots and the stem of the trees by entering through the wounds or exploiting root anastomosis between diseased and healthy trees, and it causes wood discoloration and vessel dysfunction as well as necrosis of the cambium and living bark tissues thus resulting in the death of the trees [4,5]. Spread of Ceratocystis platani to diseasefree areas can occur in a variety of ways. Foot and vehicle traffic, birds, rodents, and insects can transport either C. platani-infected sawdust from pruning of infected trees or fungal propagules that first form in the cracks between necrotized bark and wood and subsequently wash down to the base of the tree. Fungal inoculum from infected trees growing on river banks can be transported by the water flow, infecting healthy trees on the banks downstream [6]. Wind can also contribute to spread fungal propagules making possible new infection events if fresh pruning wounds are encountered [7]. Finally, Platypus cylindrus, a xylofagous frass-producing beetle, has been shown to be a vector of C. platani [8].
Native to the southeastern USA, this fungus has spread to several countries in Southern Europe and the Mediterranean basin where it is destroying urban and natural stands of highly susceptible Platanus species, P. × acerifolia (Ait.) Willd. and P. orientalis L. [3,5,9,10].
Due to the damage caused, C. platani was included by the EU in the list of quarantine pests included in the Annex II part B (Pests known to occur in the Union territory) of the COMMISSION IMPLEMENTING REGULATION (EU) 2019/2072 [11] and in the EPPO A2 list of quarantine organisms [12][13][14]. The A2 list contains by definition pests that are not widely distributed in the EPPO region and in fact the pathogen seems to be absent in northern Europe, and its presence in the African Mediterranean countries is only suspected. However, the pathogen could easily spread to Northern European countries, especially in a warming European climate.
In Italy, where C. platani is widespread, a ministerial decree established mandatory monitoring, prophylaxis, and eradication measures to safeguard plane trees [15,16].
The core of the EPPO mission is to protect plant health in agriculture, forestry, and the uncultivated environment through the development of international strategies and standards aimed at impeding the introduction and the spread of dangerous pests. Development and standardization of reliable diagnostic methods is essential to protect plant health. In this context, robust validation data is a requirement before any diagnostic method can be included in the official diagnostic EPPO protocols. Since 2014, EPPO hosts Euphresco (EUropean PHytosanitary RESearch COordination), a network of organizations (e.g., National Plant Protection Organizations (NPPO) and research institutes) that fund and coordinate research in plant health. Thus, Euphresco strongly encourages test performance studies (TPS) for interlaboratory validation of diagnostic protocols.
Traditional detection of C. platani has relied on a number of assays-microscopic observations, isolation on nutritive media, moist chamber, carrot test, and baiting testwhich are time-consuming and prone to false negative response [14,17,18]. Recently Real-Time PCR based methods for C. platani detection were developed [7,17], representing the first rapid and reliable tools for the protection of Platanus spp.
Luchi et al. [7] designed a Real-Time PCR method specifically for monitoring and quantifying aerial inoculum of the pathogen. This method can be used to monitor pathogen levels during sanitation cuttings and to investigate airborne transmission.
Pilotti et al. [17] developed a Real-Time PCR method for in-wood detection, which is appropriate for diagnostic purposes as well as investigation of host resistance. This method includes three ITS1-targeted assays based on EvaGreen and SYBR Green dyes and a Taqman probe. These assays, validated in depth by in-house comparisons, showed highlevel performance on several parameters: amplification efficiency, analytical specificity (inclusivity and exclusivity), diagnostic sensitivity and specificity, analytical sensitivity, enforceability in necrotic wood, and inter-operator, intra-laboratory reproducibility [17,18].
In this study, we report in detail the results of a TPS performed by nine European laboratories which provides further validation of the method by Pilotti et al. [17]. The EPPO guidelines for validation of diagnostic methods were used to design the TPS and calculate the performance criteria [19][20][21]. Overall, the TPS confirmed (nearly) best fit performance parameters for this method across laboratories. This shows that it is a reliable tool (i) to safeguard C. platani-free countries and zones by way of an accurate monitoring and (ii) to characterize resistance levels of potentially canker stain-resistant Platanus genotypes.

Wood Sample Collection and Assessment of Their Infectious Status
Wood samples were collected in 2017 from 15 plane trees. Trees were selected according to four different ranks (Table 1): (i) healthy trees (n = 3)-H; (ii) trees naturally infected with C. platani (n = 3)-NI; (iii) trees artificially infected with C. platani that were dead from a variable number of months, (n = 6)-AI [ these potted trees were inoculated at age four in April 2016 with tooth pick technique [4] and were killed by the disease in the same year]; and (iv) trees naturally affected by canker diseases other than canker stain (n = 3)-D. With regard to the latter rank two trees were affected by the typical cankers associated with Fomitiporia mediterranea (M. Fischer) and showed the typical fruit body of this hymenomycete [22] (referred to as D.1 and D.2 in Table 1). The third tree (D3) was affected by pruning wound-associated black/carbonaceous perennial cankers. All sampled trees were located in Rome, except for two NI trees located in Pisa. All trees labelled as NI and D were located in urban avenues and suburban roads. They were all still viable though declining at the time of sampling. In all diseased trees, discolored wood was collected from the reaction zone.
For comparison purposes, all wood samples were tested using the following diagnostic non-molecular assays to assess the presence/absence of C. platani: (i) microscope assay, (ii) isolation on nutritive medium, (iii) carrot assay, (iv) moist chamber, and (v) bait-plant assay. The specific nature of perithecia and mycelium obtained in the various assays was ascertained by microscopic observations. This enabled the identification of C. platani conidia and ascospores for morphological diagnosis [4,14]. Details on microscope and biological diagnostic assays can be found in Supplementary Text S1.

DNA Extraction
Fungal mycelium for DNA extraction was obtained by scraping the mycelium of a C. platani isolate (C.P. 32) with a sterile pipette tip from actively growing colonies cultured on PDA (Potato dextrose agar, Oxoid-Unipath Ltd., Basingstoke, Hampshire, England). The wood and mycelium extractions started from aliquots of 700 mg. Fungal mycelium and wood tissue aliquots were ground to powder with liquid nitrogen using a sterile pestle and mortar, and DNA was extracted using a DNeasy Plant Maxi Kit (Qiagen, Hilden, Germany), following the manufacturer's instructions. After the final elution, 800 µL flow-through was obtained. For samples under 800 µL (anyway not less than 790 µL), additional flow-through obtained from the second elution step was added to make up the difference in volume. DNA was quantified using a Thermo Scientific NanoDrop™ 1000 Spectrophotometer.
The focus of this interlaboratory comparison was the assessment of reproducibility and performance parameters of the Real-Time PCR method based on EvaGreen, Taqman, and SYBR Green variants [14,17,18].
Each PL chose to perform between two options: (i) one assay type (Taqman or Eva-Green or SYBR Green) with a Bio-Rad master mix and another commercial master mix specifically suited for Real-Time PCR; and (ii) two assay types using Bio-Rad master mixes. Bio-Rad master mixes were SsoFast™ EvaGreen ® Supermix (for EvaGreen assay), SsoAdvanced™ Universal Probes Supermix (Taqman assay), and SsoAdvanced™ Universal SYBR ® Green Supermix (SYBR Green assay). Master mixes by other companies were qPCR MasterMix and qPCR MasterMix No ROX (Eurogentec, Seraing, Belgium), Maxima Probe/ROX qPCR Master Mix and Maxima SYBR Green/ROX qPCR Master Mix (Thermo Fisher Scientific, Waltham, MA, USA), and Power SYBR Green PCR Master Mix (Applied Biosystem, Thermo Fisher Scientific).
In all the assays the concentration of the primers and, when used, the probe was kept fixed at 0.5 µM and 0.3 µM for each primer and probe, respectively. Use of Bio-Rad master mixes and the CFX96 TM thermocycler implied the integral application of the cycling protocol [14,17,18]: iinitial denaturation at 96 • C for 3 min; 40 cycles at 95 • C for 10 s (denaturation) and 66 • C for 20 s (annealing/extension); final extension at 72 • C for 5 min (the latter step was used exclusively for EvaGreen and SYBR Green assays). Slight changes to the amplification program were made according to the operating instructions of the commercial master mix used and to the thermocycler type. Specifically, the PL that choose master mixes other than Bio-Rad performed a preliminary test aimed at comparing different annealing/extension times-20 s (the canonical value) vs. 30, 40, and 60 s-using PAC and NAC (positive and negative amplification control). PAC was tested both undiluted and at a 1:10.000 dilution.
See Supplementary Text/Table S1 for details of the Real-Time PCR procedures, i.e., primers and probe sequences, composition of master mixes, reaction assembly, and thermal cycling conditions.
A cycle threshold (Ct) of ≤37 was suggested by the OL and was applied in the TPS as the upper limit for a positive detection. In fact, it was previously established that 3 fg of C. platani gDNA was the lowest detectable DNA dose per PCR reaction (Limit of Detection, LoD), in the approximate Ct range of 33-36, depending on the assay [18].
The fluorophores used were EvaGreen, SYBR Green I, 6-FAM (6-carboxy-fluorescein). The Real-Time PCR systems used were CFX96 TM (Bio-Rad), StepOne Plus TM (Applied Biosystem, Thermo Fisher Scientific), Rotor-Gene TM 6000 (Corbett, Life Science), Lightcycler 480 (Roche), and Applied Biosystems ® 7500 (Applied Biosystem, Thermo Fisher Scientific). In all Real-Time PCR systems, the threshold line was set in an auto-calculating modality. Details on fluorophores and Real-Time PCR systems are given in Text S2.

Diagnostic Confirmation of DNA-Stock-Samples (Dss), Aliquoting, Homogeneity Testing, and the Shipping Material
Dss were tested by the OL in triplicate with EvaGreen and Taqman assays (Bio-Rad master mixes) in order to confirm diagnosis performed with non-molecular assays.
The root square of the Dss ( √ 16= 4) was tested for repeatability with 11 replicates in the EvaGreen assay. The Dss tested for repeatability were NI.3 and AI.6 (C. platani-positive) and D.1 and H.3 (C. platani-negative).
The 16 Dss were then aliquoted by distributing a volume, for each micro tube and for each participant, that was large enough to perform the homogeneity test by the OL and all the activities planned for the PL (Table 2 and Supplementary Table S2). The quantity was adjusted to cope with failures of each activity due to common human errors. These were named DNA-aliquot-samples (Das) and they represented the material of the sensu strictu TPS, based on blind samples. The gDNA from an axenic culture of C. platani for use in the analytical sensitivity test (also supplied as a blind Das) Some Das were tested in triplicate with the EvaGreen assay to ascertain the homogeneity of Das among themselves and to compare them to the Dss from which they had been derived. A number of Das higher than the root square of their total number was tested ( Table 2). We established that homogeneity was basically achieved if Dss and their relative Das gave the same qualitative diagnostic response-positive or negative. It was considered to be fully achieved if they also gave fluorescent signals at the same or very close Ct.
All material-DNA aliquots, Bio-Rad master mixes, oligonucleotides, and the probewas dispatched in polystyrene boxes of 15.4 cc volume filled with pellets of dry ice. The Real-Time PCR master mixes from companies other than Bio-Rad were also used by the PL.

Stability Test
A stability test was performed by the PL on PAC and NAC in order to test the stability of both DNA samples and reagents after shipping. Each control was tested in triplicate with the Bio-Rad master mixes that had been shipped together with the PAC and the NAC. In addition, the OL tested all Dss 6 months after the completion of the TPS. The full compliance of the stability test was based on the same quali-quantitative criterion ruling the homogeneity test.

The Test Performance Study with Real-Time PCR: (i) Generating Standard Curves, (ii) Testing Blind-Coded DNA-Aliquot-Samples (iii) Testing Analytical Sensitivity
The TPS was executed by the PL and the OL. In order to assess the reproducibility of the amplification efficiency of the Real-Time PCR on wood extracts, the PL and the OL generated standard curves with Da.St.Cu. In addition to the undiluted DNA extract (1:1), six five-fold serial dilutions were used (1:5, 1:25, 1:125, 1:625, 1:3.125, 1:15.625) and each was tested in triplicate using 2 µL per PCR reaction. PL performed one or two real-time PCR assays chosen among Taqman, EvaGreen, or SYBR Green versions, and using Bio-Rad and/or other commercial master mixes. The OL performed all three assays.
Blind Das were tested in triplicate. Taqman assay was primarily used to test the analytical sensitivity but EvaGreen and SYBR Green assays were also used by some laboratories. The OL instructed PL to dilute the aliquot sample containing C. platani gDNA to obtain 7.5 and 1.5 fg per µL in the last two dilutions. Two microlitres for each dilution were then used to test the capacity of the method to detect 15 and 3 fg C. platani gDNA per PCR reaction. Each DNA quantity was tested in six replicates.
The complete list of the codes of the PL and the OL with the relative Real-Time PCR assay, master mix, and Real-Time PCR apparatus used is reported in Table 3, where each letter identifies a PL and the number the type of test performed. A chronological listing of scheduled TPS actions can be found in Supplementary Table S2. Table 3. Master mix/Real-Time PCR system combinations adopted in the test performance study (TPS) by the organizing laboratory (OL = Z) and the participating laboratories (PL, the other different letters). All combinations were used to obtain standard curves (except A2 and G in Taqman) and to detect the blind samples (except A3 and A4). Those tested for analytical sensitivity are marked with red asterisks. Numbering for each letter, when present, indicates that different master mix/Real-Time PCR system combinations were used by a PL with the same assay/chemistry. The names of Bio-Rad master mixes have been shortened, see the text for the full names.

Performance Criteria, Nomenclature, and Statistical Analysis
The Real-Time PCR assays were evaluated for their capacity to produce accurate and reproducible results for the detection of C. platani. We thus inferred the following performance parameters from TPS qualitative results: accuracy (AC), diagnostic sensitivity (DSE), diagnostic specificity (DSP), repeatability ('accordance') (DA) and reproducibility ('concordance') (CO) ( Table 4). AC is the closeness of agreement between detection results of the laboratories and the assigned values for the samples (i.e., their true sample status, infectious and healthy). AC is based on DSE and DSP. DSE is a measure of the ability of the method to detect the target in samples in which the target is present, i.e., those with a positive assigned value (N + ), whereas DSP is the capacity of the method to fail the detection in samples in which the target is absent, i.e., those with a negative assigned value (N − ). DA is the level of agreement between replicates of a sample tested under the same conditions; in this work, we calculated DA on results from testing Dss with 11 technical replicates (performed by OL) and the C.P. 32 gDNA with six technical replicates (analytical sensitivity). CO is the ability of the method to provide consistent results when applied to aliquots of the same sample tested under different conditions (i.e., by different laboratories). All definitions and calculations were taken from ISO 16,140 [23], OEPP/EPPO standards [19][20][21], Chabirand et al. [24,25], and Langton et al. [26]. In this work the positive and the negative detections obtained for samples for which the assigned value is, respectively, positive (infected) and negative (healthy) are referred to as true positives (TP) and true negatives (TN). False positives (FP) are results obtained for samples where a negative result is expected (given the true sample status); similarly, false negatives (FN) are results obtained for samples where a positive result is expected.
Calculations used are listed in Table 4. The qualitative input data were the positive and the negative detections derived from each technical replicate from testing the DA of Dss, the stability of the shipped material and the analytical sensitivity. For blind Das, the qualitative result was considered both at (i) the technical replicate level; and (ii) the Das level, i.e., positivity and negativity was attributed to a Das by coherent results obtained from at least two technical replicates (out of three).
The dataset is a binomial distribution, thus the binconf function of the Hmisc package of R [27] statistical software was applied to search for confidence intervals (95%) for AC, DSE, and DSP criteria, using the Wilson score method with no continuity correction [28]. Tests on the equality of AC, DSE, and DSP between laboratories, and the chemistry-based assays were performed using a one-sample t-test.
Standard curves were compared according to the following procedure: the average value at each dilution point was subtracted from the corresponding values of a theoretical curve sharing the template values in the x axis (expressed in base-10 logarithm) with experimental curves, but with an exact best fit efficiency (100%) (i.e., with a regression equation with 3.32 as the angular coefficient). We thus obtained seven normalized replication values for each curve, which were used in the statistical analysis to compare the various curves and to give a measure of the significance among them. Fisher's test for the analysis of variance (ANOVA) was developed on the dataset obtained after verifying the normality of the distribution with the Shapiro-Wilk test and the homogeneity of the variances with the Bartlett test.

Outliers
Data sets were considered outliers and were excluded from analysis when: (a) results of controls were non-concordant; (b) accuracy was statistically different from the average of accuracy obtained by all laboratories (n. FN or FP > average FN or FP ± 3 SD); (c) results of one test were incomplete (e.g., no technical repetition reported); (d) the number of undetermined results was significantly different from the other laboratories (n. undetermined/inconclusive > average undetermined ± 3 SD) [29].

Disclosure Policy by the Organiser
After contacting PL, OL distributed the 'TPS plan' describing the project. After acceptance by PL, OL also sent the 'operational instructions' on how to perform all the tests.
The 16 blind samples (Das) were labelled with numbers from 1 to 16 followed by the letter representing each PL. Numbering of Das was randomly attributed and was different among all PL. Only OL kept the correspondence between the Das of each PL and the Dss.
To maintain the confidentiality of the link between each PL and its own results, we use the letter assigned to the PL ('A-H') in this paper as well. The results of the OL ('Z') are made public.

Sample Preparation
The OL collected the wood samples for the TPS from 15 plane trees and verified their status (infected/healthy). Diagnosis performed with non-molecular assays confirmed the presence of C. platani in those samples collected from trees affected by canker stainlike symptoms and from trees artificially infected with C. platani. The pathogen was not detected in samples of healthy-looking non-inoculated trees, and from trees of urban avenues affected by disease other than canker stain (Table 1). For details on the outcome of non-molecular assays, see Supplementary Table S3.
Testing of the DNA-stock-samples (Dss), which had been extracted from the above reported trees with Real-Time PCR, confirmed the diagnosis performed with non-molecular assays. Melting peaks were detected in all C. platani-infected Dss, with a melting temperature of 81-81.5 • C as expected [18]. No peaks were detected in the C. platani uninfected Dss, confirming that the corresponding wood samples/trees were free of C. platani. The absence of peaks with a melting temperature other than that of C. platani target amplicon also confirmed the absence of primer dimers or other aspecific amplification products ( Figure 1). The coherence between traditional and molecular diagnosis thus verified the real infectious/healthy status of the TPS samples. Testing of the DNA-stock-samples (Dss), which had been extracted from the above reported trees with Real-Time PCR, confirmed the diagnosis performed with non-molecular assays. Melting peaks were detected in all C. platani-infected Dss, with a melting temperature of 81-81.5 °C as expected [18]. No peaks were detected in the C. platani uninfected Dss, confirming that the corresponding wood samples/trees were free of C. platani. The absence of peaks with a melting temperature other than that of C. platani target amplicon also confirmed the absence of primer dimers or other aspecific amplification products ( Figure 1). The coherence between traditional and molecular diagnosis thus verified the real infectious/healthy status of the TPS samples. Repeatability (DA) of detection was also fully confirmed with 11 replicates in four selected Dss (out of 16) that reached 1, i.e., a perfect fit. Figure 2a illustrates the comparison between first Real-Time PCR detections on Dss (three replicates) and detections for DA. Samples not infected with C. platani yielded no fluorescent signals. Repeatability (DA) of detection was also fully confirmed with 11 replicates in four selected Dss (out of 16) that reached 1, i.e., a perfect fit. Figure 2a illustrates the comparison between first Real-Time PCR detections on Dss (three replicates) and detections for DA. Samples not infected with C. platani yielded no fluorescent signals. Dss were thus used to make Das intended for the PL. Homogeneity was tested in selected Das (Table 2) and was fully confirmed comparing with the quali-quantitative response of the Dss and obtaining highly similar Ct values (Figure 2b). The ranges of variation were all between 0.08 and 0.29 Ct.
The stability test was performed by the PL to assess that material received by post from the OL remained unaltered after shipping. All tests gave the correct qualitative response, with full DA and similar Ct values in the PAC, i.e., with a range of variation among the average values from the different PL of 1.52 Ct in the EvaGreen assay and of 5.3 Ct in the Taqman assay (Figure 2c,d). Testing all Dss in EvaGreen after the completion of the TPS again confirmed their stability as average detections for each Dss varied between 0.6-1.5 Ct in comparison with those performed at the beginning of the preparatory phase.

Thermal Cycling Adjustments
The Real-Time PCR method was developed using Bio-Rad master mixes [17]. Use of different master mixes implied that slight changes to the amplification program were made according to the product instructions and a preliminary test aimed at identifying with EvaGreen assay on the root square of DNA stock samples (Dss) (n = 4); DA of the two C. plataniinfected Dss is shown (the C. platani-uninfected Dss-D.1 and H.3-consistently gave no signal, data not shown); black columns represent the Ct mean of 11 technical replicates and are compared with the Ct mean of three technical replicates (grey columns), which were obtained from the same Dss in the initial detection phase. (b) Homogeneity test (performed by OL): here we compare results of the EvaGreen assay from the initial detection phase on Dss (in triplicate, first column of each group) and from detection on the DNA-aliquot-samples (Das) (in triplicate) derived from Dss and intended for PL; nine Das from C. platani-uninfected samples were also tested and gave no signal, as expected (data not shown) (c,d) Stability test: results of testing a positive amplification control (PAC = NI.3) by the OL and the participating laboratories (PL) with Bio-Rad EvaGreen and Taqman assays, after they received from the OL the material, which included also the Bio-Rad Real-Time PCR master mixes. This test also included a negative amplification control (NAC = H.1), which was always negative (data not shown). The bars indicate the standard deviation of the mean.
Dss were thus used to make Das intended for the PL. Homogeneity was tested in selected Das (Table 2) and was fully confirmed comparing with the quali-quantitative response of the Dss and obtaining highly similar Ct values (Figure 2b). The ranges of variation were all between 0.08 and 0.29 Ct.
The stability test was performed by the PL to assess that material received by post from the OL remained unaltered after shipping. All tests gave the correct qualitative response, with full DA and similar Ct values in the PAC, i.e., with a range of variation among the average values from the different PL of 1.52 Ct in the EvaGreen assay and of 5.3 Ct in the Taqman assay (Figure 2c,d). Testing all Dss in EvaGreen after the completion of the TPS again confirmed their stability as average detections for each Dss varied between 0.6-1.5 Ct in comparison with those performed at the beginning of the preparatory phase.

Thermal Cycling Adjustments
The Real-Time PCR method was developed using Bio-Rad master mixes [17]. Use of different master mixes implied that slight changes to the amplification program were made according to the product instructions and a preliminary test aimed at identifying the optimal annealing/extension times. In these tests, 60 s was chosen for Eurogentec qPCR MasterMix and qPCR MasterMix No ROX to perform subsequent experiments, 30 s for Thermo Scientific Maxima Probe/ROX qPCR Master Mix and Maxima SYBR Green/ROX qPCR Master Mix, and with Applied Biosystems Power SYBR Green PCR Master Mix.
A full description of the Real-Time PCR method for Ceratocystis platani detection used in this TPS by the OL and the PL are contained in Supplementary Text/Table S1.

Test Performance Study: Generating Standard Curves
To validate reproducibility of Real-Time PCR efficiency, the OL and six PL generated a total of 20 standard curves using a wood extract DNA and different assays, master mixes and Real-Time PCR systems (Table 3). In all standard curves the most dilute replicate sample was detected. In general, efficiency and R 2 were close to the best fit except for curves obtained with Taqman and EvaGreen assays in the Lightcycler 480 system, whose efficiency was 79.7 and 74.3, respectively. As this was the only case of low performance among all standard curves obtained, we considered these data as outliers and did not include the two curves obtained with a Lightcycler 480 system (D) in further comparisons among the curves. Table 5 lists the performance parameters for each standard curve. Table 5. Performance parameters of Taqman, EvaGreen, and SYBR Green assays evaluated through the inference of standard curves by using different master mix/Real-Time PCR system combinations (which is identified by the different alpha-numerical codes; see legend of Table 3). The curves were generated using 7 5-fold serial dilutions (from 1:1 to 1:15.625) of plane tree gDNA obtained from wood naturally infected with Ceratocystis platani. Specifically, we evidenced the interlaboratory reproducibility by comparing all standard curves obtained with CFX96 TM and Bio-Rad master mixes both with Taqman and EvaGreen assays (Figure 3a,c, Table 5). Regarding the Taqman assay (five curves by five laboratories) efficiency and intercept values varied within the ranges 94.1-99.9 and 33.7-34.6 (i.e., a variation of 0.9 Ct), respectively. The same parameters of curves obtained with Eva-Green (four curves by four laboratories) varied within the range 98.3-100.7 and 33.5-34.0 (i.e., a variation of 0.5 Ct), respectively. standard curves obtained with CFX96 TM and Bio-Rad master mixes both with Taqman and EvaGreen assays (Figure 3a,c, Table 5). Regarding the Taqman assay (five curves by five laboratories) efficiency and intercept values varied within the ranges 94.1-99.9 and 33.7-34.6 (i.e., a variation of 0.9 Ct), respectively. The same parameters of curves obtained with EvaGreen (four curves by four laboratories) varied within the range 98.3-100.7 and 33.5-34.0 (i.e., a variation of 0.5 Ct), respectively.  (a,c) curves performed by different PL with the same assay, master mix, and Real-Time PCR system show full interlaboratory reproducibility as quite close/overlapping (no significant differences, p > 0.05); (b) curves obtained in a Taqman assay with different commercial master mixes and different Real-Time PCR systems show that the latter factor always determines significant differences (p < 0.05); (d) curves obtained in SYBR green assay. A distinction according to the Real-Time PCR system is also evident but not significant. The number following the letter representing the PL indicates the master mix/Real-Time PCR system adopted (see Table 3).

Laboratory
In the Taqman assay, we also compared all curves performed with any commercial master mix on the three different Real-Time PCR systems used. These comparisons showed that all curves performed with the same Real-Time PCR system, although with different master mixes, were quite close/overlapping. On the other hand, the three different Real-Time PCR systems determined three discrete parallel bundles of curves. Curves obtained with Rotor-Gene TM 6000 were the lowest and those obtained with CFX96 TM and StepOne Plus TM were the highest and intermediate, respectively (Figure 3b). Statistical analysis revealed a significant difference among the Taqman curves obtained with the different Real-Time PCR systems (p < 0.05) (Figure 3b).
Four experiments were performed in SYBR Green assay, which used two different master mixes on CFX96 TM and two on Applied Biosystems ® 7500 (Table 3). Even these curves overlapped based on the thermocycler system, largely irrespective of the master mix. Curves obtained with CFX96 TM were the lowest (Figure 3d). In this case, the curves did not show statistically significant differences.
The box plots in Figure 4 show the variability of the data series; the distance between the boxes highlights the differences. The box plots in Figure 4 show the variability of the data series; the distance between the boxes highlights the differences.  Table 3.

Test Performance Study: Blind Testing of DNA-Aliquot-Samples
All the participants laboratories were able to submit the results for a total of 20 data sets and 320 results from blind test items. Specifically, in all three assays-EvaGreen, SYBR Green, and Taqman-all Das which were N + were always correctly detected, at both the technical replicate and the Das level, namely only TP and no FN were obtained. Thus, DSE was 100% at the laboratory and the assay level. A quantitative view of fluorescent detection signals (Ct) obtained with positive Das is depicted in Appendix A, Figure A1. The Taqman assay performed by PL C with the Rotor-Gene TM 6000 thermocycler clearly revealed that this system ensured the lowest Ct while keeping the highest DSP.
Similarly, all detections performed by the laboratories on N − Das showed best fit except in the following cases: (i) six FP technical replications referring to two Das of the same experiment performed with SYBR Green assay; and (ii) two FP technical replications referring to two Das of the same experiment and performed with a Taqman assay (Table 6). In the latter case, the FP fluorescent signals were just below the threshold of Ct 37, which . Box plot of the Ct values of the standard curves carried out in the various laboratories and normalized to a theoretical best fitting curve. Curves were obtained with Taqman, EvaGreen, and SYBR Green assays. The organizing laboratory (OL = Z), participating laboratories (PL = A, B, C, E, F, G, H). For the combination laboratory + master mix + Real-Time PCR system, see Table 3.

Test Performance Study: Blind Testing of DNA-Aliquot-Samples
All the participants laboratories were able to submit the results for a total of 20 data sets and 320 results from blind test items. Specifically, in all three assays-EvaGreen, SYBR Green, and Taqman-all Das which were N + were always correctly detected, at both the technical replicate and the Das level, namely only TP and no FN were obtained. Thus, DSE was 100% at the laboratory and the assay level. A quantitative view of fluorescent detection signals (Ct) obtained with positive Das is depicted in Appendix A, Figure A1. The Taqman assay performed by PL C with the Rotor-Gene TM 6000 thermocycler clearly revealed that this system ensured the lowest Ct while keeping the highest DSP.
Similarly, all detections performed by the laboratories on N − Das showed best fit except in the following cases: (i) six FP technical replications referring to two Das of the same experiment performed with SYBR Green assay; and (ii) two FP technical replications referring to two Das of the same experiment and performed with a Taqman assay (Table 6). In the latter case, the FP fluorescent signals were just below the threshold of Ct 37, which suggests a very slight contamination. Note that such FPs do not condition the Taqman performance parameters at the level of Das (being one FP out of three technical replicates for each Das). Table 6. Performance parameters-diagnostic sensitivity (DSE), diagnostic specificity (DSP) accuracy (AC) and reproducibility (CO)-of Taqman (TM), EvaGreen (EG), and SYBR Green (SG) assays inferred for each laboratory/master mix/Real-Time PCR system combination and globally for each assay. For the legend of the combinations laboratory/master mix/Real-Time-PCR system (represented by a single letter or an alphanumeric code following the chemistry acronym), see Table 3. Values of performance parameters are those calculated based on the results at the level of technical replicates, i.e., the more stringent criterion. Confidence intervals for the laboratory combinations and the assays were 96.3-100, wherever performance values showed the best fit, i.e., = 100. Where values were not best fitting confidence intervals are reported in brackets (grey-shaded). * Statistically different value (p-value < 0.001).

Performance
In summary we obtained 2 FP out of 198 technical replicate-detections of Das which were N − in Taqman, no FP out of 90 N − in EvaGreen, and 6 FP out of 72 N − in SYBR Green. No FN were obtained out of 330, 150, and 120 technical replicate-detections of Das, which were N + in Taqman, EvaGreen, and SYBR Green, respectively. Accordingly, values of AC and CO globally referred to each assay were either high or best fit. Table 6 shows the value and the confidence intervals of the performance parameters. Presented values of performance parameters are those calculated based on the results at the level of technical replicates, i.e., the more stringent criterion. Supplementary Spreadsheet S1 reports the qualitative results detailed at the level of both technical replicate and Das, as well as the calculations made to determine the performance parameters.

Test Performance Study: Testing Analytical Sensitivity
Testing analytical sensitivity in EvaGreen and SYBR Green assays always yielded the best fit of performance parameters with both 15 and 3 fg (Table 7). In the Taqman assay, performance was at the highest level in 6 PL out of 8 with 15 fg, and in 4 PL out of 8 with 3 fg (Table 7). Specifically, (i) two FN replicates (referring to two tests) out of 48 N + replicates were detected with 15 fg; and (ii) six FN replicates (referring to four tests) out of 48 N + replicates were detected with 3 fg. However, all FN gave a fluorescent signal with Ct values in the range 37.07-37.74 (slightly above the established threshold for a positive detection).
In conclusion, reproducibility (CO) of analytical sensitivity was 100% in EvaGreen and SYBR Green assays and 92.9 and 76.2% in Taqman assays with 15 and 3 fg, respectively ( Table 7, Supplementary Spreadsheet S1).
The confidence intervals are reported in Table 7 (the different values are verified for statistical significance for p-value < 0.001). Figure 5 shows a quantitative view of the fluorescent detection signals obtained (Ct). Table 7. Performance parameters of analytical sensitivity-diagnostic sensitivity (DSE), repeatability (DA) accuracy (AC), and reproducibility (CO)-of Taqman (TM), EvaGreen (EG), and SYBR Green (SG) inferred for each laboratory/master mix/Real-Time-PCR system and globally for each assay. For the combinations of laboratory/master mix/real-time PCR system (represented by a single letter or an alphanumeric code following the assay acronym), see Table 3. Confidence intervals for the laboratory combinations and the assays were 96.3-100 or 0.96-1 wherever performance values show the best fit, i.e., = 100 or 1. When performance values do not show a best fit, the confidence intervals are reported in bracket (grey-shaded).  Figure 5. Analytical sensitivity tested in Taqman (TM), EvaGreen (EG), and SYBR Green (SG) by the organizing laboratory (OL = Z) and the participating laboratories (PL), represented by case single letters or alphanumeric codes to represent the adopted combination master mi Time PCR system (see Table 3). The black columns represent the mean Ct values at which the detectable pathogen DNA quantity i.e., 3 fg gDNA of Ceratocystis platani, was detected, as prev reported by Lumia et al. [18]. Grey columns are the average detection Ct of a quantity of DN times larger, 15 fg. The bars are the standard deviation of six replicate-based means. The num red asterisks above each column represent the number of false negatives (FN) obtained in e periment.

Performance
In this analysis, the experiment performed by D was not included as the defau tings of the Real-Time PCR system they used-Lightcycler 480 (Roche)-does not p Ct values over 35. This caused problems because the detection of 3 fg frequentl around this value in several PL and in the OL.

Discussion
This TPS, performed exclusively with laboratories proficient in Real-Time PCR firms the excellent performance of our intra-laboratory validation of the method [ The TPS was articulated in five phases: (i) preparation, (ii) the validation of the amp tion efficiency through the inference of standard curves, (iii) blind detection of sam (iv) validation of the analytical sensitivity, and (v) assessment of all validation param All verifications performed in the preparatory phase guaranteed that all materi compliant, i.e., (i) the infectious state of all samples had been correctly determined tegrating the results of different methods, (ii) the Dss were amplifiable and amplifi results were fully repeatable, (iii) Das intended for the PL were homogeneous, and dispatched material, samples, and reagents remained unaltered after delivery. Final annealing/extension time of the cycling protocol was also successfully adjusted for m mixes other than Bio-Rad.
All three tested assays-EvaGreen, Taqman, and SYBR Green-resulted in (n best fit data. Overall, the method has proven not only reproducible but also robusttransferable-that is to say that the performance did not substantially change when different master mixes and Real-Time PCR platforms, which is not a foregone conc when testing a method in collaborative studies [30].  Taqman (TM), EvaGreen (EG), and SYBR Green (SG) assays by the organizing laboratory (OL = Z) and the participating laboratories (PL), represented by upper case single letters or alphanumeric codes to represent the adopted combination master mix/Real-Time PCR system (see Table 3). The black columns represent the mean Ct values at which the lowest detectable pathogen DNA quantity i.e., 3 fg gDNA of Ceratocystis platani, was detected, as previously reported by Lumia et al. [18]. Grey columns are the average detection Ct of a quantity of DNA five times larger, 15 fg. The bars are the standard deviation of six replicate-based means. The number of red asterisks above each column represent the number of false negatives (FN) obtained in each experiment.

Discussion
This TPS, performed exclusively with laboratories proficient in Real-Time PCR, confirms the excellent performance of our intra-laboratory validation of the method [17,18]. The TPS was articulated in five phases: (i) preparation, (ii) the validation of the amplification efficiency through the inference of standard curves, (iii) blind detection of samples, (iv) validation of the analytical sensitivity, and (v) assessment of all validation parameters.
All verifications performed in the preparatory phase guaranteed that all material was compliant, i.e., (i) the infectious state of all samples had been correctly determined by integrating the results of different methods, (ii) the Dss were amplifiable and amplification results were fully repeatable, (iii) Das intended for the PL were homogeneous, and (iv) all dispatched material, samples, and reagents remained unaltered after delivery. Finally, the annealing/extension time of the cycling protocol was also successfully adjusted for master mixes other than Bio-Rad.
All three tested assays-EvaGreen, Taqman, and SYBR Green-resulted in (nearly) best fit data. Overall, the method has proven not only reproducible but also robust-hence transferable-that is to say that the performance did not substantially change when using different master mixes and Real-Time PCR platforms, which is not a foregone conclusion when testing a method in collaborative studies [30].
Analysis of the standard curves reveals that PL reproduced a high amplification efficiency and data linearity, obtaining values that approach a best fit. This also clearly suggests that the whole procedure-DNA extraction and Real-Time PCR-would not be affected by inhibitors, which are potentially present in necrotic wood infected with C. platani.
In the absence of a suitable method for the statistical comparison of the different standard curves, we developed a method that we present here for the first time. This method is based on normalizing the Ct data of each dilution point of the curve with those of a theoretical best fit curve. This results in seven replicate values (corresponding to the seven dilution points) available for each curve to infer a mean and compare it with ANOVA. Interestingly, no significant differences could be found among the curves obtained when using the same Real-Time PCR system, regardless of the master mix used. This clearly suggests that use of the different master mixes with the adjusted thermal condition did not influence PCR efficiency nor the intercept value. This provides good evidence of both the reproducibility and the versatility/robustness of the method.
In contrast, significant differences were found among the curves obtained with the Taqman assay depending on the different Real-Time PCR systems that had been used. All of the curves were essentially parallel, namely with similar angular coefficient and thus efficiency; therefore, the intercept was responsible for the diversity in curve bundles. Specifically, the curves obtained with Rotor-Gene TM 6000 had the lowest intercept, StepOne Plus TM was intermediate, and CFX96 TM was the highest. This clearly shows how the Real-Time PCR systems that enabled inference of curves with lower intercept values guaranteed the detection of the same target DNA quantity-thus also of the LoD-at a lower Ct than the other systems. The capacity of the Rotor-Gene TM 6000 for detection at a lower Ct than other platforms was also previously observed in a comparison with Lightcycler 480 [31,32]. This would also imply that Rotor-Gene TM 6000 has the potential to lower the LoD of our method, which had been established using the CFX96 TM [18]. Indeed, in the analytical sensitivity test conducted by one PL using the Rotor-Gene TM 6000, the LoD of 3 fg was detected at Ct 31.6 on average (SD = 0.5), whereas in those conducted by six PL using the CFX96 detection was at Ct 35.6 on average (SD = 1.3). This indicates that the Rotor-Gene TM 6000 could theoretically enable the detection of a LoD ten times lower than 3 fg (i.e., 0.3 fg per PCR reaction) at a Ct value roughly 3.3 cycles later than 31.6 (i.e., at Ct 34.9), which is still allowed for a positive detection. Obviously, this has to be practically demonstrated as well as a demonstration that the thermocycler does not create false positive signals at a higher Ct.
Regarding detection of blind samples, all three assays obtained either very high or best fit CO, as derived from the (nearly) best fit performance parameters obtained by each PL. In practice, we analyzed data starting either from results on technical replicates or from those at the Das level (i.e., those derived from in-agreement results from at least two replicates out of three). While the former is more stringent, the latter can mask any inconsistent results within the technical replicates, i.e., if one replicate out of the three is an FP or an FN. In Table 6, we present results obtained with the first method while in Spreadsheet S1 both are presented.
With regard to N − samples (n = 360 at the technical replicate level), a total of eight FP (2.2% of all N − ) were detected. Six of these were detected in SYBR green by a single PL, which had a substantial impact on the performance of the method by this PL. Nevertheless, diagnostic specificity (DSP), accuracy (AC), and CO of the assay remained high at 91.7, 96.9, and 93.2, respectively. Taqman and EvaGreen showed a best fit.
All N + samples (n = 600 at the technical replicate level) were detected as positive (FN were 0% of all N + ) and Ct values among the different PL were very close, regardless of the master mix used. This again confirms the robustness of the method as stated above. To corroborate the significance among standard curves obtained in Taqman with the different Real-Time PCR systems, all detections performed with Rotor-Gene TM 6000 (Taqman) were at a clearly lower Ct than the others.
The analytical sensitivity of this Real-Time PCR method (LoD of 3 fg per PCR reaction in Taqman, EvaGreen, and SYBR Green assays) has been thoroughly validated by Lumia et al. [18] in an intra-laboratory performance test. The inter-laboratory comparison of this work again confirmed the capacity of the assays to detect 3 fg gDNA per PCR reaction. While all performance parameters and CO had best fit in the EvaGreen and SYBR Green assays, they were somewhat lower in the Taqman assay, where CO was 92.9 and 76.2 when testing 15 and 3 fg, respectively (Table 7). A possible explanation of this partial lack of homogeneity among the PL might be due to the fact that each PL performed in-house the dilutions of the gDNA sample dispatched by the OL. When performing a dilution, a minimal and an inadvertent variation in the first dilution steps can significantly impact the real value of the lowest concentration. This can thus condition the DA of the detection at what is believed to be the LoD, which by definition borders on undetectable DNA concentrations.
In diagnostic methods, validation and standardization of analytical sensitivity is key. Especially in highly sensitive methods such as this, validation of LoD is necessary in order to exploit the detection potential in studies where quantification is required. One example is the accurate evaluation of the repeatability/reproducibility of the resistance response of plants to pathogens with a specific focus on a very small pathogen titer, as this is frequently linked to the condition of resistance. We have previously shown that this Real-Time PCR method enabled a dramatic distinction between resistant and susceptible Platanus genotypes in terms of the target copy number of C. platani in the inoculated trees [17].
However, it should be considered that a genuine detection at the LoD can be indistinguishable from a contamination, which typically gives late fluorescent signals similar to detection of the LoD. It is therefore important that the Real-Time PCR plate is scattered with many replicates of a no-template or healthy control, as a guarantee of the accuracy of the results when they do not yield fluorescent signals. For routine diagnosis, use of controls is a must, but the number of replicates can be limited. Practical experience indicates that testing samples extracted from heavily infected wood can cause a contamination that spreads to the wells of the plate and can show up as late fluorescent signals. Thus, we recommend that in assays performed with diagnostic purposes all samples yielding late fluorescent signals from 34/35 Ct upward should be either a) repeated with a consistent array of negative controls (no template controls and extracts from healthy wood) or b) repeated with new samples collected from tissue possibly affected by fresh pathological necrosis.
Real-Time PCR technology is still the gold standard for an early, reliable, accurate, robust, and sensitive pathogen detection. In fact, among its many applications, it is used extensively for high-throughput screening and early diagnosis of SARS-CoV-2 and other infectious human diseases [33]. However, newer but already established technologies such as third-generation sequencing technologies are now being considered for their diagnostic potential [34,35]. Developing new methods for C. platani detection by exploiting these technologies would represent a new challenge for diagnosis and research on canker stain of the plane tree. This approach could integrate the still unequalled diagnosis by Real-Time PCR with additional epidemiological data that could be obtained upon detection of intraspecific sequence variants.

Conclusions
The excellent results of the TPS confirmed this Real-Time PCR method as a primary diagnostic tool for monitoring C. platani-free zones. Thus, this method would be crucial to impede the spread of canker stain in Northern European areas, such as Austria, Germany, Belgium, and the United Kingdom, where the disease is not yet present but plane tree is widely grown in the urban environments. Moreover, this Real-Time PCR can confidently be used by all researchers working on the characterization of the resistance/susceptibility levels of Platanus genotypes to canker stain. Indeed, a consistent detection protocol will enable reliable comparisons among the results obtained in different selection programs. Figure A1. Results of testing in blind the DNA-aliquot-samples (Das) by the organizing laboratory (OL) and the participating laboratories (PL) with EvaGreen (a), SYBR Green (b) and Taqman assay (c). Das from samples not infected with Ceratocystis platani were always negative, except in very few cases (see Table 6 and main text). The bars indicate the standard deviation of the mean. The color-associated uppercase letters followed by numbers indicate the OL (=Z) or the PL and the master mix/Real-Time PCR system combination adopted (see Table 3). TPS samples are in the vertical axes (NI = naturally infected, AI = artificially infected). C.P.32 is gDNA extracted from C. platani axenic culture (strain 32).