Validation of a Commercial Indirect ELISA Kit for the Detection of Bovine alphaherpesvirus 1 (BoHV-1)-Specific Glycoprotein E Antibodies in Bulk Milk Samples of Dairy Cows

Simple Summary Recent studies have assessed the feasibility of using commercial indirect enzyme-linked immunosorbent assays (ELISA) on bulk milk (BM) samples. Today is a need to validate a diagnostic method for granting and maintaining a correct surveillance strategy for infectious bovine rhinotracheitis (IBR). However, the poor availability of reference materials for diagnostic tests on milk samples and, the low sensitivity of blocking glycoprotein E (gE) ELISAs using milk as a matrix, represent limitations for developing new assays to control infectious diseases in livestock. This study aimed to validate a commercial indirect ELISA kit to detect antibodies to gE of Bovine alphaherpesvirus 1 (BoHV-1) from BM samples, according to the OIE Manual of Diagnostic Tests and Vaccines for Terrestrial Animals. The samples were collected from an IBR outbreak. The findings demonstrated high diagnostic performances. Furthermore, the validated kit is an easy-to-use and economical method to discriminate between BoHV-1 infected or immunised animals with gE-deleted markers vaccines using BM samples. Moreover, milk sampling represents a fast and non-invasive procedure for animals because it avoids negative effects, such as stress, and reduces the total cost of surveillance. Abstract In this study, we validated a commercial indirect enzyme-linked immunosorbent assay (ELISA) to detect antibodies to glycoprotein E (gE) of Bovine alphaherpesvirus 1 (BoHV-1) in bulk milk (BM) samples using the OIE Manual of Diagnostic Tests and Vaccines for Terrestrial Animals. The assay performance characteristics were evaluated using a panel of positive (n = 36) and negative (n = 80) samples with known infectious bovine rhinotracheitis (IBR) status. The assay showed adequate repeatability (within-run and between-run), with a coefficient of variability (CV%) of replicates below 30%; only two 1:40 diluted samples had a CV% above 20%. Additionally, an agreement analysis of the qualitative results of replicates led to a Gwet’s agreement coefficient of 0.99 (95% confidence interval (CI): 0.96–1.00, p < 0.001). The estimated diagnostic sensitivity (DSe) and diagnostic specificity (DSp) were 100% (95% CI: 90.3–100%) and 97.5% (95% CI: 91.3–99.7%), respectively. Overall, a good level of agreement was observed between the assay results and the true IBR status of samples (weighted Cohen’s κ: 0.96, 95% CI: 0.78–1.00). The findings demonstrate that the indirect ELISA kit validated here is an easy-to-use and economical method to differentiate infected and gE-deleted marker vaccine-immunised animals using BM samples.


gE-ELISA (ERADIKIT)
In this study, the gE indirect ELISA kit (ERADIKIT™ Bulk Milk Surveillance Kit PLUS, In3diagnostic, Torino, Italy) was used following manufacturer recommendations (Table 1). It was then compared to the protocols described in previous studies [22,33], as slight modifications were done. The net optical density (OD) of each sample was calculated by subtracting the OD value of the negative antigen (even columns) from that of the positive antigen (odd columns). The percentage of reactivity of each sample was calculated as the ratio between the net OD value of the sample and the net OD value of the positive control as follows: (Net OD value of the sample) (Net OD value of the positive control) × 100  [22], and used in this study (gE III). According to the cut-off established by the assay manufacturers, samples with a reactivity greater than 40% were classified as positive, those with reactivity between 30% and 40% as doubtful, and those with a reactivity lower than 30% as negative.

Criteria for Assay Validation
In this study, the approach adopted to validate the indirect gE-ELISA test included the following criteria: 1. Repeatability 2.
Diagnostic accuracy For these evaluations, we primarily referred to protocols [36,37] of the OIE Manual. The outcomes of all tests were analysed quantitatively and then qualitatively, based on the cut-off values established by the assay manufacturers.

Reference Samples
In this study, all data analysed were collected as a part of the routine diagnosis and not for this experiment; therefore, according to the national legislation, ethics approval and written informed consent were not required.
A panel of negative and positive reference samples based on milk samples from farms or animals with a known history and IBR status was created to evaluate the performance of the gE indirect ELISA test.
The negative reference samples comprised 80 BM samples collected from 80 different herds in northern Italy (Piedmont Region) with official IBR-free status. The farms were certified by the regional public veterinary service in 2019 under the IBR European legislation 2004/558/CEE [38]. Each BM sample comprised a maximum of 50 milk samples.
The National Reference Centre for Infectious Bovine Rhinotracheitis Biobank (Perugia, Italy) provided a set of samples (serum and individual milk) collected during an IBR outbreak that occurred in central Italy in 2018. The outbreak involved a dairy herd of approximately 270 cows, for which no IBR vaccine had been used in the last seven years. Twelve 4-year-old lactating cows were selected from this herd and divided into three groups of four animals each. Serum and individual milk samples were collected from the three groups at 10, 21, and 63 days after the start of the outbreak (time 0, official confirmation of IBR outbreak). BoHV-1 antibodies in serum samples were evaluated by gB-ELISA (IDEXX IBR gB X3 Ab, Westbrook, ME, USA), gE-ELISA (IDEXX IBR gE Ab test), and VNT. In addition, whole-virus ELISA (IDEXX BHV1 Bulk Milk Ab test), gB-ELISA (IDEXX IBR gB X3 Ab), and gE-ELISA (IDEXX IBR gE Ab test) were used to evaluate the antibody concentration in the milk samples.
The protocols used were those described by the ELISA kit manufacturers, and VNT was performed according to the protocol reported in the OIE Manual of Diagnostic Tests and Vaccines for Terrestrial Animals, the "Infectious Bovine Rhinotracheitis/Infectious Pustular Vulvovaginitis (Chapter 3.4.11) [39]. The results are presented in Table 2.
Twelve positive individual milk samples were tested using the gE indirect ELISA kit (ERADIKIT). Based on the OD results, they were grouped into three categories: weak, medium, and high reactivity. Each positive individual milk sample was diluted at 1:10, 1:20, and 1:40 in certified IBR-free milk, thereby obtaining a total of 36 milk samples, which were used as the positive reference samples for this study (Table 3).  Total of positive reference samples 36 a OD-optical density; b Ag+-well of the plate with antigen; c Ag--well of the plate without antigen; d Net-OD Ag+ minus OD Ag-; e S/P%-sample-to-positive control ratio.

Repeatability
To evaluate within-laboratory repeatability (variation in results of replicates), each positive reference sample was tested in four replicates: two replicates within the same test run (same operator on the same day) and two replicates in different sessions (two operators on different days).
Within-run repeatability was assessed by calculating the coefficient of variability (CV) for each sample, obtained as the ratio between the standard deviation and the arithmetic mean of the two replicates within the same test run. Similarly, between-run repeatability was assessed by calculating the CV of four replicates for each sample.
Then, the average within-run CV and between-run CV were calculated for each reactivity group and dilution, respectively, and the corresponding 95% confidence intervals (CIs) were estimated. Additionally, for reporting purposes, an agreement analysis of the qualitative results of the four replicates was conducted to quantify the repeatability in terms of the agreement of test results beyond chance. To achieve this, Gwet's agreement coefficient (AC) was calculated with a 95% CI. Cohen's kappa statistic (κ) is generally the first choice for estimating agreement. However, it is known to be influenced by the prevalence of the attribute under study and, in some cases, can yield unexpected results, known as the paradoxes of kappa [40]. Gwet's AC, on the other hand, has greater stability as the prevalence of the attribute under study varies; thus, it is a more reliable tool in the cases of unbalanced distributions such as the one under examination [40,41].

Analytical Sensitivity (ASe)
To assess the assay ASe, the limit of detection (LOD) was conservatively defined as the maximum dilution at which 100% of replicates showed positive results.

Diagnostic Sensitivity (DSe) and Diagnostic Specificity (DSp)
Based on the cut-off established by the kit manufacturers, the qualitative results of the test, which were obtained from the reference sample panel, were dichotomised as negative and non-negative (doubtful or positive) and cross-tabulated in a 2 × 2 table according to the IBR status of the reference samples. Then, according to Chapter 2.2.5 of the OIE Manual of Diagnostic Tests and Vaccines for Terrestrial Animals [37], estimates of the assay DSe (proportion of positive reference samples that tested positive in the assay) and of the assay DSp (proportion of negative reference samples that tested negative in the assay) were obtained, calculating the corresponding exact binomial 95% CIs.

Diagnostic Accuracy
To evaluate the diagnostic accuracy of the assay, agreement analysis was conducted to quantify the concordance between the qualitative results of the indirect ELISA and the true IBR status of the reference samples. Weighted Cohen's kappa was computed, providing the respective 95% CI.

Statistical Analysis: Tools and Settings
Statistical analyses were conducted using Microsoft Excel 2013 (Microsoft Corporation, Redmond, WA, USA) and Stata ® 16.1 statistical software (Special Edition; StataCorp LP, College Station, TX, USA). The significance level was set at α = 0.05.

Repeatability and ASe
The quantitative results of the four replicates are shown in Table 4. Replicates I and II denote the two replicates performed in the same test run, and replicates III and IV denote the replicates performed in two different sessions. All samples were confirmed to be positive in all replicates except for the 1:40 diluted sample of ID-37, which was classified as doubtful (S/P%: 33.9).
The assay showed a LOD of 1:20 for the weakly positive samples. In contrast, the assay LOD for the medium-highly positive samples was estimated to be at least 1:40 (Table 4; Figure 1).
The within-run and between-run variabilities expressed as the CV% of replicates I and II (I-II) and the CV% of replicates I, II, III, and IV (I-IV), respectively, are shown at the sample level in Table 5 and Figure 2.
The within-run CV% ranged between 2.0% and 22.1%. The CV% of one sample (ID-42; 1:40 diluted) exceeded 20% (Figure 2A). The between-run CV% ranged between 5.6% and 24.9%; nevertheless, only one sample (ID-37; 1:40 diluted) exceeded 15% ( Figure 2B). For each reactivity group and dilution ratio, the mean value of within-run CV% and the mean value of between-run CV% are shown in Figure 3, respectively, with corresponding 95% CIs. Additionally, overall averages of CV% were computed for each dilution, regardless of reactivity.

Diagnostic Performances: DSe, DSp, and Diagnostic Accuracy
Overall, 78 out of 80 negative reference samples were confirmed as negative by gE indirect ELISA (Figure 4). Only 2 samples out of 80 generated false-positive reactions (one positive: S/P, 52.8%; one doubtful: S/P, 34.6%), although they were classified as negative when tested a second time. The agreement analysis on the qualitative results of the four replicates led to a Gwet's AC of 0.99 (95% CI: 0.96-1, p < 0.001), which, according to Altman's scale, corresponds to a 'very good' agreement [42].

Diagnostic Performances: DSe, DSp, and Diagnostic Accuracy
Overall, 78 out of 80 negative reference samples were confirmed as negative by gE indirect ELISA (Figure 4). Only 2 samples out of 80 generated false-positive reactions (one positive: S/P, 52.8%; one doubtful: S/P, 34.6%), although they were classified as negative when tested a second time.
The qualitative results of the gE indirect ELISA performed on the 116 reference samples were dichotomised as negative and non-negative and cross-tabulated according to the IBR status of the samples ( Table 6). The assay showed a DSe of 100% (95% CI: 90.3-100%) and a DSp of 97.5% (95% CI: 91.3-99.7%).
Additionally, to quantify the overall ability of the assay to correctly categorise samples, the agreement analysis of the 116 reference samples led to a weighted Cohen's κ of 0.96 (95% CI: 0.78-1; Table 7). According to Altman's scale, a value of κ ranking between 0.61 and 0.80 corresponds to a 'good' agreement, while a value of κ higher than or equal to 0.81 corresponds to a 'very good' agreement [42].  The qualitative results of the gE indirect ELISA performed on the 116 reference samples were dichotomised as negative and non-negative and cross-tabulated according to the IBR status of the samples ( Table 6). The assay showed a DSe of 100% (95% CI: 90.3-100%) and a DSp of 97.5% (95% CI: 91.3-99.7%). Additionally, to quantify the overall ability of the assay to correctly categorise samples, the agreement analysis of the 116 reference samples led to a weighted Cohen's κ of 0.96 (95% CI: 0.78-1; Table 7). According to Altman's scale, a value of κ ranking between 0.61 and 0.80 corresponds to a 'good' agreement, while a value of κ higher than or equal to 0.81 corresponds to a 'very good' agreement [42].

Discussion
The improvisation and validation of the available assays is a prerequisite for dealing with current economic insecurities. Furthermore, all assays should be validated to monitor their routine performance for an intended purpose. Validation is a process that in-

Discussion
The improvisation and validation of the available assays is a prerequisite for dealing with current economic insecurities. Furthermore, all assays should be validated to monitor their routine performance for an intended purpose. Validation is a process that includes estimates and optimisation of all analytical and diagnostic performance characteristics of a test [36]. Several commercial gE indirect ELISA kits intended to detect BoHV-1 in BM samples have been developed [20,22,23,33]. For their application in a screening test in a surveillance programme, they should be capable of detecting all levels of true positivity, especially those borderline cases where animals have low antibody levels.
In this study, we made concerted efforts to improve and validate the performance of a commercial gE indirect ELISA kit: the ERADIKIT™ Bulk Milk surveillance Kit PLUS. The protocol for the gE indirect ELISA test was applied to skimmed milk and was modified by modifying the steps for the purification and concentration of IgG to enhance the kit's performance compared to the original protocol described in Muratore et al. [33].
The protocol described earlier [33] only evaluated DSe, DSp, and ASe, and was successful.
Here, we modified the DSe (100%; 95% CI: 90.3-100%), DSp (97.5%; 95% CI: 91.3-99.7%), incubation times, room temperatures, washing step, sample number and volumes, dilution factors, and wash buffer formulation of the original protocol to ensure the fitness of the kit for the intended purpose. The protocol developed herein validated the results of the original protocol [33]. It showed similar results compared with those obtained using the original protocol (DSe = 100%; 95% CI: 89.4-100% and DSp = 98.85%; 95 CI: 95.90-99.86%). However, CIs, which indicate the precision of the DSe and DSp estimates, were wide. This could be attributed to the limited number of samples used to estimate the sensitivity and specificity, as it is difficult to find field samples with well-defined IBR status. Furthermore, the optimisation of the reagents and protocol by including samples belonging to animals with known infection status that had a definite level of activity (titre or concentration) for the pathogen could also have affected the results.
In this study, we demonstrated good agreement between the test results and those of the IBR status of samples, repeatedly achieving the same results for the milk samples examined. The evidence of adequate within-laboratory repeatability was highlighted by raw absorbance values of less than 20%, meeting the requirements of the OIE Manual of Diagnostic Test and Vaccines for Terrestrial Animals (CV < 30%) [43,44]. Only two samples generated false-positive reactions, with values of CV% approaching 20-30%, but they were restored when tested a second time. It could be due to an unavoidable minimum bias and random error [44].
Collectively, in this study, calculating beforehand the prevalence within each BM sample [45] using the preliminary knowledge of reactivity of each positive serum individual sample demonstrated more precision of the DSe and DSp estimates and adequate repeatability of the assay compared to those obtained by Muratore et al. [33]. Moreover, as described in previous studies [23], the antibody titre and dilution factor also partially influenced the analytical sensitivity in the present study. The assay ASe showed a LOD of 1:20 for the weakly positive samples compared to at least 1:40 for the medium-highly positive samples. Therefore, even if some steps for IgG purification and concentration processes were modified, the indirect gE-ELISA partially detected weakly positive samples in concentrated pooled milk samples [19,23]. Several studies have demonstrated that weakly positive samples are frequently missed before dilution [15,23]. Therefore, it has been suggested to check the performance of the competitive ELISA tests using various dilutions to identify the maximum dilution at which 100% of replicates can improve the positive results. Moreover, as suggested by Schroeder et al. [23], reducing the pool size from 50 to 20 milk samples or establishing regular testing (e.g., monthly) might improve the precision of the results. In such a case, all 1:20 dilutions of weakly positive samples would have been detected by the assay validated in this study.
In addition, several other points must be taken into account while using the indirect ELISA to detect BoHV-1-specific gE antibodies in BM samples.
Increasing the IgG concentration in milk samples to increase the sensitivity of the test is a major drawback [23]. This procedure requires a longer period for serological analysis and the use of more laboratory materials.
Moreover, under the Delegated Regulation (EU) 2020/689, some prerequisites of BoHV-1 BM testing must be satisfied. In particular, a pool of no more than 50 milk samples (individual or BM) may be used in tests for granting IBR-free status, and a pool containing no more than 100 milk samples (individual or bulk) may be used for the maintenance of IBR-free status [11]. Additionally, serological tests against gE of BoHV-1 must be performed on BM samples from farms with at least 30% of the lactating bovine. In this case, milk samples must be taken on at least three occasions at intervals of not less than three months from lactating bovine representing all epidemiological units of the herd. Moreover, blood samples must be collected from all non-lactating bovines and male bovines over 12 months of age. In a herd comprising less than 5% male bovines wherein at least 95% of females used for milk production are over 24 months of age, BM samples must be taken on at least six occasions at intervals of not less than 2 months from lactating animals representing all epidemiological units of the establishment [11].
However, milk sampling represents a fast and non-invasive procedure for animals because it avoids negative effects, such as stress, and reduces the total cost of surveillance [46]. In addition, such samples can be easily collected, pooled, and tested. This can increase the number of serological investigations per year, allowing earlier detection of new BoHV-1 outbreaks [22,33]. Therefore, further studies are required to improve the validation pathway of this innovative serological kit, such as the use of European reference materials collected from cattle and buffalo.

Conclusions
In conclusion, the results of this study indicate that the new protocol gE indirect ELISA test "ERADIKIT™ Bulk Milk Surveillance Kit PLUS" did not improve the kit's performance significantly compared to the original protocol described by Muratore et al. [33]. However, the new procedure could be considered a useful tool, provided that the applicability of indirect gE-ELISA testing of IgG concentration in milk samples could be employed in a pool of individual samples from no more than 40 lactating cows, where individual samples are available for routine analysis (i.e., somatic cell count, protein, and fat content), as suggested by Colitti et al. [22]. Furthermore, the applicability of gE-ELISA for BM testing, without the IgG concentration step, proved to be suitable to detect BoHV-1 in vaccinated herds, provided that gE prevalence is higher than 10%. Nevertheless, in this case, more frequent testing is necessary, so that most cows present in a herd will contribute to a BM sample at least once per year [20]. Further studies are required to evaluate the gE indirect ELISA test "ERADIKIT™ Bulk Milk Surveillance Kit PLUS" using European reference materials collected from cattle and buffalo.

Patents
The patent application was registered using the In3diagnostic (EP 2,942,628 A1 Kit and in vitro method for identifying BoHV-1 infection).
Author Contributions: C.R., S.P., F.F., L.F. and S.R.: conceptualisation, original draft preparation, writing, and manuscript revision; C.I., C.R., S.P., S.R. and C.N.: data curation, review, and editing; C.I. and C.N.: visualisation, data curation; C.I., C.P., E.R. and L.F.: methodology, A.D. and L.F.: statistical analysis; F.F. and S.P.: supervision. All authors have read and agreed to the published version of the manuscript. Acknowledgments: The authors are grateful to Gianfranco Corgiat Loia, Enrico Maria Ferrero, and Maria Elena Careddu of the Piedmont Regional public veterinary service for providing the negative reference samples. The authors also thank Gigliola Canepa, University of Milan, for revising the language of the manuscript.

Conflicts of Interest:
The authors declare no conflict of interest. In3diagnostic, the manufacturer of the gE indirect ELISA, had no role in the study design; the collection, analysis, or interpretation of the data; the writing of the manuscript; or the decision to submit it for publication. In this study, In3diagnostic only provided the kit for the validation process, in accordance with the formal agreement with the Istituto Zooprofilattico Sperimentale Umbria-Marche, "Togo Rosati".