Comparison of Frequentist and Bayesian Meta-Analysis Models for Assessing the Eﬀicacy of Decision Support Systems in Reducing Fungal Disease Incidence

: Diseases of fruit and foliage caused by fungi and oomycetes are generally controlled by the application of fungicides. The use of decision support systems (DSSs) may assist to optimize fungicide programs to enhance application on the basis of risk associated with disease outbreak. Case-by-case evaluations demonstrated the performance of DSSs for disease control, but an overall assessment of the efﬁcacy of DSSs is lacking. A literature review was conducted to synthesize the results of 67 experiments assessing DSSs. Disease incidence data were obtained from published peer-reviewed ﬁeld trials comparing untreated controls, calendar-based and DSS-based fungicide programs. Two meta-analysis generic models, a “ﬁxed-effects” vs. a “random-effects” model within the framework of generalized linear models were evaluated to assess the efﬁcacy of DSSs in reducing incidence. All models were ﬁt using both frequentist and Bayesian estimation procedures and the results compared. Model including random effects showed better performance in terms of AIC or DIC and goodness of ﬁt. In general, the frequentist and Bayesian approaches produced similar results. Odds ratio and incidence ratio values showed that calendar-based and DSS-based fungicide programs considerably reduced disease incidence compared to the untreated control. Moreover, calendar-based and DSS-based programs provided similar reductions in disease incidence, further supporting the efﬁcacy of DSSs.


Introduction
According to Higgins et al. [1], "meta-analysis refers to the statistical synthesis of results from a series of studies". In plant pathology, meta-analysis has become in a powerful tool [2] to address questions such as factors determining the effects of different pesticides and biological treatments for managing diseases [3][4][5]. Meta-analysis implementation requires the selection of proper statistical models to draw robust conclusions. Different types of statistical models were proposed with that regard; however, there is a risk of biased parameter estimation, misinterpretation and incorrect conclusions if a quality control of statistical techniques is not considered [6]. According to Philibert et al. [7], one

Literature Search and Data
A database was assembled based on a systematic literature review from the (i) Web of Science (WOS) and (ii) the Fungicide and Nematicide Tests (F&N Tests and Plant Disease management reports). For (i), publications were selected according to multiple search strings described in Appendix A. For (ii), reports were extracted according to multiple key words (Appendix A). Hard copies of volumes published prior to 2000 were examined directly. Relevant experiments were selected according the following criteria: (a) the experiment evaluated at least one DSS-based strategy, one calendar-based strategy and an untreated control (as a minimum each experiment included three sub-experiments according to the type of treatment); (b) all sub-experiments reported disease incidence (i.e., proportion of diseased organs) and sample size (i.e., total of organs considered to evaluate disease incidence). All experiments fulfilling those criteria were included in the meta-analysis. In calendar-based programs the number and timing of fungicide applications was fixed before the experiment, usually based on the standard practices for disease control. In DSS-based programs the number and timing of fungicide applications was decided during the course of the experiment based on risk indicators.
Each experiment was defined as a unique combination of location (country, state), year, crop, organ evaluated, disease, pathogen, treatment strategy, number of sprays and treatment id (see Table A2 for detailed information about the location, crop and disease information). Conversely, each sub-experiment was characterised by the type of treatment (DSS, calendar or untreated), the observed disease incidence, the sample size and the number of sprays. For both sources ((i) and (ii)) a total of 67 experiments, 285 sub-experiments were selected from a total of 19 publications/reports (see Table A3 for further details). The number of independent experiments among the papers varied from 1 to 11 and the number of sub-experiments among experiments from 3 to 7 (Table A3). The database included a total of 67 sub-experiments under untreated conditions, 86 under a calendar-based strategy and 132 under DSS-based strategy (Table A3). An overall and individual (per sub-experiment) description of the dataset is provided in Section 3.1 in terms of the observed disease incidence and incidence ratios (IRs) between calendar (Cal./Unt.) and DSS (DSS/Unt.) strategies against the untreated control.

Meta-Analysis
The response variable used to quantify the level of disease in all experiments (sub-experiments) was the number of diseased organs. Given the nature of this metric, meta-analysis was formulated within the familiar framework of GLMs [10] and applied to data with a binomial likelihood [14] and a logit link function. The GLM overcomes problems of models based on normal likelihood [15]. It was formulated to compare the effect of the two fungicide strategies (DSS and calendar) simultaneously in comparison to the untreated control.
This generic model was extended by the inclusion of random effects usually named as generalised linear mixed model (GLMM) [16] with the aim to account for the unobserved sources of variability among experiments beyond the fungicide treatment effects, such as the different pathogens/crops targeted, the mode of action of the fungicides, and inoculum levels. Both GLM and GLMM were fitted using two statistical methods, a frequentist and a Bayesian method. All implementations were made in the R environment (version 3.5.1). The full analysis can be reproduced using code and data archived at https://bitbucket.org/elaher/comparative-meta-analysis/src/master/.

Statistical Modelling
The logistic GLM was formulated as: where the random variable Y ij which describes the number of diseased organs in the sub-experiment j of the experiment i out of a total of n ij organs analyzed, is assumed to follow a Binomial distribution with unknown probability of disease θ ij (disease incidence); I cal(ij) and I dss(ij) are dummy variables for calendar and DSS treatments equal to 1 if the sub-experiment j of the experiment i was conducted with one of these strategies and to zero otherwise; β cal and β dss are fixed parameters (regression coefficients) which capture calendar and DSS treatment effect compared to the untreated control, β 0 . This model formulation is usually denoted in meta-analysis framework as "fixed-effects" model [5,9] due to it assumes that each treatment effect does not vary across experiments.
The logistic GLMM was formulated as an extension of the GLM defined previously by the inclusion of three random effects describing the between-experiment variability of the treatment effects. In the GLMM model, Equation (2) is modified as follows: where γ cal(i) , γ dss(i) are random parameters which capture calendar and DSS treatment effect for each experiment i in relation to the untreated control, γ 0(i) . Each individual experiment effect is assumed to be multivariate normally distributed about the means β * 0 , β * cal(i) , β * dss(i) with a 3 × 3 variance-covariance matrix Σ, defined as follows: The means (fixed parameters), β * cal and β * dss , in constrast to GLM interpretation, describe the average effects of calendar and DSS treatments within the population of experiments in comparison to the untreated control, β * 0 . Their corresponding random effects, b cal(i) and b dss(i) , capture individual experiment calendar and DSS treatment effects in relation to the untreated control, b 0(i) . Thus, under GLMM specification, each treatment effect varies from experiment to experiment i. The variances, σ 2 0 , σ 2 cal and σ 2 dss capture the extent of the above mentioned variability while covariances, σ 0,cal , σ 0,dss , σ cal,dss point correlations between sub-experiments belonging to the same experiment. This model is usually called the "random-effects" model [1] in the context of meta-analysis.

Parameter Estimation: Frequentist vs. Bayesian Approach
Inference for both models (GLM and GLMM) was carried out using frequentist and Bayesian statistics, successively, leading to two different sets of estimated parameters. Frequentist models were denoted as GLM_F and GLMM_F and their Bayesian counterparts as GLM_B and GLMM_B. Parameters of frequentist models were estimated by maximum likelihood through iterative reweighted least squares method and Laplace approximation using the glm() and the glmer() functions of the package lme4 [17] implemented in the R software [18] version 3.5.1, respectively. Likelihood ratio tests were performed to assess the significance of the fungicide treatment effects. Model comparison was done using the Akaïke Information Criterion (AIC) [19], smaller values of AIC correspond to preferred models. A rule of thumb outlined in Burnham and Anderson [20] is that models with ∆ i (AIC) = AIC i − min AIC higher than 10 have no support against a model with minimum AIC value.
In the Bayesian approach, uncertainty about quantities of interest and experimental results is always expressed in probabilistic terms. Inferential processes for learning about a quantity of interest φ always start with a prior distribution which contains all relevant information about φ, π(φ). Experimental data are related to φ via a sampling model (i.e, binomial model) which is the basis for computing the likelihood function L(φ). Both elements are formally combined by means of the Bayes' rule to obtain the posterior distribution for φ, π(φ | Data), which synthesize all the available knowledge about φ, π(φ | Data) ∝ L(φ) π(φ).
Bayesian simple inferential processes are based on conjugate families for which the prior and the posterior distribution belongs to the same distribution family. Under this specific context the posterior distribution can be calculated analytically. However, for more complex models such as ours, it needs to be approximated with numerical methods such as Markov chain Monte Carlo (MCMC) methods [21]. Then, the posterior distribution of the inference parameters is described by a random sample of parameter values. The same sample of parameters can be used to approximate the posterior distribution of any quantity of interest, for examples, the disease incidence, the odds ratio (OR), the incidence ratio (IR), etc. All these distributions can be summarised using several point estimates such as the mean, the median, the mode. Furthermore, Bayesian approach makes possible to quantify uncertainty of any posterior distribution by means of credible intervals [22].
GLM_B and GLMM_B inferential processes were performed under several independent prior scenarios. For the GLM_B, following the recommendation of Gelman et al. [23] a weakly informative prior scenario with Cauchy distributions was considered. Specifically, the prior of β 0 was defined by a Cauchy distribution with location parameter µ fixed at 0 and a scale parameter σ of 10, (C(0, 10)). Conversely, the priors of β dss and β cal were defined by Cauchy distributions also centered at 0 with scale parameter σ = 2.5, (C(0, 2.5)). This default prior scenario implies that model fitting uses an adaptation of the standard iteratively weighted least squares computation [23] which makes similar this model with its frequentist counterpart. In GLMM_B, a standard non-informative prior scenario was defined to give prominence to data and also to make it comparable with GLMM_F. The prior for β * 0 , β * dss and β * cal was defined independently by means of normal distributions centered at 0 and with a wide variance (N(0, 1000)) . An inverse Wishart distribution (IW(Ψ, ν)) was considered to specify the prior to the variance-covariance matrix, Σ. Specifically, the inverse Wishart distribution is defined with an r × r scale matrix Ψ with r equal to the number of random parameters, and with several degrees of freedom ν. Ψ was specified as an identity matrix (values of 1 in the diagonal and 0 otherwise) of 3 × 3 and ν = 3. For both models, posterior distribution was approximated by means of Markov chain Monte Carlo (MCMC) simulation methods by means of the JAGS software (version 4.3.0) through the R2jags package (version 0.5-7) [24] of the R software.
The MCMC algorithm was run with three Markov chains each including 120,000 iterations after a burn-in period of 20,000 iterations. In addition, the chains were thinned by storing one in ten iterations in order to reduce autocorrelation in the subsequent sample. Convergence was assessed via three different criteria: (i) graphically, drawing trace plots and assessing the simulated values of the chains appear overlapping one another, (ii) based on the potential scale reduction factor,R, whose values must be equal or close to 1, and (iii) by means of the effective number of independent simulation draws, ne f f , which must be >100 [25]. Regarding methods for Bayesian model choice, the deviance information criterion (DIC) [26] was considered. Smaller values of DIC correspond to preferred models, and a DIC difference of 5 or more is generally regarded as practically meaningful [27].
In addition to the standard model selection criteria (AIC or DIC), the goodness of fit for the different models was evaluated graphically comparing observed vs. fitted disease incidence in the logit scale. Note that for Bayesian models, logit fitted values were based on the median of the posterior distribution of the logit disease incidence.

Treatment Effects: Disease Incidence, Odds Ratio, Incidence Ratio and Predictive Distribution of the Odds Ratio
Effect measures were assessed by the estimation of disease incidence, OR and IR for the four models [28,29]. With the "random-effects" models (GLMM), these quantities are assumed to vary across experiments and this variability was described by computing the predictive distribution for a new experiment chosen at random.

Disease Incidence
For GLM_F models, disease incidence (in probability scale) was estimated for each specific fungicide strategy (untreated, calendar and DSS) according Equation (2) notation as follows: in whichβ 0 ,β cal andβ dss correspond to the maximum likelihood parameter estimates. Their associated 95% confidence intervals were calculated according to in which z 0.975 denotes the 0.975 quantile of the normal distribution and SE the standard error. Note that disease incidence computation was calculated in the logit scale due to errors are assumed normally distributed and then logit-transformed (inv.logit(x) = exp{x} 1+exp{x} ) [30]. Generally, SE = Var with Var denoting estimated variance.
For GLM_B, posterior distribution of the disease incidence was drawn from the posterior distribution of the model parameters following the tranformation described in Equation (6) and summarized by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.
For GLMM_F model, average "experiment" disease incidence was computed for a each specific fungicide strategy (untreated, calendar and DSS) according Equation (4) notation as follows: As with the GLM_F computation, 95% confidence intervals were estimated following the same approach described in Equation (7) but adapting it to the parameters involved. For GLMM_B, posterior distribution of the average the disease incidence was drawn from the transformation of the posterior distribution of the model parameters involved and summarised by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.

Odds Ratio
Based on the formal definition of the disease incidence described previously, OR for calendar (OR cal ) and DSS (OR dss )-based strategies against control as well as OR for calendar against DSS (OR cal/dss ) were estimated for GLM_F model as follows: Note the OR computation was carried out on a log scale [31]. Thus, based on the previous statement 95% confidence intervals were estimated according to in which z 0.975 denotes the 0.975 quantile of the normal distribution and SE the standard error.
For GLM_B, posterior distribution of the OR was drawn from the posterior distribution of the relevant model parameters based on the transformation described in Equation (9) and summarized by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.
Under GLMM_F model, average "experiment" OR for calendar and DSS-based strategy against the untreated control and OR for calendar against DSS were computed as follows: but also in the log scale with 95% confidence intervals estimated as described in Equation (10) using the relevant parameters. For GLMM_B, posterior distribution of the average OR was drawn from the transformation of posterior distribution model parameters and summarised by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty. OR cal and OR dss were used as a summary measure to quantify the efficacy of each fungicide program by means of relative changes in odds disease. OR cal/dss was used to assess differences of calendar strategy in relation to DSS in terms of relative changes in odds disease.

Incidence Ratio
Following the same strategy specified in the previous section for OR computation, IR for calendar (IR cal ) and DSS (IR dss )-based strategies against untreated control and IR for calendar against DSS were estimated for GLM_F model as follows: Also for IR computation was carried out on a log scale and then log-transformed to convert them into the original metric [31]. Thus, based on the previous statement 95% confidence intervals were calculated according to in which z 0.975 denotes the 0.975 quantile of the normal distribution and SE the standard error. For GLM_B, posterior distribution of the IRs was drawn from the transformation of posterior distribution of the model parameters involved and summarized by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.
For GLMM_F model, average "experiment" IR for calendar and DSS-based strategy against the untreated control and IR for calendar against DSS were computed as follows: but also in the log scale and with 95% confidence intervals were estimated in the same way as described in Equation (13) but adapting it to the parameters involved. For GLMM_B, posterior distributions of average IR were drawn from the transformation of posterior distribution of the model parameters concerned and summarised by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty. IR cal and IR dss were used as a summary measure to quantify the efficacy of each fungicide program by means of relative changes in disease incidence. IR cal/dss was used to assess differences of calendar strategy in relation to DSS in terms of relative changes in disease incidence.

Predictive Distribution of the Odds Ratio
To assess the extent of the heterogeneity estimates among experiments, the uncertainty of the OR cal , OR dss and OR cal/dss were also re-evaluated by calculating their corresponding predictive distributions. These predictive distributions allow us to incorporate between-experiment variability and to derive the so-called prediction intervals. Following the computation strategy described in Section 2.4.2, predictive distributions of the OR were computed in log-scale and then back-transformed. Parameters involved here were γ cal,new and γ dss,new (they represent the values of γ cal(i) and γ cal(i) for a new experiment and new sub-experiments chosen at random). According to Higgins et al. [1], their corresponding predictive distribution were defined as in which t N−2 denotes a t-distribution with N − 2 degrees of freedom with N = 19 representing the number of experiments. With GLMM_B, the OR predictive distributions were derived from the predictive distributions of γ cal,new , γ dss,new . These distributions were obtained by sampling values of γ cal(i) and γ cal(i) from their corresponding posterior distributions using the following transformations, exp{γ cal,new } and exp{γ dss,new }.
Individual disease incidence under untreated sub-experiments presented in the vast majority of experiments higher values than their calendar and DSS counterparts, with only a few exceptions (A1, C1, D1, P1 and Q9) (Figure 2a). Calendar and their corresponding DSS showed similar disease incidence and IR values, sometimes higher for calendar, sometimes lower. Most IRs for Cal./Unt. and DSS/Unt. were lower than 1 with a few exceptions (A1, C1, D1, P1 and Q9 experiments) (Figure 2b).

Statistical Modelling Evaluation
The "random-effects" models performed better than "fixed-effects" models with both frequentist and Bayesian approaches according to the AIC or DIC scores. AIC for models GLM_F and GLMM_F were estimated at 64189.00 and 26098.96 (Table 1). Their Bayesian counterparts (GLM_B and GLMM_B) got DIC scores of 64188.97 and 25860.42 (Table 2), respectively. Thus, with both inferential approaches, results were in favor of the GLMMs.

Disease Incidence, Odds Ratio and Incidence Ratio Estimates
Disease incidence, OR and IR estimates are displayed in Figure 4a-c for the four fitted models (GLM_F, GLM_B, GLMM_F and GLMM_B). Overall, all models showed similar point estimates. Confidence intervals obtained with the frequentist models were also similar to the credible intervals obtained with Bayesian models (GLM_F vs. GLM_B and GLMM_F vs. GLMM_B).
With the GLM_F, the estimated median disease incidence was higher for the untreated control, 0.43, than for calendar-based (0.116) and DDS-based strategies (0.149). The same trend was observed with the GLMM_F (median values of 0.387, 0.045, 0.078 for control, calendar and DDS strategies, respectively). GLMM_F revealed that the calendar and DDS estimates strongly overlap. Comparable results were obtained with the Bayesian models (GLM_B and GLMM_B).
With the GLM_F, OR median values for OR cal , OR dss , OR cal/dss were equal to 0.174, 0.232 and 0.750, respectively. With GLMM_F, these estimates were equal to 0.074, 0.133 and 0.55 (lower than frequentist estimates). Based on GLMM_F outputs, both calendar and DSS programs provide strong reduction of odds disease compared to the untreated controls (OR cal and OR dss < 1). Nevertheless, odds disease in calendar programs would be lower that in DSS (OR cal/dss < 1, but with some uncertainty). The level of reduction obtained with DSS is already very high and leads to low disease incidences (see Figure 4a). Similar results were obtained with the Bayesian models (GLM_B and GLMM_B) and also for the different modelling IR estimates (Figure 4c).    Figure 5 shows predictive distributions for the OR for the GLMM_F (Figure 5a-c) and GLMM_B (Figure 5d-f). The x-axis shows the possible values of the ORs and the y-axis the percentiles (%) of its corresponding predictive distribution. That is, for each specific value of the OR showed, its corresponding cumulative probability denotes the % of chance to be less than this value.

Predictive Distribution of the Odds Ratio
Predictive distributions obtained with GLMM_F and with GLMM_B showed some noticeable differences for the OR cal and OR dss , not for the OR cal/dss . For GLMM_F, the probabilities (cumulative disease incidence probabilities) to have an OR lower than one are equal to 0.99 for both treatment strategies (Figure 5a,b). These results reveal that there is more than 99% to decrease the odds of disease with calendar and DSS strategies compared to untreated controls. However, according to the GLMM_B model, these probabilities were lower (about 0.8) (Figure 5d,e). Despite this difference, both models support that calendar and DSS strategies show high capabilities in reducing odds of disease. For both models, the probability that OR cal/dss was lower than one is equal to 0.68 which means that odds of disease might be lower with calendar but with a considerable degree of uncertainty (Figure 5c,f).

Discussion
In the present study, four statistical models were assessed to evaluate the sensitivity of a meta-analysis aimed to compare the efficacy of DSSs to the standard calendar-based fungicide strategies for the control of fungal diseases in crop plants. Statistical modelling was set comparing "fixed-effects" and "random-effects" models in the framework of GLMs with frequentist and Bayesian inferential methods. Our results showed that the different treatment effect measures considered (disease incidence, OR, and IR) are highly sensitive to the integration of random-effects which may lead to different conclusions. These differences between "fixed-effects" and "random-effects" models were reproduced in both inferential methods.
Our assessments based on AIC or DIC scores and the graphical analysis of discrepancies between observed and estimated values confirmed that the dataset was better described assuming variable treatment effects among the experiments. Conceptually, the inclusion of random treatment effects was in line with the intrinsic nature of the data, which were gathered from experiments carried out by researchers operating independently. Our analyses showed that variance parameters were higher than 1 (Tables 1 and 2), indicating a substantial heterogeneity among experiments [32,33].
Our results also indicated that the use of "fixed-effects" models may lead to a strong underestimation of the uncertainty associated with the treatment effects measures considered (incidence, OR, IR) to summarise treatment efficacies. For both GLMMs, fequentist and Bayesian, the width of the uncertainty measures (confidence/credible intervals) was larger than in their GLM counterparts, due to the lack of correlation between sub-experiments observations that it is implicit in "fixed-effects" specification. Since all sub-experiments within an experiment were evaluated under the same local conditions, they cannot be considered independent. Thus, models ignoring this correlation (i.e., "fixed-effects"model), overestimate the number of independent data and thus underestimate the level of uncertainty as noted by Makowski et al. [5].
Models with "random-effects" allow deriving predictive distributions, which enable making more robust conclusions as indicated for the consideration of confidence/credible intervals, but also by the prediction intervals [33]. This interesting feature was included in our work by the derivation of the predictive distribution for the OR. For both GLMMs, frequentist and Bayesian, the estimation of treatment effects measures (disease incidence, OR, and IR) was obtained from the conditional median/posterior distribution (for Bayesian models) of the random experiment effect in which between-experiment variability for the different treatments was not considered. Predictive distributions allow the incorporation of these sources of variability and obtain a more appropriate treatment effect summary than the average [32]. Consequently, the use of predictions gained relevance in recent meta-analyses, becoming a standard statistical output due to its utility to assess heterogeneity and uncertainty of treatment effects in target populations [1,34,35]. According to the predictive distribution for the OR, both fungicide strategies showed to be equally effective in reducing disease incidence compared with the untreated control.
Frequentist and Bayesian models for both "fixed-effects" and "random-effects" produced almost identical results for disease incidence, OR and IR. Our results also highlighted the importance of properly interpret OR and IR as relative measures. These measures do not have an absolute interpretation and tend to be insensitive to differences in baseline events, so it could be recommendable to consider an absolute measure such as incidence estimates to support their interpretation. According to the previous studies of [5,32,36], our results confirmed that under a weakly informative prior scenario, frequentist and Bayesian models perform similarly. However, this was not the case of the predictive distribution for the OR which slightly varied between the two methods, although they lead to the same conclusions. The problem of finding reasonably good methods for prediction in both frequentist and Bayesian methods is an open research field, thus it is necessary to check the performance of prediction methods in practical meta-analysis and to interpret them with caution.
Our results indicated that the both inferential methodologies evaluated had similar practical advantages, but each of them with its pros and its cons. In general, the Bayesian approach is conceptually more straightforward and addresses uncertainty in a more comprehensive and interpretable way. Posterior distributions (parameter or derived quantity) can be characterised in terms of probability. Thus, a 95% credible interval is simply the central portion of the posterior distribution that contains the 95% of parameter values. By contrast, frequentist inference does not allow probability statements about the parameters, the p-value is a measure of evidence against a null hypothesis when it is assumed to be true. Thus, a 95% confidence interval means that if the same procedure to construct confidence intervals was repeated many times, then in 95% of the cases the true value will lie within the interval.
The Bayesian approach also makes easier to extend simple models to more complex formulations [37]. Currently, there are several software alternatives available such as WinBUGS [38], JAGS [39], INLA [40], Stan [41], which allow implementing those complex models much easily than in the frequentist framework. By contrast, the requirement in the Bayesian context of setting a prior specification for each parameter could be a weakness in the inference process. In some non-informative prior scenarios, different prior specifications may lead to different posterior distributions affecting inference robustness [42]. Moreover, in some cases the computational demand of Bayesian models could be extremely intensive. Thus, as a general recommendation, inferential method choice in meta-analysis should address and balance all the issues above.
In the context of fungal diseases, DSS strategies allow growers to estimate the risk of disease and devise more efficient fungicide spray programs, integrating different sources of information for control decisions including action thresholds [11]. However, currently there is a limited degree of adoption of DSSs due in part to growers' aversion to risks and a perceived lack of reliability of DSSs [13,43]. Building on numerous individual field trials conducted to evaluate DSSs in a wide range of disease-crop systems, our meta-analysis demonstrates that DSS-based programs are effective in reducing the incidence of fungal diseases with efficacy comparable to calendar-based fungicide programs. Statistical inference comparing the number of fungicide sprays in DSS-based and calendar-based strategies will be the topic of further research.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript:  Table A1. List of papers used in the meta-analysis, source and search string.