Comparison of Frequentist and Bayesian Meta-Analysis Models for Assessing the Efficacy of Decision Support Systems in Reducing Fungal Disease Incidence

Abstract

Diseases of fruit and foliage caused by fungi and oomycetes are generally controlled by the application of fungicides. The use of decision support systems (DSSs) may assist to optimize fungicide programs to enhance application on the basis of risk associated with disease outbreak. Case-by-case evaluations demonstrated the performance of DSSs for disease control, but an overall assessment of the efficacy of DSSs is lacking. A literature review was conducted to synthesize the results of 67 experiments assessing DSSs. Disease incidence data were obtained from published peer-reviewed field trials comparing untreated controls, calendar-based and DSS-based fungicide programs. Two meta-analysis generic models, a “fixed-effects” vs. a “random-effects” model within the framework of generalized linear models were evaluated to assess the efficacy of DSSs in reducing incidence. All models were fit using both frequentist and Bayesian estimation procedures and the results compared. Model including random effects showed better performance in terms of AIC or DIC and goodness of fit. In general, the frequentist and Bayesian approaches produced similar results. Odds ratio and incidence ratio values showed that calendar-based and DSS-based fungicide programs considerably reduced disease incidence compared to the untreated control. Moreover, calendar-based and DSS-based programs provided similar reductions in disease incidence, further supporting the efficacy of DSSs.

1. Introduction

According to Higgins et al. [1], “meta-analysis refers to the statistical synthesis of results from a series of studies”. In plant pathology, meta-analysis has become in a powerful tool  [2] to address questions such as factors determining the effects of different pesticides and biological treatments for managing diseases [3,4,5]. Meta-analysis implementation requires the selection of proper statistical models to draw robust conclusions. Different types of statistical models were proposed with that regard; however, there is a risk of biased parameter estimation, misinterpretation and incorrect conclusions if a quality control of statistical techniques is not considered [6]. According to Philibert  et  al. [7], one of the quality control measures which meta-analysis should incorporate is a sensitivity study of the estimated effects in relation to the statistical model characteristics.

In practice, the selection of a statistical model rises two important issues: (i) the nature of the measured variable (continuous, binary, categorical) and (ii) the consideration between-experiments variability in the measured variable which leads to the choice between fixed and random effects models [1,8,9]. Furthermore, model type will determine the “treatment effect” (also named“effect size”) choice to summary effects and assess differences between treatments evaluated. In plant pathology, meta-analysis sensitivity analyses addressed differences between fixed-effect and random-effect models within different modelling frameworks (linear models, generalized linear models)  [3,5] and also for frequentist vs. Bayesian inferential approaches [5].

In this paper, we consider experiments where the measured variable is binary and corresponds to the occurrence of a disease in treated/untreated plant materials. Within this framework, a sensitivity analysis between two different meta-analysis models (fixed effects vs. random effects) and two different estimation methods (frequentist vs. Bayesian) were proposed from a generic generalized lineal model (GLM) [10] to assess the efficacy of fungicide programs based on decision support systems (DSSs) in comparison to standard calendar-based programs.

Currently, agricultural policies promote the adoption of more sustainable low-input agricultural systems reinforcing the reduction of fungicide application among other measures. In that context, the use of predictive models (empirical or process-based) allows estimating the risk of disease and devise more efficient fungicide spray programs optimizing spray timing and avoiding unnecessary treatments  [11,12]. Building on predictive models, DSS programs integrate all types of information required for control decisions, including action thresholds [11]. However, the level of adoption of them is generally low [13]. The perceived lack of reliability of DSSs could be overcome with more studies comparing the efficacy of this strategy with the standard calendar-based fungicide programs. In this context, meta-analysis is a powerful methodology in the sense that can combine different sources of information covering a wide range of disease-crop systems.

In this paper, we compare four different GLMs to perform a meta-analysis on the efficacy of DSS and calendar fungicide programs. The four models differ in the assumption made on the variability of the treatment effect (which is assumed to be either constant or variable between experiments) and in the inferential method used for parameter estimation (frequentist or Bayesian). For all models, three treatment effects measures: disease incidence, odds ratios and incidence ratios were estimated and compared to evaluate the efficacy of DSS and calendar fungicide programs.

2. Material and Methods

2.1. Literature Search and Data

A database was assembled based on a systematic literature review from the (i) Web of Science (WOS) and (ii) the Fungicide and Nematicide Tests (F&N Tests and Plant Disease management reports). For (i), publications were selected according to multiple search strings described in Appendix A. For (ii), reports were extracted according to multiple key words (Appendix A). Hard copies of volumes published prior to 2000 were examined directly. Relevant experiments were selected according the following criteria: (a) the experiment evaluated at least one DSS-based strategy, one calendar-based strategy and an untreated control (as  a minimum each experiment included three sub-experiments according to the type of treatment); (b)  all sub-experiments reported disease incidence (i.e., proportion of diseased organs) and sample size (i.e., total of organs considered to evaluate disease incidence). All experiments fulfilling those criteria were included in the meta-analysis. In calendar-based programs the number and timing of fungicide applications was fixed before the experiment, usually based on the standard practices for disease control. In DSS-based programs the number and timing of fungicide applications was decided during the course of the experiment based on risk indicators.

Each experiment was defined as a unique combination of location (country, state), year, crop, organ evaluated, disease, pathogen, treatment strategy, number of sprays and treatment id (see 

Table A2. List of papers used in the meta-analysis. Papers are identified by its corresponding code  id. List of experiments used in the meta-analysis related to its corresponding paper. Experiments are characterised by an id and its corresponding location, crop type and disease *.

Paper	Experiment
Reference	Id	Location	Crop	Disease *
Brown-Rytlewski et al. [44]	A1	Ohio, US	Wheat	Fusarium head blight
Brown-Rytlewski et al. [45]	B1	Michigan, US	Wheat	Fusarium head blight
Brown-Rytlewski et al. [46]	C1	Michigan, US	Wheat	Fusarium head blight
Brown-Rytlewski et al. [47]	D1	Michigan, US	Wheat	Fusarium head blight
Babadoost [48]	E1	California, US	Apple	Sooty blotch complex
	E2	California, US	Apple	Flyspeck
Babadoost [49]	F1	California, US	Apple	Sooty blotch complex
	F2	California, US	Apple	Flyspeck
Gleason et al. [50]	G1	Iowa, US	Apple	Sooty blotch complex
	G2	Iowa, US	Apple	Flyspeck
Hovius and McDonald [51]	H1	Ontario, CA	Lettuce	Downy mildew
	H2	Ontario, CA	Lettuce	Downy mildew
Mcdonald et al. [52]	I1	Ontario, CA	Lettuce	Downy mildew
Averre et al. [53]	J1	North Carolina, US	Asparagus	Cercospora blight
Llorente et al. [54]	K1	Emilia-Romagna, IT	Pear	Brown spot
	K2	Girona, ES	Pear	Brown spot
	K3	Emilia-Romagna, IT	Pear	Brown spot
	K4	Girona, ES	Pear	Brown spot
	K5	Girona, ES	Pear	Brown spot
	K6	Emilia-Romagna, IT	Pear	Brown spot
	K7	Girona, ES	Pear	Brown spot
	K8	Girona, ES	Pear	Brown spot
	K9	Girona, ES	Pear	Brown spot
	K10	Girona, ES	Pear	Brown spot
	K11	Emilia-Romagna, IT	Pear	Brown spot
Bhatia et al. [55]	L1	Florida, US	Mandarin	Alternaria brown spot
	L2	Florida, US	Mandarin	Alternaria brown spot
	L3	Florida, US	Mandarin	Alternaria brown spot
	L4	Florida, US	Mandarin	Alternaria brown spot
Byrne et al. [56]	M1	Michigan, US	Tomato	Anthracnose
	M2	Michigan, US	Tomato	Anthracnose
	M3	Indiana, US	Tomato	Anthracnose
	M4	Michigan, US	Tomato	Anthracnose
	M5	Michigan, US	Tomato	Anthracnose
	M6	Indiana, US	Tomato	Anthracnose
	M7	Indiana, US	Tomato	Anthracnose
Montesinos et al. [57]	N1	Girona, ES	Pear	Brown spot
	N2	Girona, ES	Pear	Brown spot
Peres and Timmer [58]	O1	São Paulo, BR	Mandarin	Alternaria brown spot
	O2	São Paulo, BR	Mandarin	Alternaria brown spot
	O3	São Paulo, BR	Mandarin	Alternaria brown spot
Wu et al. [59]	P1	California, US	Lettuce	Downy mildew
	P2	California, US	Lettuce	Downy mildew
	P3	California, US	Lettuce	Downy mildew
Louws et al. [60]	Q1	Michigan, US	Tomato	Early blight
	Q2	Michigan, US	Tomato	Early blight
	Q3	Michigan, US	Tomato	Early blight
	Q4	Michigan, US	Tomato	Anthracnose
	Q5	Michigan, US	Tomato	Anthracnose
	Q6	Michigan, US	Tomato	Anthracnose
	Q7	Michigan, US	Tomato	Rhizoctonia fruit rot
	Q8	Michigan, US	Tomato	Rhizoctonia fruit rot
	Q9	Michigan, US	Tomato	Rhizoctonia fruit rot
Rasiukevivciute et al. [61]	R1	Kaunas, LT	Strawberry	Gray mold
	R2	Kaunas, LT	Strawberry	Gray mold
	R3	Kaunas, LT	Strawberry	Gray mold
	R4	Kaunas, LT	Strawberry	Gray mold
	R5	Kaunas, LT	Strawberry	Gray mold
	R6	Kaunas, LT	Strawberry	Gray mold
Rosli et al. [62]	S1	Iowa, US	Apple	Sooty blotch complex /Flyspeck
	S2	Iowa, US	Apple	Sooty blotch complex /Flyspeck
	S3	Iowa, US	Apple	Sooty blotch complex /Flyspeck
	S4	Iowa, US	Apple	Sooty blotch complex /Flyspeck
	S5	Iowa, US	Apple	Sooty blotch complex /Flyspeck
	S6	Iowa, US	Apple	Sooty blotch complex /Flyspeck
	S7	Iowa, US	Apple	Sooty blotch complex /Flyspeck
	S8	Iowa, US	Apple	Sooty blotch complex /Flyspeck

* APS (American Phytopathological Society) Common Names of Plant Diseases. https://www.apsnet.org/dcenter/resources/commonnames/Pages/default.aspx Accessed 08/03/20.

for detailed information about the location, crop and disease information). Conversely, each  sub-experiment was characterised by the type of treatment (DSS, calendar or untreated), the  observed disease incidence, the sample size and the number of sprays. For both sources ((i) and (ii)) a total of 67 experiments, 285 sub-experiments were selected from a total of 19 publications/reports (see

Table A3. List of papers used in the meta-analysis. Papers are identified by its corresponding reference and a code id. List of experiments used in the meta-analysis related to its corresponding paper. Experiments are characterised by an id and the number of sub-experiments for each fungicide strategy (untreated, calendar and DSS).

Paper		Experiment	Sub-Experiments
Reference	Id	Id	Untreated	Calendar	DSS
Brown-Rytlewski et al. [44]	A	A1	1	1	2
Brown-Rytlewski et al. [45]	B	B1	1	1	2
Brown-Rytlewski et al. [46]	C	C1	1	1	2
Brown-Rytlewski et al. [47]	D	D1	1	1	2
Babadoost [48]	E	E1	1	2	2
		E2	1	2	2
Babadoost [49]	F	F1	1	2	2
		F2	1	2	2
Gleason et al. [50]	G	G1	1	1	4
		G2	1	1	4
Hovius and McDonald [51]	H	H1	1	1	1
		H2	1	1	1
McDonald et al. [52]	I	I1	1	1	1
Averre et al. [53]	J	J1	1	1	1
	K	K1	1	1	2
		K2	1	1	1
		K3	1	1	1
		K4	1	1	2
		K5	1	1	2
Llorente et al. [54]		K6	1	1	1
		K7	1	1	2
		K8	1	1	1
		K9	1	1	1
		K10	1	1	1
		K11	1	1	1
Bhatia et al. [55]	L	L1	1	1	3
		L2	1	1	3
		L3	1	1	3
		L4	1	1	3
	M	M1	1	3	1
		M2	1	3	1
		M3	1	3	1
Byrne et al. [56]		M4	1	3	1
		M5	1	3	1
		M6	1	3	1
		M7	1	3	1
Montesinos et al. [57]	N	N1	1	1	3
		N2	1	1	2
	O	O1	1	1	3
Peres and Timmer [58]		O2	1	1	2
		O3	1	2	2
	P	P1	1	1	1
Wu et al. [59]		P2	1	1	2
		P3	1	1	2
	Q	Q1	1	1	4
		Q2	1	1	4
		Q3	1	1	5
		Q4	1	1	4
Louws et al. [60]		Q5	1	1	4
		Q6	1	1	5
		Q7	1	1	4
		Q8	1	1	4
		Q9	1	1	5
Rasiukevivciute et al. [61]	R	R1	1	1	1
		R2	1	1	1
		R3	1	1	1
		R4	1	1	1
		R5	1	1	1
		R6	1	1	1
Rosli et al. [62]	S	S1	1	1	1
		S2	1	1	1
		S3	1	1	1
		S4	1	1	1
		S5	1	1	1
		S6	1	1	1
		S7	1	1	1
		S8	1	1	1
TOTAL			67	86	132

for further details). The number of independent experiments among the papers varied from 1 to 11 and the number of sub-experiments among experiments from 3 to 7 (Table A3). The database included a total of 67 sub-experiments under untreated conditions, 86 under a calendar-based strategy and 132 under DSS-based strategy (Table A3). An overall and individual (per sub-experiment) description of the dataset is provided in Section 3.1 in terms of the observed disease incidence and incidence ratios (IRs) between calendar (Cal./Unt.) and DSS (DSS/Unt.) strategies against the untreated control.

2.2. Meta-Analysis

The response variable used to quantify the level of disease in all experiments (sub-experiments) was the number of diseased organs. Given the nature of this metric, meta-analysis was formulated within the familiar framework of GLMs [10] and applied to data with a binomial likelihood [14] and a logit link function. The GLM overcomes problems of models based on normal likelihood [15]. It was formulated to compare the effect of the two fungicide strategies (DSS and calendar) simultaneously in comparison to the untreated control.

This generic model was extended by the inclusion of random effects usually named as generalised linear mixed model (GLMM) [16] with the aim to account for the unobserved sources of variability among experiments beyond the fungicide treatment effects, such as the different pathogens/crops targeted, the mode of action of the fungicides, and inoculum levels. Both GLM and GLMM were fitted using two statistical methods, a frequentist and a Bayesian method. All implementations were made in the R environment (version 3.5.1). The full analysis can be reproduced using code and data archived at https://bitbucket.org/elaher/comparative-meta-analysis/src/master/.

Statistical Modelling

The logistic GLM was formulated as:

$$\begin{array}{cc}\hfill & Y_{ij}\sim \mathrm{Binomial}(n_{ij},{\theta }_{ij}),\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill & \mathrm{logit}\left({\theta }_{ij}\right)=\log \left({\displaystyle \frac{{\theta }_{ij}}{1-{\theta }_{ij}}}\right)={\beta }_{\mathtt{0}}+{\beta }_{\mathtt{cal}}{I}_{\mathtt{cal}\left(\mathtt{ij}\right)}+{\beta }_{\mathtt{dss}}{I}_{\mathtt{dss}\left(\mathtt{ij}\right)},\hfill \end{array}$$

(2)

where the random variable $Y_{ij}$ which describes the number of diseased organs in the sub-experiment j of the experiment i out of a total of $n_{ij}$ organs analyzed, is assumed to follow a Binomial distribution with unknown probability of disease ${\theta }_{ij}$ (disease incidence); $I_{\mathtt{cal}\left(\mathtt{ij}\right)}$ and $I_{\mathtt{dss}\left(\mathtt{ij}\right)}$ are dummy variables for calendar and DSS treatments equal to 1 if the sub-experiment j of the experiment i was conducted with one of these strategies and to zero otherwise; ${\beta }_{\mathtt{cal}}$ and ${\beta }_{\mathtt{dss}}$ are fixed parameters (regression coefficients) which capture calendar and DSS treatment effect compared to the untreated control, ${\beta }_{\mathtt{0}}$. This model formulation is usually denoted in meta-analysis framework as “fixed-effects” model [5,9] due to it assumes that each treatment effect does not vary across experiments.

The logistic GLMM was formulated as an extension of the GLM defined previously by the inclusion of three random effects describing the between-experiment variability of the treatment effects. In the GLMM model, Equation (2) is modified as follows:

$$\begin{array}{cc}\hfill \mathrm{logit}\left({\theta }_{ij}\right)=\log \left({\displaystyle \frac{{\theta }_{ij}}{1-{\theta }_{ij}}}\right)& =({\beta }_{\mathtt{0}}^{*}+b_{\mathtt{0}\left(\mathtt{i}\right)})+({\beta }_{\mathtt{cal}}^{*}+b_{\mathtt{cal}\left(\mathtt{i}\right)})I_{\mathtt{cal}\left(\mathtt{ij}\right)}+({\beta }_{\mathtt{dss}}^{*}+b_{\mathtt{dss}\left(\mathtt{i}\right)})I_{\mathtt{dss}\left(\mathtt{ij}\right)},\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & ={\gamma }_{\mathtt{0}\left(\mathtt{i}\right)}+{\gamma }_{\mathtt{cal}\left(\mathtt{i}\right)}{I}_{\mathtt{cal}\left(\mathtt{ij}\right)}+{\gamma }_{\mathtt{dss}\left(\mathtt{i}\right)}{I}_{\mathtt{dss}\left(\mathtt{ij}\right)},\hfill \end{array}$$

(4)

where ${\gamma }_{\mathtt{cal}\left(\mathtt{i}\right)}$, ${\gamma }_{\mathtt{dss}\left(\mathtt{i}\right)}$ are random parameters which capture calendar and DSS treatment effect for each experiment i in relation to the untreated control, ${\gamma }_{\mathtt{0}\left(\mathtt{i}\right)}$. Each individual experiment effect is assumed to be multivariate normally distributed about the means ${\beta }_{\mathtt{0}}^{*}$, ${\beta }_{\mathtt{cal}\left(\mathtt{i}\right)}^{*}$, ${\beta }_{\mathtt{dss}\left(\mathtt{i}\right)}^{*}$ with a 3 × 3 variance-covariance matrix $\mathsf{\Sigma }$, defined as follows:

$$\left[\begin{array}{c}{\gamma }_{\mathtt{0}\left(\mathtt{i}\right)}\hfill \\ {\gamma }_{\mathtt{cal}\left(\mathtt{i}\right)}\hfill \\ {\gamma }_{\mathtt{dss}\left(\mathtt{i}\right)}\hfill \end{array}\right]\sim N_3\left\{\left[\begin{array}{c}{\beta }_{\mathtt{0}}^{*}\\ {\beta }_{\mathtt{cal}}^{*}\\ {\beta }_{\mathtt{dss}}^{*}\end{array}\right],\left[\begin{array}{ccc}{\sigma }_{\mathtt{0}}^2& {\sigma }_{\mathtt{0},\mathtt{cal}}& {\sigma }_{\mathtt{0},\mathtt{dss}}\\ {\sigma }_{\mathtt{0},\mathtt{cal}}& {\sigma }_{\mathtt{cal}}^2& {\sigma }_{\mathtt{cal},\mathtt{dss}}\\ {\sigma }_{\mathtt{0},\mathtt{dss}}& {\sigma }_{\mathtt{cal},\mathtt{dss}}& {\sigma }_{\mathtt{dss}}^2\end{array}\right]\right\}.$$

(5)

The means (fixed parameters), ${\beta }_{\mathtt{cal}}^{*}$ and ${\beta }_{\mathtt{dss}}^{*}$, in constrast to GLM interpretation, describe  the average effects of calendar and DSS treatments within the population of experiments in comparison to the untreated control, ${\beta }_{\mathtt{0}}^{*}$. Their corresponding random effects, $b_{\mathtt{cal}\left(\mathtt{i}\right)}$ and $b_{\mathtt{dss}\left(\mathtt{i}\right)}$, capture individual experiment calendar and DSS treatment effects in relation to the untreated control, $b_{\mathtt{0}\left(\mathtt{i}\right)}$. Thus, under GLMM specification, each treatment effect varies from experiment to experiment i. The variances, ${\sigma }_{\mathtt{0}}^2$, ${\sigma }_{\mathtt{cal}}^2$ and ${\sigma }_{\mathtt{dss}}^2$ capture the extent of the above mentioned variability while covariances, ${\sigma }_{\mathtt{0},\mathtt{cal}}$, ${\sigma }_{\mathtt{0},\mathtt{dss}}$, ${\sigma }_{\mathtt{cal},\mathtt{dss}}$ point correlations between sub-experiments belonging to the same experiment. This model is usually called the “random-effects” model [1] in the context of meta-analysis.

2.3. Parameter Estimation: Frequentist vs. Bayesian Approach

Inference for both models (GLM and GLMM) was carried out using frequentist and Bayesian statistics, successively, leading to two different sets of estimated parameters. Frequentist models were denoted as GLM_F and GLMM_F and their Bayesian counterparts as GLM_B and GLMM_B. Parameters of frequentist models were estimated by maximum likelihood through iterative reweighted least squares method and Laplace approximation using the glm() and the glmer() functions of the package lme4 [17] implemented in the R software [18] version 3.5.1, respectively. Likelihood ratio tests were performed to assess the significance of the fungicide treatment effects. Model comparison was done using the Akaïke Information Criterion (AIC) [19], smaller values of AIC correspond to preferred models. A rule of thumb outlined in Burnham and Anderson [20] is that models with ${\Delta }_i\left(\mathrm{AIC}\right)={\mathrm{AIC}}_i-\min \mathrm{AIC}$ higher than 10 have no support against a model with minimum AIC  value.

In the Bayesian approach, uncertainty about quantities of interest and experimental results is always expressed in probabilistic terms. Inferential processes for learning about a quantity of interest $\varphi$ always start with a prior distribution which contains all relevant information about $\varphi$, $\pi \left(\varphi \right)$. Experimental data are related to $\varphi$ via a sampling model (i.e, binomial model) which is the basis for computing the likelihood function $\mathrm{L}\left(\varphi \right)$. Both elements are formally combined by means of the Bayes’ rule to obtain the posterior distribution for $\varphi$, $\pi (\varphi \mid \mathrm{Data})$, which synthesize all the available knowledge about $\varphi$, $\pi (\varphi \mid \mathrm{Data})\propto \mathrm{L}\left(\varphi \right)\pi \left(\varphi \right)$.

Bayesian simple inferential processes are based on conjugate families for which the prior and the posterior distribution belongs to the same distribution family. Under this specific context the posterior distribution can be calculated analytically. However, for more complex models such as ours, it  needs to be approximated with numerical methods such as Markov chain Monte Carlo (MCMC) methods [21]. Then, the posterior distribution of the inference parameters is described by a random sample of parameter values. The same sample of parameters can be used to approximate the posterior distribution of any quantity of interest, for examples, the disease incidence, the odds ratio (OR), the incidence ratio (IR), etc. All these distributions can be summarised using several point estimates such as the mean, the median, the mode. Furthermore, Bayesian approach makes possible to quantify uncertainty of any posterior distribution by means of credible intervals [22].

GLM_B and GLMM_B inferential processes were performed under several independent prior scenarios. For the GLM_B, following the recommendation of Gelman et al. [23] a weakly informative prior scenario with Cauchy distributions was considered. Specifically, the prior of ${\beta }_{\mathtt{0}}$ was defined by a Cauchy distribution with location parameter $\mu$ fixed at 0 and a scale parameter $\sigma$ of 10, ($\mathrm{C}(0,10)$). Conversely, the priors of ${\beta }_{\mathtt{dss}}$ and ${\beta }_{\mathtt{cal}}$ were defined by Cauchy distributions also centered at 0 with scale parameter $\sigma =2.5$, ($\mathrm{C}(0,2.5)$). This default prior scenario implies that model fitting uses an adaptation of the standard iteratively weighted least squares computation [23] which makes similar this model with its frequentist counterpart. In GLMM_B, a standard non-informative prior scenario was defined to give prominence to data and also to make it comparable with GLMM_F. The prior for ${\beta }_{\mathtt{0}}^{*}$, ${\beta }_{\mathtt{dss}}^{*}$ and ${\beta }_{\mathtt{cal}}^{*}$ was defined independently by means of normal distributions centered at 0 and with a wide variance ($\mathrm{N}(0,1000)$). An inverse Wishart distribution ($\mathrm{IW}(\mathsf{\Psi },\nu )$) was considered to specify the prior to the variance-covariance matrix, $\mathsf{\Sigma }$. Specifically, the inverse Wishart distribution is defined with an $r\times r$ scale matrix $\mathsf{\Psi }$ with r equal to the number of random parameters, and with several degrees of freedom $\nu$. $\mathsf{\Psi }$ was specified as an identity matrix (values of 1 in the diagonal and 0 otherwise) of $3\times 3$ and $\nu =3$. For both models, posterior distribution was approximated by means of Markov chain Monte Carlo (MCMC) simulation methods by means of the JAGS software (version 4.3.0) through the R2jags package (version 0.5-7) [24] of the R software.

The MCMC algorithm was run with three Markov chains each including 120,000 iterations after a burn-in period of 20,000 iterations. In addition, the chains were thinned by storing one in ten iterations in order to reduce autocorrelation in the subsequent sample. Convergence was assessed via three different criteria: (i) graphically, drawing trace plots and assessing the simulated values of the chains appear overlapping one another, (ii) based on the potential scale reduction factor, $\widehat{R}$, whose values must be equal or close to 1, and (iii) by means of the effective number of independent simulation draws, $neff$, which must be >100 [25]. Regarding methods for Bayesian model choice, the deviance information criterion (DIC) [26] was considered. Smaller values of DIC correspond to preferred models, and a DIC difference of 5 or more is generally regarded as practically meaningful [27].

In addition to the standard model selection criteria (AIC or DIC), the goodness of fit for the different models was evaluated graphically comparing observed vs. fitted disease incidence in the logit scale. Note that for Bayesian models, logit fitted values were based on the median of the posterior distribution of the logit disease incidence.

2.4. Treatment Effects: Disease Incidence, Odds Ratio, Incidence Ratio and Predictive Distribution of the Odds  Ratio

Effect measures were assessed by the estimation of disease incidence, OR and IR for the four models [28,29]. With the “random-effects” models (GLMM), these  quantities are assumed to vary across experiments and this variability was described by computing the predictive distribution for a new experiment chosen at random.

2.4.1. Disease Incidence

For GLM_F models, disease incidence (in probability scale) was estimated for each specific fungicide strategy (untreated, calendar and DSS) according Equation (2) notation as follows:

$${\widehat{\theta }}_0=\frac{\exp \left\{{\widehat{\beta }}_{\mathtt{0}}\right\}}{1+\exp \left\{{\widehat{\beta }}_{\mathtt{0}}\right\}},{\widehat{\theta }}_{cal}=\frac{\exp \{{\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{cal}}\}}{1+\exp \{{\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{cal}}\}},{\widehat{\theta }}_{dss}=\frac{\exp \{{\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{dss}}\}}{1+\exp \{{\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{dss}}\}},$$

(6)

in which ${\widehat{\beta }}_{\mathtt{0}}$, ${\widehat{\beta }}_{\mathtt{cal}}$ and ${\widehat{\beta }}_{\mathtt{dss}}$ correspond to the maximum likelihood parameter estimates. Their  associated 95% confidence intervals were calculated according to

$$\begin{array}{cc}\hfill 95\%{\mathrm{CI}}_{{\widehat{\theta }}_0}& =\mathrm{ino}.\mathrm{logit}\{{\widehat{\beta }}_{\mathtt{0}}\pm {\mathrm{z}}_{0.975}\mathrm{SE}\left({\widehat{\beta }}_{\mathtt{0}}\right)\}\hfill \\ \hfill 95\%{\mathrm{CI}}_{{\widehat{\theta }}_{cal}}& =\mathrm{inv}.\mathrm{logit}\{({\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{cal}})\pm {\mathrm{z}}_{0.975}\mathrm{SE}({\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{cal}})\}\hfill \\ \hfill 95\%{\mathrm{CI}}_{{\widehat{\theta }}_{dss}}& =\mathrm{inv}.\mathrm{logit}\{({\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{dss}})\pm {\mathrm{z}}_{0.975}\mathrm{SE}({\widehat{\beta }}_{\mathtt{0}}+{\widehat{\beta }}_{\mathtt{dss}})\}.\hfill \end{array}$$

(7)

in which ${\mathrm{z}}_{0.975}$ denotes the 0.975 quantile of the normal distribution and SE the standard error. Note  that disease incidence computation was calculated in the logit scale due to errors are assumed normally distributed and then logit-transformed ($\mathrm{inv}.\mathrm{logit}\left(x\right)=\frac{\exp \left\{x\right\}}{1+\exp \left\{x\right\}}$) [30]. Generally, $\mathrm{SE}=\sqrt{\widehat{\mathrm{Var}}}$ with $\widehat{\mathrm{Var}}$ denoting estimated variance.

For GLM_B, posterior distribution of the disease incidence was drawn from the posterior distribution of the model parameters following the tranformation described in Equation (6) and summarized by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.

For GLMM_F model, average “experiment” disease incidence was computed for a each specific fungicide strategy (untreated, calendar and DSS) according Equation (4) notation as follows:

$${\widehat{\theta }}_{0M}=\frac{\exp \left\{{\widehat{\beta }}_{\mathtt{0}}^{*}\right\}}{1+\exp \left\{{\widehat{\beta }}_{\mathtt{0}}^{*}\right\}},{\widehat{\theta }}_{calM}=\frac{\exp \{{\widehat{\beta }}_{\mathtt{0}}^{*}+{\widehat{\beta }}_{\mathtt{cal}}^{*}\}}{1+\exp \{{\widehat{\beta }}_{\mathtt{0}}^{*}+{\widehat{\beta }}_{\mathtt{cal}}^{*}\}}{\widehat{\theta }}_{dssM}=\frac{\exp \{{\widehat{\beta }}_{\mathtt{0}}^{*}+{\widehat{\beta }}_{\mathtt{dss}}^{*}\}}{1+\exp \{{\widehat{\beta }}_{\mathtt{0}}^{*}+{\widehat{\beta }}_{\mathtt{dss}}^{*}\}}.$$

(8)

As with the GLM_F computation, 95% confidence intervals were estimated following the same approach described in Equation (7) but adapting it to the parameters involved. For GLMM_B, posterior distribution of the average the disease incidence was drawn from the transformation of the posterior distribution of the model parameters involved and summarised by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.

2.4.2. Odds Ratio

Based on the formal definition of the disease incidence described previously, OR for calendar (OR${}_{cal}$) and DSS (OR${}_{dss}$)-based strategies against control as well as OR for calendar against DSS (OR${}_{cal/dss}$) were estimated for GLM_F model as follows:

$$\begin{array}{cc}\hfill {\widehat{\mathrm{OR}}}_{cal}& ={\displaystyle \frac{{\widehat{\theta }}_{cal}/(1-{\widehat{\theta }}_{cal})}{\widehat{{\theta }_0}/(1-{\widehat{\theta }}_0)}}=\exp \left\{{\widehat{\beta }}_{\mathtt{cal}}\right\}\hfill \\ \hfill {\widehat{\mathrm{OR}}}_{dss}& ={\displaystyle \frac{{\widehat{\theta }}_{dss}/(1-{\widehat{\theta }}_{dss})}{{\widehat{\theta }}_0/(1-{\widehat{\theta }}_0)}}=\exp \left\{{\widehat{\beta }}_{\mathtt{dss}}\right\}\hfill \\ \hfill {\widehat{\mathrm{OR}}}_{cal/dss}& ={\displaystyle \frac{{\widehat{\theta }}_{cal}/(1-{\widehat{\theta }}_{cal})}{{\widehat{\theta }}_{dss}/(1-{\widehat{\theta }}_{dss})}}=\exp \{{\widehat{\beta }}_{\mathtt{cal}}-{\widehat{\beta }}_{\mathtt{dss}}\}\hfill \end{array}$$

(9)

Note the OR computation was carried out on a log scale [31]. Thus, based on the previous statement 95% confidence intervals were estimated according to

$$\begin{array}{cc}\hfill 95\%{\mathrm{CI}}_{{\widehat{\mathrm{OR}}}_{cal}}& =\exp \{{\widehat{\beta }}_{\mathtt{cal}}\pm {\mathrm{z}}_{0.975}\mathrm{SE}\left({\widehat{\beta }}_{\mathtt{cal}}\right)\}\hfill \\ \hfill 95\%{\mathrm{CI}}_{{\widehat{\mathrm{OR}}}_{dss}}& =\exp \{{\widehat{\beta }}_{\mathtt{dss}}\pm {\mathrm{z}}_{0.975}\mathrm{SE}\left({\widehat{\beta }}_{\mathtt{dss}}\right)\}\hfill \\ \hfill 95\%{\mathrm{CI}}_{{\widehat{\mathrm{OR}}}_{cal/dss}}& =\exp \{({\widehat{\beta }}_{\mathtt{cal}}-{\widehat{\beta }}_{\mathtt{dss}})\pm {\mathrm{z}}_{0.975}\mathrm{SE}({\widehat{\beta }}_{\mathtt{cal}}-{\widehat{\beta }}_{\mathtt{dss}})\}\hfill \end{array}$$

(10)

in which ${\mathrm{z}}_{0.975}$ denotes the 0.975 quantile of the normal distribution and SE the standard error.

For GLM_B, posterior distribution of the OR was drawn from the posterior distribution of the relevant model parameters based on the transformation described in Equation (9) and summarized by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.

Under GLMM_F model, average “experiment” OR for calendar and DSS-based strategy against the untreated control and OR for calendar against DSS were computed as follows:

$$\begin{array}{cc}\hfill {\widehat{\mathrm{OR}}}_{calM}& ={\displaystyle \frac{{\widehat{\theta }}_{calM}/(1-{\widehat{\theta }}_{calM})}{{\widehat{\theta }}_{0M}/(1-{\widehat{\theta }}_{0M})}}=\exp \left\{{\widehat{\beta }}_{\mathtt{cal}}^{*}\right\}\hfill \\ \hfill {\widehat{\mathrm{OR}}}_{dssM}& ={\displaystyle \frac{{\widehat{\theta }}_{dssM}/(1-{\widehat{\theta }}_{dssM})}{{\widehat{\theta }}_{0M}/(1-{\widehat{\theta }}_{0M})}}=\exp \left\{{\widehat{\beta }}_{\mathtt{dss}}^{*}\right\}\hfill \\ \hfill {\widehat{\mathrm{OR}}}_{calM/dssM}& ={\displaystyle \frac{{\widehat{\theta }}_{calM}/(1-{\widehat{\theta }}_{calM})}{\widehat{{\theta }_{dssM}}/(1-{\widehat{\theta }}_{dssM})}}=\exp \{{\widehat{\beta }}_{\mathtt{cal}}^{*}-{\widehat{\beta }}_{\mathtt{dss}}^{*}\}\hfill \end{array}$$

(11)

but also in the log scale with 95% confidence intervals estimated as described in Equation (10) using the relevant parameters. For GLMM_B, posterior distribution of the average OR was drawn from the transformation of posterior distribution model parameters and summarised by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.

OR${}_{cal}$ and OR${}_{dss}$ were used as a summary measure to quantify the efficacy of each fungicide program by means of relative changes in odds disease. OR${}_{cal/dss}$ was used to assess differences of calendar strategy in relation to DSS in terms of relative changes in odds disease.

2.4.3. Incidence Ratio

Following the same strategy specified in the previous section for OR computation, IR for calendar (IR${}_{cal}$) and DSS (IR${}_{dss}$)-based strategies against untreated control and IR for calendar against DSS were estimated for GLM_F model as follows:

$${\widehat{\mathrm{IR}}}_{cal}={\displaystyle \frac{{\widehat{\theta }}_{cal}}{{\widehat{\theta }}_0}},{\widehat{\mathrm{IR}}}_{dss}={\displaystyle \frac{{\widehat{\theta }}_{dss}}{{\widehat{\theta }}_0}},{\widehat{\mathrm{IR}}}_{cal/dss}={\displaystyle \frac{{\widehat{\theta }}_{cal}}{{\widehat{\theta }}_{dss}}},$$

(12)

Also for IR computation was carried out on a log scale and then log-transformed to convert them into the original metric [31]. Thus, based on the previous statement 95% confidence intervals were calculated according to

$$\begin{array}{cc}\hfill 95\%{\mathrm{CI}}_{{\widehat{\mathrm{IR}}}_{cal}}& =\exp \{\log \left({\widehat{\mathrm{IR}}}_{cal}\right)\pm {\mathrm{z}}_{0.975}\mathrm{SE}\left(\log \left({\widehat{\mathrm{IR}}}_{cal}\right)\right\}\hfill \\ \hfill 95\%{\mathrm{CI}}_{{\widehat{\mathrm{IR}}}_{dss}}& =\exp \left\{\log \right({\widehat{\mathrm{IR}}}_{dss}\pm {\mathrm{z}}_{0.975}\mathrm{SE}\left(\log \left({\widehat{\mathrm{IR}}}_{dss}\right)\right\}\hfill \\ \hfill 95\%{\mathrm{CI}}_{{\widehat{\mathrm{IR}}}_{cal/dss}}& =\exp \left\{\log \right({\widehat{\mathrm{IR}}}_{cal/dss}\pm {\mathrm{z}}_{0.975}\mathrm{SE}\left(\log \left({\widehat{\mathrm{IR}}}_{cal/dss}\right)\right\}\hfill \end{array}$$

(13)

in which ${\mathrm{z}}_{0.975}$ denotes the 0.975 quantile of the normal distribution and SE the standard error.

For GLM_B, posterior distribution of the IRs was drawn from the transformation of posterior distribution of the model parameters involved and summarized by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.

For GLMM_F model, average “experiment” IR for calendar and DSS-based strategy against the untreated control and IR for calendar against DSS were computed as follows:

$${\widehat{\mathrm{IR}}}_{calM}={\displaystyle \frac{{\widehat{\theta }}_{calM}}{{\widehat{\theta }}_{0M}}},{\widehat{\mathrm{IR}}}_{dssM}={\displaystyle \frac{{\widehat{\theta }}_{dssM}}{{\widehat{\theta }}_{0M}}},{\widehat{\mathrm{IR}}}_{calM/dssM}={\displaystyle \frac{{\widehat{\theta }}_{calM}}{{\widehat{\theta }}_{dssM}}},$$

(14)

but also in the log scale and with 95% confidence intervals were estimated in the same way as described in Equation (13) but adapting it to the parameters involved. For GLMM_B, posterior distributions of average IR were drawn from the transformation of posterior distribution of the model parameters concerned and summarised by means of the median as a point estimate and the 95% credible intervals as a measure of uncertainty.

IR${}_{cal}$ and IR${}_{dss}$ were used as a summary measure to quantify the efficacy of each fungicide program by means of relative changes in disease incidence. IR${}_{cal/dss}$ was used to assess differences of calendar strategy in relation to DSS in terms of relative changes in disease incidence.

2.4.4. Predictive Distribution of the Odds Ratio

To assess the extent of the heterogeneity estimates among experiments, the uncertainty of the ${\mathrm{OR}}_{cal}$, ${\mathrm{OR}}_{dss}$ and ${\mathrm{OR}}_{cal/dss}$ were also re-evaluated by calculating their corresponding predictive distributions. These predictive distributions allow us to incorporate between-experiment variability and to derive the so-called prediction intervals. Following the computation strategy described in Section 2.4.2, predictive distributions of the OR were computed in log-scale and then back-transformed. Parameters involved here were ${\gamma }_{\mathtt{cal},\mathtt{new}}$ and ${\gamma }_{\mathtt{dss},\mathtt{new}}$ (they represent the values of ${\gamma }_{\mathtt{cal}\left(\mathtt{i}\right)}$ and ${\gamma }_{\mathtt{cal}\left(\mathtt{i}\right)}$ for a new experiment and new sub-experiments chosen at random). According to Higgins et al. [1], their corresponding predictive distribution were defined as

$${\displaystyle \frac{{\gamma }_{\mathtt{cal},\mathtt{new}}-{\widehat{\beta }}_{\mathtt{cal}}^{*}}{\sqrt{{\widehat{\sigma }}_{\mathtt{cal}}^2+\mathrm{SE}{\left({\widehat{\beta }}_{\mathtt{cal}}^{*}\right)}^2}}}\sim t_{N-2}{\displaystyle \frac{{\gamma }_{\mathtt{dss},\mathtt{new}}-{\widehat{\beta }}_{\mathtt{dss}}^{*}}{\sqrt{{\widehat{\sigma }}_{\mathtt{dss}}^2+\mathrm{SE}{\left({\widehat{\beta }}_{\mathtt{dss}}^{*}\right)}^2}}}\sim t_{N-2}$$

(15)

in which $t_{N-2}$ denotes a t-distribution with $N-2$ degrees of freedom with $N=19$ representing the number of experiments. With GLMM_B, the OR predictive distributions were derived from the predictive distributions of ${\gamma }_{\mathtt{cal},\mathtt{new}}$, ${\gamma }_{\mathtt{dss},\mathtt{new}}$. These distributions were obtained by sampling values of ${\gamma }_{\mathtt{cal}\left(\mathtt{i}\right)}$ and ${\gamma }_{\mathtt{cal}\left(\mathtt{i}\right)}$ from their corresponding posterior distributions using the following transformations, $exp\left\{{\gamma }_{\mathtt{cal},\mathtt{new}}\right\}$ and $exp\{{\gamma }_{\mathtt{dss},\mathtt{new}\}}$.

3. Results

3.1. Descriptive Analysis of the Database

Disease incidence data are shown in Figure 1a and IRs for calendar and DSS strategies against the untreated control and for calendar against DSS in Figure 1b. Individual disease incidence data and IR values are shown in Figure 2a and Figure 2b, respectively. Note that some experiments show more than one IR (Cal./Unt. and DSS/Unt.) (Figure 2b) because more than one calendar and/or DSS strategies were evaluated (Table A3).

The disease incidence in untreated sub-experiments ranged from 1.5% to 100%. The mean and the median values were 43.17% and 39.00% (Figure 1a). The disease incidence for calendar and DSS-based strategies, showed overlapping distributions with similar mean (11.94% and 13.92%) and median (4.06% and 4.56%) values (Figure 1a). For Cal./Unt. and DSS/Unt. ratios the vast majority of observations were concentrated below the 1 with third quantile values ($Q_3$) equal to 0.430 and 0.610 (Figure 1b, respectively, which shows a tested efficacy of both programs. Furthermore, both IR distributions overlap partially which reflects a similar efficacy between the two strategies ($Q_3$ for Cal./DSS ratio was equal to 1).

Individual disease incidence under untreated sub-experiments presented in the vast majority of experiments higher values than their calendar and DSS counterparts, with only a few exceptions (A1, C1, D1, P1 and Q9) (Figure 2a). Calendar and their corresponding DSS showed similar disease incidence and IR values, sometimes higher for calendar, sometimes lower. Most IRs for Cal./Unt. and DSS/Unt. were lower than 1 with a few exceptions (A1, C1, D1, P1 and Q9 experiments) (Figure 2b).

3.2. Statistical Modelling Evaluation

The “random-effects” models performed better than “fixed-effects” models with both frequentist and Bayesian approaches according to the AIC or DIC scores. AIC for models GLM_F and GLMM_F were estimated at 64189.00 and 26098.96 (

Table 1. Parameter estimates, standard error and 95% confidence intervals for GLM_F and GLMM_F.

GLM_F			GLMM_F
${\beta }_{\mathtt{0}}$	$-0.282$ *	$(-0.302,-0.262)$	${\beta }_{\mathtt{0}}^{*}$	$-0.461$	$(-1.205,0.282)$
${\beta }_{\mathtt{cal}}$	$-1.748$ *	$(-1.781;-1.714)$	${\beta }_{\mathtt{cal}}^{*}$	$-2.604$ *	$(-3.944,-1.264)$
${\beta }_{\mathtt{dss}}$	$-1.460$ *	$(-1.487,-1.433)$	${\beta }_{\mathtt{dss}}^{*}$	$-2.014$ *	$(-3.077;-0.951)$
${\sigma }_{\mathtt{0}}^2$				2.677
${\sigma }_{\mathtt{cal}}^2$				8.369
${\sigma }_{\mathtt{dss}}^2$				5.387
${\sigma }_{\mathtt{0},\mathtt{cal}}$				−1.066
${\sigma }_{\mathtt{0},\mathtt{dss}}$				−1.453
${\sigma }_{\mathtt{cal},\mathtt{dss}}$				6.387
AIC	64,189.000			26,098.962

* 0 outside the confidence interval.

). Their Bayesian counterparts (GLM_B and GLMM_B) got DIC scores of 64188.97 and 25860.42 (

Table 2. Median of the posterior distribution and 95% credible intervals for GLM_B and GLMM_B.

GLM_B			GLMM_B
${\beta }_{\mathtt{0}}$	$-0.282$ *	$(-0.302,-0.262)$	${\beta }_{\mathtt{0}}^{*}$	$-0.462$	$(-1.252,0.337)$
${\beta }_{\mathtt{cal}}$	$-1.748$ *	$(-1.781;-1.714)$	${\beta }_{\mathtt{cal}}^{*}$	$-2.585$ *	$(-4.131,-1.189)$
${\beta }_{\mathtt{dss}}$	$-1.459$ *	$(-1.486,-1.432)$	${\beta }_{\mathtt{dss}}^{*}$	$-2.017$ *	$(-3.222;-0.877)$
${\sigma }_{\mathtt{0}}^2$				2.778	$(1.453,6.057)$
${\sigma }_{\mathtt{cal}}^2$				8.717	$(3.961,22.126)$
${\sigma }_{\mathtt{dss}}^2$				5.726	$(2.849,13.340)$
${\sigma }_{\mathtt{0},\mathtt{cal}}$				−1.061	$(-4.512,1.301)$
${\sigma }_{\mathtt{0},\mathtt{dss}}$				−1.433	$(-4.562,0.292)$
${\sigma }_{\mathtt{cal},\mathtt{dss}}$				6.614	$(3.167,15.701)$
DIC	64,188.970			25,860.42

* 0 outside the credible interval.

), respectively. Thus, with both inferential approaches, results were in favor of the GLMMs. This result was confirmed by the graphical analysis of logit(observed) vs. logit (estimated) of GLMs against GLMMs (Figure 3a,c and  Figure 3b,d) in which differences between observed and estimated (in logit scale) for GLMM were smaller. Figure 3a,c and  Figure 3b,d also showed that similar inferences were obtained between frequentist models and their Bayesian counterparts.

3.3. Statistical Modelling Inference Results

3.3.1. Parameter Estimates

Estimates for model parameters are shown in Table 1 (for frequentist, GLM_F and GLMM_F) and Table 2 (for Bayesian, GLM_B and GLMM_B). ${\beta }_{\mathtt{0}}$, ${\beta }_{\mathtt{cal}}$ and ${\beta }_{\mathtt{dss}}$ (for GLMs) and (${\beta }_{\mathtt{0}}^{*}$, ${\beta }_{\mathtt{cal}}^{*}$ and ${\beta }_{\mathtt{dss}}^{*}$) (for GLMMs) had different point estimates. However, the signs of the coefficients were consistent between the GLM and GLMM. On the other hand, ${\beta }_{\mathtt{0}}^{*}$, ${\beta }_{\mathtt{cal}}^{*}$ and ${\beta }_{\mathtt{dss}}^{*}$ showed wider confidence intervals (credible intervals) than ${\beta }_{\mathtt{0}}$, ${\beta }_{\mathtt{cal}}$ and ${\beta }_{\mathtt{dss}}$. For the “random-effects” models (GLMM_F and GLMM_B), variance parameters (${\sigma }_{\mathtt{0}}^2$, ${\sigma }_{\mathtt{cal}}^2$ and ${\sigma }_{\mathtt{dss}}^2$) were far from 0 indicating a strong between-experiment  variability.

3.3.2. Disease Incidence, Odds Ratio and Incidence Ratio Estimates

Disease incidence, OR and IR estimates are displayed in Figure 4a–c for the four fitted models (GLM_F, GLM_B, GLMM_F and GLMM_B). Overall, all models showed similar point estimates. Confidence intervals obtained with the frequentist models were also similar to the credible intervals obtained with Bayesian models (GLM_F vs. GLM_B and GLMM_F vs. GLMM_B).

With the GLM_F, the estimated median disease incidence was higher for the untreated control, 0.43, than for calendar-based (0.116) and DDS-based strategies (0.149). The same trend was observed with the GLMM_F (median values of 0.387, 0.045, 0.078 for control, calendar and DDS strategies, respectively). GLMM_F revealed that the calendar and DDS estimates strongly overlap. Comparable results were obtained with the Bayesian models (GLM_B and GLMM_B).

With the GLM_F, OR median values for OR${}_{cal}$, OR${}_{dss}$, OR${}_{cal/dss}$ were equal to 0.174, 0.232  and 0.750, respectively. With GLMM_F, these estimates were equal to 0.074, 0.133 and 0.55 (lower than frequentist estimates). Based on GLMM_F outputs, both calendar and DSS programs provide strong reduction of odds disease compared to the untreated controls (OR${}_{cal}$ and OR${}_{dss}$ < 1). Nevertheless, odds disease in calendar programs would be lower that in DSS (OR${}_{cal/dss}$ < 1, but with some uncertainty). The level of reduction obtained with DSS is already very high and leads to low disease incidences (see Figure 4a). Similar results were obtained with the Bayesian models (GLM_B and GLMM_B) and also for the different modelling IR estimates (Figure 4c).

3.3.3. Predictive Distribution of the Odds Ratio

Figure 5 shows predictive distributions for the OR for the GLMM_F (Figure 5a–c) and GLMM_B (Figure 5d–f). The x-axis shows the possible values of the ORs and the y-axis the percentiles (%) of its corresponding predictive distribution. That is, for each specific value of the OR showed, its  corresponding cumulative probability denotes the % of chance to be less than this value.

Predictive distributions obtained with GLMM_F and with GLMM_B showed some noticeable differences for the OR${}_{cal}$ and OR${}_{dss}$, not for the OR${}_{cal/dss}$. For GLMM_F, the probabilities (cumulative disease incidence probabilities) to have an OR lower than one are equal to 0.99 for both treatment strategies (Figure 5a,b). These results reveal that there is more than 99% to decrease the odds of disease with calendar and DSS strategies compared to untreated controls. However, according to the GLMM_B model, these probabilities were lower (about 0.8) (Figure 5d,e). Despite this difference, both models support that calendar and DSS strategies show high capabilities in reducing odds of disease. For both models, the probability that OR${}_{cal/dss}$ was lower than one is equal to 0.68 which means that odds of disease might be lower with calendar but with a considerable degree of uncertainty (Figure 5c,f).

4. Discussion

In the present study, four statistical models were assessed to evaluate the sensitivity of a meta-analysis aimed to compare the efficacy of DSSs to the standard calendar-based fungicide strategies for the control of fungal diseases in crop plants. Statistical modelling was set comparing “fixed-effects” and “random-effects” models in the framework of GLMs with frequentist and Bayesian inferential methods. Our results showed that the different treatment effect measures considered (disease incidence, OR, and IR) are highly sensitive to the integration of random-effects which may lead to different conclusions. These differences between “fixed-effects” and “random-effects” models were reproduced in both inferential methods.

Our assessments based on AIC or DIC scores and the graphical analysis of discrepancies between observed and estimated values confirmed that the dataset was better described assuming variable treatment effects among the experiments. Conceptually, the inclusion of random treatment effects was in line with the intrinsic nature of the data, which were gathered from experiments carried out by researchers operating independently. Our analyses showed that variance parameters were higher than 1 (Table 1 and  Table 2), indicating a substantial heterogeneity among experiments [32,33].

Our results also indicated that the use of “fixed-effects” models may lead to a strong underestimation of the uncertainty associated with the treatment effects measures considered (incidence, OR, IR) to summarise treatment efficacies. For both GLMMs, fequentist and Bayesian, the  width of the uncertainty measures (confidence/credible intervals) was larger than in their GLM counterparts, due to the lack of correlation between sub-experiments observations that it is implicit in “fixed-effects” specification. Since all sub-experiments within an experiment were evaluated under the same local conditions, they cannot be considered independent. Thus, models ignoring this correlation (i.e., “fixed-effects”model), overestimate the number of independent data and thus underestimate the level of uncertainty as noted by Makowski et al. [5].

Models with “random-effects” allow deriving predictive distributions, which enable making more robust conclusions as indicated for the consideration of confidence/credible intervals, but  also by the prediction intervals [33]. This interesting feature was included in our work by the derivation of the predictive distribution for the OR. For both GLMMs, frequentist and Bayesian, the  estimation of treatment effects measures (disease incidence, OR, and IR) was obtained from the conditional median/posterior distribution (for Bayesian models) of the random experiment effect in which between-experiment variability for the different treatments was not considered. Predictive distributions allow the incorporation of these sources of variability and obtain a more appropriate treatment effect summary than the average [32]. Consequently, the use of predictions gained relevance in recent meta-analyses, becoming a standard statistical output due to its utility to assess heterogeneity and uncertainty of treatment effects in target populations [1,34,35]. According to the predictive distribution for the OR, both fungicide strategies showed to be equally effective in reducing disease incidence compared with the untreated control.

Frequentist and Bayesian models for both “fixed-effects” and “random-effects” produced almost identical results for disease incidence, OR and IR. Our results also highlighted the importance of properly interpret OR and IR as relative measures. These measures do not have an absolute interpretation and tend to be insensitive to differences in baseline events, so it could be recommendable to consider an absolute measure such as incidence estimates to support their interpretation. According  to  the previous studies of [5,32,36], our results confirmed that under a weakly informative prior scenario, frequentist  and Bayesian models perform similarly. However, this was not the case of the predictive distribution for the OR which slightly varied between the two methods, although they lead to the same conclusions. The problem of finding reasonably good methods for prediction in both frequentist and Bayesian methods is an open research field, thus it is necessary to check the performance of prediction methods in practical meta-analysis and to interpret them with caution.

Our results indicated that the both inferential methodologies evaluated had similar practical advantages, but each of them with its pros and its cons. In general, the Bayesian approach is conceptually more straightforward and addresses uncertainty in a more comprehensive and interpretable way. Posterior distributions (parameter or derived quantity) can be characterised in terms of probability. Thus, a 95% credible interval is simply the central portion of the posterior distribution that contains the 95% of parameter values. By contrast, frequentist inference does not allow probability statements about the parameters, the p-value is a measure of evidence against a null hypothesis when it is assumed to be true. Thus, a 95% confidence interval means that if the same procedure to construct confidence intervals was repeated many times, then in 95% of the cases the true value will lie within the interval.

The Bayesian approach also makes easier to extend simple models to more complex formulations [37]. Currently, there are several software alternatives available such as WinBUGS [38], JAGS [39], INLA [40], Stan [41], which allow implementing those complex models much easily than in the frequentist framework. By contrast, the requirement in the Bayesian context of setting a prior specification for each parameter could be a weakness in the inference process. In some non-informative prior scenarios, different prior specifications may lead to different posterior distributions affecting inference robustness [42]. Moreover, in some cases the computational demand of Bayesian models could be extremely intensive. Thus, as a general recommendation, inferential method choice in meta-analysis should address and balance all the issues above.

In the context of fungal diseases, DSS strategies allow growers to estimate the risk of disease and devise more efficient fungicide spray programs, integrating different sources of information for control decisions including action thresholds [11]. However, currently there is a limited degree of adoption of DSSs due in part to growers’ aversion to risks and a perceived lack of reliability of DSSs [13,43]. Building on numerous individual field trials conducted to evaluate DSSs in a wide range of disease-crop systems, our meta-analysis demonstrates that DSS-based programs are effective in reducing the incidence of fungal diseases with efficacy comparable to calendar-based fungicide programs. Statistical inference comparing the number of fungicide sprays in DSS-based and calendar-based strategies will be the topic of further research.