Sample Size Requirements for Assessing Statistical Moments of Simulated Crop Yield Distributions

Mechanistic crop growth models are becoming increasingly important in agricultural research and are extensively used in climate change impact assessments. In such studies, statistics of crop yields are usually evaluated without the explicit consideration of sample size requirements. The purpose of this paper was to identify minimum sample sizes for the estimation of average, standard deviation and skewness of maize and winterwheat yields based on simulations carried out under a range of climate and soil conditions. Our results indicate that 15 years of simulated crop yields are sufficient to estimate average crop yields with a relative error of less than 10% at 95% confidence. Regarding standard deviation and skewness, sample size requirements depend on the degree of symmetry of the underlying population’s distribution. For symmetric distributions, samples of 200 and 1500 yield observations are needed to estimate the crop yields’ standard deviation and skewness coefficient, respectively. Higher degrees of asymmetry increase the sample size requirements relative to the estimation of the standard deviation, while at the same time the sample size requirements relative to the skewness coefficient are decreased.


Introduction
Mechanistic crop growth models are of high importance in agricultural research.They offer a cost-effective tool for simulating plant growth under a wide range of management options and environmental conditions [1].The field of application of crop models is wide.For instance, new management technologies can be tested in quasi-field trials and agro-environmental problems can be addressed at field-, farm-or watershed-level [2].Crop growth simulation models can also be used to identify critical traits with respect to survival rates and yield levels (e.g., [3,4]).They are further extensively employed for climate change (CC) impact assessments (e.g., [5][6][7][8][9][10]), in which the goal is often to derive crop yield distributions for varying climate conditions and management options, in particular with respect to irrigation, fertilization intensity or soil cultivation (e.g., [5,9,11,12]).
Optimal crop-and site-specific management patterns highly depend on the prevailing climate conditions.Reliable information concerning the distribution of yields can thus be obtained only from simulations spanning a sufficiently large number of years [13].In principle this is not a problem, in particular if climate records are developed with the help of stochastic weather generators [14].In practice, however, the computational burden can easily become a critical issue.For instance, the simulation of crop growth during a single vegetation period with the crop model CropSyst requires about 7 sec on a common PC (Intel Pentium Core(TM) i5 at 3.33 GHz).Thus, the decision to run one hundred or one million simulations is not without consequences.Computational constraints are even more relevant if crop models are applied in a spatially explicit setup (e.g., [15]) or if a large number of management options is optimized simultaneously by heuristic optimization techniques (e.g., [11,16]).
The choice of an adequate sample size is a well-known problem in statistics ( [17][18][19]) and is crucial for the analysis of yield distributions. Yet, it has never been addressed in a systematic way in agronomic studies and climate change impact assessments.A review of the existing literature indicates a wide range of assumptions made at this stage.For instance, Moriondo et al. [20] consider yield records extending over 100 years to derive information on mean and standard deviation of crop yields, whereas Tingem et al. [21] rely on 50-year records to simulate mean yield levels.Next, Thornton et al. [22] use 30 repetitions to estimate the first two statistical moments, while only 25 runs are used by Finger and Calanca [23] to estimate also skewness.Finally, Kapphan et al. [24] use 1000 crop yield simulations in order to estimate climate-related risks and to design optimal crop yield insurance contracts.
Under the assumption that samples are normally distributed, statistical theory provides solutions regarding the sample size requirements for the estimation of both the mean as well as the standard deviation.In the former case, the minimum sample size n necessary to obtain an estimate with an maximum absolute error of d at a confidence level can be given following Cochran's sample size formula [17]: where z /2 is the upper (1 /2) quantile of the standard normal distribution and 2 is the population variance.Regarding standard deviation, the minimum sample size n s to obtain an estimate with the relative error d s can be determined from the method of Thompson and Endriss [25]: where s is the sample standard deviation, which represents an unbiased estimate for the underlying population statistics .Distributions of crop yields, however, only seldom follow the normal distribution [26].Furthermore, Equations ( 1) and (2) do not provide guidance concerning the sample size requirements relative to the estimation of higher statistical moments, in particular skewness, which is of great importance for many applications in agricultural economics [27].
Against this background, the aim of this study was to investigate sample size requirements for the estimation of the first three statistical moments of crop yield distributions.The analysis is based on a large simulation experiment conducted with the crop growth model CropSyst [28].Given the fact that yield distributions may vary considerably in shape depending on crop, climate and soil characteristics, we set up our simulation study as a combinatory experiment with two crops, viz.winterwheat (Triticum aestivum L.) and maize (Zea mays L.), two sites at the Swiss Plateau, viz.Payerne and Uster, and two climate scenarios, viz. a baseline scenario reflecting current climatic conditions and a future scenario characterized by markedly higher temperature and reduced summer precipitation amounts.

Methods
CropSyst is a deterministic, process-based crop growth model, which simulates crop growth at a daily time scale [28].In order to drive the biological and environmental processes, CropSyst requires daily weather data along with the specification of soil and crop characteristics [28].
The crops considered for our analysis are grain maize, a warm season crop being particularly sensitive to drought at flowering [29], and winterwheat, a cool season crop being prone to excess temperature [30], which is sown in autumn and harvested in summer.Site specific crop parameters for maize and winterwheat were obtained from results of the calibration exercise described in Klein et al. [31].Following Lehmann et al. [11], the sowing date of winterwheat was fixed at 10 October whereas grain maize was sown when the 5-day average air temperature exceeded 10 °C.Furthermore, standard nitrogen fertilization amounts of 140 kg•ha and 110 kg•ha were assumed for winterwheat and maize, respectively [32].In addition, identical initial soil conditions with respect to the concentrations of organic matter and nitrogen were used in each year in order to avoid distortions due to dynamic effects in soil nutrient availability.Thus, all variations in simulated crop yields were only due to differences in weather conditions.Effects of elevated CO 2 concentrations on crop growth were not taken into account, because its quantification is still highly uncertain [33].
One thousand, five hundred years of synthetic daily weather data was generated consistently with observations for the reference period of 1981-2009 (Baseline) as well as for a climate change (CC) scenario valid for 2036-2065 using the stochastic weather generator LARS-WG [34,35].As detailed in Lehmann et al. [11], the CC scenario was specified according to simulations performed with the ETHZ-CLM regional climate model [36] in the context of the ENSEMBLES experiment [37] assuming a A1B emission pathway [38].The scenario projects increases in monthly average temperatures between 2.0 °C in winter and 4.0 °C in summer months.Regarding average precipitation, only in summer months ying assumption in LARS-WG [34,35], the simulated data can be considered as representing 1500 independent realizations of annual weather states.
For each combination of crop × location × scenario, the synthetic weather data was used as input to CropSyst for the simulation of 1500 crop yields.These were assumed to represent the underlying yield population, with corresponding statistical moments denoted as ref (mean yield), ref (standard deviation of crop yields) and ref (skewness of crop yields).In order to analyze the effect of different sample sizes on the robustness and accuracy of the estimated statistical moments and to determine minimum sample size requirements, the following procedure was implemented: (1) 5000 samples of crop yields were drawn without replacement from the population for sample sizes i = 5,10,15,….,1500.(2) For each sample size i and realization j, mean ( ), standard deviation ( ) and skewness ( ) were estimated based on the drawn sample.
(3) Relative deviations of the individual estimates from their reference values were computed for all moments as: ( with analogous equations for the standard deviation and the skewness.Here, is the relative difference of the mean yield in the sample j = 1,2,3,…,5000 of size i = 5,10,15,…,1500 from the population's mean yield ref .(4) Finally, upper 95%-quantiles for the distributions of , and were computed for each sample size i.This measure can be used to determine the minimum sample size leading for 95% of all 5000 samples to a relative error smaller than a pre-= 5%, 10%, 15% and 25%.The 95%-quantile has been chosen because it represents a robust measure and can directly be compared with the 95%-confidence interval usually applied in conjunction with Equations ( 1) and (2).

Results
Figure 1 shows the distributions of the simulated 1500 crop yields for all considered simulation settings.The corresponding statistical moments are presented in Table 1.Since winterwheat is already harvested in early summer, it is less exposed to summer droughts and exhibits a narrower yield distribution than grain maize.Figure 1 further suggests a smaller spread of yields at Uster than at Payerne owing to the relatively more humid climate conditions at the former location.Average yield levels are reduced for both crops and at both locations under CC.The null hypothesis of normality is rejected by a Kolmogorov-Smirnov test for all scenarios except for grain maize at Payerne and winterwheat at Uster both simulated under CC climate conditions.Figure 2 shows the relationship between the number of considered weather years and the 95%-quantile of the relative errors of the estimated statistical moments at the example of the winterwheat simulation at Payerne under Baseline climate conditions.In all circumstances, relative errors of moment estimates computed according to Equation (3) decrease with increasing sample size.
As expected from statistical considerations, relative errors of mean yields are smaller than relative errors of estimated standard deviations, which in turn are smaller that relative errors of skewness coefficients.Overall, the results in Figure 2 suggest that while about 10 samples are sufficient to estimate average yields with a relative error of less than 10% at 95% certainty, considerably larger samples are needed to estimate higher order moments at the same level of accuracy.Sample size requirements to obtain moments estimates with a relative accuracy of better than 25%, 15%, 10% and 5% at 95% confidence are summarized in Tables 2 and 3.These figures suggest that a sample size of 15 observations is for all scenarios sufficient to obtain estimates of the mean yield with a relative error of less than 10%.With respect to mean yield estimations, differences in sample size requirements between the two sites reflect the overall smaller coefficients of variation at Uster than at Payerne.
Much larger sample sizes are required in order to obtain reliable estimates of crop yields' standard deviations, with substantial differences depending on scenario.For instance, 675 observations are required to estimate the standard deviation of maize yields in the Baseline scenario at Uster to within 10% of the reference at 95% certainty.Conversely, already 120 observations are sufficient to estimate the standard deviation of maize yields at Payerne with the same accuracy and certainty level.a All values shown in Table 3 correspond to the 95%-quantile.b = mean yield; = standard deviation of yields; = skewness of yields.
Concerning skewness, sample size requirements are even larger and can vary substantially depending on crop, site and scenario.With the exception of maize at Uster, the results indicate that more than 1000 samples are needed to estimate the skewness coefficient with a relative accuracy of less than 10% with 95% certainty.
While it is difficult to discern more specific patterns in Tables 2 and 3, concerning the estimation of the standard deviation and the skewness coefficient there are systematic tendencies that appear when plotting minimum sample size requirements against reference values of the skewness coefficient (Figure 3).Within the examined range of reference values, the minimum sample size relative to the estimation of the standard deviation increases with the degree of asymmetry.As opposed to this, minimum sample sizes for the estimation of the skewness decrease with increasing degree of asymmetry.This latter feature, however, can be explained by the fact that in the limit of a symmetric distribution the skewness coefficient is equal to zero, and therefore relative errors are in principle always infinitely large.Returning to the estimation of mean yields, we notice that minimum sample sizes listed in Tables 2 and 3 are always very close to the values obtained from applying Cochran's sample size formula (Equation (1)), in spite of the fact that only in two scenarios simulated crop yield distributions follow normality.
Lower agreement is found between our empirical estimates and the theoretical derived minimum sample sizes for estimating standard deviations obtained from Equation (2).For a relative error of 10% at 95% confidence, the latter results for all scenarios in a minimum sample size of 194 observations.At Payerne this figure lies within the range of the empirical data.On the other hand, with the exception of winterwheat yields simulated under the CC scenario, the method of Thompson and Endriss [25] largely underestimates the required sample size at Uster.

Discussion and Conclusions
In our analysis we addressed the evaluation of the minimum sample size required to estimate mean and higher statistical moments of crop yield distributions with given accuracy and confidence.While the sample size required to estimate mean yields did not show large differences across the range of combinations of crop × location × scenario, the required sample sizes for higher statistical moments was found to be extremely sensitive to the characteristics of the population from which the samples are drawn.More specifically, our results indicate that the minimum sample size required for estimating the standard deviation and skewness can be related to the degree of asymmetry of the underlying distribution, at least for the range of skewness coefficients implied by our simulations.
Relatively to the eight simulation setups considered in this study, the following conclusions can be drawn: A sample size of 15 yield observations is sufficient to obtain estimates of mean yields with a relative error of less than 10% at 95% confidence.200 realizations are in general sufficient to obtain estimates of the standard deviation of crop yields with a relative accuracy of better than 10%.The sample size should be increased to roughly 500 when it can be assumed that the crop yield distribution is strongly skewed (absolute skewness value > 1).At least 1000 realizations are needed in most cases to reliably characterize the skewness of the distribution.When a high degree of symmetry is suggested by the available information, much larger samples are needed.This implies that in the absence of prior information, risk analyses should always be based on very large sample sizes.In practice, simulating 1000 or more years of crop yields may not always be feasible.In these cases compromises between the computation time and the accuracy of the estimated statistical moments have to be made.For instance, the required sample size is reduced by a factor of about 5 with respect to the estimation of the standard deviation, if the allowable relative estimation error is increased from 10% to 25%.This can be meaningful in studies aiming e.g., at optimizing crop management (e.g., [11,16]) or in studies simulating crop yields in a spatially explicit manner (e.g., [15]).
Concerning the estimation of mean yields, we noticed that Cochran's formula (Equation (1), [17]) provides a reliable starting point for the determination of the required sample size, in spite of the fact the assumption of normality does usually not apply to crop yields.A further drawback of Equation ( 1) is that in principle knowledge of the population standard deviation is needed, whereas in practice only the sample standard deviation s is available.This difficulty can be overcome by considering two-stage sampling procedures [39,40].Although preliminary tests using Stein's two-stage sampling procedure conducted with our data suggest that there is no necessity for taking a second sample if more than 5 observations are already considered in the first step, double sampling is simple enough to be implemented in impact assessments.
This kind of drawback is not found with the method of Thompson and Endriss [25], since it depends only on the choice of the relative accuracy and confidence level.However, our results suggest that its outcomes tend to largely underestimate sample size requirements, in particular when the distributions are narrow and the coefficients of variation are low.
Despite the fact that we considered two different crops, two sites and two climate scenarios, our study cannot have the pretension of being exhaustive and further work is needed to develop general rules.Future research should consider other crop types and extend the analysis to geographic areas characterized by more extreme conditions than examined in this study.Furthermore, analyses of simulated crop yields should be complemented with more theoretical studies referring to standard distributions other than the normal one.Insights could be gained, e.g., from consideration of the beta distribution.Apart from the fact that it has been shown suitable for describing crop yields [41], the beta distribution is flexible enough to mimic distributions with various degree of asymmetry and spread.Moreover, exact formulas are available for evaluating the statistical moments of interest in impact assessments.

Figure 2 .
Figure 2. Relative error of estimated statistical moments as a function of sample size.Simulations of winterwheat yields at Payerne under Baseline climate conditions.

Figure 3 .
Figure 3. Relationship between sample size required for the estimation of standard deviation (triangles) and skewness (squares) and absolute values of the reference skewness.Required sample sizes refer to relative errors of less than 10% at 95% certainty.

Table 1 .
Statistical moments of simulated yield distributions.

Table 2 .
Minimum sample sizes for different relative errors at Payerne.
a All values shown in Table2correspond to the 95%-quantile.b μ = mean yield; = standard deviation of yields; = skewness of yields.

Table 3 .
Minimum sample sizes for different relative errors at Uster.