3.1. Participation, Management Methods, and Experience of Citizen Scientists
Between 2017 and 2021, a total of 4408 trials were requested by citizen scientist participants (Table 1
). The most popular trials, averaging over 100 requests annually, were flowers (211), summer squash (151), tomato (126), and bean (122). The least popular trials, averaging less than 50 requests annually, were carrot (29), corn (3), leafy greens (41), and pumpkin (42). A total of 478 unique individuals have participated in the trial program. Of these, 124 participated in two or more years, while 354 participated in a single year. Participation was capped at 45 individuals in the pilot year (2017). Since 2017 was a pilot year designed to evaluate the program methods, experiences and general program evaluation that helped guide the project are included. However, individual cultivar evaluation data is only presented from the full project years of 2018–2020. The number of participants grew to 135 in 2018 and 171 in 2019. Participation dipped slightly in 2020 due to a recruitment window reduced by COVID-19 restrictions (139 participants) but recovered and was at its highest level in 2021 (186 participants).
On average, 41% of participants returned trial data at the end of the season (Table 2
). Return rate ranged from 32% to 55%, depending on the crop. Excluding 2017, which was the program evaluation year, first-time participants tended to have lower average return (33%) rates compared to those who had participated in the program for more than one year (53%) (Table 3
). Average return rates were lower in years three and four of the program compared to the preceding years. This is likely due to advertising to a broader demographic, resulting in participants representing a wider range of gardening experience.
Participation numbers and return rates illustrate both the opportunity and the challenge of using a citizen science project to educate and engage participants and efficiently collect trial data and evaluations. Following the 2017 program evaluation year, when participation was limited to EMG volunteers, no set requirements have been placed upon participants. Those interested have been able to request on a first-come, first-served basis, and the only limits have been seed quantities and the number of orders that could be filled and shipped in the early spring to enable arrival before ideal planting dates. This means that as the program has become more well-known and interest in food gardening has increased, new participants and those with less experience have been able to participate.
While the proportion of first-time participants has varied throughout the trial years, the return rate for new participants has lagged those of returning participants (Table 3
). This may be due to a lack of complete understanding of the project’s scientific goals or lack of commitment by new gardeners or those less connected to land-grant universities. This fits with similar citizen science programs that have found participant knowledge has increased through citizen science engagement, but participation alone is likely insufficient to provide a foundation for understanding of scientific research and efforts overall [13
]. In the 2019, 2020, and 2021 trial years, there has been an increased focus on communicating with participants and providing educational resources through email, video, and social media engagement throughout the growing season. These efforts were designed to support knowledge gain while also explaining the value of the participant role and data in the project. More concerted efforts and formalized processes are needed to introduce new gardeners to both key management practices in the garden and the importance of their follow-through in returning data on both crop successes and failures.
Beginning in 2019, participants were asked to report crop failure that prevented the complete evaluation of a trial. This was done to increase data return rates as it was anticipated that participants might not consider data return relevant to the trial program if a crop had failed. While this did not provide the expected increase in data return rates, it did provide information on which crops were more challenging for citizen scientists and why. Reported crop failure rates varied widely by crop (Table 4
). Leafy greens had the highest reported failure (44%), while winter squash had the lowest (2%). While many crop failures were equally spread across germination, human error, insect/disease, and wildlife issues, certain crops exhibited higher rates within specific categories. Pumpkins and corn appeared to be more prone to wildlife issues, corn and flowers to human error, flowers, herbs, and melons to germination issues, and squash and pumpkin to insects/diseases.
These rates and reasons of failure have been used to guide crop and cultivar selection in subsequent years. Over time, pumpkin and sweet corn trials have been replaced by those with higher levels of success and participation. Conversely, flower and herb trials have experienced high enough participation rates to continue to be included despite germination issues. However, as discussed above, more time and communication efforts have been spent in recent years to educate and support gardeners in starting transplants and seeding practices and timing to support better success in trial establishment. It may also be helpful in the future to assess citizen science participant motivation and focus recruitment and engagement efforts based on key motivations, as has been investigated in other citizen science projects [19
]. While this project has been designed primarily based on techniques to enhance data gathering and quality, it may be valuable in the future to focus more on the social dimensions.
As the HGVT program grew and expanded, new questions were added to the evaluation to better understand participant techniques and gardening experience and practices. Site and nutrition management data collection began in 2019. These questions included fertility type, site type, and years of gardening experience. Most participants grew their trials in-ground (58%), followed by raised beds (31%) (Table 5
). Only 9% of trials were grown in containers. Categorical analysis of crop failure rates by site was not significant (p
= 0.8090), with sites averaging a 20% failure rate across all the growing systems. The fact that the failure rate is not significantly different between garden sites supports the inclusion of a range of sites in the trial. The practice of sharing data and cultivar suggestions with gardeners across growing systems is also supported by the results of Crane et al. [4
], who found that in-ground performance of cucumber cultivars in trials was similar enough to be useful in suggesting cultivars to gardeners who grow in containers.
Fertility methods were close to equally distributed among compost, compost + organic, synthetic, and none, averaging 22% to 29% of trials (Table 6
). Compost + synthetic was a much smaller proportion of trials (4%). Crop failure rates by fertility were not significant (p
= 0.1498). While failure rates for compost + synthetic or synthetic averaged 11 to 13%, failure rates were nearly double (19% to 25%) for the remaining categories, which did not include synthetic fertilizer. Questions targeted at better understanding participant fertility use could provide greater insight into impact on crop failure rate as well as indicate if education materials are needed on how to optimally combine compost use with fertilizer to best meet crop nutrient needs.
The majority of participants had 20 to 30 years of gardening experience (27%) (Table 7
). Only 8% of participants had less than five years of gardening experience. Crop failure by years of gardening experience was significant (p
= 0.0795). Trials executed by participants with less than five years of gardening experience had a failure rate of 45%. This was three times higher than trials executed by those with 10 to 40 or greater than 50 years of gardening experience. While not surprising, the higher level of failure among participants with less gardening experience indicates that more education may be warranted in order to build success among participants and produce a more extensive data set. In future trial years, first-time participants may need to be engaged earlier in the trial and more directly, potentially with an introductory visit or training to best support their first-year experience.
3.2. Crop and Cultivar Evaluations
In the past four years (2017 to 2020), 158 cultivars have been evaluated through the Tennessee Home Garden Variety Trial program. Performance ratings differed (p
< 0.0001) among cultivars. Table 8
provides a summary of cultivar trials as well as statistical analysis for years 2018–2020. Individual cultivar evaluations from program evaluation year 2017 have not been included in the tables below. Performance ratings were on a 1 to 10 scale in all years of the trial, with 1 indicating poor performance and 10 indicating excellent performance. The most highly ranked cultivars across all years, with ratings of 9.0 or greater, were ‘Ambrosia’ (melon), ‘Desert’ (summer squash), and ‘Golden Star’ (summer squash), and ‘Maxibel’ (bean). The least popular cultivars, with ratings of 5.1 or less, were ‘Sugar Baby’ (watermelon), ‘Malibu’ (bean), and ‘Pepitas’ (pumpkin).
The proportion of respondents who stated they would recommend a cultivar to other home gardeners is also presented in Table 8
. While this number tended to follow the same trend as performance ratings, a few high-performance scores were matched with a low recommendation and vice versa. This is very interesting and the reason for these dual questions in the evaluation. It is a goal of this evaluation to separate personal preferences for cultivars with overall performance. While it is likely that overall rating and recommendation for others will be similar, it is interesting that some cultivars were rated as better for others than the participant rated their own approval and satisfaction.
Participants were asked to specify qualities that influenced that recommendation in order to provide more insight into why participants preferred or did not prefer cultivars for themselves or others. Reasons listed by the highest proportion of respondents included yield (58%), flavor (55%), appearance (53%), health (51%), and size (42%). Less than 5% of respondents noted other reasons like texture, shape, early maturity, germination, shelf life, canning/cooking, production season length, and vigor. While non-commercial food producers use a range of factors in evaluating cultivars, it is interesting to see that appearance, health, yield, and flavor were all quite similar in the percentage of influence on ratings. These percentages illustrate the importance of trials that evaluate attributes other than yield, the most common data gathered in traditional cultivar trials. These reasons also line up with national surveys of gardeners who often rank taste and quality as high or higher in importance than productivity or money savings related to crop yield [1
]. In many ways, these data on appearance, health, and flavor, as rated by real home gardeners, would not be accessible using traditional trial techniques.
Performance ratings by crop also exhibited significant differences (p
< 0.0001) (Table 9
). Okra, herbs, and summer squash had higher performance ratings (7.5 to 7.9) than flowers, pumpkin, melons, winter squash, and corn (6.4 to 6.9). Cucumber (7.2) and bean (7.1) fell in the middle of these two groups (7.2). The proportion of participants that would recommend cultivars within these groups closely followed rankings of performance scores. Some crops that did not perform well in the program evaluation year were not trialed again. For example, carrots were trialed only in 2017, and there were timing challenges in getting seeds to participants in the spring early enough for good production. As a general trend, longer season and more space consumptive crops, such as pumpkin, winter squash, and corn, tended towards lower ratings overall.
Of the cultivars evaluated, 11 were evaluated over two or more years in 2018–2020 (Table 10
). Performance ratings between years within these cultivars were very similar with none of the cultivar rating showing significantly different results in different trial years. Differences between years ranged from 0.1 to 2.2 points, averaging 0.7 points. The strong internal consistency of the data for cultivars across years suggests reliable evaluations within our trial participants even with fluctuations in the individuals participating and the number of evaluations completed in a given trial.
Six cultivars were evaluated in both named and blind trials to determine if cultivar name biased participants evaluations (Table 11
). Only Marketmore 76 cucumber exhibited a significant difference in performance rating between the named and blind trial. This cultivar ranked higher (9.1) in the blind trial compared with the named trial (7.8). Among most cultivars, the difference in performance rating between blind and named trials did not exceed 1.3 points. The pumpkin ‘Cinnamon Girl’ exhibited the largest difference, with the blind trial rated at an 8.2 for performance and the named trial rated at a 6. These results indicate name recognition bias appears to be a minor factor.