We engaged citizen science volunteers to successfully extract information from phenology camera imagery that cannot presently be extracted automatically. This information included vegetative and reproductive states of landscape vegetation, occurrence of snow, and the locations of individual trees. Additionally, we used citizen science to ground-truth automated algorithms that determine the start and end of spring and autumn dates directly from phenology camera imagery. Here we discuss possible applications for phenology data produced by citizen science data-processing projects like Season Spotter and provide guidance for the design and implementation of future image-based online citizen science projects.
4.2. Recommendations for Citizen Science Data Processing of Remote Sensing Imagery
Building an online citizen science project for data processing of images has never been easier. Using the Zooniverse Project Builder (www.zooniverse.org/lab
), it is possible to prototype a new project in under an hour [21
]. Currently, the Project Builder supports single-answer and multiple-answer questions and drawing tasks. The next step is creating project content: the full text and many images necessary for task completion as well as help, tutorial, and educational materials. Beta testing the project, potentially multiple times, is vital to ensure high quality and usability of the resulting classifications [36
]. For example, Mountain Watch, a ground-based phenology citizen science project, found that initial identifications by volunteers were inconsistent, due to misidentification of species or inaccurate location descriptions. Consequently, the project altered its methods to use permanent plots and better-trained personnel [38
]. As it is for any citizen science project, the main ongoing investment is recruiting, retaining, and communicating with volunteers as well as in good data-management practices [39
Season Spotter was successful in engaging volunteers and producing valuable data by following some straightforward guidelines [22
]. We ensured that our questions were short, simply worded, and lacked jargon. We provided comprehensive supplementary material behind “help” buttons. This material included example images, additional instructional text, and directions on what to do if an unusual image was presented. We also provided background and reference materials so that volunteers understood how they were contributing to science and could explore plant phenology in more depth. Importantly, we maintained communication with volunteers throughout, from initial testing of the project throughout its running. This allowed us to incorporate feedback into the project design initially and alerted us when volunteers were having difficulty with particular question types. Crall et al. found partnering with organizations that share common goals to reach a broad audience was key for recruiting volunteers and that ongoing engagement of existing volunteers was important for volunteer retention [22
We learned that Season Spotter was not as efficient as it might have been, though. Ideally, citizen science projects should be set up to maximize volunteer effort and speed data-classification time. Having multiple volunteers classify every available image decreases random error, but increases the amount of overall volunteer effort required. For data that are temporally correlated, it may be possible to exclude sets of images and subsample from the rest. For example, if one were only interested in extracting reproductive states from vegetation at a site where plants flower only in summer, weeks that have daily temperatures always below freezing can be excluded before sending images to volunteers. Another strategy might be to initially present only every fifth image to volunteers when a phenological state such as flowering typically happens over several days or weeks. Then a second round of classification could be conducted consisting of only the unused images immediately before and after those that were positively classified as flowering. For a site in which flowering lasts two weeks out of the year, this method would require just 25% of the effort needed to inspect every image over the year. A more advanced approach might use machine-learning algorithms to provide initial classifications and then ask volunteers to weigh in on images for which the algorithm has low certainty.
Another area for ongoing online citizen science research is in design techniques for sustaining volunteer interest and engagement [42
]. While we actively engaged with volunteers on social media and in a dedicated chat forum, we found that Season Spotter’s volunteer retention rate was somewhat lower than those of other Zooniverse projects. We believe that this may have been due to a lack of variety in images seen by volunteers. In order to develop full datasets per site, we selected a subset of PhenoCam sites to use in Season Spotter. This meant that our volunteers saw images from the same sites repeatedly, potentially leading to boredom, as was suggested by some volunteer comments in the chat forum. Future projects may want to assess volunteer response to image variety during beta testing. Exactly how to balance volunteer interest with researcher data needs is a topic in need of more research.
Citizen science projects analyzing remote sensing imagery will need to take into account the size of the features to be classified relative to the full image and the viewing device used by volunteers. In Season Spotter, volunteers were often able to accurately classify flowers and cones in images (Figure 6
, Figure 7
, Figure 8
and Figure S2
), but they were unable to correctly identify images containing small grass seedheads. Additionally, images containing very few flowers were sometimes misclassified as not having any flowers. Because PhenoCam images comprise entire landscapes, extraction of information about small features such as flowers and seeds is likely going to succeed or fail based on the size of the individual features relative to the size of the whole image and the pixel resolution of the image (as is the case for all fine-scale remote sensing [43
]). For example, volunteers were able to identify cones in a hemlock forest. However, the most easily seen cones were quite close to the camera and therefore reasonably large relative to image size (Supplementary Figure S2
). A similar view that contained only trees farther away would have been more difficult for volunteers, and it is likely that the rate of cone detection would have dropped. By contrast, the reproductive phenology of monocultures like crops may be obvious to volunteers, even if individual flowers or seeds are relatively small; volunteers were readily able to differentiate vegetative and reproductive stages of corn and soybean (Figure 8
On average, volunteers viewed images at 40% their original resolution, due to device and browser limitations. Several volunteers noted in the Season Spotter discussion forum that the size of the images made it difficult to see small features like flowers and cones, and they requested a zoom tool that would magnify images on-screen so that they could more carefully inspect images. Such a tool must be considered carefully, because it presents the dilemma that all volunteers may not see the same things when viewing the same image, potentially biasing the resulting data. Recording whether or not each volunteer uses such a zoom tool may help to properly combine classifications. Another method for increasing the ability for volunteers to correctly identify small features is to pre-select a region of the full landscape image that is likely to contain those features and then crop images to just this region, in effect automatically zooming in for all volunteers. The amount of zoom that can be achieved will ultimately be limited by the resolution of the initial image; higher-resolution images are more likely to elicit good citizen science classifications for small things like flowers and cones than lower-resolution ones.
As with all scientific endeavors, it is important for citizen science projects to assess the quality of collected data and account for bias [36
]. In Season Spotter, we found a tendency for volunteers to select the right-hand image when presented paired images for one spring question, but not three autumn questions. We were able to avoid bias in the resulting spring data by showing every set of paired images in both orientations, and we suggest all projects showing images side by side do likewise. Further research into the causes of perception bias in online citizen science projects would be valuable to help guide project design to ensure high data quality [44
Citizen science for Earth observation at large spatial scales holds promise for increasing basic understanding of Earth’s biological and physical systems as well as advising management and policy objectives. Season Spotter and similar online data-processing citizen science projects make it feasible to analyze large image sets, whether they are from near-Earth sensors like phenology cameras, traditional satellite sensors like Landsat (e.g., [45
]), or specialized picosatellite sensors. The resulting data can be used to support Earth observation for climate change research and land management both alone and in combination with satellite remote sensing.