It’s Virtually Summer, Can the Zoo Come to You? Zoo Summer School Engagement in an Online Setting

: Zoological collections are rapidly changing from a place of entertainment to centers of education. Many zoological collections run holiday and weekend clubs with activities aimed at inspiring and enthusing the next generation. The COVID-19 pandemic saw zoological collections across the world closing, leading a need for alternative educational content. Edinburgh Zoo, UK converted their summer school to a virtual provision. This provided a unique opportunity to determine the effectiveness of online zoological education by investigating if engagement levels differ for family groups when education is ‘live’, ‘recorded’ or ‘activity’ based. A total of 235 participants signed up for the Virtual Summer School, which comprised of 46 separate activities. Submissions, comments and polls were coded for content level and activity type. Results show that the overall engagement was higher for the live sessions compared to the recorded content; however, the content level was higher for activities. Content level increased over the week and there was a higher reported nature appreciation at the end of the Virtual Summer School. These findings provided evidence to suggest that online holiday zoo education can engage and inspire and gives insight on how to maximize the engagement and knowledge acquisition when using these online platforms.


Introduction
Conservation education is part of the remit of the modern zoo and by fostering pro environmental behaviors, zoos fulfil their role in support of conservation [1]. Educational outputs offer a diversity of learning opportunities but are broadly split into two categories; informal (interpretation, demonstrations, visitor-led) and formal (taught, educator-led) [2]. Informal education is delivered via immersive exhibits and interpretation, and public engagement via keeper talks and handling sessions; whereas formal education includes sessions catered for schools and other groups, outreach sessions and regular clubs [3]. The latter, being run either weekly, monthly or during the holidays, can provide an opportunity for children to learn, not just animal and conservation related information but social skills and cooperation in a fun and engaging environment [4,5]. The varied approach to education allows for many learning styles to be accommodated, however it is important to understand how effective these approaches are in allowing visitors to acquire and retain the information. In order to meet the requirements of the World Association of Zoo and Aquaria (WAZA) Conservation Education Strategy [6], zoo education should strive to engage by inspiring awe, provide an opportunity to learn. By encouraging these feelings of awe and increased knowledge, zoo educators aim to bring about attitude and behavioral changes [7]. The measure of this is difficult to obtain but attempts have been made to evaluate public engagement through recording the number of people at exhibits and interacting with exhibit interpretation as well as recording dwell times at exhibits [8] or through conversation analysis [9].
The effects of teaching can be translated through measuring knowledge acquisition and learning retention, either actual (short-or long-term) [10,11] or self-reported and through measuring conversations [11], interest, emotions [12] and motivation/intention to change their behavior [13]. Whilst there are studies investigating these outputs, they are limited in comparison to other areas of zoo research and systematic evaluation during all stages of educational programs is needed for a full understanding of the educational benefits [7]. In particular, there is very little evaluation of the impacts of zoo clubs and even less on online education.
The COVID-19 pandemic resulted in the closure of zoological collections and the need for the international zoo community to adapt quickly. Reductions in staff and researchers no longer able to access onsite resulted in alternative methods to evaluate the effect of the closures on animal behavior [14][15][16]. Equally, the international zoo education community moved to delivering content and engaging with the public in different ways [17][18][19][20].
Family researchers have found that learning within the family is a crucial educational experience [21]. Parents who are involved with their children's education have resulted in positive associations and children show higher receptive vocabulary and taskpersistence [22]. However, online learning during the COVID-19 pandemic changed the dynamic of families, meaning parents spent a large proportion of time monitoring their child during an online class but not interacting [23].
During the first COVID-19 UK lockdown, families struggled with well-being and increased disruptions to their day to day [24]. There is already research to suggest time spent out of school results in lower physical activity [25] and that confinement can also affect mental wellbeing too [26]. This has led to a greater concern in the overall wellbeing of children [27]. Access to nature benefits both physical and mental wellbeing [28], however, during the pandemic many were not able to access the outdoors and this number was even greater for those from urban areas or high deprivation [29]. Taking this into account RZSS Edinburgh Zoo, developed a week-long Virtual Summer School (VSS) to bridge multiple generations and connect families with nature. The VSS comprised of four hours of content per day delivered live, recorded and as set activities that involved interaction with the outdoors [19].
The VSS is a mixed aged group program for 5-15 years old children at RZSS Edinburgh Zoo. This provided an opportunity to investigate family engagement with zoo education in an online setting and to evaluate the program's aim; to connect people with nature and to provide awe, wonder and creativity, in line with the WAZA Conservation Education Strategy [6].
With the closure of zoos, due to the pandemic and the increase of zoo education available online, the main aim of this study was to evaluate this novel form of informal zoo education delivery. Specifically, the following objectives were explored:

•
To investigate the difference in participant engagement when education is delivered live online compared to asynchronous online resources.

•
To consider the quality of learning throughout the VSS.

•
To determine if there was a change in time spent as a family as a result of the VSS.

•
To evaluate if a virtual version of the summer school was successful in fostering a connection with nature.

•
To see what categories of activities resulted in higher participant engagement.

Virtual Summer School Program
The one weeklong Virtual Summer School (VSS) was repeated over four separate weeks during July/August 2020 (during Scotland's phase 2 and 3 of lockdown) and resulted in 235 participants. In total there were 46 main 'discovery activities' linked to the natural world and its conservation delivered via a Google Classrooms page. A mixture of online and offline activities were employed, to maximize effective use of screen time (Table 1 and Supplementary Material Figure S1). These activities (Table 1) were either self-led, educator-led or family-led experiences, equating to roughly four hours of activities per day. Family submissions (i.e., Photos of the craft, written or video submission), comments, surveys, live polls and 'check-ins' provided data to evaluate love for nature, engagement, complexity and satisfaction (Supplementary Material Figure S1). Five live sessions were also available and recorded for flexibility, ensuring full access for everyone. Only six activities required additional resources from the participants, with five just using everyday recycled materials and none requiring printing. All activities were optional for further flexibility.
Every activity aimed to include as many of the qualities of 'Significant Nature Situations' (as characterized by Giusti et al. [30]) as possible to maximize the nature connections made (Table 1). This table also shows the wide range of experiences included during VSS, specifically chosen to increase families' breadth of connections and knowledge of nature

Data Collection
A pre-and post-questionnaire created with Google Forms was disseminated to the participants (Supplementary Material Questionnaire). They entailed seven questions on the pre-questionnaire and twelve on the post-questionnaire. The questions consisted of ordinal scale questions including rating an appreciation for nature, respondents' opinions of zoo collections and numerical questions on time spent together completing activities as a family. Free text questions were included to gauge the opinions on how the families enjoyed the VSS and how they found the online format.
Activities were categorized by mode of delivery, either; prescribed activities (e.g., Zoo Cupcakes), recorded content (e.g., Animal Adventures) or live content (e.g., LIVE: Behind the Scenes). Comments and submissions on the Google Classrooms by participants were also put into a ranking system of high, medium and low content level with classifications (Table 2), adapted from Khali et al. [12]. This ranking system allowed data to be collected on how an individual understood and engaged in a task through how this was presented in submitted work or comments. A low-level submission gave the general assumption that an individual only understood the basics of a task, for example what species they are seeing. Whereas a high-level submission showed that an individual has a higher understanding and was more intellectually engaged, therefore conducting further research into a species for example, to find out the endangerment level and general structure of a species group.

Code Description Example Comment
Low Identifies a species, discussing the species basic needs or habitat A comment only stating the individual's enjoyment or not fully completing an activity 'This is a deer, they are prey' 'This craft was fun to make' Medium A comment identifying the; biological, social or ecological needs of the species. A comment explaining why the individual enjoyed the task or fully completing a task with an addition to the set activity 'This is a lion; they live in groups called prides to make hunting more successful' 'I enjoyed this task because I learnt about giraffes, and I made a hanging feeder for enrichment' High A comment identifying the individual behaviors of a species and why they perform them. A comment asking in-depth questions about a species and or producing a craft/enrichment for a species and explaining why they chose components for it 'This is a meerkat, they live in groups called mobs, each meerkat has a role, and all perform sentry duty to watch for potential predators' 'When I go to a Binturong enclosure I smell popcorn, what does that do?' 'I made an enrichment item for a robin; I chose to make it like this because...here are some instructions on how to make it'

Data Analysis
Coding was carried out independently by the primary researcher and a pilot sample was checked for inter-observer agreement. Coded agreement was greater than 80% and thus deemed robust as a method to use. Total number of engagements, as measured by number of submissions or number of comments were compared across modes of delivery using a chi squared goodness of fit test. Level of submission (high/medium/low) between the different modes of delivery were compared with a chi squared test for association. An ordinal regression was run to test for relationships between the time from the start of the VSS (as measured by the delivery of each activity) and the level of submitted content or comments.
As questionnaires were anonymized it was not possible to match responses pre and post VSS. As such, the data was treated as two independent samples. Daily time spent together on activities pre and during the VSS was not normally distributed but of equal variance (p > 0.05) and so was analyzed for significant difference using a Mann-Whitney U test. Love for nature scores were of unequal variance (p = 0.006) and thus analyzed with an unequal variance t test.
Finally, a chi squared goodness of fit test using specific proportions of activities per SNS category was employed to compare the number of submissions/comments for each category. All analysis was run on Minitab 19 with a significance value <0.05.

Results
In total, 46 activities were offered, of which five were live sessions, 16 recorded sessions and 25 sessions with an option for submission of work. Over the four weeks of the Virtual Summer School, 395 interactions were coded (in the form of comments or submitted items) from the 235 participants that registered for the Virtual Summer School. From the questionnaire responses 19% were ages 7 and under, 33% were 8-9 year-olds, 24% 10-11-year-olds and 24% 12-15-year-olds; 33 families also reported adult involvement.

Activity Type
Participant engagement, as measured by number of submissions and number of comments differed between the modes of delivery. On average, live sessions saw a significantly greater total number of submissions (χ 2 = 63.87, p < 0.001) in comparison to recorded activities and prescribed activities (Figure 1).
Participant engagement, as measured by number of submissions and number of comments differed between the modes of delivery. On average, live sessions saw a significantly greater total number of submissions (χ 2 = 63.87, p < 0.001) in comparison to recorded activities and prescribed activities (Figure 1).

Submission Level
However, higher numbers than expected of 'high' and 'medium' level complexity within submissions were found within the prescribed activities (χ 2 = 34.482, p < 0.001, Figure 2). .

Submission Level
However, higher numbers than expected of 'high' and 'medium' level complexity within submissions were found within the prescribed activities (χ 2 = 34.482, p < 0.001, Figure 2).
Participant engagement, as measured by number of submissions and number of comments differed between the modes of delivery. On average, live sessions saw a significantly greater total number of submissions (χ 2 = 63.87, p < 0.001) in comparison to recorded activities and prescribed activities (Figure 1).

Submission Level
However, higher numbers than expected of 'high' and 'medium' level complexity within submissions were found within the prescribed activities (χ 2 = 34.482, p < 0.001, Figure 2).
.  The level of complexity of submissions also changed as the week progressed with activities being numbered in order of delivery from 1-46 (Z = −4.02, p < 0.001; Figure 3), with higher levels of submission more likely towards the end of the Virtual Summer School week.

Family Time
Questionnaire responses showed that sports, going for walks and spending time outdoors were popular activities carried out together by families, this was closely followed by a baking, crafts, playing games and watching wildlife. In the pre VSS questionnaire participants reportedly spent, on average, 5 h completing activities together as a family. During the VSS, participants self-reported an average of 3.3 h completing the VSS activities together as a family. There was significantly less time spent together completing the activities during the VSS than on family activities Pre VSS (W = 12575, p < 0.001; Figure 4). J. Zool. Bot. Gard. 2021, 2,6 The level of complexity of submissions also changed as the week progressed with activities being numbered in order of delivery from 1-46 (Z = −4.02, p < 0.001; Figure 3), with higher levels of submission more likely towards the end of the Virtual Summer School week.

Family Time
Questionnaire responses showed that sports, going for walks and spending time outdoors were popular activities carried out together by families, this was closely followed by a baking, crafts, playing games and watching wildlife. In the pre VSS questionnaire participants reportedly spent, on average, 5 h completing activities together as a family. During the VSS, participants self-reported an average of 3.3 h completing the VSS activities together as a family. There was significantly less time spent together completing the activities during the VSS than on family activities Pre VSS (W = 12575, p < 0.001; Figure 4).

Nature Appreciation
A two-sample t-test found a significant increase in self-reported love for nature after the VSS (t = 3.14, df = 164, p = 0.001, Figure 5), with a medium effect size (Hedges' g = 0.44).

Nature Appreciation
A two-sample t-test found a significant increase in self-reported love for nature after the VSS (t = 3.14, df = 164, p = 0.001, Figure 5), with a medium effect size (Hedges' g = 0.44).

Nature Appreciation
A two-sample t-test found a significant increase in self-reported love for nature after the VSS (t = 3.14, df = 164, p = 0.001, Figure 5), with a medium effect size (Hedges' g = 0.44).

Activity Category
Chi-squared goodness-of-fit tests to compare the number of submissions for each category based on the number of activities per category found a significant difference in submissions. The categories that involved child-driven and awe activities provided a higher number of submissions and involvement of mentors and entertainment resulted in a lower number of submissions for the overall number of activities that were involved in each category (χ 2 = 75.30, df = 14, p < 0.01; Figure 6).

Activity Category
Chi-squared goodness-of-fit tests to compare the number of submissions for each category based on the number of activities per category found a significant difference in submissions. The categories that involved child-driven and awe activities provided a higher number of submissions and involvement of mentors and entertainment resulted in a lower number of submissions for the overall number of activities that were involved in each category (χ 2 = 75.30, df = 14, p < 0.01; Figure 6).

Discussion
The results suggest a mixture of discovery types enables both higher participant engagement with nature-based activities (by including live sessions) and higher levels of complexity in work produced (by including prescribed discoveries), ensuring families are both inspired and have the opportunity to take their learning deeper. They also suggest

Discussion
The results suggest a mixture of discovery types enables both higher participant engagement with nature-based activities (by including live sessions) and higher levels of complexity in work produced (by including prescribed discoveries), ensuring families are both inspired and have the opportunity to take their learning deeper. They also suggest VSS discoveries enabled knowledge and understanding to build over time, as evidenced by increasingly complex submissions as the week progressed.
Dettmann-Easler & Pease [31] found that summer schools were effective in enhancing positive environmental attitudes the longer individuals participated and Kruse and Card [10] also found individual's biodiversity knowledge increased after just one week. The findings here, suggest that the online nature of this provision has not affected the knowledge and attitude changes that would be expected from live delivery of these types of educational programs.
Our results show a higher number of interactions in the live session compared with the recorded or prescribed activities, as also found by Miller et al. [32], who demonstrated a reported increased engagement and empathic concern towards species in live sessions. However, this mode of delivery did result in the lower quality submissions suggesting a more superficial level of learning was occurring, despite suggestions of engagement correlating with knowledge retention [3,32,33].
Concerns over the impact of digital education during COVID-19 have been raised [34] and research has shown that hands-on learning through interactive activities significantly increases learner's knowledge of conservation [10,35] and the learners' overall value of biodiversity [36]. Despite this being a virtual program, the mix of activities allowed for time away from the computer, social interactions and interactions with nature. The higher level of content in the submissions for the prescribed activities support this as a valuable aspect to the program design. Online zoo education programs should ensure interactive activities such as 'Enclosure Design Challenge', are included to improve knowledge and attitudes and ensure a higher continuous engagement.
The categories labelled 'entertainment' and 'involvement of mentors' had a much lower than expected participant engagement. These specific categories required the participation of a parent. While it is possible that the participants engaged with these activities, there were fewer interactions or online engagement to evidence this. The time spent as a family on the VSS activities, though less than the self-reported time spent together as a family on the whole, still demonstrates a high proportion of that family time was given to VSS activities. Time spent with a parent during the lockdown periods averaged around 30% with some variation between countries [37]. Most of this time was spent monitoring their child during an online class but not necessarily interacting [23] and parents reported being busier than usual due to the lack of family support and therefore had less time to spend with their child(ren) [38].
Therefore, it is encouraging that the VSS supported families to spend some of their time together on activities and provided some evidence that this program was not just an isolated screen-based activity. However, feedback from some parents, noted the additional pressure to be available for these tasks and so setting out activities and resources in advance can help working parents to plan for the next day. Aiming to incorporate activities that encourage children to discuss issues within nature with family members may be another way forward to bring in more involvement of mentors and further research to investigate this would be welcome.
Developing a human-nature connection (HNC) in children is embedded into the World Zoo and Aquarium Conservation Education Strategy which sets out to "create connections with nature and foster empathy to wildlife" [6]. Giusti et al. [30] found that human-nature connection (HNC) develops in children when they perform certain actions and experience certain emotions. Child-driven, awe and involvement with animals had the highest level of engagement in the VSS. These categories included lessons such as all three Animal Encounters, Enrichment, Creative Creatures' competition and Enclosure Design Challenge all of which allow children to interact or stimulate emotions, thus linking and backing up the self-reported increase in connection to nature that the participants had. Increase in concern for nature due to educational interventions is well documented [39]. However, these gains in knowledge and attitudes are often observed in those that already have had experience and connections with nature [40,41].
A high average appreciation for nature is seen even in the pre VSS questionnaire and it is likely that many of the families participating had visited Edinburgh Zoo beforehand and or attended a previous summer school therefore already had an interest. In addition, though it has been suggested that video content cannot replicate the up-close zoological experience [32], online content can lower barriers; either financial, physical or mental, that may otherwise prevent those from accessing the real time zoo experience. The general feedback on the VSS demonstrated the accessibility of the online format, with asynchronous content allowing participation at a time to suit and in reaching families that wouldn't normally be able to attend a live holiday club due to time or distance.
Several of the VSS lessons were categorized as 'thought-provoking', which included the sessions; Commentary Challenge, Scottish Discoveries and Zoo Q&A. All these lessons included instructions which asked participants to reflect on a species and discuss the conservation issues. These 'thought-provoking' lessons had higher numbers of interactions compared to what would have been expected if all lesson engagement was equal. The nature of the family format of submissions through the Google Classroom prevented the data being analyzed by age. Without relating these engagements to age categories, we cannot reflect on age effect specifically; however, 52% of participants were under the age of 10. Due to development of theory of mind and linguistics, younger participants can engage in though provoking activities, but older children from the age of 10 are more likely to express a wider range of emotions [12]. Mid-teens were under-represented during VSS and Hughes et al. [42] showed they demonstrated the lowest connection to nature. Future programs with similar levels of evaluation are needed to ensure strategies of engagement, with this demographic, have an equal level of success at bringing about connections with nature and behavioral change.

Conclusions
Adapting to online education in short timeframes is challenging but important in order to continue to inspire a love for nature and allow families with children of all ages to engage with conservation organizations. Education is a key role in zoos in order to move people to action to help protect habitats and species. Despite some concerns as to the effect of lockdown on education and, in particular, the increased use of digital technology, these results demonstrate that it is possible to help families appreciate and connect to nature, using a mixed activity approach, positively via digital interactions without large amounts of screen time, while at the same time lowering barriers to engagement and increasing accessibility for all.

Institutional Review Board Statement:
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of the University Centre Sparsholt (16th Dec 2020, approval code: UCSEC 0220).
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
A file with prepared data will be available on the ResearchGate profile of the corresponding author (MSF).