The Gamification of XFEL Education Using XFEL Crystal Blaster

Abstract

Novel groundbreaking techniques, such as serial femtosecond crystallography (SFX), which utilizes X-ray free-electron lasers (XFELs), have led to impressive advances in the field of structural biology. However, educating the next generation of scientists on this complex, advanced, and continuously evolving field can be challenging. Gamification has been shown to be an effective strategy for engaging new learners and has a positive influence on knowledge acquisition, student satisfaction, and motivation. Here, we present an educational game, XFEL Crystal Blaster, aimed at increasing middle and high school students’ exposure to advanced topics in crystallography. This simple and accessible game is available on multiple platforms, is intuitive for gamers, and requires no prior knowledge of the game’s content. The assessment of students’ experiences with the game suggests that the XFEL Crystal Blaster game is likely to develop some introductory knowledge of XFELs and X-ray crystallography and increase interest in learning more about X-ray crystallography. Both of these outcomes are key to engaging students in the exploration of emerging scientific fields that are potential career pathways.

1. Introduction

The recent development of X-ray free-electron lasers (XFELs) [1,2,3] and their application to novel techniques in the emerging field, such as serial femtosecond crystallography (SFX) [4,5,6], has allowed researchers to push the boundaries of protein structure analysis. Empowered by new science and technology, researchers can now probe biochemical reactions on femtosecond timescales, using nanocrystals at room temperature, and stitch together movies of molecular mechanisms in action [7,8]. In order to achieve this major milestone in structural biology, interdisciplinary teams of scientists worked together to develop novel sample handling and delivery systems [9,10], data analysis and processing algorithms [11,12], and time-resolved triggering mechanisms [6,13] to enable the widespread use of SFX. Though incredibly complex and cutting-edge, the techniques and practices developed for SFX are becoming more mainstream and are being adopted by synchrotron facilities [14,15,16].

While XFEL methods and technology are becoming more accessible, education on these new and complex techniques and their capabilities lags behind. For this reason, considerable efforts have been made by the NSF-funded Science and Technology Center (STC), Biology with X-ray Free-Electron Lasers (BioXFEL), to provide progressive education on advancing technologies [17]. Across science, technology, engineering, and mathematics (STEM) fields, these challenges are being addressed through the use of new pedagogies, employed to increase the rate of adoption and engagement among younger scientists. Perhaps most notably, creating engaging and meaningful digital learning environments that implement game-based education within STEM fields has recently captured the attention of students born into the age of digital technology [18]. Generation Z, the first true digital natives, are often categorized as having shorter attention spans, being capable of multitasking, and learning through visual graphics, interactivity, and games [19]. As cohorts of learners evolve, so should the approach to teaching them. In fact, changes in science education have included the development of novel teaching methods that meet the needs of today’s learners, such as instructional approaches that use video and virtual technologies. One new digital approach, namely gamification, has been tested in multiple education settings and with different age groups. Gamification typically includes gameplay using elements such as point scoring, peer competition, teamwork, and score tables to drive engagement, with the goal of helping students assimilate new information and/or testing their knowledge [20,21]. One major challenge for educators seeking to use educational games in their classrooms lies in game creation, as most educators do not have the skills, knowledge, or resources to create customized games. To facilitate the process, researchers have created innovative gamification networks [22] and collaborative authoring systems [23] designed to assist educators.

In general, gamification has been found to positively impact students’ motivation for learning science and results in a positive attitude towards gamification in science education [24]. Early evidence also suggests that gamification can have a positive influence on knowledge acquisition, as well as influence student satisfaction and motivation, when used as a supplement to, not a replacement for, traditional learning models [25]. Furthermore, the integration of gamification in science education has been shown to significantly increase students’ engagement and support learning of related classroom activities [20]. These benefits, coupled with increased accessibility through ubiquitous technology such as smartphones and web browsers, make education games an attractive supplement to learning, particularly for complex scientific topics.

The emerging field of interdisciplinary XFEL sciences promises to change the landscape of structural studies and has created an opportunity to develop a new and diverse scientific workforce. Foundational education on the scientific principles of and exposure to XFEL science concepts are critical to both attracting new researchers to the field and to the long-term success of the field. It is in these early stages that the infusion of talent and diversity will be most influential to the future direction of the field. Unfortunately, early exposure to advanced science topics, such as structural biology and macromolecular biochemistry, is uncommon in high school and undergraduate science courses. While some structural biology programs, activities, and lesson plans are tailored to undergraduate students [26,27,28,29], typically students encounter these topics later in their educational pathways, after they have made important decisions about their future careers. To effectively increase interest in the field, it is critical that the importance of structural biology and related disciplines are impressed upon students at an earlier age.

Here, we present a modest educational game, XFEL Crystal Blaster, aimed at increasing middle and high school students’ exposure to advanced science concepts associated with XFEL science and techniques, particularly SFX and crystallography. This simple and accessible game is available on multiple platforms, is intuitive for gamers, and requires no prior knowledge of the game’s content. It employs retro-style graphics that demonstrate the SFX data collection process, the rationale for such experiments, and a brief explanation of real-life experimental results and their significance. The game starts by setting the scene and goals for the “experiment”—to solve the crystal structures of three important proteins so researchers can determine how they work. The simple gameplay mechanics are appropriate for younger audiences and encourage gamers to play often to improve their scores.

2. Materials and Methods

2.1. Game Design and Creation

The game was developed using the Unity^® Game Engine by former University of Buffalo undergraduate student and BioXFEL intern, Fiacre Kabayiza. This platform was chosen because it is a free game engine with built-in, cross-platform support. This allows developers to build and maintain one codebase, while being able to deploy to iOS, Android, Windows, Mac, and the web. It was coded in C#, while Git was used for version control with source code hosting provided by Bitbucket™ from Atlassian™ [30]. Music and sound effects were downloaded from Freesound.org [31]. Art assets were custom-made or obtained from the OpenGameArt [32] website, with some reference art from The Spriters Resource [33] (Figure 1A). The game code was original with the exception of the “UI Modal” asset by Gravitons, downloaded from the Unity Asset Store [34]. Currently, the game is freely available but closed-source; however, the developers plan to make it open-source in the near future to allow others to make modifications and additions to the game. The game is also hosted for web play and download on the BioXFEL website using itch.io [35], which allows developers to upload content, design a page for it, and track views and downloads.

2.2. Gameplay

The XFEL Crystal Blaster game is designed as a simple yet engaging game that integrates scientific and educational content in a manner that emulates an actual beamtime experiment at an XFEL facility. The virtual setup includes a pulsing XFEL source, displayed as a brief flash of rainbow light, the serial delivery of protein crystals, and an X-ray detector (Figure 1B). As with a typical SFX experiment, a nozzle continuously streams protein crystals through the path of the X-rays and the X-ray pulses (sometimes) diffract off of the passing crystals. As the crystal passes through a set of crosshairs, the user must tap to fire the XFEL, a slight deviation from a realistic setup that affords an interesting gameplay mechanic. If timed properly, the crystal is hit by the XFEL beam and subsequently destroyed, producing a diffraction pattern. The player is guided through three different levels of increasing difficulty, where the crystals move faster each time the player levels up. Upon the collection of a complete dataset (an unrealistic 10 frames), the user is rewarded with a video animation of the resulting protein structure and a brief description of the protein target and why it is of interest to researchers (Figure 1C). Protein targets were selected to represent realistic light-activated proteins of interest to BioXFEL and the greater scientific community: Rhodopsin (pdb 4X31) [36], Photosystem II (pdb 4UB6) [37], and PYP (pdb 5HD5) [38]. These canonical protein targets of initial SFX experiments not only provide realistic representations of BioXFEL scientists’ work, but also conveniently exist in three different colors (purple, green, and yellow, respectively), helping to visually distinguish the different crystals within the game. The animated protein structures were created in PyMol [39]. Upon completion, the player is able to enter a name on the leader board of “Top Scientists” and compare their score to previous playthroughs (Figure 1D).

2.3. Implementation

In an attempt to reach the broadest audience possible, XFEL Crystal Blaster was simultaneously released on multiple platforms, including the iTunes Store on iOS, Google Play for Android, and the BioXFEL website (www.bioxfel.org/game, accessed on 30 March 2022) in September 2016. All versions are free to download and play. Users are not required to watch advertisements, and there are no opportunities to make in-app purchases. As of March 2022, approximately 294 and 288 downloads were reported on iTunes iOS and Google Play, respectively. The game has 35,200 impressions on the iOS App Store, and more than 6800 users have accessed the game via the BioXFEL website.

In addition to the game’s online presence, the game also has been featured at several in-person events including a poster presentation at the 2018 LCLS/SSRL Users meeting and at the 2017 Annual International BioXFEL Conference in Las Vegas. The game has been integrated into a virtual arcade game system using the platform gather.town and has been showcased at several online BioXFEL events (Figure 2A). However, the game had its largest audience at the USA Science and Engineering Festival Exposition in Washington DC in 2018. The Exposition is the country’s largest science and engineering fair, which boasts more than 350,000 attendees and over 3000 hands-on exhibits. During this event, middle school and high school students played the game, competed against each other, and downloaded XFEL Crystal Blaster on their personal devices (Figure 2B). It is estimated that several thousand students engaged with the exhibit during the three-day event. BioXFEL representatives also led demonstrations on molecular and protein model building, streamed BioXFEL educational videos (available on www.youtube.com/BioXFEL, accessed on 30 March 2022), and talked with students about what it is like to be a scientist (Figure 2C).

XFEL Crystal Blaster has been featured on the popular PDB101 Education Corner managed by the Research Collaboratory for Structural Bioinformatics (RCSB), which operates the US data center for the global Protein Data Bank (PDB) archive [40]. The PDB101 Education Corner newsletter and website helps teachers, students, and the general public explore the 3D world of proteins and nucleic acids. XFEL Crystal Blaster is also installed as an exhibit at the Buffalo Museum of Science and was briefly featured on the touch screen display in the lobby of the Stanford Linear Accelerator Center (SLAC). The game is frequently used in local high school outreach activities and for open house events hosted by BioXFEL to provide a hands-on introduction to the use of XFELs and SFX.

2.4. Game Assessment

The effectiveness of the game’s approach to introducing students to XFELs and X-ray crystallography was assessed through an online survey of 78 high school students from three different schools who were asked to play the game as part of their regular science lessons (Supplementary Materials). Student feedback was used to make improvements to the game, analyze educational outcomes, and assess the overall effectiveness of this educational game.

The XFEL Crystal Blaster Game Assessment survey was designed to collect students’ views of the educational value and game functionality of the XFEL Crystal Blaster Game. The survey also collected students’ recommendations for improving the game. While the XFEL Crystal Blaster Game had been publicly available since September 2016, no formal assessment of it had been conducted to determine its impact on game players.

The XFEL Crystal Blaster Game Assessment survey was developed by the BioXFEL Education and Diversity Director and modified by the evaluator prior to its distribution in Fall 2021. The game and survey were introduced to high school teachers via an email invitation which requested that teachers incorporate the game into relevant science lessons and allow their students to play the game and provide feedback by completing the survey. Teachers were compensated for their time introducing the game to students and providing class time for students to participate in the evaluation. Teachers from three high schools facilitated students’ participation in the evaluation, resulting in 78 survey responses from students.

The survey consisted of three sections. The first section collected brief demographic information from students and included two items asking about their views on learning science in school and outside of school. The second section of the survey collected data on students’ perceptions of the educational value of the game, including their prior knowledge of XFELs and crystallography, and the extent to which they learned from and became more interested in the field of X-ray crystallography as a result of playing the game. Section three collected students’ views of game functionality, including difficulty and recommendations for improvement. The findings and conclusions of the evaluation are presented here.

3. Results

Assessment Results

The majority of students responding to the survey were in Grade 10 (Figure 3A) and were nearly equally distributed by gender. Most students (90%) responding to the survey indicated that they liked learning science in their classrooms at least some of the time, and 40% of students indicated that they also enjoyed learning science outside of school time.

All students who played the game reported that they had no prior knowledge of X-ray crystallography, and 40% of them responded that they had no prior knowledge of proteins or protein structure. On a scale of 1 to 10, with 1 being “too easy” and 10 representing “too difficult”, students’ average rating of the game’s difficulty was 3.8, indicating that the game content was neither too difficult for students to understand, nor so easy that the game might be unappealing to high school students. Overall, a majority (87%) of students reported that the level at which science content was presented was appropriate for their understanding, demonstrating that the XFEL Crystal Blaster game is accessible to students with limited or no knowledge of the game’s content. There also were no differences in the perceived difficulty of the game by students’ grade level or gender.

Among all students taking the survey, 92% reported that the game was somewhat or very effective as an introduction to the field of XFELs and X-ray crystallography. Students’ reports of effectiveness were very similar regardless of students’ grade level or gender. The impact of the game on students’ interest in learning more about the field of X-ray crystallography was also measured, with 62% of students reporting that the game had increased their interest in learning more about the science content presented in the game. The game was particularly effective at increasing the interest of students who reported engaging in science learning outside of school, for example watching science programs, performing experiments at home, or participating in an out-of-school science club. Of this group of 32 students, 84% indicated that the game had increased their interest in learning more about X-ray crystallography. The game had no impact on students’ interest in learning more about the field for students who indicated that they were not interested in science learning in school or outside of school.

Students also offered suggestions for improvement to the game. Most recommendations were regarding game functionality, such as adding more levels to the game and differentiating between levels, slowing down and/or shortening the text descriptions, and increasing the difficulty of the game. Students’ insightful observations and questions about the game indicated that they likely reviewed the game multiple times. As an example, one student asked if the speed at which the crystals dropped was an indication of difficulty in X-raying the crystals (i.e., slow-moving crystals were harder to X-ray). While this is not a direct recommendation, this question provides an opportunity to improve the game in ways that contribute to students’ accurate conceptions. One potential drawback of gamification in science is the unintentional introduction or affirmation of students’ naive conceptions of science, similar to the negative impact of poorly developed models. Providing students the opportunity to offer recommendations and ask questions in the context of game use in a classroom setting will contribute to the improvement of the game; therefore, the assessment of the game should be conducted periodically.

4. Discussion

Recent Updates and Future Design

In response to feedback provided by our users, the game was updated to address some of the issues mentioned in the survey. The User Interface (UI) resolution was increased to better scale across different platforms, the introductory cutscene was made faster, and individual dialogue lines are now skippable. In addition, a “Hard” mode was added, where the crystals are released from the sample injector at a faster and more variable speed. More work is planned to offer different gameplay variations and new crystal structures that reflect the current state of the field. This will include more examples of GPCRs and common drug targets that are now routinely used in XFEL studies.

5. Conclusions

Findings of the assessment of the XFEL Crystal Blaster game suggest that the game provides an accessible learning experience to high school students regardless of the students’ grade level, gender, or prior knowledge of the game’s scientific content. Further, the game was an effective vehicle for introducing students to the field of XFELs and X-ray crystallography and for increasing students’ interest in learning more about these fields. Interacting with the game was somewhat more impactful for students who enjoyed learning science in out-of-school activities. Overall, the assessment suggests that students who engage with the XFEL Crystal Blaster game are likely to develop some introductory knowledge of XFELs and X-ray crystallography and increase their interest in learning more about X-ray crystallography. Both of these outcomes are key to engaging students in the exploration of emerging scientific fields that could be future career pathways.