QuarkNet: A Unique and Transformative Physics Education Program

Abstract

The QuarkNet Collaboration has forged nontraditional relationships among particle physicists, high school teachers, and their students. QuarkNet centers are located at 50+ universities and labs across the United States and Puerto Rico. We provide professional development for teachers and create opportunities for teachers and students to engage in particle physics data investigations and join research teams. Students develop scientific knowledge and habits of mind by working alongside scientists to make sense of the world using authentic experimental data. Our program is based on a classroom vision where teaching strategies emulate closely the way scientists build knowledge through scientific and engineering practices. Program outcomes show that student engagement in research and masterclasses does develop an understanding of the process of scientific discovery and science. Teachers provide classroom environments that model scientific discovery and science practices and take advantage of opportunities to expand their involvement in the profession of science teaching. We describe success factors that enhance local center programs, and discuss important benefits of the program that flow on to the university faculty.

1. Introduction

In its 20th year, QuarkNet, a teacher professional development program, is embedded in the particle physics research community. QuarkNet partners high school science teachers and their students with physicists working in experiments at the scientific frontier. These experiments are searching for answers to fundamental questions about the origin of mass, the dimensionality of spacetime and the nature of symmetries that govern physical processes. Physicists and teachers bring the excitement of modern physics—dark matter, dark energy, the Higgs boson, even gravitational waves—to high school physics classrooms.

QuarkNet is committed to building learning communities associated with scientific research groups. Physicists make long-term commitments to mentor high school teachers, to provide opportunities for their QuarkNet colleagues to learn how scientists make discoveries and to share large datasets and analysis tools in appropriate ways for high school classrooms. We develop personal relationships between physicists and teachers, and among teachers themselves. Our centers are places where teachers are comfortable revisiting physics content (modern and classical), discussing teaching and learning, reviewing/exploring instructional materials and ideas, and asking the “silly” questions that otherwise might never be asked. Change takes time, and teachers need support as they implement new teaching strategies. QuarkNet provides that space and time.

2. Background

QuarkNet is based upon and evolved from experiences that are unique to experimental particle physics research:

• Typically, experiments are of long duration, with extensive lead times, construction periods and data-collection and analysis periods. Experiments can be several decades long from inception to execution to publication.
• A well-trained, enthusiastic workforce of scientists and technical personnel (in significant numbers) is needed to effectively carry out such experiments. Experiments range from several hundred to several thousand participants.
• Proponents of scientific and technical proposals to the National Science Foundation and other governmental agencies must identify, plan for and carry out a significant effort in education and outreach as a component of their programs.
• National and international efforts continue to underscore the urgent need to attract young people to scientific and technological (STEM) careers and to train them well. For example, in the United States, the Next Generation Science Standards (NGSS) [1] call for science education to more closely resemble science. We understand from colleagues that scientists and educators in other countries have similar interests. However, our program focus is on the United States.

QuarkNet provides a coherent response to these and other issues and develops a national community of researchers and educators working together on frontier scientific research. QuarkNet is led by a national group of educators and physicists with many years of experience in professional development workshops and institutes, instructional materials development, and teacher research programs. The project includes over 50 centers at universities and research labs in 27 states, the District of Columbia and Puerto Rico, where 82 physicist mentors and over 550 teachers collaborate locally to enhance the teaching and learning of physics.

The program develops teachers’ knowledge and skills to support student investigations that model scientific research. QuarkNet offers research internships for teachers and students, masterclasses [2] for students, research-based workshops and ongoing support for teachers, and provides instructional materials and access to online datasets. Instructional materials are presented in an online Data Portfolio (DP) [3] that organizes them by data strand and level of student engagement. Level descriptions explain the data analysis skills that students apply at each level ranging from building basic analytical skills to designing scientific investigations. Level 3 e-Labs [4] use the internet and distributed computing to provide an opportunity for students to organize and conduct authentic research, to experience the environment of scientific collaborations, and possibly to make real contributions to a burgeoning scientific field.

3. QuarkNet Framework

Like the United States particle physics research program, QuarkNet is a national program implemented at universities and laboratories across the country. A research group joins the program by creating a local QuarkNet center that offers opportunities for high school teachers and their students to share the excitement of current physics research. Centers support national program goals and objectives, offer programming within the national program framework and participate in national program components such as workshops and student activities. Centers receive program assistance from a staff of master teachers and IT specialists. An instructional designer provides quality control for instructional materials using science-education-based principles [5,6]. Outside evaluators assess the effectiveness and outcomes of the national and local programs using professional-development-based principles [7,8].

A local QuarkNet center program develops in three stages. Year 1 provides an opportunity for two teachers to hold summer internships working on the mentors’ research project and to attend a week-long data camp at Fermi National Accelerator Laboratory (Fermilab) with teachers from other centers. In Year 2, these lead teachers and their mentors offer a three-week research-based program for other local teachers. Recently, these sessions have focused on studying cosmic rays using a QuarkNet classroom detector [9]. In Year 3 and beyond, local centers conduct a one-week follow-on program that often includes offerings by national staff and fellows (master teachers steeped in QuarkNet activities).

3.1. Scientific Discovery

QuarkNet engages teachers and students in research, masterclasses, and e-Labs to develop an understanding of the process of scientific discovery using current scientific data. What do we mean by scientific discovery? For most people, including most teachers, new scientific discoveries are something they learn about as news—like an event reported online, in a newspaper, magazine or a television broadcast. While scientific breakthroughs may seem like eureka moments, years of research are behind every discovery. Scientific discovery is actually a journey, not an event. It was 50 years from the time Peter Higgs proposed a new particle responsible for generating mass to its discovery at the European Organization for Nuclear Research (CERN). It took 100 years for scientists to see evidence for gravitational waves predicted by Einstein’s theory of general relativity. What took so long?

Our teachers and students learn about and experience part of these scientific journeys. We present scientific discovery as a research cycle [10]. Research starts with what we know and the questions we ask. In particle physics, the Standard Model, a framework for our current understanding of matter, leaves fundamental questions unanswered such as: What are dark matter and dark energy? What happened to antimatter? Are there extra dimensions of space? Scientists build tools to study these questions, but some discoveries must wait for more sophisticated tools. Particle physicists turn to instruments that break the old barriers of energy, precision and intensity to explore new frontiers. Scientists look for new science by looking for the unexpected. They publish claims based on evidence and reasoning, to use terminology in the NGSS. These claims lead to new questions, and the journey continues. Offshoots of the journey are the ideas and technologies of particle physics that enter the mainstream of society to transform the way we live. For example, Fermilab’s superconducting wire and cable are at the core of MRI diagnostic imaging tests, and the World Wide Web was developed by particle physicists at CERN to facilitate communication in experiments with thousands of members.

3.2. Center Success Factors

Given this design, QuarkNet is actually more than 50 programs that meet the needs and interests of local teachers. Over nine years of evaluating centers (observations, teacher and mentor interviews, classroom visits, etc.), evaluators have developed success factors that clearly characterize effective local programs [11]. Staff and fellows assist centers in meeting these factors. Briefly, factors include:

• A strong teacher in aspects such as program development, delivery, classroom implementation and follow-up leads the center;
• A strong mentor who is not only a researcher but is familiar with physics education and teacher professional development leads the center. (We have found that “success” is having either a strong teacher leader or a strong mentor; both are helpful but do not appear to lead to greater success.)
• Participants who meet regularly;
• Centers that offer meaningful activities, not just talks and trips;
• Participants that directly address classroom implementation of instructional materials and activities.
• Centers that receive specific support and/or follow-up from staff such as helping troubleshoot cosmic ray detectors;
• Some centers that may raise money for additional activities and/or have additional grants;
• The participant base that is stable. (However, there is turnover, as older teachers retire or move to new positions and new teachers join.)
• Activities that address and support teacher professionalism such as attending meetings of professional organizations;
• Successful centers that establish a local learning community.

4. Evaluation Activities

The QuarkNet program is not static, but has evolved to reflect changes in the education context in which we operate and in response to findings from formative evaluation. QuarkNet today has become better because of this input, but we strive continually to be a stronger program.

We discuss the evaluation approach from the most recent five-year grant. Based on program goals, the following questions guided evaluation for teacher and student program components:

• Study Question 1: To what extent does QuarkNet create opportunities for teachers and support them in providing opportunities for students with a goal to increase their scientific literacy and learn cutting-edge physics and research?
• Study Question 2: To what extent do QuarkNet teachers create opportunities by providing the necessary environments for students to learn particle physics content and research?
• Study Question 3: To what extent do students learn particle physics content and research, and, more particularly, LHC physics and research with a goal toward increasing students’ awareness and knowledge?
• Study Question 4: To what extent do teachers provide workshops and presentations to their colleagues, share ideas with colleagues, and provide services in their schools with a goal toward becoming more professional, including contributing to the quality and practice of colleagues in the field of science education?

4.1. Evaluation Methods for Teacher Components

Each year, the evaluators prepare informal reports after observing a sample of professional development workshops providing information for staff to make decisions about program improvement. Since 2014, evaluators have used an observational protocol based on 13 criteria for general best practices for all professional development and 6 specific criteria for science education/QuarkNet. These criteria address the ultimate purpose to create an effective learning environment for participants, to engage participants in science process that increases scientific literacy (particularly with regard to data use), and to provide mechanisms to ensure that what they are learning is transferred to the classroom.

Evaluators conducted a three-year implementation study and provided an informal report on classroom implementation by teachers participating in the 2015–2017 Data Camp and workshops. Over several years of evaluating QuarkNet, there have been multiple attempts at collecting student data and various methods used to obtain implementation data. Three methods were developed and tested in a pilot study during the 2014–2015 and 2016–2017 school years as possible strategies for probing implementation. The method used in the 2015–2016 school year, utilizing teacher logs, student work and student reflections, proved to provide the most data-rich and cost-effective way to gather information.

4.1.1. Data Camp

Data Camp is an introductory workshop for both new and veteran teachers of physics and physical science who either have had little to no experience with particle physics and are looking for a kick-start, or who have had little experience with quantitative analysis of particle physics data. This camp is different from a traditional teacher workshop because it puts emphasis on an authentic data analysis experience, in which the teachers get to play the student role by learning a challenging topic they may initially have known very little about. A total of 102 teachers from 31 centers have attended Data Camps 2013–2016, led by 4 fellows. Evaluation data gathered from the 2014 and 2015 Data Camps offered by the same instructors resulted in determining the workshops were well run and stable.

4.1.2. Workshops

Workshops offer teachers an opportunity to experience an investigation, as their students would. Each provides structure and guidance for teams of teachers to work through a physics “research” project from designing an investigation, through analyzing data to presenting findings. Throughout the process, teachers reflect on teaching and learning, discussing classroom management strategies and skills that teachers need to teach, and what students need to learn in this environment.

We offer two- or three-day workshops at centers. We select the instructor based on experience and availability. In 2015, the workshops adhered to “scripts” that were developed to more closely follow the evaluation criteria; 77% of the general criteria for effective workshops and 66% of specific criteria were “met” or “met to a great extent.” Staff have implemented an annual facilitators workshop to improve these findings.

4.2. Evaluation Methods for Student Research Programs

As with the implementation study, several methods were used to assess the most cost-effective method for collecting data. The study question governing the final choice was, “To what extent do students increase scientific literacy as a result of participating in a student research project?” Possibilities included those used in an evaluation study of DOE student programs by the National Center for Improving Science Education [12]. It was clear from the study that pre- and post-conceptual maps using guidelines and ratings techniques adapted from Novak and Gowin [13] provided cost-effective answers to the question when the maps were assessed qualitatively and quantitatively. Quantitatively maps were scored based on concepts included and, more importantly, appropriate links among the concepts. Since students included “propositional statements” (statements about how the concepts are linked), the extent to which the link was appropriate could be assessed, as well as students’ understanding of scientific process. Post-maps showed a much more sophisticated or scientifically literate understanding of scientific process. Students began (pre) mostly with a map showing “the” scientific method, then had a post showing an iterative process of findings and further questions, decisions based on data, and a community of scientists and other professionals who engaged in experiments.

5. International Outreach

While primarily a national program, QuarkNet has had international connections and collaborations since 1998, early in the program. For example, the initial idea for QuarkNet masterclasses came from meetings between staff teachers and their European counterparts in the International Particle Physics Outreach Group (IPPOG) [14]. QuarkNet staff created the Compact Muon Solenoid (CMS) masterclass as requested by the CMS Collaboration and manage masterclasses offered in the United States. QuarkNet has exported masterclasses to Latin American countries and beyond.

Four countries, Germany, Taiwan, Japan and the United Kingdom, have adapted the QuarkNet program model using cosmic ray studies as the content. Denmark is planning a similar adaptation, and France has enhanced its early efforts in cosmic ray studies outreach from discussions with QuarkNet staff at IPPOG.

With support from the U.S. NSF International Linear Collider (ILC) grant to the University of Oregon, QuarkNet has participated in the ILC broader impacts program, reaching out globally with our cosmic ray studies program and strengthening several international alliances. Today, over half of the QuarkNet cosmic ray detector data acquisition cards (DAQs) are located abroad, allowing our students access to data from around the world.

6. Program Outcomes

We explore the outcomes for teachers and their students, and mentors and their colleagues. Our evaluators report outcomes for teachers and students. Mentors and others in the research groups gain important benefits from their interactions with high school learners. PIs document these through a survey sent to mentors.

6.1. Teacher Outcomes

QuarkNet offers effective learning environments where teachers learn how scientists make discoveries and about particle physics content and research. They share effective teaching practices with one another and become more professional. Overall, professional teachers see themselves as educators, not just instructing one or more groups of students in a particular classroom in one school. Professional teachers are more likely to talk about what they do in more global terms as instruction that affects the larger community, whether it be the local community, school district or state. Also, teachers tell us that participation in QuarkNet helps motivate them to stay in teaching.

It appears to evaluators that teachers who have had successful experiences with QuarkNet, and/or have been participating in effective centers, report continuing their participation and increasing their roles in leadership activities over time (Figure 1). QuarkNet can be said to encourage a number of these professional behaviors in teacher-participants, although the extent to which the behaviors are encouraged appears to vary from center to center. What QuarkNet seems to encourage, as a network of teachers and scientists, is increased professional responsibility, such as:

• Acting as mentors and providing professional development opportunities beyond the QuarkNet centers;
• Becoming members of and attending meetings of professional organizations;
• Serving their schools and districts in various capacities such as working on committees;
• Acting empowered to adjust their teaching and curriculums to serve the needs of their students, such as having students work on long-term projects.

Our program is based on a classroom vision where teaching strategies emulate closely the way scientists build knowledge through science and engineering practices. Teachers create classroom learning environments using instructional materials that provide particle physics examples appropriate for momentum/energy-related topics. Teachers have access to authentic experimental data and offer classroom projects that model scientific research.

Since teachers participate over several years (a few have participated from the beginning, i.e., 20 years), it is possible to see changes over time. As with teacher professionalism, teachers increasingly engage students in long-term projects using e-Labs and cosmic ray detectors peaking at 6–10 years of participation (increasing from about 5% at baseline to 30% at 6–10 years), use higher-level DP activities at a 15–35% increase (thereby engaging students in higher levels of data use), use particle physics examples in teaching classic physics (increasing from 40–60% in the first year to 50–85% by 3–5 years of participation). On average, teachers use best practice strategies (as in NGSS), such as data analysis weekly and design experiments once or twice a month.

The implementation studies included teachers who taught mainly in advanced classes, and therefore could be generalized to about 65% of teachers, as identified in the teacher survey, who also taught at least one advanced class. The teachers in the study reported on DP activities they used with their students. Specific use of data that has been greatly emphasized by QuarkNet was reflected in the classrooms, as was science as it is actually done and authentic science practices. Teachers taught the DP activities as intended, which they found “effective,” in that the activities model the actual process of scientific research.

6.2. Student Outcomes

We evaluate the outcomes for students who participate in activities offered by the national program. Implementation studies show that students in advanced classes appeared to enjoy learning cutting–edge physics using QuarkNet resources. With regard to data use, students recognize that repeated measures, error checking, peer review, and comparison of findings to others’ work improves reliability and validity.

6.2.1. Summer Research

Summer interns with at least four weeks of experience became more scientifically literate, learning for the first time how scientists make discoveries. Student enthusiasm for the summer research program was contagious. Post-test scores on concept maps showed greater understanding of authentic science process, especially linking conclusion back to hypothesis, publishing results, and appropriate or actual use of the library/reference materials. Students who had an average to high score pre-map often had an even higher score on the post-maps, showing that they came to the program well prepared. However, some students with lower pre-map scores achieved a much higher score post, meaning all students can achieve a greater scientific literacy when provided with an authentic research experience.

Results showed that student research experiences increased participating students’ scientific literacy. Pre- and post-program concept map comparisons showed that students increase their understandings of authentic science process to a statistically significant degree when engaged in one of the research programs for four or more weeks. Chains of thought indicated a better idea of how scientists engage in actual research, as opposed to a linear concept of “The” scientific method. Since the “pre” indicated what students learn in school, it is clear that this research experience led to a more complete understanding of how scientific research is actually carried out. QuarkNet student research programs, therefore, can be a model for effective outreach.

6.2.2. Masterclasses

Students who attended masterclasses with at least three days of preparation increased physics content knowledge and data analysis skills. In March 2016 QuarkNet centers with 582 students and 74 teachers participated in U.S. Masterclasses. Years 2013–2015 had similar numbers. We asked all teachers to give three hours of preparation prior to the masterclasses and provided them both specific objectives and activities they could use. Outside evaluation validated the effectiveness of masterclasses. When students were asked the extent to which they liked the masterclass they attended, the overall mean was 4.0 out of a possible “5” and a standard deviation of only 0.9, meaning relative consistency among responses. After attending the masterclass, the students reported being more interested in physics in general. (The mode was 4 out of a possible 5 from 124 responses out of 320 possible.)

According to the Masterclass Study in 2012, changes staff had made to the masterclass and masterclass preparation made a difference in student learning based on the overall effect sizes and statistically significant differences between tests given at three points in the activity—pre, midpoint, and post. Overall, there were statistically significant differences (level < 0.001) between pre-test and midpoint tests, between midpoint tests and post-tests and between pre- and post-tests. That means that masterclass preparation and participating in masterclass were effective in increasing content knowledge and skills included on the test. It corroborates a former finding that students enrolled in physics classes, especially advanced (e.g., honors and AP) physics, are the best candidates to achieve the intended goals at the highest levels. Improvements included: providing an orientation, focusing on and providing effective resources, videos and activities, and providing suggestions to sites for conducting an effective masterclass.

6.2.3. e-Labs

The e-Labs had 1640 “teacher” accounts as of October 2017. These teachers supported 1066 “student” accounts; a student account may support one to many actual students. Other important measures of e-Lab activity involved the creation of e-Lab posters and plots. This year users saved 1387 plots; historical total was 19,430. These plots appeared in some of the 289 posters written by teachers and students; the historical total was 1670. Uploads to the cosmic ray database were 8829 in 2016, adding to the overall total of 78,654. These numbers indicated a solid use of the e-Labs by a broad base of users.

A 2014 study compared student CMS and Cosmic Ray e-Lab posters with contextual data provided by teachers. Of two factors often expressed by teachers in how students would be able to conduct their investigations, one—achievement level—was not a defining indicator of poster ratings. The other—sufficient time to conduct the study—indicated that teachers should choose other instructional options should time be short. Teachers using e-Labs as an introduction to research showed fewest to no discernable gains. It is clear that this level of student engagement requires extensive prior knowledge such as going through the Data Portfolio levels. Teaching practices that appeared to affect higher student scores included introducing all aspects of the e-Lab first or better yet, going through one full example, providing background information, using guided inquiry, reviewing student work, especially students reviewing one another’s work.

The most important common student mistake was not providing evidence to support the hypothesis/claim or making a claim based on limited data. Other areas where students had difficulty meeting expectations included not giving background information or reasons for their hypothesis, not discussing why weaknesses and possible confounding variables could be an issue, perfunctory or incomplete discussion and conclusion sections, and incorrect information in introductions.

6.3. Mentor Outcomes

For the most part, the QuarkNet faculty mentors are self-selecting volunteers. (Mentors do not receive remuneration from the program.) QuarkNet has the advantage of being a well-established and well-regarded program within the high-energy physics community. Most physics departments and the majority of high-energy physics groups in the United States know about QuarkNet. For the past several years, two–three new groups approach us each year to become part of the QuarkNet collaboration. We try to accommodate as many new centers as possible, given our budgetary constraints.

While we do not provide formal training for new mentors, our staff teachers work very closely with them and their lead teachers during their first years with QuarkNet. All of our centers are assessed as part of an overall, annual evaluation of the national program. This includes the leadership provided by both the mentors and lead teachers. Staff follow up with suggestions for improvements.

Several factors motivate these physicists to contribute their time and effort to the program. First is a genuine interest in education and outreach, and a desire to work with teachers and students in their community. Second, for many of these scientists, it is important to have a “broader impacts” component in their research programs. Participation in QuarkNet provides them the opportunity to join an established, credible network of like-minded physicists that is well known within the United States funding agencies. Also, working in QuarkNet helps the mentors gain recognition within both their university and broader communities, and can be helpful for young faculty members under consideration for tenure and promotion. Often, the work done by teachers and students during the summer contributes to the development and testing of hardware for experiments. These efforts are known and appreciated by the leadership of these experiments.

Mentors report that they become the nexus of a community that:

• Improves their teaching.
• Benefits the university through broader impacts activities.
• Enriches their research programs.
• Gives them access to new friends, contacts and collaborators in education and research.
• Benefits their grant writing.

Mentors report that other group members learn that education, research and mentoring are lifetime activities for all.

• Everybody enjoys the opportunity to meet teachers.
• Students share their love of physics and pride in research.
• Younger professionals polish presentation, communication, mentoring and teamwork skills.
• Researchers see how far high school students can go with proper support.
• Researchers learn to value and support the work of teachers. Since burnout/retention is a huge issue for secondary teachers, this is quite significant.

7. Summary

QuarkNet is a unique and transformative physics education program that embeds teachers and their students in a science research community. This strong national program builds long-term relationships that provide opportunities for teachers to create learning environments that more closely model how scientists work; for students to learn how scientists make discoveries, something they do not learn at school; and for physicists to benefit their communities and their research. Each year we strive to improve our overall program. The Data Portfolio was the latest improvement. In our new grant proposal, we have added content for neutrino physics, which would be supported by our partners at Fermilab.