Assessing Conceptual Understanding via Literacy-Infused, Inquiry-Based Science among Middle School English Learners and Economically-Challenged Students

Abstract

The overarching purpose of our study was to compare performances of treatment and control condition students who completed a literacy-infused, inquiry-based science intervention through sixth grade as measured by a big idea assessment tool which we refer to as the Big Ideas in Science Assessment (BISA). First, we determine the concurrent validity of the BISA; second, we investigate the differences in the post-test of the BISA between treatment and control English Learners (ELs), controlling for their performance in the pre-test; third, we analyze the differences in the post-test of the BISA between treatment and control non-ELs, controlling for their performance in the pre-test; and fourth, we examine the relationship between students’ English language proficiency as measured by standardized assessment, and their performance in the BISA among ELs and non-ELs, respectively. Our findings indicate: (a) literacy-infused science lessons with big ideas, implemented through the tested intervention, improved students’ language acquisition and science concept understanding for ELs and economically challenged students (ECs); (b) there was a positive relationship between language and content for both ELs and non-ELs, with a similar magnitude, suggesting that students with a higher level of English proficiency score higher in science assessment; and (c) the lesson plans prepared were successful for promoting a literacy-infused science curriculum via a 5E Model (Engage, Explore, Explain, Elaborate, and Evaluate) that includes three to five of the Es used daily. A pedagogical approach for a literacy-infused science model with big ideas is proposed.

In 2014–2015, the number of English language learners (ELs) in U.S. public schools was estimated at 4.6 million (9.4%), up from 4.3 million (9.1%) in 2004–2005 [1]. English language learners are those individuals who are learning English in school and who speak a native language other than English. In the state of Texas, where this study was implemented in 2016, there were 980,487 ELs in Texas public schools (18.5% of the student population) [2]. Additionally, 51% of U.S. students in pre-kindergarten through 12th grade in the 2012–2013 school year were classified as economically-disadvantaged students (From this point on in the paper, we refer to economically disadvantaged students as economically challenged students, or EC students) [3]. In Texas, there were 59% economically-challenged (EC) students enrolled in public schools in 2016–2017 [4]. EC students are those who qualify for the free or reduced lunch program under the National School Lunch and Child Nutrition Program. Compared to their English-speaking peers, EL and EC students underperform on content-area assessments, particularly in science, at national and state levels. For example, according to the most recently published data on science, the 2015 National Assessment of Educational Progress (NAEP), the percentage of ELs at or above proficiency in fourth-grade science was 9% compared with 40% for non-EL students; the percentage of ELs at or above proficiency in eighth-grade science was 4% versus 34% for non-ELs [5]. In Texas, for the 2016–2017 academic year, only 58% of ELs passed the state standardized science assessment in fifth-grade, compared with 74% of all students; the gap is wider in eighth grade, with 48% of ELs passing, compared with 76% of all students [4]. Also, during the same time period, fifth-grade EC students scored a 66% passing rate and eighth grade EC students scored a 67% passing rate compared to a 74% overall passing rate at fifth grade and a 76% overall passing rate at eighth grade at the state level.

1. Literacy-Infused, Inquiry-Based Science for ELs and ECs

Science is content-area reading, and for ELs and non-EL EC students, it can be arduous, because the language used in science is frequently information-dense, abstract, and technical [6]. Huerta et al. [6] noted that simple or common words may be confusing as the meanings change in the new academic context (i.e., mass or wave); moreover, others (i.e., [7,8]) have indicated that scientific discourse relies on complex linguistic structures and new vocabulary to convey sophisticated ideas and questions. In addition to language, science involves a certain set of practices referred to as science and engineering practices [9] and new ways of thinking about the natural world [7]. Therefore, to facilitate students’ learning, researchers have advocated the integration of literacy and science and determined that this integration can promote improved English literacy skills and scientific thinking for EL and EC students (e.g., [10,11,12,13,14,15,16,17,18,19]). Engaging in inquiry-based instruction has resulted in or promoted improved student learning for the general student population and has been noted by several researchers and scholars for over 50 years (e.g., [20,21,22,23,24,25]). Despite this, the research is scant specifically regarding EL and EC students and the exploration of the relationship between academic language and overarching conceptual understanding in science. However, a few researchers (e.g., [12,13,26,27,28,29,30]) have provided solid evidence that inquiry-based instructions increase, in particular, ELs’ and ECs’ conceptual understanding of science, particularly when literacy is connected to science instruction. The inquiry-based instruction referred to in our study includes engaging ELs and ECs in practices such as asking questions, planning investigations, developing models, and interpreting data, all while promoting language and literacy skills.

In this paper, we present research that is part of a randomized control trial (RCT) longitudinal evaluation in which we investigated the effects of implementing a literacy-infused, inquiry-based science curriculum with middle school Spanish-speaking EL and EC students and how to assess such a curriculum for ELs and ECs [12,31]. We define literacy-infused science as interventions or innovations that combine the learning of the academic language of science with the conceptual, contextual, and content understanding of science. Teaching the academic language of science for ELs and ECs includes direct and discovery learning of science content and science literacy via reading strategies, listening techniques, oral language development skills, and writing conventions. In literacy-infused science for ELs and ECs, students are engaged with the materials which afford them a physical hook to remember the terms in English. They use their full senses to recall academic language. From engaging in such activities, students develop ideas about the natural world and begin to construct their own ideas and meanings in the way described by Piaget [32] and Bruner [33]. Therefore, when students, along with their classroom peers and their teachers, think about learning science, part of it is the envisioning of a process of making meaning and developing science concepts from concrete experiences. Thus, the students engage in science as practice, which refers to “doing something and learning something in a relevant and meaningful way so that the doing and the learning cannot be separated” [34] (p. 34). In this process, it is critical to engage students in doing science, along with the detailed discussion and reflection that accompanies it as they develop overarching science concepts and understanding of facts [35].

2. Conceptual Understanding: Big Ideas in Science for ELs and ECs

We were interested in the teaching of science concepts which we noted as big ideas in science for ELs and ECs and which were taught, learned, and assessed. Bianchini noted that “big ideas draw upon fundamental concepts and methods of a science discipline or are built around real-life issues and dilemmas” [36] (p. 40). She further said that “big ideas are different from topics or themes, which are other common organizing device” [36] (p. 40); this was further supported by the American Association for the Advancement of Science (AAAS) when the AAAS began to use big ideas developed for literacy in science, mathematics, and technology, instead of the term, theme or topic [37]. Bianchini stated that:

… big ideas help students better understand and connect science content and processes. They serve as motivators that pull students into learning about science; they help answer the “So what?” question. Big ideas are organizing principles around which students can arrange facts, concepts, processes, and applications they encounter during learning. Big ideas also help students make connections within and across science units, as well as with other areas of interest and learning.

[36] (p. 43)

Big ideas in science are related to the disciplinary core ideas (DCIs) in the Next Generation Science Standards [9], and represent overarching principles in life, physical, and earth and space sciences. The big ideas develop coherently from kindergarten to Grade 12. They are statements about the major concepts that are encompassed in the curriculum. They are relevant key concepts that should have meaning and connection to a student’s life, and they should promote understanding of events and phenomena in the world [38]. Big ideas are overarching core concepts that, when unpacked, lead to an understanding of an area of scientific knowledge. These big ideas are where content material from differing branches of science strands are brought together with common themes that thread the subjects together. The NGSS [9] has three components: disciplinary core ideas, science and engineering practices, and cross cutting concepts. In this paper, we address the DCIs that are related to the science big ideas for sixth-grade science classrooms as part of a longitudinal randomized controlled trial (RCT) targeted to improve science literacy among EL and EC middle school students (This project was designed and funded by the National Science Foundation [DRL-0822343] to engage ELs and EC non-EL students in middle school science). The approach focuses on literacy-infused science in which (a) teachers were provided with training every two weeks; and (b) students were engaged with science lessons that infused English-as-second-language (ESL), reading, and writing strategies into the curriculum, thus promoting literacy-infused science education.

According to Duncan and Cavera, “there are DCIs for each of the four major disciplines: physical sciences, life sciences, earth and space sciences, and engineering (engineering, technology, and applications of science” [39] (p. 67). These are addressed by the NGSS. Furthermore, Duncan and Cavera indicated that, to be a DCI, an idea must meet four criteria and stated:

First, it must be a key organizing principle within the discipline or across several disciplines; that is, it should be a core idea in the eyes of scientists. Second, it must have broad explanatory power: It should help learners understand and be able to reason about an array of phenomena and problems in the discipline. In this sense, it needs to be a useful thinking tool that is generative for students, and it should help them think about phenomena and problems they may encounter in and out of the classroom, both now and in their future. Third, a DCI needs to be relevant and meaningful for students. It should relate to phenomena and problems that students find intriguing. Fourth, the idea needs to have depth that allows for continued learning over the course of schooling.

[39] (p. 68)

We consider big ideas in science for ELs and ECs to be related to phenomena that students first think about in the world around them—their world. In fact, very young children enter school with daily observations that make them curious about their world as they begin to develop science understandings [40,41,42,43]. The natural curiosity of children and their abilities to think about the world around them are fundamental to the exploration of the big ideas in the current project. Not only has Donnovan and Bransford [44] commented on, but also Koch [35] (pp. 12–15) has shared how new understandings are based on the foundation of prior understandings and experiences. They used the story by Lionni, Fish Is Fish [45]. In this children’s book, a fish learns that he is different from his friend the frog based on his lived experiences as a fish and the tales the frog tells him about dry land and the creatures that live there. New understandings are illustrated as a small minnow encounters a tadpole in a pond. Initially, they think they are both fish; however, eventually, the tadpole grows little front legs and then becomes a frog. The frog then ascends from the water in order to live on land. The tadpole not only grew into a frog, the minnow also grew into a fish. The fish missed his friend. The frog returned for a visit to the pond to see his friend, the fish, and he described all the he had seen on land: birds, cows, men, women, and children. But, as the fish heard the story, he thought about them as fish with wings and feathers, or as fish with legs and hats and coats. Then, the fish decided to come out of the water to see the frog had described, which illustrates his lack of understanding of the environment. However, he began to gasp for air, and the frog gently pushed his friend back into the water. The thankful fish began to develop new understandings that a fish cannot be a frog and that, for certain, a fish is a fish. Koch noted that just like the fish, students begin with what they know as they build new understandings. More specifically, she related the story to big ideas as follows:

Sometimes what children know is rooted in a misconception, and this becomes a barrier to new learning. Childhood misconceptions can become barriers to adult science learning unless they are addressed directly. …In addition to the role of prior knowledge, learning researchers have discovered that factual knowledge must be placed in a conceptual framework to be well understood. This is the big idea or the overarching core concept that, when unpacked, leads to discrete facts.

[35] (pp. 12–15)

Koch specified that one of the big ideas is that different vertebrate groups (fish, amphibians, birds, reptiles, and mammals) have differing identifiable characteristics which provide them the ability to live in different environments, relating their structure to function. In this project, we specified these overarching core concepts for the curriculum and called them big ideas in science for ELs and ECs. Hence, we refer to the assessment of these concepts as the Big Ideas in Science Assessment (BISA). One of the issues lacking in the research literature is the assessment of such literacy-infused, inquiry-based, big ideas in science. Perhaps, the outcome of such research would be surprising to researchers and practitioners; in fact, Olsen et al. stated that, “When teachers focus on assessing central concepts rather than vocabulary or trivial details, they are often surprised at how much their ELs know” [46] (p. 48). Furthermore, big ideas can assist students in seeing the connections between different ideas in science, which helps them be prepared for engagement within their world and even their work [47]. According to Harlen, such connections may increase creativity and innovation and can “prepare students to participate in, rather than being at the mercy of, the rapid changes in occupations and communication using technologies developed through engineering and the applications of science” [47] (p. 5).

3. Purpose of the Study

The overarching purpose of our study was to compare performances of treatment and control condition students who completed a literacy-infused, inquiry-based science intervention through sixth grade as measured by the big idea assessment tool, BISA).. Therefore, the specific purpose of the study was to first determine the concurrent validity of the BISA; second, to investigate the differences in the post-test of the BISA between treatment and control ELs, controlling for their performance in the pre-test; third, to analyze the differences in the post-test of the BISA between treatment and control non-ELs, controlling for their performance in the pre-test; and fourth, to examine the relationship between students’ English language proficiency (also equated with English literacy) as measured by standardized assessment, and their performance on the BISA among ELs and non-ELs, respectively.

4. Methods

This study, part of a larger randomized, longitudinal, field-based research of literacy-integrated science instruction, was provided to ELs and ECs. The broader study was implemented in a school district in Southeast Texas in which 85% of the students were on free or reduced lunch [48]. In addition, 25.9% of the students in the district were African-American and 69.7% Hispanic [48]. The district was chosen because of its (a) learning program (i.e., regular and English-learning) accessibility within the district; (b) positive experience educating EC students of color; (c) consistency in educational philosophy and operations; and (d) standing related to student academic achievement.

5. Participants

In this study, we included participants who were ELs or ECs. The ELs were classified as monitored and supported in English language development by the English as a second language (ESL) program on the campuses. These students were taught in English; however, the students, among themselves, also used Spanish (their native language) in clarifying concepts with each other. The EC students (native language was English) spoke English within the classroom. From 10 intermediate schools, four schools were randomly assigned to treatment (enhanced practice, n = 2) and comparison (typical practice, n = 2) conditions. Both ELs and ECs in a specified school received the same enhanced or typical practice. This was implemented in order to avoid contamination of the intervention between experimental and comparison classrooms. Teachers were randomly selected within the school for participation. Four were in treatment, and eight were in comparison schools (see [31] for more details about the design). Teachers averaged 8.6 years in the profession, and two of the 12 were beginning teachers. Each teacher taught two to three sections. Our study designed to be experimental at the school level, and quasi-experimental at the student level, abiding by the state law [49] that prohibits the random selection or assignment of individual EL students to a program. At the student level, the total number of students within the classrooms at the beginning of sixth grade was 383, with an attrition rate of 28% (which is comparable to the mobility rate of this urban school district as reported by TEA), which resulted in a total of 276 students at the end of sixth grade. For the purpose of this study, only data collected from EL and EC students in the sixth grade were included in the analysis; specifically, 160 ELs (105 treatment; 55 control) and 116 non-EL EC students (48 treatment; 68 control) who completed the intervention at the end of sixth grade were included in the analysis.

6. Research Questions

We responded to four research questions. They are as follows:

• Is there concurrent validity of the BISA as determined by a relationship between the students’ scores on the BISA and their science benchmark assessment among sixth grade ELs and non-EL-EC students, respectively?
• Is there a statistically significant difference in the post-test of the BISA between treatment and control ELs, controlling for their performance in the pre-test?
• Is there a statistically significant difference in the post-test of the BISA between treatment and control non-EL EC students, controlling for their performance in the pre-test?
• Is there a statistically significant relationship between students’ English language proficiency as measured by a standardized assessment, and their performance in the BISA among ELs and non-EL-EC students, respectively?

7. Intervention as It Relates to the BISA

Professional development. The intervention, implemented 28 of the 36 weeks of school during sixth grade, was delivered at the teacher level through professional development (PD) and at the student level via literacy-infused science curriculum and instructional activities. During the three-hour biweekly teacher PD sessions, teachers overviewed upcoming lessons and accompanying instructional materials; reviewed science concepts, clarified their own misconceptions; and conducted experiments and inquiry activities. They were also instructed on ESL strategies that were integrated into our lessons, which are introduced in the next section. The 45-minute daily science instructional intervention included: (a) a warm-up activity for students to think, record, and discuss responses to a science-based scenario or prompt; and (b) a lesson structured with an engaging instructional cycle that encourages students’ inquiry. We focused the literacy-infused science curriculum on academic vocabulary development and extended literacy via science-related expository text. In this way, we sought to improve students’ understanding of science concepts; as well as to facilitate students’ learning of science content through written academic science vocabulary via individual science notebooks (see [12] for a detailed description of the intervention).

Control teachers received 30 hours of district-provided how to teach science professional development, but it was not specifically stated how to implement a literacy-infused inquiry-based science curriculum with big ideas and how to integrate technology in lessons. They were not provided with the literacy-infused, inquiry-based science curriculum; rather, they were provided with the district science curriculum that did not infuse literacy skill development.

Instructional activities. In this project, we adhered to the NGSS, which call for a turn from teaching facts, to helping students construct a rational discussion of phenomena [50]. This shift formed the basis of the literacy infused inquiry-based science curriculum. Herein, we provide an example that was focused on Grade 6 ELs and ECs in their development of the understanding of the Law of the Conservation of Energy. There are five steps in the lesson development for incorporating big ideas for literacy-infused science.

Step 1: Identify national and state science standards that promote understanding of the science concept. Big ideas served as the foundation for the science lessons. However, as big ideas are overarching and often abstract, it was important to identify national and state science standards that address a specific component of the target big idea, and because of that type of relationship, we thought it necessary, then, to assess the concurrent validity between the two. For example, thermal energy transfer is one subcomponent supporting the eighteenth and twentieth century scientist’s, Lavoisier’s and Einstein’s original big idea concept laws that matter and energy cannot be created or destroyed, but they can change form. National and state science standards related to thermal energy transfer are listed in

Table 1. Standards for a Big Idea.

Standard Type	Big Idea and Related National Standards and State Standards
	Big Idea: The total amount of energy in the Universe is always the same but can be transferred from one energy store to another during an event.
NGSS standards	Samples of disciplinary core ideas from NGSS [51]: PS3.B: Conservation of Energy and Energy Transfer (p. 124) Energy is present whenever there are moving objects, sounds, light, or heat. When objects collide, energy can be transferred from one object to another, thereby changing their motion. In such collisions, some energy is typically also transferred to the surrounding air; as a result, the air gets heated and sound is produced. (PS3B-End of grade 5; p. 125) Light also transfers energy from place to place. (PS3B-End of grade 5; p. 125) Energy is transferred out of hotter regions or objects and into colder ones . (PS3B-End of grade 8; p. 126)
State science standards	Texas Essential Knowledge and Skills, Science, Grade 6 [52] 6.9 Force, motion, and energy. The student knows that the Law of Conservation of Energy states that energy can neither be created, nor destroyed, it just changes form. (p. 7) 6.9A The student is expected to investigate methods of thermal energy transfer, including conduction, convection, and radiation. (p. 7) 6.9B The student is expected to verify through investigations that thermal energy moves in a predictable pattern from warmer to cooler until all the substances attain the same temperature, such as an ice cube melting. (p. 7)

.

Step 2: Analyze science standards. Once the related national and state standards are identified, the next step is to analyze the standards by breaking them down to determine the specific concepts and academic science vocabulary required for each standard. For example, Texas Essential Knowledge and Skills (TEKS) 6.9A includes investigate as the cognitive verb, or performance expectation, and includes the words thermal, energy, transfer, conduction, convection, and radiation, as academic science vocabulary. Analyzing standards helps determine where the concepts fit into the scope and sequence of lessons. It is important to determine students’ prior knowledge and beliefs about the concept and to consider potential misconceptions. This helps to define the background knowledge that needs to be taught to provide a foundation for the target concept(s). The entire curriculum is, what we consider to be, literacy-infused science (see [12] for a full discussion of the curriculum with supporting documentation). In the lessons, there is a distinction between everyday language and the academic language of science [53,54,55]. Gee reported that science language includes technical vocabulary and discourse patterns that linguistically differ from daily language [53]. According to Lemke, specific discourse structures including comparisons, descriptions, and definitions are produced within scientific language [55]. Such science language is expected of students in the classroom where the expectation is for them to understand and to employ discourse structures [7]. That may be difficult for ELs and ECs. Fang noted that the specialized features of science language, “… can and do present significant comprehension challenges to adolescent students, especially struggling readers and English language learners who do not have sufficient experience with academic texts in content areas such as science” [7] (pp. 505–506).

Step 3: Formulate student science objectives. The third step involved formulating a series of daily student objectives that align to science standards and support the science concept. For this project, we had one student objective for each daily lesson—three lessons per week. These student objectives guided the development of the lessons to build an understanding of science concepts in the TEKS and DCIs, as summarized in the big idea. An example of a daily student science objective focused on the topic of conduction is, “The student will investigate thermal energy transfer between two objects that touch using an investigation and recording observations in a science journal.”

Step 4: Select and develop instructional activities and materials for lessons. The lessons incorporate the 5E instructional model [56], allowing students to be actively engaged in constructing science concepts as they move toward deepening their understanding through their experiences. Daily student science objectives guide the selection and development of instructional activities and materials incorporated into the 5Es (Engage, Explore, Explain, Elaborate, and Evaluate). How we used each “E” follows. These lessons were implemented three days a week, for 80 min each day. Unlike many lessons that include the 5E model in which one of the Es is used daily, our lessons included between three and five Es daily. We found that this was a critical component to push the ELs (and EC students) to move ahead in their understanding of science and academic language. By using the 5Es, we integrated the practices recommended by the National Research Council [57].

Engage activities help create interest in a topic and allow teachers to access prior knowledge. An example of an engage activity related to thermal energy transfer is showing students a glass of room temperature water and asking students how the temperature of the water can be changed. After adding ice, allow students to describe what is happening and predict the temperature of the water after the ice is added.

Explore activities allow students to interact in collaborative groups to investigate and generate ideas using hands-on science materials. Students explore the transfer of thermal energy using a heat transfer kit to observe the temperature of two containers of water as heat transfers across an aluminum bar.

During Explain, concepts and definitions are clarified. The Explain portion of the lessons included vocabulary preview in which cognitive verbs and academic science vocabulary were introduced with student-friendly definitions, visuals, and a sentence using the vocabulary word in context. The teacher modeled and students practiced their pronunciation of vocabulary words and tricky words that appeared in the text. Strategically-paired student partners read science text directly related to target science concepts. Only reading passages that related to the target concept was assigned. Finding tightly-aligned, grade-appropriate expository text sometimes requires, in addition to science textbooks, supplementary or modified resources. In our lessons, partners took turns reading paragraphs and discussing scripted questions that guide comprehension. Then, the teacher asked the comprehension questions to the class using questioning strategies that allowed all students to respond through simultaneity and randomization techniques so that all students had the opportunity to participate and develop oral academic language skills. Technology, via Edusmart [58], was also integrated into the Explain portion of the lessons through the use of standards-aligned educational software that provided explanations of concepts through animation and simulation [12].

Elaborate experiences further extend student learning and understanding. An elaboration activity related to thermal heat transfer was student brainstorming and providing other examples of thermal energy transfer. Teachers formatively assessed student understanding during Evaluate activities in which students applied new concepts and skills. For example, asking students to write a journal response explaining why a spoon gets hot when placed in a hot cup of coffee, applying the concept of conduction. Lesson closure included open-ended questions that summarize activities and investigations while relating concepts back to the big idea. At the end of the lesson on conduction, students were asked to draw a conclusion about how thermal energy transfers from one object to another. The teacher guided the discussion to connect concepts back to the big idea of conservation of energy.

Step 5: Select ESL strategies and English language proficiency standards (ELPS) to scaffold instruction for ELs (and ECs) for literacy-infused science. While selecting and developing instructional materials, it is important to consider scaffolding instruction to meet student needs. These lessons are what we consider literacy-infused science [18,31], with the curriculum being interdisciplinary between science and literacy. We embedded ESL strategies into the lessons throughout the 5Es, including questioning strategies, academic language scaffolding, visual scaffolding, use of manipulatives and realia, collaborative and cooperative grouping, content connections, and integration of technology [12]. Further, the principles behind state-mandated English Language Proficiency Standards are incorporated throughout the lesson via strategic listening, speaking, reading, and writing activities.

The students in the control classrooms received the traditional science curriculum without a focus on literacy infusion. Control lessons did include some inquiry-based lessons; however, lessons were not presented with big ideas, did not include three to five E’s per lesson, and did not include literacy specific infusion.

8. Instruments

BISA. The goals of science education include exposing students to a progression of concepts that leads to the formulation of big ideas that facilitate understanding natural phenomena related to the students’ lived experiences. Our development of the BISA was supported by the work of Gotwals et al., who said, “Tests that have items that are designed based on a solid theory of cognition allow us to make better and more valid inferences about how students’ performance on the items relates to what they know and can do more generally” [59] (p. 21).

We developed the BISA related to the big ideas in science based on standards and kept in mind what Russ et al. noted, which was that science teaching and assessment should not only be about the discipline and content, but also about students developing “the understanding that evaluation of ideas in science also requires judgments of reasoning as plausible, sensible, and mechanistic” [60] (p. 889) and about helping students “to construct scientific explanations, anticipate arguments, evaluate ideas and claims, and engage in sound and productive argumentation” [60] (pp. 889–890). Additionally, with such instruction and authentic assessments, students can “be better prepared to learn canonically correct content, as they will be able to better make sense of new information” [60] (p. 890). Four items as a sample of the BISA are found in Appendix A.

Table 2. Sixth-Grade Science Concept (Big Idea) Assessment Aligned with Next Generation Science Standards.

Big Idea	Assessment Item Number	Next Generation Science Standards
		Crosscutting Concepts	Disciplinary Core Ideas
The diversity of organisms, living and extinct, is the result of evolution.	28, 29	Structure and Function	LS1: From molecules to organisms: Structures and processes
	1	Structure and Function	LS4: Biological Evolution: Unity and Diversity
The total amount of energy in the Universe is always the same but can be transferred from one energy store to another during an event.	12	Energy and Matter: Flows, cycles, and conservation	PS1: Matter and its interactions PS3: Energy
	2, 8, 15, 16, 20, 21, 22, 23	Energy and Matter: Flows, cycles, and conservation	PS3: Energy
All matter in the Universe is made of very small particles.	3	Cause and effect: Mechanism and explanation	ESS3: Earth and human activity
	11	Cause and effect: Mechanism and explanation	PS1: Matter and its interactions
	9, 10, 13, 14	Structure and Function	PS1: Matter and its interactions
The composition of the Earth and its atmosphere and the processes occurring within them shape the Earth’s surface and its climate.	4	Patterns	ESS2: Earth’s systems
	5	Energy and Matter: Flows, cycles, and conservation	ESS2: Earth’s systems
	24, 25, 26, 27	Structure and Function	ESS2: Earth’s systems
Organisms require a supply of energy and materials for which they often depend on, or compete with, other organisms.	6	Cause and effect: Mechanism and explanation	LS2: Ecosystems: Interactions, energy, and dynamics
Changing the movement of an object requires a net force to be acting on it.	7	Cause and effect: Mechanism and explanation	PS2: Motion and stability: Forces and interactions
	17, 18, 19	Scale, Proportion, and Quantity	PS2: Motion and stability
Organisms are organized on a cellular basis and have a finite life span.	30	Structure and Function	LS1: From molecules to organisms: Structures and processes

Note: The Crosscutting Concepts and the Disciplinary Core Ideas are referenced from the Next Generation Science Standards [9].

lists the item number and the science concept (the big idea) that was being assessed.

Pilot of the BISA. The BISA was developed and piloted by the research team with 25 students prior to the sixth-grade implementation. Internal consistency was established at 0.70. The test consists of 30 multiple-choice questions with a compilation of questions from the Texas Education Agency’s sixth-grade state science test. Drafts were sent to the project science specialist for feedback. Minor tweaks to the assessment were made in terms of language usage. The total score was derived on a percentage scale of 100.

Science benchmark assessment. The benchmark tests were developed by the district in alignment with the state science curriculum—TEKS. Similar to the state assessment, these benchmark tests are also criterion-referenced in which cut-off scores are used to measure students’ mastery of the knowledge and skills. The benchmark test was used since sixth grade does not have a state science assessment provided. The tests are given every six weeks, and topics covered over the year are: physical science including energy sources and force and motion (test 1); physical science including force, motion, and energy, and simple machines (test 2); chemistry including matter, and accumulative of topics covered during the first three six-week periods; earth and space including organization of the solar system and earth science (test 4); earth science (test 5); and life science including organisms, and an accumulation of topics covered during the second three six-week periods (test 6). Face validity and content validity have been reported by Lara-Alecio et al. [12].

Woodcock Language Proficiency Battery-Revised. Students’ English literacy was measured by Woodcock Language Proficiency Battery-Revised WLPB-R [61], a standardized assessment battery, on a broad range of language proficiency in speaking, listening, reading, and writing. Subtests, or areas of literacy measured, included Verbal Analogies, Oral Vocabulary, and Passage Comprehension. Adequate construct, content, and concurrent validity and reliability information are reported in the test manual [61]. The average reliability coefficient of Cronbach’s alpha based on our sample was 0.78 for Oral Vocabulary, 0.83 for Verbal Analogies, and 0.78 for Passage Comprehension. Age-based scale scores for these subtests were collected at the end of sixth grade.

9. Data Collection and Analysis

To provide further evidence regarding the concurrent validity of the big idea assessment, we proposed the first research question to perform a correlation analysis between students’ performance on the big idea assessment and their district-administered science benchmark tests. The BISA was given at the beginning and end of grade 6, and data on six district benchmark tests were collected at the end of the school year.

To address questions 2 and 3, an analysis of covariance (ANCOVA) was performed with the pre-test as the covariate and the post-test as the outcome variable to monitor students’ progress and make comparisons between treatment and control conditions. To address question 4, a Pearson’s r correlation analysis was conducted to examine the relationship between students’ English language proficiency and their science achievement. The percent of items answered correctly in BISA was used in data analysis, with 160 ELs (105 treatment; 55 control) and 116 non-EL EC students (48 treatment; 68 control) who completed the intervention through sixth grade. For all questions, separate analyses were conducted on ELs and non-ELs, respectively.

10. Results

The goal of our study was to compare performances of treatment and control condition students who completed a literacy-infused science intervention through sixth grade as measured by the BISA tool. Therefore, the specific purpose of the study was to determine (a) the concurrent validity of BISA; (b) the differences in the post-test of the BISA between treatment and control ELs, controlling for their performance in the pre-test; (c) the differences in the post-test of the BISA between treatment and control non-ELs, controlling for their performance in the pre-test; and (d) the relationship between students’ English language proficiency as measured by a standardized assessment, and their performance in the BISA among ELs and non-ELs, respectively. Results are presented in the order of the research questions.

Question 1:

Is there concurrent validity on the big idea science assessment as determined by a relationship between the students’ scores in the BISA and their science benchmark assessment among ELs and non-EL-EC students, respectively?

Correlation analysis was conducted between BISA (pre and post) and the six district science benchmark tests, and

Table 3. Correlation between the BISA and District Benchmark Science Assessment by Language Status.

	Benchmark 1	Benchmark 2	Benchmark 3	Benchmark 4	Benchmark 5	Benchmark 6
EL
Pre-test	0.407 **	0.274 **	0.254 *	0.226 *	0.306 **	0.295 **
Post-test	0.547 **	0.465 **	0.485 **	0.280 **	0.627 **	0.754 **
Non-EL, EC
Pre-test	0.442 **	0.250 **	0.308 **	0.378 **	0.425 **	0.261 **
Post-test	0.438 **	0.389 **	0.231 *	0.494 **	0.384 **	0.591 **

Notes: ** Correlation is significant at the 0.01 level (two-tailed); * Correlation is significant at the 0.05 level (two-tailed).

shows that such relationships between students’ scores in the BISA and the benchmark assessments are all statistically significant and positive, ranging between 0.226 and 0.754 for ELs, and 0.250 and 0.591 for non-EL ECs, in the form of Pearson r coefficients, particularly with the end-of-year science assessment (benchmark 6), suggesting a strong concurrent validity of BISA.

Question 2:

Is there a statistically significant difference in the post-test of the BISA between treatment and control ELs, controlling for their performance in the pre-test?

Results, from the ANCOVA suggest that treatment ELs (82% accuracy) outperformed control students (66% accuracy) in the post-test after adjusting for the pre-test performance (F = 95.10, p < 0.001), with an effect size (partial eta squared) of 0.294. The pre- and post-assessments of the BISA for treatment (blue) and control students (orange) are noted in graphic form (Figure 1) for ELs based on the average scores of BISA.

Question 3:

Is there a statistically significant difference in the post-test of the BISA between treatment and control non-EL-EC, controlling for their performance in the pre-test?

Results from the ANCOVA suggest that treatment non-EL EC students (84% accuracy) outperformed their control peers (79% accuracy) in the post-test after adjusting for the pre-test performance (F = 22.43, p < 0.001), with an effect size (partial eta squared) of 0.083. The pre- and post-assessments of the BISA for treatment (blue) and control students (orange) are noted in graphic form (Figure 2) for non-EL ECs based on the average scores of BISA.

Question 4:

Is there a statistically significant relationship between students’ English language proficiency as measured by the standardized assessment, and their performance in the BISA among ELs and non-EL-EC students, respectively?

Results derived from correlation analysis are presented in

Table 4. Correlation between the BISA and English Language Literacy by Language Status.

	Oral Vocabulary	Verbal Analogies	Passage Comprehension
EL
Pre-test	0.301 **	0.346 **	0.323 **
Post-test	0.260 *	0.314 **	0.303 **
Non-EL, EC
Pre-test	0.309 **	0.370 **	0.312 **
Post-test	0.208 *	0.372 **	0.316 **

Notes: ** Correlation is significant at the 0.01 level (two-tailed); * Correlation is significant at the 0.05 level (two-tailed).

, which shows that relationships between students’ scores in the BISA and the WLPB-R sub-tests are all statistically significant and positive, ranging between 0.260 and 0.346 for ELs and 0.208 and 0.372 for non-EL ECs, in the form of Pearson r coefficients.

11. Discussion

We have determined that conceptual understanding of science from a big idea basis can be taught and assessed via literacy-infused science curriculum for ELs and ECs. Though we have found this to be the case in our work with students, Harlen suggested that typical exams include items that are disconnected parts of knowledge that has been taught [47]. Without exams being connected to what has been taught and without such big idea concepts to help students in understanding and developing key abilities, the impact of assessment and the level of student understanding is diminished. Additionally, Ketelhut et al. noted that the isolated fact-based questions poorly represent the complexity in which real-world science is situated and contextualized [62]. Our attempt to assess the big ideas via the BISA directly addresses the challenge as noted by Harlen, as well as Ketlhut et al.

Harlen further indicated that another challenge is that teachers may not be prepared to teach in such a manner. He stated:

When planning lessons, it is important for teachers to have in mind how the goals of individual lessons fit into a wider picture of more powerful ideas that can help students make sense of a broad range of related phenomena and events.

[47] (p. 6)

Our study included bi-weekly professional development over 28 weeks with teachers and the development of their curriculum that they were taught. This provided them with a developmental direction for the big ideas and how the ideas connected throughout the lessons so that conceptual understanding could be developed.

The layered challenge with the development of teachers and the assessment of big ideas in science is the development of academic language and literacy in the area of science with children who have English as a second language or with those who are economically challenged. Thus, in our study, it was the combination of big idea development in the curriculum for students, professional development, and literacy-infused and inquiry-based science that aided in improving science education for ELs and ECs. Consequently, there are four main points based on our study. Those are:

• Literacy-infused science lessons with big ideas, implemented through the tested intervention, improved students’ language acquisition and science concept understanding for ELs and ECs. These learning experiences focused closely on what EL and EC students needed to learn in a specific science unit and how teachers needed to scaffold the concepts and the academic language of science with them.
• There was a positive relationship between language and content for both ELs and non-EL ECs, with a similar magnitude, suggesting that students with a higher English proficiency (literacy) score higher in science assessments. This seems to be a logical finding as it seems reasonable that higher rates of literacy in the language of instruction of a subject area would yield higher rates of achievement in the content subject. It is reasonable then to consider that students would be able to demonstrate literacy (reading, speaking, listening, and writing) in the specific subject. ELs whose native language is Spanish and EC students whose native language is English, as presented in the beginning of the paper, generally underperform in national assessments (e.g., NAEP) compared to their non-EL and non-EC peers. Therefore, we advocate that academic literacy is critical not only for ELs, but also for EC students, since our finding is that a higher proficiency in academic English literacy is associated with higher science achievement. This finding aligns with a previous study derived from the same project with grade 5 students [6], and can be supported by theorists (e.g., [63]) and researchers (e.g., [64]) who have reported the association between academic language and conceptual understanding. Such a finding further indicates the importance of integrating literacy into science teaching for both ELs and EC students, because with such integrated instructional practices, students are encouraged to practice the four language skills (listening, speaking, reading, and writing), while simultaneously developing a strong base for establishing background knowledge and vocabulary, thereby promoting academic achievement [27].
• Our findings for ELs and EC students indicate that the PD and lesson plans were successful for promoting literacy-infused science via a 5E model that includes three to five of the Es used daily as opposed to using such practices with one to two Es per day, as is usually practiced. Big ideas were able to be conveyed and assessed through the project curriculum, literacy-infused science with the components depicted as a summary of our project in Figure 3: (a) national standards; (b) state standards; (c) DCIs; (d) 5E Model and hands-on activities; and (e) ESL and content reading strategies.
• The instructional intervention included the engagement of ELs and EC students in science as practice. We placed an emphasis on the cognitively-demanding task of considering content in context; such emphasis helped treatment students to demonstrate a higher level of understanding of big ideas over those control students who did not participate in the intervention.

Ultimately, the BISA was designed to check for conceptual understanding in science. Thus, our findings support the view that if big ideas are presented and taught as students learn and contextualize science, they can be assessed. We found that such a big idea assessment had concurrent validity with the district science benchmarks test, which is based on the high-stakes, state assessment. Therefore, we contend that teachers can incorporate big ideas which promote the holistic and broad value of key learning concepts in science into their teaching and then in their assessment of those concepts, since doing so would have no negative impact on outcomes of their standards-based and state-based evaluative measures.

12. Conclusions

Via our study, we further the importance, especially for ELs and ECs, of teaching science for understanding rather than for memorization and rote acquisition of vocabulary. We advocate, based on the research, that better understanding can be gained by students if the curricular lessons are built to include big ideas, overarching science concepts. Such big ideas can assist students, particularly among those populations included in our study, to make connections with more holistic science concepts in the process of gaining new understandings.

Furthering the importance of our study is the fact that no studies could be found that were based on an RCT with a curriculum that was literacy-infused science with technology infusion as described in the lesson development. Additionally, we found there to be few researchers who have studied conceptual understanding via the assessment of big ideas in science in schools with ELs and EC students. We believe that further study should be undertaken to expand the study while comparing ELs and EC student growth in science and literacy to their non-EL and non-EC peers’ growth. We advance the notion that a BISA based on the curriculum presented should be incorporated into the science curriculum. It is important that assessment is aligned with the big ideas presented and where students are engaged in an assessment that calls for critical thinking and the task of making connections in understanding as opposed to simply responding to isolated questions based on pieces of knowledge. We encourage future use and study of big idea assessments that are developed and used in science classrooms with ELs and EC students, ensuring that assessment matches instruction.

Based on our study, teachers’ ongoing professional development as a part of the intervention was noted in the implementation of the lessons. Therefore, we encourage the development of teachers’ understandings of big ideas and their connectedness to the broader world. Teachers’ understanding of instruction via ongoing professional development that is targeted to the big idea concepts and the lessons is recommended for school districts. As such, it may be necessary for departments of literacy, multilingualism, and science to work together to develop such professional development sessions. Just as science education requires a change to incorporate inquiry approaches in science that are built on big ideas for understanding and on the infusion of literacy for the development of science concepts and academic language, particularly for ELs and ECs, teacher professional development also requires a change to reflect these alterations. These actions would require a district or school level policy change for professional development to be on-going, targeted, and integrated between disciplines and to eliminate professional development as business as usual in isolated science or language/literacy silos.

We, like Harlen [47], believe that the curriculum should be written to include big ideas, but foremost, that teachers should be advocates of connected learning. In order to implement this type of instructional approach, as indicated, teachers need professional development so that they can learn how to teach and promote connected science concepts through inquiry-based strategies. Additionally, such instruction that infuses literacy with science is important for academic language development and understanding of science for ELs and ECs. This infusion can assist teachers in helping their students understand the connected world in which they live. They will then have the ability to apply big ideas in order to make informed decisions as they approach adulthood with opportunities to transform their world.