Scientific Impact and Its Role in Scientific Reasoning

We tested 75 participants in a selective university near the East Coast of the United States for their skills in scientific reasoning. We used scientific reasoning assessments for Generating Hypotheses, Generating Experiments, and Drawing Conclusions. To measure scientific reasoning skills, we also used a task involving analyzing scientific impact based on titles of published studies (which were either highly cited or scarcely cited), and another task involving creating what participants believed might be high-impact scientific studies in three subject matter areas. Participants further completed two fluid intelligence tests: Number Series and Letter Sets. They also filled in demographic information, including self-reported SAT/ACT scores and college GPA. (We cannot obtain actual grades at our university because of student-confidentiality issues.) We found that the scientific reasoning tests for Generating Hypotheses, Generating Experiments, and Drawing Conclusions clustered into a single factor, and the task for creating high-impact studies was also factored with these scientific reasoning tests. The two fluid ability tests—Number Series and Letter Sets—clustered into a distinct single factor. The task of analyzing impact seemed to be in between the other tasks, showing characteristics of not only the scientific reasoning tasks but also of the fluid intelligence tasks.