Scaffolding Rubrics to Improve Student Writing: Preliminary Results of Using Rubrics in a Sociology Program to Enhance Learning and Mechanical Writing Skills

Abstract

In the era of accreditation, academic accountability and transparency within curriculum is becoming a desired standard within and across disciplines. Through the use of course learning outcomes, program outcomes can be strengthened. Scaffolding within curricula can benefit both accountability and assessment goals. Through the use of Bloom’s cognitive taxonomy, scaffolding within the course can be used to aid in the accomplishment of the course learning outcomes. Scaffolding within the course curriculum can move students through Bloom’s cognitive taxonomy levels toward the mastery of specific skills. Writing is a major area of assessment as an indication of learning across disciplines. Scaffolding rubrics were used within sociology courses to specifically address both student learning and mechanical writing skills. Preliminary results of using rubrics to enhance student learning and scaffolding in eight courses (one 100-level, one 200-level, two 300-level, and four 400-level sociology courses) will be presented.

1. Introduction

Writing is a necessary skill set needed by undergraduates as they pursue careers. However, writing skills for undergraduates have to meet employers’ expectations for many careers [1,2,3,4,5]. For example, law enforcement and correctional officers need strong writing skills because they will often be required to write incident reports. Likewise, social workers will need strong writing skills in case they ever have to prepare case notes for court appearances. Writing skills have become an issue for all faculty members. No longer can other disciplines expect the English faculty to be the only faculty to teach writing skills [6]. Many professions require writing skills in daily activities and depend on employees to write using critical thinking skills. Undergraduate students continue to struggle with the transfer of knowledge through the use of critical thinking skills for a variety of situations [4].

Writing Across the Curriculum (WAC) is a pedagogy movement developed during the 1970s to address writing skills in higher education [3]. Two major issues stressed in WAC were Writing in the Disciplines (WID) and Writing to Learn (WTL). Writing in the discipline involves “basic writing skills and critical thinking, problem solving and specialized knowledge such that students’ command of subject and writing makes it possible for them to write well for different purposes and different audiences” [5]. “Writing to learn advocates support implementing certain types of writing exercises that appear to enhance student learning and writing performance” [5]. This pedagogy treats writing as an essential element of critical thinking both as a learning skill and a professional skill [1,3]. These two issues are beneficial when determining course and outcome goals. Higher education is currently in the era of assessment through course outcomes and program outcomes.

Embedding and scaffolding Bloom’s cognitive taxonomy into course development can assist in course and outcome goals to address the continued need for assessment. Bloom’s cognitive taxonomy is a six-level progression of higher learning skills originally developed in 1956 [7]. The taxonomy has since been revised and modified several times [8,9]. According to this taxonomy, the six levels of higher learning are remembering, understanding, applying, analyzing, evaluating and creating. Remembering—students will recall and facts and basic concepts; Understanding—students can explain concepts; Applying—students can apply basic concepts in various situations; Analyzing—students can identify and draw connections between concepts and ideas; Evaluating—students can critique information; and Creating—this is the pinnacle of the taxonomy—students have the ability to develop new ideas based on information and concepts identified. The six levels cover three larger areas of skills: knowledge, skills, and attitudes. Through the use of Bloom’s cognitive taxonomy, students are expected to move through all levels of the learning process.

2. Review of Literature

The purpose of designing scaffolding within a discipline is for the student to move through the levels of Bloom’s cognitive taxonomy to develop “deep learning” of material by graduation [10,11]. Teater [10] suggested that “Incorporating aspects of doing, feedback and digesting into the learning activities and teaching methods is congruent to and supportive of the different learning styles of the students” [10] (p. 5). Thus, these levels of learning skills can be embedded in course curriculum to gauge student’s progress of learning [12]. Student writing is an assessment method used within disciplines to evaluate students’ levels of understanding presented material. Freshman college students should not, and do not, have the same cognitive skills as graduating seniors. According to the National Center for Education Statistics [13], “52 percent of twelfth-graders performed at the Basic level in writing in 2011. The Basic level denotes partial mastery of the prerequisite knowledge and skills that are fundamental for proficient work at each grade” (p. 2). Scaffolding writing assignments from 100-level courses through the curriculum until graduation can increase a student’s mechanical writing abilities and their understanding of material from superficial learning to deep learning. By scaffolding writing assignments, course data and program data can be collected and used to make adjustments in both the course and goals. Course assessment goals can be embedded within individual courses [14,15] and these outcome goals can be combined with additional outcome goals to establish areas of strengths within a course or department [14]. When embedded assignments and tests are entwined with course, program, and university goals, faculty are not overburdened to report data for all levels of competency assessments [16]. Thus, scaffolding holds two distinct and important justifications: First, students have the ability and opportunity to develop “deep learning”, and second, data can be easily gathered for both course and discipline assessment.

Writing is a progressive skill, and students need the opportunity to practice writing, receive feedback, and rewrite to develop and improve their skills and understanding of the material [17]. Rubrics can be used successfully to increase both student’s writing skills and their understanding of the material. Rubrics are developed to incorporate both mechanical and material understanding expectations to students prior to their writing assignments [18]. Within many disciplines, writing rubrics have been used both to assess and provide feedback to students’ understanding and knowledge of key issues to meet course and program goals [10,14,15,19,20,21,22]. These rubrics are designed to provide feedback to students regarding their strengths and areas they need to strengthen; they assist in the development of students’ cognitive and writing skills [18]. Writing rubrics also benefit faculty with consistency in grading, while providing data for course goals and program goals [23,24]. Researchers have addressed assessment and student learning goals, better known as course outcome goals, through the use of rubrics [19,24,25,26].

Faculty use numerous methods to improve students’ writing and self-assessment skills; peer-review of written material, drafts of written assignments, written feedback on assignments, and a combination of the aforementioned methods. Using writing rubrics alone to enhance students’ self-assessment of written work has not improved either student learning or future written assignments [22,25,26]. Kinne and colleagues [24] found that students have difficulty gaining enough adequate information from peer-reviewed material alone. Peer reviews improved with specific rubrics for writing assignments [27]. Faculty report that students do not use written feedback on returned assignments to make adjustments in future assignments [6]. Researchers have added writing rubrics along with feedback on written assignments to provide information to students for future writing assignments [18,22,24,26,27]. Writing rubrics can be used by students to understand the feedback provided on the written assignment (thus improving their self-assessment and understanding of the material) and improve future written assignments.

2.1. Combining Scaffolding with Competency Assessment

There are important writing skills required at each course level for undergraduates to progress through from their first year to graduation. The scaffolding of skills in each required course teaches students writing skills needed to successfully complete the degree in their discipline. Faculty can assist the students’ understanding of material at all course levels through the embedding of specific skills in both writing assignments and testing [28]. This increases both students’ grades and competency assessment scores for the department. Hooper and Butler [1] summarize the benefits of scaffolded writing when they note that “Student writing can be scaffolded to move students toward more complex thinking and stronger compositional skills throughout the course or program of study” [1] (p. 7). An added bonus of having important writing skills embedded in scaffolded assignments and tests is that it allows faculty to focus on the material a student presented versus the technical issues related to writing. The specific writing skills to scaffold and embed in all course work are based on Bloom’s cognitive taxonomy, beginning by focusing on remembering and understanding, while briefly introducing applying and analyzing at the 100-level freshman courses. The 200-level courses reinforce remembering and understanding with a more thorough introduction of applying and analyzing. The 300-level courses continue to reinforce remembering and understanding but focus on application with the incorporation of analyzing and evaluating. Finally, the 400-level courses involve the use of all six levels of Bloom’s cognitive taxonomy through creating a discipline-specific product.

The scaffolded and embedded writing skill results can be used for university-level and program-level assessments. Scaffolded writing skills are necessary for students to meet both university- and program-level competencies [12]. Without scaffolding beginning in the 100-level courses, it becomes very difficult for students to meet the competency requirements in higher-level courses. Scaffolded writing skills are developed using individual course objectives [29] with the focus of addressing both program- and university-level competencies. The use of Bloom’s cognitive taxonomy is important in developing scaffolding writing assignments for the development of students’ competencies [19]. Program assessment is vital to identifying the strengths and needs of student learning within a program [21].

Scaffolded writing assignments can be used to meet program assessment competencies related to four key program goals: Program goal one: Students will demonstrate the ability to describe and appropriately use key sociological concepts. Program goal two: Students will demonstrate the ability to design, implement, and analyze sociological research. Program goal three: Students will demonstrate the ability to apply sociological theory to the analysis and understanding of concrete social phenomena. Program goal four: Students will demonstrate knowledge of career opportunities in the profession of sociology and related fields.

Program assessment competencies are necessary to establish four university assessment competencies. Examples of university-level competency assessments are Competency 1—Gaining university-level knowledge and comprehension of the information; Competency 2—Application of the information; Competency 3—Analysis of the information; and Competency 4—Communicating effectively and appropriately.

2.2. Assignment Rubrics and Feedback

According to Mansilla and colleagues [20], there are two very significant rationales for using grading rubrics at all levels of undergraduate assignments. Grading rubrics provide transparency and objectivity for all students in the course. A grading rubric also allows students to review their strengths of the assignment and the areas they need to improve prior to the next assignment [20]. All writing assignments have rubrics for students to review the first day of the course to begin discussion of faculty expectations of student’s writing skills [23]. Providing a grading rubric in advance of assigning the writing assignment allows faculty to discuss different aspects of the writing assignment during regular lectures, and to begin the connection between definitions and concepts.

Providing the grading rubric early in the semester also allows students the ability to begin making connections between the presented material and the course learning objectives. Lipnevich and colleagues [26] reported that the use of a rubric produced the greatest improvement in a student’s writing skills. Nordrum and colleagues [27] combined the use of in-text feedback and assignment rubrics for writing assignments and found students used the two forms of feedback differently. Students used the in-text feedback to correct common writing errors, and the rubric was used to not only identify writing errors but also to give the student a better understanding of where improvement was needed for future writing assignments. Ash and colleagues [19] focused on the use of a rubric with three writing attempts (a raw, revised, and final version) prior to a grade being given. Both the raw and revised writing assignments were given feedback along with feedback from the rubric. The authors found that all scores improved from the raw draft to the final version. Horton and Diaz [2] and Lipnevich and colleagues [26] also found increased writing skills when students had a rubric prior to the writing assignment. Students had the opportunity to submit writing drafts for feedback, but the faculty did not grade the drafts until the final submission [2,26].

3. Study Methods

The current study was developed to explore if using feedback comments, a graded rubric and scaffolding within a discipline will increase writing skills for future writing assignments. One of the keys to scaffolding of academic skills is to build each level of courses on key concepts, ideas and skills from previous courses within their discipline. As students’ progress in their discipline, additional writing skills are scaffolded by building on previous writing skills.

Scaffolding begins in every course with a plagiarism assignment. This assignment is done through a website which offers a tutorial of plagiarism, followed by a quiz to gauge students’ understanding of plagiarism. Upon successful completion of the quiz, a certificate is produced for the student to copy and submit as the student’s first writing assignment. Successful completion of this assignment begins teaching and building writing skills for the student. To reinforce completion of this assignment, no written work will be graded without having a certificate for the student in the grade book.

Beginning with freshman- or 100-level courses, the students’ first written assignment is to locate a newspaper article that represented a discipline concept or idea presented in their current course. This assignment is described in detail both during the course, verbally, and written in the syllabus. An ungraded rubric giving specific details of the required components of the assignment are posted on the university’s digital platform to allow students the ability the review the grading requirements for each component of the assignment prior to submission. Students graded assignments are returned with comments on their written assignment and a graded rubric identified the components from “exceeds” to “does not meet” scoring. The components are listed on the back of the graded rubric with space to the left of the component for students to provide the appropriate correction for that component. Students have the opportunity to identify appropriate corrections to all areas that did not receive an “exceeds” on the rubric. This serves both short and long-term goals for the student: (1) to identify how each component was scored; (2) to reinforce an appropriate correction for future assignments; and (3) to receive bonus points for providing appropriate responses to issues identified on the graded rubric. A student receives the bonus points if he/she received an “exceeds” in all components and submitted the graded rubric. Submitting the graded rubric is not a requirement of the course, however many students like the idea of receiving the bonus points associated with this process.

In the 100-level course, writing assignments are used to begin teaching students how to take information from the class and textbook and see the connection in their everyday world through recent newspaper articles. Specific website news outlets are provided along with citation examples for website news outlets. Students are required to select a news event and connect it with discipline definitions and concepts for a specific textbook chapter. These writing assignments use Bloom’s cognitive taxonomy to focus on remembering, understanding and introduce application.

The writing assignments in 200-level courses continue to build writing and competency skills through the use of two peer-reviewed journal articles (provided by the faculty) and one peer-reviewed journal article that students locate through the library database. Students are required to read the peer-reviewed journal articles and develop an essay to describe discipline specific concepts to explain a situation. This course level continues to build on remembering, understanding and application while beginning to add analysis of the material. In-text citation and reference examples are provided to students in this course level. Through the use of in-text citations and a reference page, students begin to understand their opinions need to have support from other academic sources.

The 300-level courses become more complex and detailed in writing skills. This course level requires students to produce an argumentative essay with a thesis statement, a theory, and a literature review, using all levels of Bloom’s cognitive taxonomy. Students also are required to find peer-reviewed articles from a library database and the Publication Manual of the American Psychological Association (American Psychological Association (APA) 2010) is a required textbook. At this course level, students experience several new writing skills and begin to struggle with the assignments. Many students have little or no experience with the development of a thesis statement, literature review, or locating peer-reviewed journal articles. To aid in the skill building process, the final product is broken down into individual assignments, so students can build on their writing skills while adding new and more complex skills [30].

The 400-level courses require students to produce a research product which builds on all the writing skills learned in previous courses. These courses require students to successfully use all levels of Bloom’s cognitive taxonomy. These courses require the submission of an Institutional Review Board (IRB) application as part of their research product. Students must have effective and appropriate communication writing skills at this level to pass the IRB application process. Now the question becomes: How can successful writing skills be reinforced?

4. Results

In the 100-level course, students were graded on a summary of the news article, connection to a sociological concept, grammar and organization, APA format, and guidelines from the syllabus. These categories were given in specific detail on the rubric. Students’ first writing assignments were returned with in-text comments and a scored rubric (the rubrics used are available upon request). On the back of the rubric, students were asked how they were going to correct any category where they did not receive an “exceeds” (the highest score) for the second writing assignment. Not all of the students submitted a corrected rubric. Twenty-seven students submitted the first written assignment with a mean score of 32.759 out of 50. Twenty-one students submitted a corrected rubric with a mean increase to 48.654.

In the 200-level course, rubric categories included: summary of a peer-reviewed journal article, connection to a theory, grammar and organization, and APA format and guidelines from the syllabus. These categories were given in specific detail on the rubric. The students’ first writing assignments were returned with in-text comments and a scored rubric. On the back of the rubric, students were asked how they were going to correct any category where they did not receive an “exceeds” (the highest score) for the second writing assignment. Twenty-two students submitted the first library assignment with a mean score of 29.750 out of 50. The second library assignment was submitted by 22 students with a mean increase to 45.341.

For the 300-level course, students developed a thesis statement. Grammar, wording of the statement, and statement length were areas on the grading rubric. These categories were given in specific detail on the rubric. The students’ first attempts at this assignment were returned with in-text comments and a scored rubric. On the back of the rubric, students were asked how they were going to correct any category where they did not receive an “exceeds” (the highest score) for the final writing assignment. A thesis statement was submitted by 16 students with a mean of 7.938 out of 12. Sixteen students submitted a correct thesis rubric with a mean increase of 2.313. The second 300-level course had five students—only four students submitted the thesis written assignment with a mean of 9.50, and only one student submitted a corrected thesis rubric with an increase of 1.25. Another writing assignment at the 300-level was the development of a theoretical basis for a longer paper. The theory assignment was graded based on the identification of theory, the theorist, the connection to the students’ project, and APA citations. These categories were given in specific detail on the rubric. The students’ first attempt of this assignment was returned with in-text comments and a scored rubric. On the back of the rubric, students were asked how they were going to correct any category where they did not receive an “exceeds” (the highest score) for the final writing assignment. The theory written assignment was submitted by 17 students with a mean of 3.265 out of 10. The corrected theory rubric was submitted by 17 students with a mean increase of 2.941. The second 300-level class had four students that submitted the theory written assignment with a mean of 4.50. All four students submitted the corrected theory rubric with a mean increase of 3.125.

The 400-level classes involved independent research and the submission to an IRB. The IRB submission was graded based on the project description, research procedures, materials, confidentiality, and the consent form. These categories were given in specific detail on the rubric. The students’ first attempts at this assignment were returned with in-text comments and a scored rubric. On the back of the rubric, students were asked how they were going to correct any category where they did not receive an “exceeds” (the highest score) for the second writing assignment. Fourteen students submitted a draft IRB with a mean of 31.5 out of 50. These students had two more attempts to submit an IRB for approval. Thirteen students submitted a corrected IRB with a mean increase of 4.643. Twelve students submitted a corrected IRB for approval with another mean increase of 4.286. In a second class, three students submitted the IRB assignment with a mean of 27.917; the corrected IRB submission increased the mean by 5.00. This course also required a theory section. The theory assignment was graded on the identification of theory, the theorist, the connection to the students’ project, and APA citations. These categories were given in specific detail on the rubric. The students’ first attempt at this assignment was returned with in-text comments and a scored rubric. On the back of the rubric, students were asked how they were going to correct any category where they did not receive an “exceeds” (the highest score) for the final writing assignment. Fourteen students submitted the theory written assignments with a mean score of 6.214 out of 10; only seven students submitted a correct theory rubric receiving all available points. This course also required a “methods” section. This assignment is graded on the collection of data, identification of variables, operationally defined variables, and the statistical method identified. These categories were given in specific detail on the rubric. The students’ first attempts at this assignment were returned with in-text comments and a scored rubric. On the back of the rubric, students were asked how they were going to correct any category where they did not receive an “exceeds” (the highest score) for the final writing assignment. Seven students submitted a written methods assignment with a mean of 3.0 out of 5. The same seven students submitted a correct methods rubric with increasing the mean to 3.857.

5. Discussion

It is obvious from the above increases that providing a rubric for all writing assignments early in the semester, allowing students to submit a draft in which faculty provides in-text feedback, and making corrections on the grading rubric all had a positive impact on students’ grades. The increase in appropriate writing skills makes reading and grading final papers much easier. Therefore, the combined use of scaffolded writing assignments, rubrics to identify deficiencies in those writing assignments, and bonus points for corrected mistakes may serve as a promising method of improving student writing skills. Furthermore, the use of these rubrics allows for a more streamlined and empirical approach to program assessment.

This study took place at a small regional state-sponsored university located in the southeastern region of the United States. The small size of the department was advantageous for the scaffolding of writing assignments because the same instructor taught the core sociology classes all sociology students had to take. This university context is important because it may be more difficult to scaffold rubrics across an entire curriculum in large departments with many different full-time and part-time instructors.

The scope of the analysis was limited to findings derived from one instructor’s classes. Thus, results and expectations might be different in other instructor’s classes. Likewise, the initial writing skills of students could vary based on the quality of writing instruction students receive while in high school. This is especially true for smaller regional universities and junior colleges that tend to rely on local high schools to act as “feeder” schools for their programs. However, it is reasonable to assume that the scaffolding of rubrics would be beneficial to all students, regardless of the initial writing skill of students when they begin their higher education.

The scaffolding of rubrics may be more difficult and time-consuming for instructors with larger class sizes or greater scholarly research expectations. However, this highlights a potential benefit of scaffolding rubrics. Rather than one instructor scaffolding rubrics across all of their classes, smaller departments and programs may be able to coordinate the scaffolding of rubrics across an entire department or specific program. Therefore, it is possible for departments or programs to create flexible “program-specific” rubrics. Future research should examine the feasibility and success of scaffolding rubrics throughout an entire department or program.

Finally, Bloom’s taxonomy has been used throughout this paper to reference the revised cognitive taxonomy. However, this focus ignores the affective and psychomotor domains proposed by Bloom. It is possible these other two domains could provide a mechanism or tool for improving student writing but this was not the focus of the current study. Future innovative studies could seek to explore how Bloom’s other proposed domains could be used to improve student writing within the discipline.

6. Conclusions

In an era of competency assessment, the need for creativity in developing assignments that teach students how to be critical thinkers and at the same time improve effective and appropriate communication skills is difficult. Many faculty members are teaching large numbers of students, and they have research and service obligations. To make the challenge more difficult, many students are entering colleges and universities not prepared to meet the rigors of academia. Connecting writing assignments to course objectives, program goals, and university competencies not only provides easy access to available data, but this data can also be used to strengthen the course, the program, and the university. Furthermore, students’ abilities to develop effective critical thinking and writing skills can be improved. Although the method described and outlined in this paper is time-consuming, the benefits for increasing the student’s ability to write effectively using critical thinking skills is worth the effort.