Two Oral Exam Formats for Literary Analysis in the Tertiary English as a Foreign Language Seminar

Abstract

For novice students, developing disciplinary literacy in literature courses in English as a Foreign Language education (EFL) at university entails mastering a number of skills. The purpose of this small-scale action research study is to investigate the extent to which two different oral exam formats can serve to make explicit commonly held warrants in the discourse community of literary studies. The material consists of observation notes from Socratic seminars and Thought-Question-Epiphany (TQE) seminars, both of which are analyzed using qualitative content analysis. The results show that most students adopt disciplinary conventions, such as building on each other’s ideas, using critical lenses, showing contextual awareness, and supporting claims with textual evidence. While the Socratic seminar format generates lively discussions, the sole focus on questions prevents students from preparing textual evidence for specific literary elements in the analysis. In the TQE seminar, some students react negatively to the forced inclusion of an epiphany, but the format also gives an opportunity to identify significant quotes in advance and to expand on interpretative ideas prompted by the three components.

1. Introduction

In an effort to investigate language practices in English classrooms, the present study deals with classroom-based assessment practices in tertiary education in the teaching of English literature. After the development of new literacy studies in the 1980–1990s (Barton 2007; Baynham 1995; Street 1984), the range of precisely what is expected to be mastered by novice university students has been discussed in terms of “academic literacies” instead of the earlier focus on “study skills” and “academic socialization” (Lea and Street 1998, pp. 158–59). In fact, these skill-sets are integrated and should not be considered separately, according to Lea and Street (1998). Such a synthesis is nonetheless expressed differently across fields, which came to the fore when Shanahan and Shanahan (2008) first coined the concept of disciplinary literacy arguing that “there are differences in how the disciplines create, disseminate, and evaluate knowledge” (p. 48). Not surprisingly, then, the epistemology of literary studies differs from that of other fields. Goldman et al. (2016) point out the following distinct features “[w]arrants and reasoning in literary argumentation can be grounded in personal beliefs and life experiences, in critical theory, in philosophical tenets ... [and conclusions are] always open to contestation” (p. 227). Apart from this plurality and the tendency in many subfields to avoid claiming to be a method or a theory at all, there is also the focus on debatable claims. Furthermore, having large chunks of texts without subheadings eschews many of the standard formats for academic writing in the social sciences, which by itself could easily lead to students feeling overwhelmed.

Bearing in mind that assessment may represent “the single biggest influence on how students approach their learning (Rust et al. 2005, p. 231), it follows that the design of written and oral assignments becomes a vital factor to consider for instructors in higher education. One such design principle is constructive alignment (Biggs 1996), which relies on a logical sequence from clear learning objectives guiding the planning of classroom activities to the tested skills in the cumulative assessment. For the individual student, it may still be unclear what leads to a high grade, especially in subjects comprising more than one discipline. A case in point is the field of literary studies in the English as a Foreign Language (EFL) context. Since the learning objectives in the syllabus may comprise linguistics, literature, and language proficiency, it may not be entirely clear how these relate to each other.

Mastering complex sets of skills represent a steep learning curve for EFL students who already contend with the language barrier. Goldman et al. (2016) outline the high level of tasks demanded from students in literature courses, such as how to “notice rhetorical and structural moves prototypically made by authors to convey meanings and make their argument” and also “construct oral and written arguments regarding their interpretations” (p. 226). Not surprisingly, mismatches will occur if students think that it is enough to formulate personal response statements, but the conventions of the field dictate that the reader makes a persuasive case for an interpretation of pertinent aspects of the text. The current inquiry aims at studying how EFL students engage in discourse practices in the field of literary studies when participating in two different oral exam formats, namely Socratic seminars (National Paideia Center 1988) and Thought-Question-Epiphany (TQE) seminars (Thompson 2017).

Conventions such as those mentioned above, and others are part of the tacit knowledge that professors hold about how the discourse communities of literary studies are organized and evolve. Fish (1980) suggests that this process stems from how the act of reading is conceptualized, highlighting that “meanings are the property neither of fixed and stable texts nor of free and independent readers but of interpretive communities that are responsible both for the shape of the reader’s activities and for the texts those activities produce” (p. 322). In order to pass their literature classes, novice students thus need to learn not only how to read in a new way but also arrive at analyses of literature that conform to what literary critics would recognize as valid contributions to the discourse community and to do so in writing, class discussions, and oral exams or presentations.

Developing disciplinary literacy relies on a series of steps, but the scope of the present study will be limited to the following three learning objectives: interpersonal skills, analytical skills, and the ability to use textual evidence. According to Goldman et al. (2016), identifying appropriate textual evidence for interpretative claims constitutes a difficult task for students that requires scaffolding. Such scaffolding can be designed as exercises to make visible the process of identifying passages in the text that may serve to substantiate claims about the effect of symbolism, etc. Along similar lines, Corrigan (2019) acknowledges that he often asks his literature students to add “more complexity, more nuance, more insight, more depth, more critical and creative thought, more attention to the details of the text and to the context in which those details exist” (p. 3). These last two points refer to close reading skills, which are essential for high-quality literature papers. Clearly, students need explicit instruction on how to arrive at a literary analysis that passes muster.

One example of explicit instruction comes from Chick et al. (2009) who conducted a lesson study of ways to have students in literature courses experience “the pleasures of difficulty, complexity, paradox, ambiguity, and the multilayered meanings in literary texts” (p. 401). In their investigation of student responses to a poem, they found that “data support the value of collaborative learning as well as students’ recognition of this value” (Chick et al. 2009, p. 425). This is in line with what Weissman (2010) proposes, namely, that it is not enough to teach students close reading, rather “[a] more multifaceted, self-reflective, and collaborative reading practice is required, one that addresses how a literary text both constrains and exceeds the meanings its readers make of it” (p. 44). Such an endeavor accommodates the barrage of literary theories introduced in the field ever since 1938 when. Rosenblatt ([1938] 1995) posited that reading is not only an event but a transaction between the reader and the text. She was one of the first to challenge the focus on the unity of the text proposed by New Criticism, even if it took until the 1970s and 1980s for the field to diversify and more fully embrace new approaches (Parker 2014).

In recent years, approaches focusing on active learning and student-centered pedagogy in higher education have been lauded as more in tune with the needs of college students entering a rapidly changing global market (Misseyanni 2018). Moving away from the supposedly neutral position of “the informed reader” in formalist theories (Fish 1980, p. 48) and instead introducing multiple theoretical lenses, may allow students to read against the text in a literature classroom based on “critical inquiry” (Beach et al. 2016, p. 5). Within literature pedagogy, using critical inquiry in the classroom entails a power shift from the instructor to the students regarding what is considered meaningful learning (Beach et al. 2016). While reading against the text may seem quite advanced for undergraduate students, Appleman (2015) argues that even in secondary school “[o]ur responsibility as literature teachers is to help make the ideologies inherent in those [literary] texts visible to our students” (p. 3). Likewise, Corrigan (2017) states “teaching literary studies means teaching the ways of reading and writing that constitute our discipline, particularly critical reading and persuasive writing about texts” (p. 550). When using theoretical approaches, students can learn to identify ideological layers of a literary text that are not immediately apparent; a critical skill that can serve them well also in other areas of their lives.

In this approach, conversations about literary texts can instead deal primarily with what students bring to the table. This is especially prominent in Socratic seminars, which is one of the formats investigated in the present study, and where the type of discourse differs from other established academic formats in the sense that it is neither a deliberative dialogue with consensus as the objective nor a debate where winning over opponents matters most. On the contrary, it has been designed to encourage students to build on each other’s ideas (Davies and Sinclair 2014). The range of discussion formats used in a critical inquiry classroom may, for instance, cover literature circles (Daniels 2002), book talks (Chambers 2011), and similar peer-led group constellations.

In terms of assessment of learning objectives, Shanahan and Shanahan (2014) propose that explicit instruction may allow students to recognize more fully what is valued in various learning situations, equipping them to succeed to a higher degree. The main idea stated by Shanahan and Shanahan (2014) is that disciplinary literacy “invites students to join the disciplinary field itself. It’s a kind of invitation to join a club” (p. 629). Students are welcomed as potential members, which aligns with the principles of equity and inclusion.

Viewing discipline-specific conventions as a club may not be prevalent among instructors as of yet. Wilder (2011) found in an empirical interview study of 13 professors of literature and composition that “disciplinary discourse practices appear to function largely implicitly in the course’s background” (p. 60). Furthermore, the only professor who claimed to use their course as an introduction into the discipline still refrained from mentioning to students exactly what literary theory they were using (Wilder 2011). When theoretical underpinnings are hidden, the authority of the professor is upheld, and students may be less able to question propositions made (Parker 2014). Such taken-for-granted attitudes to epistemological matters may also impact how students view the degree of subjectivity and bias in grading. In a study on assessment at a university in the U.K., MacLellan (2001) found that a majority of the students believed that their professors used criteria not stated on syllabi when grading. It is possible that if instructors are unaccustomed to clarifying what the conventions of the field are, they may be equally vague when it comes to grading criteria.

Increasing the transparency within literary studies in tertiary education may be fraught with difficulties due to teaching traditions. Wilder and Wolfe (2009) suggest that in undergraduate writing courses explicit instruction can be quite a contentious issue: “[w]hile many scholars and instructors have advocated explicitly teaching these formerly tacitly imparted conventions, others have strongly argued against such explicit instruction” (p. 171). The reasons might be that professors fear that students will not be able to implement the strategies because the discourse practices are too complex and can only be learned through mentoring (Wilder and Wolfe 2009). In addition, instructors may want to protect students’ unique voices and prevent formulaic writing (Wilder and Wolfe 2009). Along similar lines, abstract qualities, such as style and originality that are valued in literary studies perhaps due to their close vicinity to the expressions of elegant prose in literature, may also influence the grading process if instructors see these qualities as innate rather than acquired.

As noted above, the present study compares the result from Socratic seminars and TQE seminars both of which fall under the umbrella term of dialogic teaching. Dialogic teaching, as defined by Alexander (2020), “harnesses the power of talk to engage [students’] interest, stimulate thinking, advance understanding, expand ideas and build and evaluate arguments, empowering them for lifelong learning and democratic engagement” (p. 1). This requires fine-tuned orchestration by the teacher, who needs “a broad array of interactive skills, strategies and moves [and] to exercise their own judgment about how these are most effectively applied to the particular context in which they are working ...” (Alexander 2020, p. 3). For teachers using constructive alignment (Biggs 1996) and transparent grading procedures, this type of engagement and student-centered learning can also be implemented in the process of assessment in the form of Socratic seminars.

1.1. Definition of Socratic Seminar

The Socratic seminar method has been used for decades in the United States, most commonly under the auspices of Paideia seminars. The National Paideia Center (1988) was started in North Carolina by Adler and Friday in an attempt to cultivate critical thinking skills in a “public school system that was at once intellectually rigorous and fundamentally equitable”. Originally a technique used to boost the performances of underserved student populations, it has recently been implemented in advanced placement courses thus effectively catering to privileged groups (Getachew 2021).

In a quasi-experimental study of Socratic seminars, students engaged in more complex discussions and used more textual evidence (Davies and Sinclair 2014). Davies and Sinclair (2014) clarify that “[c]entral to Socratic questioning is the provision of a thought-provoking, open-ended question, which promotes inquiry and allows ideas to be probed, grappled with and tested” (p. 23). According to Chisholm and Quillen (2016), dialogic teaching is one way to increase collaborative learning and student engagement. In an action research study on Fishbowl discussions, which entails that there is one group of students engaged in discussion and another group of students observing and taking notes, they found that

students responded in ways that were (a) dialogic (e.g., posing questions about the text and engaging in multiple perspectives), (b) metacognitive about participation structures, (c) supportive of other student voices, and (d) engaged in complex dialogic exchanges across discussion platforms—all functions of discussion that helped promote both meaning making about the text and relationship building.

(Chisholm and Quillen 2016, p. 89)

Similar to what Chisholm and Quillen (2016) found when employing the Fishbowl format, a Socratic seminar will also be conducive to dialogic learning and active listening skills. In addition to the fact that the Socratic seminar can function to increase student engagement, it also qualifies as a relatively safe examination form in comparison to other group-based projects. Social loafing will be less of an issue in a Socratic seminar since each student is assessed on their individual contributions and skills. Even in a group that might be less inclined to cooperate, it is possible for a student to engage in the behaviors and demonstrate the required skills without having to worry about others’ performances reflecting negatively on them.

Nonetheless, as in any examination procedure, there are safeguards that may have to be put in place to counteract undesirable behaviors. In her English classroom, Walsh-Moorman (2016) found that “the seminar sometimes supported pre-existing social dynamics allowing some voices to dominate” (p. 72). Another problem concerned the fact that students introduced textual evidence rather seldomly. In fact, Walsh-Moorman (2016) reports being “disappointed by actual time spent in an instructional period talking about the text” and she “noted that most discussion is more universal to the text; about big ideas and themes, rather than specific passages and details” (p. 73). In an article reporting on the work of two English Language Arts (ELA) teachers, Barker (2017) outlines their strategies to counteract similar problems, for instance interrupting the discussion when the talkative students have said enough and organizing digital comments for students observing the discussion in a Fishbowl activity thus allowing shy or reticent students a chance to succeed as well. The careful wording of instructions and rubrics as well as strategic preparations, such as setting ground rules for group work may further minimize negative behaviors.

In the courses that I teach, I have experimented with different kinds of oral exams but the one method that has consistently received positive feedback in student evaluations has been the Socratic seminar, adapted from the format developed by the National Paideia Center (1988). Since I specialize in teacher education, my aim is to model a variety of instructional strategies and assessment tasks for pre-service teachers. Therefore, I was intrigued when I came across a new format for small-group literary discussions in my Professional Learning Network. During the 2020/2021 academic year, I used the Socratic seminar in the fall term as usual, but in the spring term, I implemented this new format called the TQE seminar (Thompson 2017) in one course (38 students) but continued to use the Socratic seminar in my other class (62 students). The reason for using this material for research purposes was to investigate whether these oral exam formats are conducive to enculturating students into the discipline of literary studies.

1.2. Definition of the TQE Seminar

The TQE seminar format was developed by ELA teacher Thompson in 2017 in an attempt to move away from teacher-written questions to more meaningful learning by having the students themselves generate the questions. On her website, Thompson defines TQE as “a student-driven and teacher supported exploration of a text through individual small group and whole class curated inquiry.” Thompson (2017) explains that “Students write their OWN Thoughts, Lingering Questions, and Epiphanies about the text.” In the original format, there are multiple steps to the process. Step one is assigning a text for students to read at home. Step two is for students to formulate a thought, a question, and an epiphany based on the reading. Thompson has devised a list of question stems moving from basic to more advanced that she uses to model the kind of cognitive skills she is looking for in this step. In step three, students discuss their TQEs in a small group and decide on the best two of each category. Step four consists of a whole-class discussion of the selected TQEs from each group. When I used it with my students, I omitted the ranking of TQEs as well as the whole-class discussion. I naturally also shared the blog post that Thompson wrote with my students.

1.3. Aim and Research Questions

The purpose of this small-scale action research study is to investigate how two different formats of oral exams may serve to implement disciplinary literacy in the tertiary EFL seminar. To that end, the Socratic seminar and the TQE seminar are evaluated concerning their potential to introduce students into the academic seminar culture of literary studies. The research questions are as follows:

• To what extent do students’ rhetorical moves in the oral exam formats of Socratic seminars and TQE seminars align with discipline-specific conventions?
• What strengths and weaknesses of the two formats can be detected in the oral exam setting?

2. Materials and Methods

The reason the present study can be called action research (Mertler 2019) is that it is based on questions that arose from the practice of my own teaching and seeks to validate participant voices. The method used for data analysis is qualitative content analysis (Brennen 2013). In this research design, credibility will depend on to what extent the researcher is knowledgeable about the setting and thus able to identify a relevant focus for the study as well as suitable methods for the data collection (Graneheim and Lundman 2004). As regards transferability (Graneheim and Lundman 2004), it may be possible at least to replicate the assessment procedures. The study follows the same subset of students over the course of one term and, for the majority of students, two examinations, which has the potential to yield results of interest for practitioners in higher education perhaps considering implementing similar discussion formats in their oral examinations. In alignment with the Swedish Research Council’s ethical guidelines (Swedish Research Council 2017), informed consent was obtained from all participants involved in the study. Furthermore, students were informed that participation in the study was voluntary and could be terminated at any time. All student data are presented in a way that protects individual student integrity and anonymity.

2.1. Setting and Materials

The Swedish university where this study was carried out has five faculties, 33 departments, and an enrollment of circa 44,000 students. The prerequisite for admission to English studies courses is English 6 from the Swedish upper secondary school; the equivalent of B2 on the Common European Framework of Reference (Council of Europe 2020) scale. The material consists of what could be called observation notes somewhat similar to the unstructured field notes linguistic ethnographers take in participant observations (Copland and Creese 2015). These notes were taken during oral exams in a small group setting from two literature modules in English studies, term 2. I will also refer to written communication with students.

2.2. Courses and Participants

In the English Department, we strive to give extensive feedback on oral and written assignments. We have designed the assessment in this way because we hope it will help novice students learn the conventions of the field of literary studies. The grading criteria we use are meant to conform to the higher-order thinking skills outlined in the upper tiers of the revised Bloom’s taxonomy: analyze, evaluate, and create (Bloom and Anderson 2014), aiming to train students in critical reading.

Participants included all students enrolled in the second term of English studies in the academic year 2020/2021 and who signed up to take the oral exam, 62 students, see

Table 1. Course overview.

Course	Number of Students	Socratic Seminar Participants	Thought-Question- Epiphany (TQE) Seminar Participants
African American literature	62	52	10
Literature instruction in the EFL classroom	38	0	38

. Due to the pandemic, all classes and examinations were held online via Zoom. The time limit for the oral exam was 45 min per group.

In the oral exam in the course on African American literature, which was the last of four modules in the literature course in the second term, 62 students participated. Out of the 62 students, 24 were students in English, and 38 were pre-service teachers with English as one of their two subjects. The pre-service teachers took an additional course: Literature instruction in the EFL classroom, which was given in the first half of the semester. It was in this course that I introduced the format of the TQE seminar. In the TQE seminar, the students analyzed a short story that they had chosen from a list of six stories as well as drew on extracts from a theoretical text. The assignment was to prepare TQEs (thoughts, questions, and epiphanies) both for the short story and for the theoretical reading.

Since an overwhelming majority of the teacher trainees preferred the TQE to the Socratic in the evaluations, I gave them the option to do a TQE seminar in the course on African American literature as well. Among the nine groups of pre-service teachers, three groups chose the TQE format, ten students in total. The students formed groups themselves and signed up via the learning management system (LMS) for a time slot on a given day. On that day, they had to submit three higher-order and open-ended questions in writing (or their TQEs) via the LMS.

In preparation for the oral exam in the course African American literature, the students had studied Lorraine Hansberry’s play A Raisin in the Sun (1959) and had selected a poem or a song from the first workshop (Hansberry [1959] 2001). In the Zoom meeting, after I had given them the instructions, I turned my camera off and took notes on my laptop as the seminar unfolded, hence the discussions were entirely student-led. The reason for not recording the Zoom sessions was to alleviate the stress level for students. These exams took place during the pandemic and the faculty was informed by the Student Health Center that many students were suffering from anxiety and high levels of stress. Therefore, I emphasized that the Socratic seminar was a learning opportunity; a chance to speak about literature with others. Afterward, I shared my notes with students together with the results of the grading. These observation notes are limited to what I was able to take down during the conversation.1 As the students discussed, I wrote down as much as I could capture what they said in a document in which each student had one column. At the end of the day, I went through my notes and expanded on abbreviations, and corrected spelling mistakes. I also created one document for each student with only their utterances included. This document was subsequently used for grading the student’s performance. For the Socratic seminar, the rubric contained nine criteria, namely interpersonal skills, listening skills, ability to pose questions, dialogic skills, analytical skills, textual evidence, intertextuality, historical and cultural context, and terminology. I used a rubric with eight criteria for the TQE format: interpersonal skills, listening skills, ability to verbalize thoughts and convey epiphanies, textual evidence, intertextuality, historical and cultural context, and terminology. I also pasted their written assignment into this document so that they could receive feedback on the questions or the TQEs that they had uploaded to the LMS.

2.3. Data Analysis

The qualitative content analysis aims at bringing “out the entire range of potential meanings in texts” (Brennen 2013, p. 194). In accordance, the data analysis was conducted in three steps. The first step in the analysis of the notes was to construct a concept-driven coding frame based on previous knowledge of the discourse community of literary studies all the while upholding the principles of “unidimensionality” and “mutual exclusiveness” (Shreier 2013, p. 175). Unidimensionality refers to how “main categories should cover one aspect in the material only” whereas mutual exclusiveness concerns subcategories in which “any unit can only be coded only once under one main category” (Shreier 2013, p. 175). The main category was identified as conventions in the field of literary studies. Three subcategories were chosen in conjunction with the grading criteria: interpersonal skills, analytical skills, and textual evidence. The second step comprised coding notes based on the three subcategories and the third step consisted of a latent analysis. This was an attempt to “find the underlying meaning of the text” (Bengtsson 2016, p. 10). I wanted to investigate whether the notes of the Socratic seminars and the TQE seminars showed alignment with the conventions of literary studies. In the analysis of the Socratic seminars, I could relate my findings to previous research, but this was not possible for the TQE seminars. Therefore, I relied on the topoi identified in a replication study conducted by Wilder (2005) in order to relate the students’ disciplinary literacy practices to conventions in the field. Wilder’s (2005) formulation of topoi builds on the results by Fahnestock and Secor in 1991 (Fahnestock and Secor 1991). According to Wilder (2005), the following topoi indicate common warrants in the field of literary studies: appearance/reality, paradigm, ubiquity, paradox, and Contemptus mundi as well as the three topoi that have emerged in the field since then, namely social justice, mistaken critic, and context (pp. 84–85, 91).

3. Results and Discussion

In this section, the results from the study of classroom-based assessment practices will be presented and discussed. More specifically, I will focus on how disciplinary literacy skills are enacted in conformity to the discourse community of literary studies, which represents the main category in the qualitative content analysis.

3.1. Observation Notes from the Socratic Seminars

The presentation of the results stems from the three subcategories identified in the material when using the concept-driven coding frame. Extracts from the notes are meant to serve as an illustration for the reader of what each subcategory comprises.

3.1.1. Interpersonal Skills

The first research question (about the extent to which students’ rhetorical moves in the oral exam formats of Socratic seminars and TQE seminars align with discipline-specific conventions) deals with the conventions of the field of literary studies, i.e., disciplinary literacy (Shanahan and Shanahan 2008). As in all human interaction, there is a risk of enactment of power dynamics to the detriment of reticent or introverted students (Walsh-Moorman 2016; Barker 2017). By including interpersonal skills in the rubric, I tried to alleviate such concerns. If anybody were to dominate the discussion, they would receive the lowest point for that score. If students showed leadership skills and took care to invite other students into the dialogue, they were rewarded with the highest point. It could be argued that the rubric promotes shallow compliance with values, such as cooperation, respect, and leadership. On the one hand, the rubric makes the discipline-specific actions and behaviors explicit leading to gradual enculturation into this discourse community. On the other hand, it may reinforce status inequities since high-achieving students are more prone to carefully study the rubric and deliberatively try to engage in the desired actions. There is no denying that an oral group exam will by default reward extroverted students who enjoy talking in groups. Still, the practice of inviting fellow students into the conversation and keeping to a small-group format instead of conducting the exam as a formal individual presentation could benefit introverted or communication apprehensive students.

In the data analysis, the category of interpersonal skills in the Socratic seminar rubric concerns to what extent the students are able to take on leadership and establish a safe and encouraging group culture. The rubric included this skill because I wanted students to become enculturated into a seminar culture in literary studies where dialogue and critical inquiry are at the forefront (Beach et al. 2016).

Subcategory 1 Interpersonal skills

• Student 1 (S1): “Are you ready? So, in that case, I can start presenting the questions and you can start talking about them and we will all build on each other’s comments.”
• Pre-service teacher 1 (PST1): “Okay like I mentioned earlier I’m excited to have this discussion because I know you always have interesting things to say. So, who wants to kick this off?”
• S2: “The surveillance society that Kiser mentions, could that be like the influence why she doesn’t want to invest in the liquor store? What do you think?”
• PST2: “[Name], have you said your question?”
• S3: “You did very well, I had also identified these themes. You mentioned all of them.”

In the first example, we see how this student took charge of the discussion and helped establish the rules of the conversation, and in the second example how one student adopted a positive attitude. The third example shows how students made an effort not only to build on each other’s ideas (Davies and Sinclair 2014) and contribute new insights but also to invite others to participate. In the fourth example, the student ensured that everybody would be able to share the floor, thus conveying a sense that each person had a unique contribution to make in a collaborative reading practice (Weissman 2010). For reticent students, an invitation to participate may be the only way to gain a voice in the discussion (Barker 2017). The fifth example shows conformity with the academic seminar culture in literary studies in how students often started their comments on a positive note. Apart from being gratifying for the individual, these rhetorical moves can alleviate the pressure of being examined if students experience a boost in confidence.

3.1.2. Analytical Skills

Regarding analytic skills, several students used theoretical concepts from secondary sources when analyzing the primary texts. Some were also able to show awareness of contextual factors, such as historical context and author biography as well as make intertextual connections in a way that shed new light on the play or the poem. The rubric refers to contributing new perspectives or insights to the conversation and showing evidence of independent reasoning. The latter criterion is a sophisticated skill that is hard to define (Corrigan 2019), but I tend to see it as an ability to offer original in-depth interpretations.

Subcategory 2 Analytical skills

• S4: “I wrote down Kimberlé Crenshaw, ((quotes)), so it’s important for the sake of understanding how individual all cases are. Not enough to say that this is a person of color but it’s also a woman or related to reasons why this person would be disadvantaged in this dominant society.”
• S2: “I think these people try to dream and breathe fresh air, not allowing yourself to die from within. Beneatha and Asagai are culturally engaged and focused on their heritage. They embrace their true selves.”
• S5: “‘All Children Got Wings,’ African Americans would recognize the song as a spiritual the white audience might not, this serves as double meaning.”

Example one indicates the fulfillment of a learning objective in the course syllabus for term 2, to “apply different literary theories and methods in analysis of fiction.” In the material, several students made use of Critical Race Theory as a critical lens (Beach et al. 2016). In example two, we see how some students also addressed the wider significance of their claims when pondering how the characters represent anti-racist strategies. Example three fulfills another learning objective stated in the syllabus for term 2, the ability to “analyze literary texts in a way that takes cultural and historical contexts into account.” In terms of disciplinary literacy, intertextual references allow for deeper analysis in that they reveal the significance of various elements in the plot that novice readers may overlook. To sum up, students were able to engage in the discipline-specific discourse practice of how to analyze poetry and drama by using a critical lens, addressing possible ramifications of claims, and drawing parallels to other works of art.

3.1.3. Textual Evidence

Bearing in mind that Goldman et al. (2016) have highlighted the difficulty for novice students to provide textual evidence, it is noteworthy that all EFL students participating in the oral exam were able to fulfill that criterion. As this practice is also required in written assignments in the literature courses, students no doubt recognized its necessity for passing the exam, although the level of difficulty was shown in that many students only paraphrased. While their paraphrasing often showed extensive knowledge of the texts, students who were able to use direct quotes got higher points since this practice is more convincing in terms of supporting one’s claim. Therefore, indicating a specific passage in the text aligns more closely to discourse conventions regarding close reading (Corrigan 2019).

Subcategory 3 Textual evidence

• S4: “When he’s about to give up, when he screams and sits on his knees, and he tells his family how he’ll act this enhances the message from Hansberry where there will be these times when you want to give up but it’s still important to take your stand and use your voice ... I couldn’t see him as a good person with dreams until the last act.”
• S4: “I had a quote about why it was connected to Tupac ((quotes from play)) about stubbornness and reminder of her dreams of a garden.”
• S6: “In Tupac’s poem, ((quotes)) how will you connect that to Walter’s dream of a new business? ... Why isn’t he satisfied with what she and his father did for them? In the beginning of the play, why keep ((quotes)) dreaming?”

Example one shows what a typical paraphrase looked like in the material when used to substantiate claims about authorial intention and dynamic characterization. In example two, there is stronger alignment with the rubric since it includes a direct quote. Being able to paraphrase or quote from the text adds to the complexity of the discussion so it is an important disciplinary literacy skill (Davies and Sinclair 2014). Example three outlines how some students seamlessly integrated the evidence into their questions or claims and how meaning-making about the text was a collaborative practice (Chisholm and Quillen 2016). The reason direct quotes will make claims more persuasive is that they allow the student to point to exact wording in the primary text (Corrigan 2019). From an assessment perspective, this practice qualifies as more advanced since it entails being able to identify poignant passages at the same time as taking part in the discussion (Goldman et al. 2016). All in all, the ways in which students relied on textual evidence to ponder ideas and strengthen the validity of claims indicate that students are cognizant of this subset of discourse conventions.

3.2. Observation Notes from the TQE Seminars

As stated above, notes from the oral exams were categorized in alignment with the concept-driven coding frame. In the presentation of the results below, examples have been selected from the ten pre-service teachers who chose this format for the oral exam in the course African American literature. Since this is a new method that has not been empirically studied, I will use the topoi found by Wilder (2005) as the basis for the analysis of the data.

3.2.1. Interpersonal Skills

Subcategory 1 Interpersonal skills

• PST3: “This connects well with my thought. What happens when a dream is deferred? Like you said, ((Name)). He starts to see and long for the happier days of his life. A bird that does not lose hope. Hope is significant in both texts. The Youngers never lose hope, the bird sings and has hope. Connected to my epiphany.”
• PST4: “I really agree, this leads to my epiphany. Similarities between the poem and the play, metaphors in the poem similarities between the caged bird and black Americans.”
• PST5: “Do you want to discuss this thought?”
• PST3: “((Name)), you mentioned family and history melded together and Walter realizes this in the end. Very good epiphany.”
• PST6: “We found a lot of similarities. Strong metaphors here, more mundane with ironing. I see more that I didn’t see before. Thank you!”

In example one we see how students built on each other’s statements also in the TQE seminars while engaging in the discourse practice of finding patterns in the literary texts, which Wilder (2005) calls the ubiquity topos. Example two indicates that the students engaged in collaborative reading practices (Weissman 2010) when they used other students’ ideas to make connections and to move forward in the discussion. There are also invitations to share the floor by students as in example three (Barker 2017). Consequently, even if the TQE format has less focus on questions (one among three contributions), students still used questions as a means to increase the dialogic nature of the discussion (Chisholm and Quillen 2016). In alignment with the academic seminar culture, example four showcases how students took care to keep a positive tone in the discussion. Example five is proof of student feelings as the discussion drew to an end. These recurring words of encouragement and praise to fellow students worked as attempts to create a relaxed ambiance and alleviate the pressure of being examined.

3.2.2. Analytical Skills

Based on the pre-service teachers’ experience from the TQE seminar in the previous course, one group decided to start with the epiphanies so that they would have time to discuss them better, which shows how strategic students may be when given the chance to control their own assessment practice.

Subcategory 2 Analytical skills

• PST3: “My epiphany came from seeing the sorrow in both texts, the poem “Sympathy” the caged bird, how the bird feels after having ... imagery of being a bird placed in a cage that has the imprisoning effect on body and soul of being a metaphorical symbol of slavery lives on also in the next generation lives out in the play prejudice, discrimination. There is a happy ending, but sorrow lingers on, what will happen next? Even in a happy life, it is filled with sorrow.”
• PST5: “Mama has a conversation with Walter ((quotes)) money is life, freedom used to be life. Mama refers to historical oppression Jim Crow laws and the Harlem Renaissance in the 1920s–1930s. Many blacks moved north to escape this. For the Younger family in the play, feel trapped in the ghetto. The caged bird. The free birds dare to claim the sky. Even if Hansberry wrote the play 30 years after Harlem Renaissance. What happens when a dream is deferred?”
• PST6: “In Dunbar’s poem, the cage is both metaphorical and physical, how people were treated, not allowed to mingle at all with white people, they were punished for using the same fountains. The cage is similar for Travis, he cannot go anywhere without the family.”

In terms of disciplinary literacy, example one shows evidence of awareness of not only literary devices, such as metaphors and symbolism, but also the wider significance of inter-generational trauma and existential issues, and this could be conceptualized as Wilder’s (2005) topos of social justice. In example two, we find the topos of context (Wilder 2005) in the references to literary period as well as intertextuality. In example three, the student was able to point to the deeper meaning in the selected passage, as in the appearance/reality topos (Wilder 2005). The student used both knowledge about the historical context and the extended metaphor of the caged bird in Dunbar’s poem to build the case for the impact of structural racism on the youngest character in the play. In short, the variety of analytical skills found in the student discussion aligns with typical rhetorical moves in the discursive practice of literary studies.

3.2.3. Textual Evidence

Just as in the Socratic seminar, all students refer to the primary texts in their discussions using either paraphrase or direct quotes.

Subcategory 3 Textual evidence

• PST4: “I would say that he tried to break free from the cage but ended up in a different one. This white man who tries to buy them out from the house. The racism is put into words, integration is not possible if you want to succeed as a black man in this time. Hansberry tries to show that as a black man you have to break free in a different way.”
• PST7: “Interesting when Walter says that money is life. In a way, they are not slaves anymore, but money represents freedom because they don’t feel free. You can’t live without money so Walter has a point. You can’t become the free bird without the money.”
• PST8: “((Quotes)) so with Critical Race Theory in mind, regarding how racism is normal in the US for every person of color. This can be observed in “Caged Bird,” the free bird can go ... Fitzgerald states that ((quotes)).”
• PST4: “Also Kiser talks about how you should not run from surveillance but stay and fight instead. How strong that is. Racism is still a problem. What responsibilities we have regarding literature and what we can do about it.”

Example one comprises the social justice topos (Wilder 2005) when relating the poem to the play, contextual knowledge of the time period in the reference to the white character, as well as authorial design in regard to what the playwright is doing. In example two, a paraphrase is used as the basis for the paradigm topos (Wilder 2005) about being trapped in the proletariat in a capitalist society and there is also a reference to the extended metaphor in Dunbar’s poem. In example three, we see that some students quoted or paraphrased both from the primary and secondary sources and the latter would be in accordance with the paradigm topos (Wilder 2005). As comes across in example four, these pre-service teachers address the wider significance of literary interpretation, and they also draw parallels to their future careers. In this way, disciplinary literacy for pre-service teachers encompasses pedagogical matters.

3.3. Strengths and Weaknesses of the Two Formats

The second research question (about the strengths and weaknesses of the two formats that can be detected in the oral exam setting) centers on what advantages and disadvantages the Socratic seminar and the TQE seminar may have. Since my design of the two formats was built on dialogic teaching principles (Alexander 2020), both formats had the capacity to increase student engagement by relying on collaborative learning (Chisholm and Quillen 2016). In my student evaluations, there are often comments on how interesting it is to hear others’ interpretations of the same text. Centering the conversation on student-selected passages and text-to-self references also conforms to discourse conventions in the field of reader-response theories (Rosenblatt [1938] 1995).

Despite the fact that 79 percent of the pre-service teachers preferred the TQE method to the Socratic seminar, only three out of nine groups chose this format for the oral exam in the course on African American literature. It is possible that they chose the Socratic seminar since that was the format stated in the LMS while the option of having a TQE instead was mentioned in class. In connection to their future profession, several pre-service teachers proposed in written communication with me that the TQE method could work well in the EFL classroom for developing close reading and critical thinking skills and they also commented on how the method could be used to gradually build such skills by offering question stems as scaffolding. In terms of disciplinary literacy, there were engaged discussions in the TQE seminar as well. Students built on each other’s contributions and analyzed the texts as much as they did in the Socratic seminar. Likewise, students gave each other praise and encouragement also in the TQE seminar although there was less emphasis on that in the rubric.

The format of TQE could potentially have led to more of a monologue than dialogue style conversation with each student stating their own thoughts and epiphanies in a sequential manner. Yet the data does not support this hypothesis as students do build on fellow students’ thoughts and move the discussion forward by making connections. The method of the TQE seminar transcends the sole focus on questions in the Socratic seminar. It has been my experience that the questions students pose in the Socratic seminar are sometimes so convoluted that they need to be read several times and explained by the interlocutor, which somewhat defeats the purpose of generating a lively discussion. It is also a common occurrence in Socratic seminars that students answer their own questions. This presents less of a problem in the TQE seminar where students are supposed to expound on their own thoughts and epiphanies. What is more, posing a question becomes one of several contributions instead of the single mode.

Another important point that could be a strength for the TQE method is how it allows students to do the work of interpretation before the seminar while the Socratic seminar requires students to think on their feet and formulate answers to their fellow students’ questions. The Socratic method, therefore, makes it difficult for students to provide textual evidence, which is also what Walsh-Moorman (2016) found in her study.

The problems listed in the students’ written communication to me with the TQE method were mostly connected to terminology. Several students were skeptical of the epiphany component. Before the seminar, I communicated with a student who did not feel comfortable claiming to have had an epiphany while reading the text. They argued that this will not happen with every text. Another complaint, which came up for the Socratic seminar as well, concerned the instructions given for the task. Students requested clearer and more extensive instructions. In order to increase transparency, MacLellan (2001) advocates “central involvement of students themselves in the assessment process” (p. 317). This may also preempt suspicions that implicit criteria are used. In the instructions, the epiphany can perhaps be explained as an insight to prevent students from thinking they should come up with something original or ground-breaking. Another idea could be to ask students to base the thought on a particular aspect of the text, thereby staying within the fictional world, while the epiphany could relate to the wider significance of the text, as in contextual awareness. In short, the TQE format is preferable to the extent that it offers a variation with three different modes of response and gives students a chance to locate textual evidence for claims in advance without feeling rushed in the moment.

4. Conclusions, Suggestions for Future Research, and Pedagogical Implications

This investigation of language practices in tertiary English studies focused on classroom-based assessment practices. The aim of this small-scale action research study was to study discipline-specific rhetorical moves in two oral exam formats. The research questions concerned to what degree students will engage in disciplinary literacy by analyzing literature according to field conventions, and what advantages and disadvantages Socratic seminars and TQE seminars may have.

In both assessment formats, students performed discipline-specific rhetorical moves, interacting in a respectful manner and using active listening skills, which allowed them to build on each other’s ideas. The material shows that students analyzed the literary texts using a critical lens as well as showing contextual awareness, and they also provided textual evidence for claims in a mostly persuasive manner.

Nonetheless, it is important to note that no generally applicable conclusions can be drawn given the limitations of the material and design of the present study. A major flaw in the data collection concerns using observation notes as a documentation method. More specific data on the rhetorical moves the students made in their discussions could have been attained if the sessions had been recorded and transcribed in full. The notes could also have been subjected to the highly detailed and structured procedures of conversation analysis in order to better reflect the dynamics of group interaction.

While there are advantages and disadvantages concerning both oral exam methods, the TQE seminar tended to generate more varied discussions and enabled students to identify textual evidence prior to the exam thus reducing the stress level. Nonetheless, the Socratic seminar is a widely used format that fits well into an inquiry-driven teaching practice where students learn to develop critical thinking skills (Beach et al. 2016) even if there is a risk that students may pose such complicated questions that the discussion suffers. Given that the Socratic seminars were entirely peer-led conversations based on the students’ own questions, it may have a high probability of being perceived as relevant by students in the spirit of dialogic teaching with a focus on empowerment (Chisholm and Quillen 2016). The breadth of interpretations may also encourage a view of literature as multilayered (Chick et al. 2009; Weissman 2010). A caveat when using this method of assessment is that in a Socratic seminar, students are assessed on their response to questions that they may not have had the chance to prepare for. This means that asking students to support their claims in textual evidence, especially direct quotes, is quite a tall order. It is difficult to quickly find a quote in the literary text in real-time. Naturally, some groups share questions with each other in preparation for the oral exam, but not everybody does.

The TQE method allows more freedom for students to ponder text-to-self and text-to-world associations (Rosenblatt [1938] 1995) and it moves away from the sole focus on questions. However, the terminology of the TQE seminar can be intimidating. Some students told me that the word epiphany conjured up a type of contribution that was difficult to arrive at by necessity. Perhaps the word insight could be used instead, in order to lessen the pressure on students to come up with such a profound contribution.

Both discussion methods rely on a student-centered approach in which students gain voice and agency to explore aspects of literary texts that they find meaningful and relevant, thus transforming an exam into a learning opportunity (Rust et al. 2005). This classroom-based assessment practice may show students that close reading skills are not enough, since it is in this collaborative reading practice (Weissman 2010) that they discover how the multidimensional quality of literary interpretation (Chick et al. 2009) enriches their own understanding. In order for students to move toward the position of the expert, this focus on student voice could be complemented with strategic and explicit instruction on the conventions of the field (Wilder and Wolfe 2009). It is therefore recommended that these assessments practices are used in tandem with carefully designed rubrics, which would increase the transparency of the grading practices for students (MacLellan 2001).2 One way to engage in explicit instruction in the field of literary studies could be to teach the topoi from the replication study by Wilder (2005). If the purpose is to enculturate novice students into the discourse community of the field of literary studies, then educators would do well to openly discuss what skills are needed to become a member of that club (Shanahan and Shanahan 2014).

Future research studies could further elucidate classroom-based assessment practices in the EFL context in higher education, especially concerning how various rhetorical moves the students make in the different assessment methods would be graded by a group of instructors. Another question to investigate deals with whether students become enculturated into this type of dialogic discourse to the extent that they continue using interpersonal and analytical skills as well as textual evidence in subsequent literature courses and eventually in their own teaching practice. It would also be interesting to study other less structured oral exam formats, such as oral presentations, regarding how each method determines what topoi and other rhetorical moves students use.