Comparing Teacher Priorities and Student Uptake in EMI Lectures: An Exploratory Study

Abstract

English medium instruction (EMI) has been increasing in higher education with broad intentions of stimulating internationalization and cross-cultural learning experiences. This form of education presents opportunities and challenges for teachers and students alike. Key challenges involve various levels of second language (L2) speaking and listening abilities among teachers and students operating in EMI contexts. This exploratory study therefore examines the relationship between the main ideas two EMI lecturers in Sweden intended for their students to learn during lectures and the main ideas that EMI students report learning in the same lectures. Prior to six lectures, the teachers summarized to the researcher the main ideas to be included in the respective lecture. Immediately following the lecture, students provided their own summaries of the main ideas. A keyword analysis comparing the teachers’ intended messages and students’ reports shows that students may not be recognizing and acquiring the main ideas that the teacher intends. Further analysis distinguished two sub-groups of students: those with self-reported Swedish as a first language (L1) and those with self-reported L1s other than Swedish. A binomial proportion test showed that L1 impacted the amount of lecture main idea key words reported by the students in this study. The paper closes with a pedagogic perspective encouraging EMI lecturers to monitor student uptake on a regular basis and adjust their lecture delivery to support better learning and retention of content delivered via EMI.

1. Introduction

As institutions of higher education continue drives for globalization and internationalization, so too has English medium instruction (EMI) maintained a high rate of influence, development, and demand worldwide. EMI, defined as “The use of the English language to teach academic subjects (other than English itself) in countries or jurisdictions in which the majority of the population’s first language is not English” (Macaro 2018, p. 15), has become established on several continents and within a number of national contexts (see Ozer 2020 for a summary). With increasing numbers of university faculty and students engaging in EMI to deliver and learn course content, this manner of education has received research attention at several levels, including broad language policies, stakeholder perceptions, and contextual comparisons. Attitudes of lecturers, challenges facing students, and potential strategies adapted by both groups have been surveyed and reported.

However, despite their appeal and popularity, the quality of EMI programs and the learning they might facilitate has often been overlooked (e.g., Aguilar 2017; He and Chiang 2016). Furthermore, global inconsistencies in relation to EMI pre- and in-service linguistic and pedagogic support exist (e.g., Macaro 2018), perhaps due to financial benefits and/or reaching self-proclaimed internationalization goals. In other words, universities may need to take more responsibility for training, supporting and monitoring the quality of EMI lecturing and the learning it stimulates (Ozer 2020).

At the individual course level, EMI typically involves the integration of several second language (L2) English language skills, including reading (through course literature), speaking (while engaged in seminars, group work, etc.), and writing (via the production of term papers, essays, and examinations). Listening comprehension is also a vital component of EMI, as students are often required to attend lectures, listen to instructors for extended periods of time, maintain their attention, take notes, and learn the lecture content. That content may be delivered by lecturers whose first language (L1) is not English to a student group with wide-ranging L2 English listening proficiency levels. These realizations have led to investigations and descriptions of challenges faced by students when listening to EMI lectures and to the strategies they use to cope with and learn from lecture content. In addition, the role of the lecturer and the obstacles they sometimes face, such as being forced to teach in a non-native language, has attracted research interest. To offset these challenges and better prepare EMI lecturers for this task, some form of linguistic and pedagogic training specifically for those teaching in EMI contexts has been recommended (e.g., Morell Moll et al. 2020). Given the variations in EMI lecturer and student English abilities, coupled with viewpoints that indicate L2 listening as the most challenging of the four skills in another language (e.g., Field 2008), specific research attention on the relationship between lecturer output and student learning at the individual lecture level is urgently needed in order to provide a localized and contextualized experiential understanding of the extent to which learning is taking place within EMI lectures.

Therefore, the purpose of this paper is to explore the premise that the content delivered by EMI lecturers is learned by students attending the respective lectures. While student learning can be measured in a number of ways, this paper focuses on the fundamental relationship between content the teacher intends for the students to learn and what the students actually report learning. To do so, it reports on a study from two EMI courses at a Swedish university. The study compares the main points of lectures as described by the teachers who planned and delivered the lectures with descriptions of main ideas elicited from students immediately following those lectures. The study aims to explore the extent to which the intended main ideas and the received content are the same and considers whether students’ self-reported L1s influence the amount of corresponding main ideas they report.

The paper begins with an overview of EMI lectures with a particular focus on the roles of the students and teachers who engage in the learning and teaching. A description of EMI in the Swedish context is also provided, leading to relevant research questions. The research design is next described, which is followed by a quantitative and qualitative illustration of the data. Comparisons of pre-lecture teacher interviews and brief post-lecture student reports help to shed light on how much of the intended content is being recognized and retained with an emphasis on main ideas expressed in lectures. The findings, in turn, lead to some tentative conclusions about the relationship between teaching and learning in “the trenches” of EMI.

2. Background

2.1. Second Language Listening

Listening involves an interplay of both bottom-up and top-down processing. The former involves analyzing and generating meaning from the actual acoustic input to which the listener attends. Individual phonetic sounds combine to form syllables, which form words, clusters, and utterances. These utterances eventually form larger stretches of discourse. The processing essentially begins at the “bottom”, with perceiving at the phoneme level before moving “upward” to parsing the speech stream into meaningful chunks, and so on (e.g., Lynch and Mendelsohn 2002). The latter, top-down processing, involves experiences, perceptions, and characteristics that each person brings to the listening event. These elements include background knowledge on the topic, understanding of genre conventions, expectations, and predictions about what they will hear, and previous interactions with the speaker(s) (Vandergrift 2004). When L2 learners are listening to the L2, challenges with either type of processing, or both, can lead to misunderstandings or lack of comprehension in terms of the content as well as potential negative affective factors such as listening anxiety, lack of motivation, and frustration. All of these various components of listening may bear on listening performance in EMI lectures.

L2 listening proficiency is particularly relevant in EMI lectures, where students need to potentially deal with, among other factors, high-stakes evaluation based on lecture content, unfamiliarity with both the lecture genre and content, and idiosyncratic English use by individual lecturers. Academic lecture listening is typically viewed as one-way listening, where the student does not have the opportunity to interrupt the lecturer and ask for clarification, although this more traditional perception is gradually changing (e.g., Siegel and Wang forthcoming). As such, listeners’ phonemic perception and utterance parsing skills are needed to access the meaning contained in individual sounds and words, which also trigger previous knowledge and contextual expectations (e.g., Clark and Clark 1977; Lynch and Mendelsohn 2002). One study of listener experiences in L2 academic lectures reveal challenges both in bottom-up processing (e.g., understanding blended phonemes and elisions) as well as top-down aspects such as unfamiliar cultural references and missed transitional markers (Sheppard et al. 2015). To help develop their English listening abilities, many students will have taken English for academic purposes (EAP) courses, which often feature lecture-listening and notetaking activities to practice and prepare for EMI situations (e.g., Aizawa and Rose 2020). These preparation courses are typically taken either prior to entering university (i.e., high school or upper secondary school) or during the first year of university (e.g., an academic English bridge course) with the purpose of preparing students for EMI in their major subjects later.

2.2. EMI Lecture Listening

The academic lecture within EMI has been an object of investigation for decades and previous research has provided ample information about various structural components of lectures, which Young (1994) outlines as macro- and micro-elements. At the macro-level are typical lecture structures such as moving from theory to applied examples; comparisons and contrasts; and argument-refutation (e.g., Dudley-Evans 1994). Recognizing these different lecture organization patterns and drawing on previous experiences in attending lectures (whether in the L1 or the L2) indicates the use of top-down listening processes and can help prepare EMI students for listening success in lectures.

At the more micro-level, lectures may consist of announcements, definitions, theoretical explanations, practical examples, descriptions of processes and/or concepts, asides, anecdotes, reminders, and so on (e.g., Young 1994), although not every lecture will include each of these elements. While these various components of lectures can be identified and defined, it is important to note that lectures “usually have no distinctly recognizable sections, but instead contain a series of interweaving phases that do not appear in a particular order and can resurface throughout a lecture at any time” (Crawford Camiciottoli and Querol-Julián 2016, p. 312). To indicate shifts between different parts of a lecture, speakers usually employ transition signals, which help listeners attend to these shifts. Identification of “topic-shift markers should provide a structural basis for dividing up a lecture into smaller units” (Hansen 1994, p. 133). This observation implies that L2 listeners need to be aware of and able to recognize these markers, which play a more prominent role in lectures than in other spoken interactions (Rodgers and Webb 2016).

Each of these micro-elements offers the lecturer a variety of linguistic and pragmatic options. Here, bottom-up listening is relevant since the student needs to decode the incoming input by first processing the sounds, chunks, and so on. Teachers make quick (often unconscious) decisions about how to express ideas and concepts, how to define terms, which examples to use to illustrate points, and which parts of their lecture to paraphrase, simplify and reiterate to encourage student learning. Decisions and patterns related to rate of speech, pace of pedagogic delivery, use of multimodal avenues, and pausing, among others, certainly influence the listening comprehension, comfort, and content uptake on the part of students.

2.3. Challenges in Learning in EMI Lectures

As suggested above, students listening to EMI lectures face a number of possible challenges in their efforts to understand and learn from their respective content teachers. Low levels of student listening proficiency represent a major hurdle, one that can only be exacerbated by EMI teachers who might be unaware of or who do not fully account for L2 development and the L2 English proficiency levels of their student groups (Siegel 2020a). Among the aspects of listening related to understanding lecture delivery are dealing with high and varying rates of speech, attempting to understand unfamiliar accents, and recognizing technical and subject-specific vocabulary. To illustrate the listening comprehension struggles of some EMI students, one interviewee in Yeh (2012) estimated that only one-third of students could follow and comprehend their lecture (p. 219), likely due to lower than desirable L2 listening proficiency.

The content of EMI lectures, which typically includes advanced and abstract concepts and discipline-specific jargon, can present comprehension-related challenges even in L1 contexts; therefore, it should be no surprise that learning this type of content in an L2 is even more difficult. The type of English students are expected to understand goes well beyond the more common everyday topics that are usually covered in general English classes. Cultural differences between students and the lecturer in terms of, for example, the role the lecturer is expected to play and varying views of the lecture genre only add to the potential layers of complexity in attaining sufficient comprehension (e.g., Aguilar 2016; Flowerdew and Miller 1995).

Several recent studies provide a generally unified perspective on the aspects of EMI lecture comprehension that students perceive to be challenging. Ali (2020) reports that Omani students studying engineering via EMI have difficulty with teacher accents, speed of input, and notetaking. The act of simultaneously listening, prioritizing information, and deciding when, where and how to take notes is a particularly sophisticated activity (e.g., Piolat et al. 2005), particularly when operating in an L2 (e.g., Siegel 2020b). Understanding technical vocabulary is another obstacle students report facing across different disciplines (e.g., Blackwell 2017; Jarvis et al. 2020). Others have found that teacher accent (e.g., Bolton and Kuteeva 2012; Ali 2020) and rate of speech (e.g., Blackwell 2017) are often problematic. Siegel’s (2020a) study on estimated comprehension of EMI lectures and their understanding of their teachers’ English use showed wide variations. Many of these issues relate to challenges that L2 English listeners face even when operating in relation to more common everyday instances; that is, L2 English listeners often report problems in recognizing words in speech that they know in print, inability to parse the speech stream into meaningful chunks, and comprehending spoken input at high rates of speech (e.g., Goh 2000; Field 2008). However, the majority of these studies are based on self-report data from students and few, if any, studies in the field focus on the uptake of specific items of information. Instead, they focus on general impressions of comprehension.

In order to cope with and learn from content delivered in EMI lectures, students have adopted a range of strategies. Students may, for example, support their learning by completing pre- and post-lecture reading tasks, which would allow them to preview and/or review key vocabulary and concepts. They also avail themselves to technology in the form of translation software and bilingual dictionaries to access meaning (e.g., Ali 2020). Social strategies such as group work and asking classmates or family members for help are also options (e.g., Ali 2020). Taking notes is another common approach to help listeners maintain their attention, prioritize information from lectures, and store it for later use. While taking notes might seem an obvious and expected student action in lectures, this complicated academic activity becomes more challenging when listening to a lecture in an L2 (e.g., Siegel 2020b). When possible, students may interrupt to ask for clarification and/or do so after a lecture has concluded or during office hours; however, these opportunities are not always guaranteed. Moreover, teachers may expect that students are able to comprehend the majority if not all of the lecture content, especially if there is reading assigned.

Pedagogic support in EAP courses can also work to offset the numerous challenges and prepare students for listening success in EMI. Pedagogic attention to academic listening received an initial boost in Richards’ (1983) seminal work on listening, which provided an extensive list of academic listening sub-skills. Among them are identifying topic, following topic development, recognizing discourse markers, attending to multimodal input, and being familiar with various organizational styles of lecture (Richards 1983, pp. 228–29). This initial venture into detailing academic listening combined with the notion of language learning strategies has generated strategy instruction for academic listening (e.g., Graham and Santos 2015; Siegel 2015). Such instruction aims to increase student awareness and use of strategies such as recognizing transition markers, predicting upcoming input, and identifying rhetorical devices.

Another common pedagogic approach to supporting EAP students’ future EMI lecture comprehension involves developing effective and efficient notetaking skills. A range of studies has detailed gains in notetaking performance and lecture-content test performance following intensive periods of notetaking instruction (e.g., Dunkel et al. 1989; Hayati and Jalilifar 2009). Techniques from corpus linguistics have also generated pedagogic options for helping EMI lecture comprehension. Zare et al. (2021) conducted research that tested whether data-driven discovery learning that focuses on concordance lines including “importance markers” yielded better lecture comprehension than explicit instruction. Test results showed a statistically significant advantage for those learning via the data-driven method.

This range of pedagogic approaches and tools aims to prepare students to comprehend EMI lectures and other forms of academic listening that may strain students’ L2 listening comprehension skills. At the same time, on the opposite side of the learning-teaching relationship, EMI lecturers face challenges of their own.

2.4. Challenges in Teaching in EMI Lectures

While students may feel some trepidation when entering a lecture hall to attend an EMI lecture, lecturers themselves may also experience feelings of hesitation. They may, for example, be compelled by broader educational policies to teach in their non-native language even though they prefer to teach in the L1 (e.g., Yeh 2012). Also related to policies, teachers may be unaware of the L2 English language requirements their programs or universities have in place for matriculation. Awareness of such policies as well as knowledge of language proficiency descriptors (e.g., CEFR) would provide lecturers with at least basic information with which to inform their pedagogic and linguistic choices. Furthermore, though qualified experts in their respective content fields, some EMI teachers may lack familiarity with theories related to L2 development and/or overlook the notion that their linguistic and pragmatic choices affect student comprehension a great deal (Björkman 2010; Flowerdew and Miller 1996; Siegel 2020a). Teachers have also reported, among other hurdles, that it can be difficult to simplify complex content, encourage participation, access accurate L2 English terminology, and provide spontaneous meaningful output in EMI lectures (Ozer 2020, p. 622).

Despite these challenges, teachers have developed coping strategies to facilitate student understanding. These unsurprisingly include code-switching between the L1 and L2 English (assuming there is a shared L1), simplifying language, and using visual aids to reinforce content (e.g., Yeh 2012). Ozer (2020) reports teacher strategies such as “using analogies or real-life examples” and “re-explaining the content more slowly” (p. 621) as additional options. These strategies seem to suggest that EMI lecturers are aware of the impact their use of English has on student comprehension; that is, they are aware that adjustments in rate of speech, employing simpler language, and repetition can increase learner understanding.

Training specifically designed for EMI lecturers has been emphasized as support lecturers themselves view as valuable. Teachers have indicated that they would welcome professional development in both linguistic (e.g., speaking, pronunciation) and pedagogic competence (Ozer 2020, p. 622). Björkman (2010) also suggests that language training, pragmatic strategies, and lecturing patterns in particular could lead to improvements in the quality of EMI lecturing. Morell Moll et al. (2020) emphasize the valuable role that multimodality plays in communication in EMI. Their analysis of EMI teaching samples shows that a combination of verbal and non-verbal factors (e.g., gaze, gesture, and written language) results in improved comprehension. However, as O’Dowd (2018) states, such training does not always receive the financial backing and prioritization that it likely deserves, given the potential loss of learning and the undue stress placed on both lecturers and students.

2.5. The Swedish EMI Context

The present study took place at a university in Sweden, a country that has a more than two decades of experience with EMI (Kuteeva 2018). As Björkman (2018) points out, EMI needs to be interpreted and set within the educational context of the respective country in focus. The use of English in higher education in Sweden has been the topic of much debate in recent years, particularly in relation to broader sociolinguistic questions of the status afforded to Swedish and English within education, language policies such as teaching through parallel languages, and whether students and teachers have sufficient language abilities to be effective in EMI (Kuteeva 2018). The present study places this latest issue in focus by examining the learning, the actual transfer of information, from teacher to students in specific classroom instances. Even the use of the term EMI within research conducted within Sweden is somewhat contentious, as some courses taught in Swedish have course literature in English and code-switching, either through speaking or in writing (e.g., on Powerpoint slides) blurs the lines as to what is EMI. Policies developed to improve EMI in the Swedish context have often been top-down and problematic to implement (e.g., Björkman 2014; Kuteeva and Airey 2014). These broad strategies “tend to be detached from individuals’ linguistic practices and the reality of specific disciplines” (Kuteeva 2018, p. 48). As such, the present study aims to understand the extent to which individual EMI students are able to process lecture content in English and report the main ideas contained therein.

According to enrollment policies at the university where the study took place, to be eligible for an EMI course, students must have completed one of the following, per the university admissions office: (a) if the student comes from Sweden, they are required to pass two mandatory English courses at upper secondary school (with the expectation that their L2 English proficiency level is approximately B2); (b) if the student comes from outside of Sweden as an exchange student, the sending institution sets the L2 language proficiency requirements. Examples of the latter could include a strong score on IELTS or TOEFL or previous credits and/or graduation from another institution where English is used. Since the students involved in this small-scale study were enrolled in EMI courses at this university, those with Swedish as an L1 likely met the requirements outlined in (a) and those with other L1 backgrounds satisfied the criteria listed under (b).

2.6. Research Questions

As noted by Zare et al. (2021), noticing relevant information in English academic lectures is a prerequisite for learning that information (p. 2). In order to investigate the relationship between teachers’ intended learning uptake and students’ perceived and reported learning uptake in university EMI courses, this study addresses the following two research questions:

1. To what extent do teachers and students in EMI agree on the main ideas expressed in lectures?

2. To what extent does self-reported native language (i.e., L1) affect recognition of main idea key words in EMI?

3. Methods

3.1. Participants

Two types of participants were involved in the study: (a) two lecturers who teach EMI courses at a Swedish university and (b) students attending those respective lectures. The teachers taught courses on sociology (Lecturer A) and robotics/artificial intelligence (Lecturer B). Both teachers used English as an L2 for instructional purposes and had been doing so for more than five years at the time of the study. Both described their L2 English abilities as sufficient for teaching at tertiary level; however, more specifics about the lecturers’ L2 English proficiency were not available. Two lectures given by Lecturer A and four lectures given by Lecturer B were included in the study. Despite this slight imbalance in the number of lectures, this exploratory study aims to illustrate student recognition of main idea key words in multiple courses.

The number of students attending the lectures ranged from eight to 14 (Despite the small number of students, these sessions are considered “lectures”). Per university enrollment regulations, students either had completed requisite English classes in Sweden in upper secondary school or satisfied the basic English proficiency requirement in another way (e.g., previous university study in English or an internationally recognized test). Individual measurements of L2 English proficiency, beyond the university requirements, were not used. The following L1s were reported by students: Croatian, French, German, Hungarian, Korean, Kurdish, Persian, and Swedish, the last of which was reported the most often (as to be expected, since the study was conducted in Sweden).

3.2. Data Collection and Analysis Procedures

Participating lecturers responded to an open call for participants at the university and signed letters of consent. Prior to each lecture, the researcher briefly met with the lecturers and asked them to summarize the three-to-five main ideas that they planned for and expected students to learn and take away from the respective lecture. At the conclusion of each lecture, students anonymously responded to an online survey in which they were asked to type out the three-to-five main ideas from the day’s lecture. The survey also consisted of a closed item where students reported their native language (Swedish, English or Other). This question was then used to separate an L1 Swedish group and an Other L1 group (no students reported English as their L1) in comparison with the open question that reported main ideas. This rudimentary grouping system was used in order to distinguish those students with the community language as an L1 (i.e., Swedish) from those with less common L1s in this context. It must be acknowledged that some of the reported L1s are linguistically similar to Swedish (e.g., German) while others are more dissimilar, including typologically, such as Korean.

The goal of the data analysis was to better understand the relationship between the teachers’ intended messages and students’ reported learning and uptake. The data collection generated from six lectures (two by Lecturer A and four by Lecturer B) consisted a total of 19 distinct main ideas that contained 35 respective key words, as described by the lecturers (note “organization”, “sustainability”, and “rule-based” are listed multiple times as main idea key words). These key words were identified by the lecturers themselves; as such, these decisions were made by experts in the disciplines rather than by the researcher. In consultation with the participating lecturers, synonyms were not recognized, as the lecturers felt the terms they wanted students to learn should be clear enough to uptake.

A total of 66 unique student reports were collected from the set of six lectures, 42 of which were given by L1 Swedish students and 24 by students with L1s other than Swedish. This study aimed to focus on the immediate recall and reformulation of main lecture points as expressed by students. To confirm that these key words were used in lectures and were not simply lecturer self-reports, all lectures were digitally recorded and transcribed. A cross-analysis of lecture-identified key words with the transcripts confirmed that all key words were part of the lecturers’ spoken delivery. Visual aids such as Powerpoint slides were not collected as part of this study.

In order to understand the calibration between these two sides of the teaching-learning relationship, key words in the lecturers’ set of intended main ideas were identified and confirmed with the lecturers themselves, similar to “keyword analysis” as outlined by Groom and Littlemore (2011, p. 162). The focus of keyword analysis is often on nouns (Groom and Littlemore 2011, p. 163), which constitute the majority of key words identified by the lecturers. Using frequency of key words as a measurement, searches for these key words were made within the students’ set of reported main ideas. The frequency of the key word tokens, as well as the co-text of those tokens, was examined in order to better understand the proximity between what the teacher thought they were teaching and what the students thought they had learned in terms of main ideas from the respective lectures (i.e., RQ1). It must be emphasized that this analysis focused on student ability to recognize main idea key words rather than to display understanding of the relevant concepts and ideas themselves.

Table 1. Illustration of key word identification.

Lecturer A’s First Main Idea (12 September)	Key Word Identification	Example 1 of Student Responses including the KW	Example 2 of Student Responses including the KW
Concepts of social structure and technology within organizational studies	“organizational” or any derivative	“differentiation, integration, organization social structure, the organization as a technical system”	“differentiation organizational structure(vertical horizontal)”
Lecturer A’s Second Main Idea (12 September)	Key Word Identification	Example 1 of Student Responses including the KW	Example of Student Responses NOT including the KW
Basic understanding of different theories and perspectives focused on social structure and technology	“technology” or any derivative	“Intensive technologies, risk and organizational social structures”	“A more in depth view in how organization might work and what potential future they may behold”

below provides an illustration of this process. Further analysis using the binomial test of proportions in R (Crawley 2013) was then used to ascertain the role that students’ self-reported L1s played in their ability to report key words (i.e., RQ2).

The following excerpts from the transcript of Lecturer A’s session referred to in Table 1 illustrate the presence of these key words in the lecture:

Lecturer A:

“so organizational social structure that would be relationship among people who assume the roles of the organisation. And the organisational groups or units to which they belong, like departments, divisions, etc.”

Lecturer A:

“So … and there can be … core technology, there can be service technology, there can be different types of technologies and you … you will see in the chapter there are different ways to conceptualize it.”

Some data from the student surveys included irrelevant information. Such responses were not included in further analysis.

4. Findings

Table 2. Lecture 1 main idea key words comparison.

Lecture 1 Main Points (n = 10 Students)	Key Words	Ratio of Reported Key Words (Total)	L1 Swedish	L1 Other than Swedish
Concept of organizational environment	Organizational	6/10	1/3	5/7
Overview and basics of different theories, organizational theories, organizational environment	Environment	4/10	2/3	2/7
Ideas and reflections on how to apply theories to sustainability management, issues and challenges	Theory(ies) Sustainability	5/10 6/10	3/3 0/3	2/7 6/7
	Overall ratios	21/40 (53%)	6/12 (50%)	15/28 (54%)

,

Table 3. Lecture 2 main idea key words comparison.

Lecture 2 Main Points (n = 7 Students)	Key Words	Ratio of Reported Key Words (Total)	L1 Swedish	L1 Other than Swedish
Concepts of Social structure and technology within organizational studies	Social Structure Organizational	3/7 4/7 6/7	1/3 1/3 2/3	2/4 3/4 4/4
Basic understanding of different theories and perspectives focused on social structure and technology	Technology	4/7	1/3	3/4
Ideas and reflections on how to apply theories to sustainability management, issues and challenges (same as 11 September, but different content)	Sustainability	1/7	0/3	1/4
	Overall ratios	18/35 (51%)	4/15 (27%)	14/20 (70%)

,

Table 4. Lecture 3 main idea key words comparison.

Lecture 3 Main Points (n = 14 Students)	Key Words	Ratio of Reported Key Words (Total)	L1 Swedish	L1 Other than Swedish
Feedback control (or closed-loop control) is a general concept that applies to “systems” in many fields: biology, medicine, economics, engineering, etc.	Feedback Loop Control	6/14 3/14 7/14	6/10 3/10 6/10	0/4 0/4 1/4
It is important to clearly define: the system, its inputs, its outputs	Define Input Output	0/14 0/14 0/14	0/10 0/10 0/10	0/4 0/4 0/4
In the case of robot motion control (go-to) the “system” is not just the robot, but the robot together with its environment	Go Robot Environment	3/14 5/14 0/14	1/10 4/10 0/10	2/4 1/4 0/4
In robotics, it is important to distinguish between the “work space” and the “joint space”	Space	0/14	0/10	0/4
	Overall ratios	22/140 (15.8%)	20/100 (20%)	4/40 (10%)

,

Table 5. Lecture 4 main idea key words comparison.

Lecture 4 Main Points (n = 14 Students)	Key Words	Ratio of Reported Key Words (Total)	L1 Swedish	L1 Other than Swedish
Rule-based systems is a technique from AI that allows us to encode “human knowledge” into a machine.	Rule-based System	12/14 4/14	9/11 4/11	3/3 0/3
We can use rule-based systems to encode “human knowledge” about how to control a physical system, e.g., a robot.	Encode Human Knowledge	0/14 0/14 0/14	0/11 0/11 0/11	0/3 0/3 0/3
In a rule-based controller, one can implement strategies that are too complex to encode in a PID controller, e.g., for obstacle avoidance.	PID Avoidance	2/14 3/14	2/11 3/11	0/3 0/3
	Overall ratios	21/98 (21%)	18/77 (23%)	3/21 (14%)

,

Table 6. Lecture 5 main idea key words comparison.

Lecture 5 Main Points (n = 11 Students)	Key Words	Ratio of Reported Key Words (Total)	L1 Swedish	L1 Other than Swedish
Fuzzy sets allow us to represent “graded” membership in sets, and properties that have “degrees”.	Fuzzy Graded Membership Degrees	11/11 0/11 0/11 0/11	8/8 0/8 0/8 0/8	3/3 0/3 0/3 0/3
Fuzzy sets are simply extensions of standard sets: everything you can do with standard sets, you can do with fuzzy sets.	Standard	1/11	1/8	0/3
Using fuzzy sets in rule based control leads to smoother control and smoother behavior of the robot.	Rule-based Control	4/11 2/11	2/8 1/8	2/3 1/3
	Overall ratios	18/77 (23%)	12/56 (21%)	6/24 (25%)

and

Table 7. Lecture 6 main idea key words comparison.

Lecture 6 Main Points (n = 10 Students)	Key Words	Ratio of Reported Key Words (Total)	L1 Swedish	L1 Other than Swedish
Planning is based on “mental simulation” of future states and actions	Planning Simulation	4/10 0/10	3/7 0/7	1/3 0/3
Breadth-first search is good because it is complete and optimal, but it can be slow	Breadth-first	6/10	5/7	1/3
Heuristic search can be fast, provided you can define a good heuristic function	Heuristic	4/10	4/7	0/3
	Overall ratios	14/40 (35%)	12/28 (43%)	2/12 (17%)

compare the lecturers’ intended main ideas with those reported by students immediately following the respective lectures. Of particular interest are the ratios presented towards the right sides of the tables, where the total number of reported key words as well as a break-down between L1 Swedish users and those with L1s other than Swedish is presented. Overall ratios for each lecture, based on the total possible key word recording opportunities is displayed at the bottom of each table. The data in these tables addresses RQ1 regarding the amount of agreement between the lecturers’ intended main ideas and the students reporting thereof.

Based on the overall ratios for possible key word reporting in Table 2, both the L1 Swedish and the Other L1 group included roughly 50% of the main idea key words. However, some discrepancies exist, such as 100% (3/3) for “theory” by the Swedish users compared to 29% (2/7) by the Other L1 group. In contrast, nearly all of the Other L1 group members (6/7) recognized “sustainability” as a core concept from the lecture while none of the three Swedish users did so.

Table 3 shows a broader contrast in reported uptake between the two groups, with the L1 Swedish users reporting 27% of the key words and the Other L1 group 70%. In comparison to the findings in Table 2, however, the number of Other L1 group members is noticeably fewer at n = 4. At the same time, the group sizes are more equal in Table 3.

Whereas in Lectures 1 and 2, both groups of students combined for approximately 50% of reported key words, Table 4 shows students reporting far fewer key concepts after Lecture 3. The L1 Swedish group had more success in reporting key words (20%) than their Other L1 counterparts; however, the combined 16% suggests that several key words (and thus main ideas) may have been overlooked, as they were not even partially reported. There are also several instances where students from neither group included a key word in their reports (“define”, “input”, “output”, “space”). In particular, main ideas two and four appear to have eluded both groups of listeners, at least in relation to their reports.

Table 5 shows a higher percentage of key word reporting (23%) by L1 Swedish students than by the Other L1 group (14%). The most-frequently reported key word was “rule-based”, which registered on 86% of reports, the second-highest result of any key word (see Table 6, “fuzzy” below). This strong recognition was consistent between both groups: L1 Swedish at 82% and Other L1 at 100%. However, the Other L1 group struggled to report any other key words for the entire lecture, including no key words for either the second or third main idea expressed by the lecturer. The L1 Swedish group also struggled to report any key words for the second main idea.

Table 7 shows another high recognition rate for both groups, this time the word “fuzzy” from the first main idea. All students in both groups reported this in their main idea summaries. Interestingly, no other key words were reported by either group in relation to the first main idea. Students in both groups struggled to express key words from the other main ideas, with “rule-based” (4/11) and “control” (2/11) registering as the next highest. Still, at this point, relatively low numbers (23% in total) of key words from the main ideas were reported, both for the L1 Swedish and the Other L1 groups. It is worth notetaking that “fuzzy” is a key concept for the lecture as a whole, since it appears in all three main ideas. While students have registered that “fuzzy” is crucial, they seem to be unable to report subtle differences in how the concept has been explained and used in the lecture, as evidenced by the lack of accompanying key words in the student reports.

The highest scores among Lecturer B’s sessions are shown in Table 7, where 35% of key words were reported overall. There is a stark distinction, however, between the L1 Swedish group (43%) and the Other L1 group (17%). In relation to the second and third main ideas, the L2 Swedish group reported the two different types of searches (namely, breadth-first and heuristic) whereas only one student in the Other L1 group was able to report one of those respective terms.

Overview of Lectures

In order to determine the overall performance of both groups throughout all six lectures, a binomial test was run in R version 3.2.1 (2015). The results of this test address RQ2, which considers the role that students’ L1s play in their ability to report key words. When combined, the L1 Swedish group reported 72 key words out of a possible 288; the Other L1s group reported 44 out of 117, respectively. Thus, the formula run in R was prop.test (c (72, 44), c (288, 117)).

Table 8. Results of binomial test in R.

Data	c (72, 44) out of c (288, 117)
95% Confidence Interval	0.01 0.22
p-value	0.015 *
Prop 1	0.37
Prop 2	0.25

* Statistically significant at p < 0.05.

displays the results of this test, which indicate a statistical difference between the L1 Swedish and Other L1s groups.

The p-value (0.01) is less than 0.05; thus, the null hypothesis that the two proportions are equal is rejected. Further, since 0 does not lie between the two confidence intervals, the proportions are not equal. Based on this result, the two proportions (0.37 and 0.25, respectively) are statistically different; thus, the L1 Swedish group produced a statistically signficantly higher propotion of key words than the Other L1s group.

Moreover, while a comprehensive qualitative analysis of the student reports is beyond the scope of the present paper, an initial holistic inspection of unaltered individual student main idea summaries demonstrates wide variation in students’ apparent understanding of EMI lecture content. Focusing on three student responses from Lecture 2, a wide discrepancy in terms of fully articulation of the ideas is evident:

Student 7 (Other L1 group): “incorporation of technologies within an organization and reflecting on it from positive and negative perspectives how an organization can be organized (flat, steep hierarchy etc.) where to put the sustainability administration body.”

Student 3 (Other L1): “different types of structure within organizations (social, physical...) webers bureaucratical view (pros and cons) technology impact on society (pros and cons).”

Student 4 (L1 Swedish): “A more in depth view in how organization might work and what potential future they may behold.”

Student 7 has reported several key words and provided a main idea summary that comes closest to the lecturer’s intended learning outcomes. Student 3 provides some details and specific examples (e.g., social and physical as different types of structures within organizations). Student 4 gives a very general summary of the lecture that lacks specific information and may be able to summarize multiple lectures on the same course. In other words, while key words may appear in all of three reports, some are clearly more extensive and nuanced while others are expressed rather vaguely.

A similar situation is evident in reports from Lecture 6, where various search algorithms were in focus, per Lecturer B’s main points. Students were able to articulate these different formulas to varying extents, as displayed below.

Student 10 (L1 Swedish): “Different heuristics searches, Breadth first search, Depth first search, greedy search. Manhattan distance. We ended with slightly go through A* search.”

Student 5 (L1 Swedish): “Basis uninformed search; BFS, DFS, Iterative serpentins search. Informed search; Greedy, A*.”

Student 4 (L1 Swedish): “Path planning, BFS and DFS algorithms, Heuristic search algorithms.”

Student 9 (Other L1s): “different algorithms.”

The four student responses above are loosely ranked in order by the amount of specific information about the various algorithms they contain. Student 10 named several specific types of searches as well as other related concepts. Students 4 and 5 both also specified different types of searches by name. In contrast, Student 9 expressed the comparatively general “different algorithms” without specifying any particular type or expanding on what content was covered in relation to the algorithms. Based on this comparison, Student 10 would seem to have comprehended the specific information in the lecture to a higher extent than Student 9, with Students 4 and 5 in middle positions.

Likewise, in Lecture 5, where the term “fuzzy logic” was emphasized, several students recorded this key term but to varying degrees. In a similarly vague way to the “different algorithms” report discussed above, three students only included the term “fuzzy logic” in their reports without any other details or expansion. Others reported with more detail:

Student 2 (L1 Swedish): “Fuzzy rules compared to standard rules -fuzzy sets and -Fuzzy stimulus.”

Student 9 (L1 Swedish): “We talked about fuzzy-stimuli and how to calculate different values depending on what kind of statement the exercise was.”

Student 10: (L1 Swedish): “Fuzzy logic not being much different from other logic in a purely mathematical view. IR Sensors on the epuck robots are bad.”

Students 2, 9, and 10 all included the concept of “fuzzy” in their reports but also provided additional details and context, suggesting that they gained more knowledge from the lecture than those who only reported the term itself. Since the term “fuzzy” is used in all three main ideas expressed by the teacher, the general use and lack of distinction by some students suggests that while they recognize this as an important concept, they are not able to express the main ideas related to the concept with precision.

When viewed from a presumptive perspective of the EMI lecturer, whose main purpose is the transference of knowledge to students through the lecture format, these qualitative examples may be encouraging on one hand and disappointing on the other. Those students who were able to report and provide relevant details related to core concepts and main ideas from the lectures have demonstrated their learning. However, for those who only reported a bare minimum in relation to key terms and who failed to provide more refined demonstrations of their understanding of how the key terms had been used, teachers may be less pleased that their intended learning points were not consistently acknowledged among the student population. In other words, at least in some cases, what was intended to be taught was apparently not learned.

5. Discussion

The data generated by the comparisons of teachers’ intended main ideas and students’ reporting of the received main ideas have been presented above via a quantitative perspective in relation to key word analysis. The ratios of intended-to-received key words address RQ1 which investigates the extent to which the lecturers and students agree on the main ideas from each respective lecture. Statistical analysis in the form of a binomial test of proportions was made to determine any differences between two groups of students who attended the lectures: self-reported L1 Swedish and self-reported Other L1s.

Regarding RQ1, the data suggest that there is a noticeable difference between what the teacher intends to communicate as main ideas and what the students report to recognize as main ideas. In Lectures 1 and 2 (from sociology), students seemed to be more capable of reporting main ideas using the same key words as stated by the teacher than the students in Lectures 3–6 (from robotics) did. The overall ratio of actual to possible key words recorded was around 50% in the former but averaged just under 25% for the latter. These differences could be related to student background knowledge (potentially more familiar with terms and concepts from sociology than with robotics), individual L2 listening proficiency, lecturer style (e.g., Flowerdew and Miller 1995, 1996), lecturer use of English (e.g., Siegel 2020a), and/or ability to express the main ideas in writing (i.e., a student may have understood the main idea but not expressed it in the written report). Still, the findings tentatively indicate that students in EMI lectures may take away only 25–50% of what the teacher considers main ideas and key words. Thus, the data suggest that students are not necessarily learning what teachers intend, an issue that some institutions around the world have begun to address through recommended training for EMI lecturers (e.g., Morell Moll et al. 2020; Ozer 2020).

When it comes to the role the L1 may play in EMI lecture comprehension, there was a clear statistically significant difference between the self-reported L1 Swedish users and the self-reported Other L1 group. That is, the L1 Swedish users, as a group, reported more main idea key words than their counterparts whose L1s were other than Swedish. Interestingly, in the sociology lectures, the Other L1 group typically recorded more key words than the L1 Swedish group, but the reverse was true for the robotics lectures, where the L1 Swedish group registered higher ratios. Given that the sub-group sizes ranged between 3–14 students, these findings are suggestive but far from conclusive. Again, individual student proficiency and previous topic knowledge played a role, and these variables were not controlled for in this study apart from the relevant university enrollment policies. At the same time, the results raise the issue of how much EMI lecturers know about and account for the L1s of their student groups when crafting and delivering lectures (e.g., He and Chiang 2016).

In addition to addressing RQs 1 and 2, the quantitative findings for Lecturer B’s four lectures on robotics are indicative of a potential accent awareness effect. Lecture 6 represents the fourth lecture in a sequence given by Lecturer B and the generally higher results suggest that students may have become familiar with this lecturer’s EMI teaching and lecture style and/or with the course content. This tentative conclusion is supported by the gradual increase in key word reporting from the first lecture (Lecture 3) to the final lecture (Lecture 6) in the sequence; the percentages of reported key words rose from 16% to 21% to 23% to 35% combined. This progression is suggestive of accent familiarity, although this is only speculation, as accent familiarity was not in focus in the study. In theory, the more familiar the listener is with the speaker’s idiosyncratic patterns, the easier aural comprehension is likely to be; thus, the first time one hears an unfamiliar accent, it may be more difficult to understand compared to after attending five previous lectures given by the same speaker. Such a view is supported by research that suggests that a listener has an easier time understanding familiar accents and may struggle with unfamiliar accents and/or those different from their own (e.g., Tauroza and Luk 1997; Ockey and French 2016). Furthermore, background knowledge, motivation, and attention on that day could have led to this gradual increase. Another possibility could be differing difficulties of the lecture content.

Another point to consider within the data set are words that consistently appeared on the students’ reports of main ideas but which were not expressed by the teachers prior to the lectures. In some cases, several students included what they viewed as key words and main ideas but that did not align with the teachers’ intentions. For example, in Lecture 3, 4/14 students used the word “position” and 5/14 wrote “PID controller” (PID stands for “Proportional-Integral-Derivative”, a term used in robotics); however, the instructor included neither in their main idea summary (although PID is included in the Lecture 4 key words; see Table 5). Likewise, in Lecture 6, “algorithms” appeared in 5/10 reports and “Depth-first search” (or the abbreviation “DFS”) was written on 6/10 despite these key words not being used in the lecturer’s own summaries. The relatively consistent appearance of such words in the student reports but not in the teacher’s main idea summaries suggests that students may be recognizing different items of information as primary even though these may not be outcomes necessarily intended by teachers.

Limitations

At a broad level, EMI lecture listening is not a wholly isolated part of the EMI experience. Learning in EMI is a broad concept that includes a student’s background knowledge of the subject, motivation for learning on the course, and general orientation towards academic success. Reading course literature also contributes greatly to the learning on such courses. This present study has deemphasized these aspects of EMI learning in order to focus on aural recognition of main ideas in lectures, which itself is part of the broader concept of education via EMI. In addition, the data rely on student reports of the main ideas from lectures. It is possible that students actually had a better understanding of the main ideas than is reflected in their written reports. Moreover, on the student survey, participants were given a broad prompt to write between 3–5 main points, which may have led to students possibly combining or amalgamating discreet ideas. Further drawbacks include the small number of participants (both lecturers and students), the imbalance in the data sets (i.e., two and four lectures, respectively), and the loose key-word analysis method.

To expand on the findings reported in this paper, future research may apply a similarly simple design of asking teachers about intended learning and comparing these with post-instruction student reports with more students and teachers, across a wider array of disciplines, and in other national and educational contexts. This study has provided a brief insight into students’ reported learning but does not go beyond these comparisons to examine the teaching that took place and led students to these conclusions. Therefore, examination of data such as that presented here could take place in relation to transcripts and/or recordings of lectures so that the relation between EMI lecturers’ output (e.g., repetition, signposting, sentence structure, intonation, gestures, pausing, etc.) might lead students to report or overlook certain main ideas.

6. Implications and Conclusions

This study set out to investigate a fundamental relationship between teaching and learning: that students leave a lecture with the same main ideas that the lecturer intends to deliver. The complexity of this relationship likely increases when teachers and students are operating in higher education EMI contexts. To examine this complexity, a comparison was made between the main ideas expressed by two EMI lecturers in Sweden and the main ideas their students reported learning. Results showed that students were only able to report corresponding key words to relatively low levels overall. When students’ self-reported L1s were factored into the analysis, a significant difference was identified between the L1 Swedish group and the Other L1s group; at the same time, these groups, especially the latter, were quite small and thus no generalizations can be made based on these data. On the whole, these results may suggest that student learning in EMI be more closely monitored, perhaps on the individual lecture basis, in order to ensure that students are learning the intended material.

Since EMI involves a wide range of language skills and proficiencies in these respective skills, lecturers in such contexts may want to include low-stakes lecture follow-up activities similar to that used in the data collection for this study. Doing so would allow lecturers to: (a) confirm that students learned and can report the main ideas; (b) determine what material needs to be reviewed in subsequent lectures; (c) assign relevant readings to reinforce the concepts in another mode (i.e., rather than only through English speaking and listening). In other words, teachers would get valuable feedback that would enhance future teaching. Students could also receive formative feedback on their summaries. Teachers may also consider periodically recording and analyzing their spoken output in EMI lectures so they can monitor linguistic aspects such as articulation, rate of speech, lexical selection, and grammatical formulations (e.g., Siegel 2020a). By doing so, they can raise their awareness of their own output and make modifications to support student learning (e.g., simplifying and/or repeating ideas, clearly signaling primary versus less relevant information, etc.). Institutions can also support EMI teachers and students by providing systematic and structured pre- and in-service training and support for EMI teachers in relation to linguistic and pedagogic aspects with an emphasis on teaching students who are learning in an L2, as promoted by Ozer (2020) and Morell Moll et al. (2020). The findings from this small-scale study confirm that more attention needs to be paid to the quality of teaching and learning happening “in the EMI trenches” and not only at broader national and institutional policy levels.