Establishing Social Learning in an Engineering MOOC: Benefits for Diversity and Inclusion in Engineering Education

Abstract

Recent Higher Education Statistics Agency data shows that only 20% of engineering students at UK Universities are female, despite the hard work being undertaken by many educational institutions to address this gender imbalance via outreach events and special interventions focussing on girls/women in STEM. It has been argued that student-centred teaching methods, together with changes in the engineering curriculum itself, which emphasise the social, creative, and human-centred aspects of the discipline, are required to effect real change in engaging with those from traditionally underrepresented groups. Through analysing quantitative data on age, gender, learner type, and commenting rates in peer-to-peer discussions, we examine the development and delivery of an engineering MOOC, before, during, and after COVID-19-related lockdowns in the UK, to identify what aspects of online learning might be harnessed to improve diversity in engineering education. The results show that the MOOC attracted a better gender balance than reported for UK-based in-person engineering programmes. In addition, we show that careful structuring of discussion prompts encouraged higher levels of social learning. We recommend the continued use of interactive and discursive elements within a blended learning environment to positively impact diversity and inclusion in engineering education specifically, and STEM education in general.

1. Introduction

It is well known that engineering, both as a study discipline and as a profession, lacks diversity. This is most obvious with regards to gender diversity; it is a heavily male-dominated field, reflected in both the numbers of and ratio of women studying and working in engineering- and technology-related fields. For example, in the academic year 2018–19 women made up only 20.7% of students entering higher education (HE) courses in engineering and technology in the UK (even though women make up more than 57% of HE student population) [1]. Globally, only 30% of those enrolled in higher education courses in engineering, manufacturing, and construction are female; enrollment rates differ by county, but the European average is between 25–29% female participation in engineering-related degree programmes [2].

A similar ratio can be seen in the wider workforce. In 2017, women accounted for approximately 47% of the overall UK workforce, but only 11% of those working in engineering and related occupations are women [3]. In the same year in the UK, 12.7% of the workforce in non-engineering professions were from ethnic minority groups, in contrast to just 8.1% of those employed in engineering and related occupations [4].

If engineering fails to attract a broad diversity of entrants, the problem is not only that untapped reserves of human creative potential are being squandered, but that the discipline itself suffers harm due to the risk that innovative and effective solutions to societal problems will not be found [5].

Researchers and engineering educators have posited a variety of reasons for this apparent lack of diversity in engineering which include [6,7,8,9,10]:

• the origins of engineering rooted in the history of the industrial revolution;
• a lack of diverse role models both in engineering education and industry;
• preconceptions of engineering as a discipline that requires physical strength; and
• a stereotypical image of engineering as a subject only for the “brainy”;

Of particular interest, with relation to the latter two bullet points above, which are based on unhelpful misunderstandings of engineering, is the contrast between public perceptions of engineers and public attitudes to engineering as a profession, as reflected in a large-scale survey conducted in 2007. On the one hand, the general public seems to have little awareness or understanding of engineering, with narrow definitions being limited to “construction and manual professions… building and fixing things rather than design, innovation or creativity” [7]. On the other hand, despite the lack of general understanding of what engineering is and what engineers do, engineering was viewed as an important profession that dealt with issues affecting society and made a good contribution to people’s lives [7].

Furthermore there remains a somewhat stubborn perception of engineering as being for “the man who is ‘in love’ with technology but rather socially withdrawn if not socially inept” [6]; a stereotypical view of engineering which portrays engineers as “anti-social”, concerned only with machines, engines, and structures; spending their time working alone on complex calculations that only those with highly developed, almost genius-like levels of mathematical ability are able to understand. Such a damaging stereotypical image plays out in direct contrast to the socially accepted gender norms for women and girls as being social, collaborative, and caring, and (entirely false) perceptions of women having lower levels of ability in math and science.

Faulkner [6] posits that the arguably artificial dichotomy between “the technical” and “the social” “by which men/masculinities are so readily associated (symbolically) with technology and women/femininities with people” is, in fact, itself a major barrier to inclusion in engineering, as this contrast leads to the perception of engineering as “gender-inauthentic” for women, whereas it is a readily accepted gender norm for men.

Gender authenticity for women is reflected in the subjects that female students tend to choose to study at A Level or Level 3 (qualifications taken generally by 18-year-old pupils in the UK prior to further study at college or university) [11,12]. In the UK in 2017, science, technology engineering, and mathematics (STEM) subjects made up only 27% of girls’ A level choices, in contrast to 46% for boys [4].

Although there is still much debate around gender and sex differences in ability and achievement in math and science subjects at school and in further/higher education (for example, in [13,14]), it is clear from a number of studies that STEM subjects are perceived as predominantly masculine [9,11,15].

This gendered perception appears to feed into girls’ future career choices, with just 34% of 7–11-year-old girls surveyed by Engineering UK reporting that they would “like to be an engineer” in comparison to 59% of boys. Perhaps more alarmingly, the percentage of girls considering a career in engineering drops to just 25% by the age of 16–19; a crucial time point in terms of choosing next study steps which lead to future careers [4].

It also seems that women and girls are making study and career choices while not being aware of the diversity of careers in engineering [16,17,18], and the aspects of engineering that are social, creative, and human-centred, and therefore potentially more fitting to their gender-authentic choices, are not always salient.

Clearly, that barriers to inclusion in engineering and the causes of underrepresentation are numerous, systematic, and embedded from a young age. Moreover, despite various efforts to address the imbalance over a number of years, there does not seem to be much evidence of change. For example, the proportion of students taking A level physics who are female is around 20% and has been around this figure for the last 30 years [19]. Of course, the lack of women choosing careers in engineering (or STEM more generally) has a self-sustaining effect, meaning that there are then fewer role models in the industry to encourage and inspire the next generation—a situation that some predict will not change without radical reform in education [20,21,22].

A number of reforms to engineering education have been posited with the aim of increasing diversity in engineering; we consider just three of them here.

First, the need for engineering teaching and learning activities to emphasise that engineering encompasses technical, social, and creative aspects [8,10,23]. Indeed, the recently introduced UK Design and Technology A-level has shifted the focus away from engineering science (maths and physics) toward design engineering/applied engineering, and currently attracts 30% female students [24].

Secondly, the need to make the public more aware of the societal contribution that engineering can make to world challenges, which forms a key recommendation in the Lloyd’s Register Foundation/UCL report on preparing engineers for the 21st century [25].

A final set of reforms focuses on the presumed potential of online learning to widen access for traditionally underrepresented groups, partially through its ability to “democratise participation” [26]. However, the evidence for better diversity in terms of gender for online learning courses is not clear cut. A study of massive open online courses (MOOCs) at Delft University of Technology found much lower registration rates and participation by female students in technical online courses [27]. Nevertheless, Jiang et al. [28] found that, even though female students were less likely to enrol, they were just as likely to complete online STEM courses. In fact, Crues et al. [29] also suggest that despite lower enrollment rates in a computer science MOOC, female students’ participation in posting to the forum improved completion rates.

Similarly, looking particularly at online learning communities, Rovai [30] found that female students in an asynchronous online course had a more positive perception of the online classroom community than the male students. Consequently, there is some emerging evidence that participation in social online learning has the potential to improve gender balance in engineering education. Of particular interest also is the finding that female participation in an engineering MOOC hosted by Delft University of Technology increased by 3.4% during COVID-19 lockdown [31].

The aim of this paper is to examine the development and delivery of an engineering MOOC, before, during, and after COVID-19 lockdowns, and to identify what aspects of online learning might be harnessed to improve diversity in engineering education for the future. In particular, two main research questions are investigated. First, did the MOOC attract a diversity of learners? Secondly, how engaged were the learners in the social learning aspects of the online course?

It then goes on to highlight some lessons learned from facilitating the MOOC at four different time points: one before, two during, and one after the UK COVID-19 lockdown restrictions. Better understanding of learners’ needs during a time of major upheaval and rapid “pivoting” to online teaching and learning in the UK has the potential to positively impact future “blended learning” in engineering education in HE and more widely. In particular, examining four different instances of the course allows a comparison of learner participation during and outside of lockdown restrictions, as well as evaluating the impact of modifications made to the course to encourage greater engagement with social learning.

2. Materials

2.1. Engineering MOOC

In 2017, the University of York established its first MOOC, a course on organic chemistry (Exploring Everyday Chemistry) hosted on FutureLearn [32,33]. Since then, the university has expanded its provision and now runs over 20 MOOCs which aim to increase awareness of University of York study programmes.

The engineering MOOC was developed in 2019 by staff in the Department of Electronic Engineering and ran on four separate occasions: October 2019, April 2020, June 2020, and June 2021. The three-week-long course was designed to provide on average around three to four hours per week of learning, including videos to watch, articles to read, and practical tasks. Each week was structured as three main learning “activities”, with each activity consisting of between five and seven learning “steps”.

On each step, learners are encouraged to engage in discussions with fellow learners via the comment function below the main text content of each learning step’s web page. Learners are guided through the course with a weekly “to-do” list, and are encouraged to mark each step complete (through clicking a tick box on the page) as relevant, so that progress can be tracked. Marking steps as complete also enables the learner to pick up where they left off if they are engaging with learning at various points throughout the week.

The overall aims of the Engineering MOOC were to:

• engage with school students and recent school leavers who might consider studying engineering at university;
• ensure participation from a wider diversity of learners than those typically found on engineering higher or further education study programmes in the UK; and
• counteract the stereotypical view of engineering as focussing on machines and technology rather than people, and that engineers are introverted and antisocial.

We addressed these aims in terms of curriculum design by means of a broadly constructivist approach [34] by including real-world applications (as outlined in Figure 1) as recommended by Margaryan, Bianco, and Littlejohn [35]. Learning steps were built around a framework of engineering design and state-of-the-art technology in a visually attractive way (including the use of high-quality images), designed to appeal specifically to young learners who might be considering university study and career options. We aimed to not only include technical and scientific knowledge of how engineering systems are built and operated, but also to highlight the human-centred nature and day-to-day relevance of engineering applications.

We attempted to link seemingly disparate sub-disciplines of engineering via a systems-engineering approach to creating new technology: considering sensors, signals and systems. All modern engineering systems interface with the external world via sensors, sensors collect data in the form of continuous signals, and signals captured by the sensors must be analysed, filtered, categorized, and made useful for the human user by the engineering system.

This framework led us to organise the weekly learning steps not around engineering sub-disciplines but rather around application areas, while making sure that the links between the technical fundamentals of engineering, which we flagged to learners as drawing on their knowledge of science and math, and engineering applications were clearly articulated as recommended by Busch-Vishniac and Jarosz [20]. We achieved this by highlighting current research and development at the University of York and further afield, as well as encouraging learners to share their own experiences and knowledge of real-world engineering.

We also included positive images of modern engineering, to attract students from a wide range of backgrounds, as well as emphasising the societal impact of engineering, as recommended by the UCL Centre of Engineering Education 2018 report on engineering education innovations [25].

Another key consideration of the learning design was to engage learners in online social collaborative learning; this reflects the design ethos of all FutureLearn courses which aim to “create a community of FutureLearners who share ideas, hold engaging discussions, and support each other” [36]. As educators, the MOOC design team were very much aware of the responsibility on us to build a course structure and content, and to fully engage with our learners in order to exploit the huge potential of the online platform to support collaborative learning [30].

2.2. Data Sources

In order to discover whether the aims of the engineering MOOC were achieved, we collected and analysed various types of learner data to allow us to address the following research questions:

• Research Question A: Did the MOOC attract a diversity of learners?
• Research Question B: Did our learners engage in social collaborative learning?

We collected learner demographic and course engagement data, using the statistics dashboard provided by FutureLearn for each of four course runs with the following start dates:

• 7 October 2019
• 27 April 2020
• 29 June 2020
• 21 June 2021.

MOOC participants are able to enrol at any point after the course is made available for registration, i.e., before the facilitated/supported run takes place. FutureLearn makes a distinction between those participants who enrol but do not go on to engage with the course (“joiners”) and those who view one or more course step (“learners”). Participants are also able to enrol once the run has started at any point until the run is withdrawn. For this reason, in order to capture all learners who were potentially participating in a specific course run, we included data from learners joining the course from the start date of the course run, throughout the three-week duration of the run, plus another two weeks. This chosen time period allows comparison with FutureLearn-published data on specific course runs.

It should be noted that the statistical data provided by FutureLearn encompasses users of any “role” such as course participants, educators, and course administrators. We therefore further processed the data further in order to remove MOOC team members and administrators from these data categories.

3. Research Question A: Diversity of Learners

3.1. Methods

All learners who enrol for a FutureLearn course are requested to fill in a learner survey which collects, among other data, age range and gender, as well as answers to broader questions about learner motivations for signing up for the course, and what they hoped to achieve. Responses to the latter questions are collated to evaluate a Learner Archetype [37], briefly summarised as follows:

• Advancers—already working, hoping to advance their career through learning;
• Explorers—may be looking for a career change, or making choices about their chosen career path;
• Preparers—starting their job, career, or study;
• Fixers—those undertaking learning to better understand an aspect of their personal life;
• Flourishers—using self-help to enhance their personal or professional life;
• Hobbyists—learning to support a current hobby or past-time;
• Vitalisers—learning as a hobby, lifelong learners learning for personal interest; and
• Other.

However, it is necessary to note that the survey is not compulsory, and learners are sent only one email prior to the course run requesting completion; as such, completion rates are low. Indeed, only 12.6% of learners who went on to engage in the course actually answered any survey question across the four course runs, which meant that we were unable to make any meaningful course-by-course comparisons in respect to age, gender, or learner archetypes. Instead, learner data is totalled across the four MOOC course runs, giving a total of just over 300 data points.

3.2. Results

3.2.1. Age

The survey question on age was answered by 297 learners and the breakdown of responses is shown in

Table 1. Age ranges of learners on the four engineering MOOCs, and the reported FutureLearn average age profiles, where # indicates number and % indicates percentage.

Age Range	Engineering MOOCs		FutureLearn Average
	# Learners	% Learners
<18	33	11%	1%
18–25	130	44%	18%
26–35	57	19%	27%
36–45	36	12%	16%
46–55	17	6%	10%
56–65	8	3%	7%
>65	16	5%	6%

. The majority of learners on the engineering MOOC runs were in the 18–25 years age range (44%); the least well-represented age range was 56–65 years. A total of 11% of our learners were under the age of 18, a much higher percentage than the 1% calculated across a random sample of all FutureLearn enrolments in the last 6 months (only visible to FutureLearn Partners).

3.2.2. Gender

The survey question on gender was answered by 307 learners, and the breakdown of responses is shown in

Table 2. Number (#) and percentage (%) of engineering MOOC learners by gender.

Gender	Engineering MOOCs
	# Learners	% Learners
Female	117	38%
Male	187	61%
Nonbinary	3	1%

. A total of 38% of learners across the four runs were female, 61% male, and 1% nonbinary. FutureLearn does not make average gender data available, so we are unable to include it for comparison.

3.2.3. Learner Archetype

The survey questions relating to learner archetype were answered by 217 learners, and the breakdown of responses is shown in

Table 3. Number (#) and percentage (%) of engineering MOOC learners by learner archetype.

Learner Archetype	Engineering MOOCs
	# Learners	% Learners
Preparers	19	9%
Vitalisers	39	18%
Hobbyists	19	9%
Explorers	89	41%
Fixers	3	1%
Advancers	34	16%
Flourishers	6	3%
Other	8	4%

. The largest group of learners (41%) fitted the Explorers archetype, followed by Vitalisers (18%) and Advancers (16%). Again, FutureLearn does not make average archetype data available, but it could be argued that this would depend heavily on the course content and so would not be a useful comparison.

3.3. Discussion

Our first research question focuses on the diversity of MOOC learners, both in terms of age and gender. One of the main aims of the engineering MOOC was to attract school students and recent school leavers who might be interested in studying engineering further. Considering the data on learner age ranges and learner archetypes, it appears that the MOOC was successful in this aspect.

The MOOCs were successful in attracting a good proportion of learners aged under 18, and fulfilled the aim of engaging with school pupils who were at the point of making choices about furthering their education at university or college. The learners on the engineering MOOCs were predominantly younger than the average for FutureLearn courses, with over 50% of learners under the age of 25, whereas the FutureLearn average for this age group is just 27%.

Explorers (41%) made up the largest proportion of learner archetypes. Explorers are defined by FutureLearn as those who are “evaluating their options and want to inform their decisions about what to do next.” [38], which reflects the target group of the MOOC. This is particularly interesting, because FutureLearn research found that Explorers are more likely to be in the 26–35 age group [38], whereas we see the largest proportion of Explorers in the <18 age group, and equal proportion in 18–25 and 26–35 age groups (Figure 2).

It is worth comparing learner data between course runs that took place during UK COVID-19 lockdown periods (April 2020 and June 2020), and those which happened before (October 2019) and after restrictions had been eased, when most UK-based schools and pre-18 years educational settings had reopened for in-person teaching (June 2021).

We saw a large increase in learner numbers between October 2019 (pre-COVID) and April 2020 (first run during UK COVID-19 lockdown), a pattern which was reflected more widely across the University of York MOOC offering. For example, the Department of Chemistry MOOC saw a 132% increase [39], and the Department of Language and Linguistic Science MOOC on sociolinguistics saw a 53% increase in learners between summer 2019 and summer 2020 courses (C Childs 2021, personal communication, 27 August 2021). In addition, there was a marked change in the age profile (Figure 3) and learner archetype (Figure 4) breakdown of learners when comparing MOOC course runs outside of COVID-19 lockdown with those during COVID-19 lockdown periods.

The large increase in the 18–25 age group, and increase in the Explorer archetype participating in the April and June 2020 course runs is most probably due to the closure of schools, and the cessation of sports and social activities whereat this age group might otherwise have spent its time.

A further aim of the engineering MOOC was to encourage participation from a wider diversity of learners than those typically found in engineering higher or further education study programmes in the UK.

It does appear that emphasising the social, creative and human-centred aspects of engineering has attracted a better gender balance of learners, because 38% of the engineering MOOC learners were female. Our MOOC also appeared to attract a higher proportion of female students than has been reported for STEM-focussed MOOCs. For example, Jiang et al., [28] report 24% female participants out of 224,318 worldwide learners; similarly, Crues et al. [29] report 23% female learners on a computer science MOOC and Rayyan et al. [40] saw only 16% female learners on a physics MOOC offered by Michigan Institute of Technology (MIT).

It is more difficult to make a clear-cut comparison with the proportion of female students enrolled in engineering and technology degrees in the UK in the academic year 2020–21, which stands at just 20% [41], because there is some suggestion that female students are more likely to complete surveys than male students (e.g., [42,43], meaning that responses to the non-compulsory learner survey may, in fact, overstate the true proportion of female learners participating in the MOOC. However, Crawford, Couper, and Lamias [44] found no statistically significant difference in response rates by gender, and other research suggests that men are more likely than women to complete web-based surveys [45]. By using response rate data from Lefever, Dal, and Matthíasdóttir [42] and Hutchison, Tollefson, and Wigington [43], studies which both relate to educational settings, we estimate that women might be 1.5 times more likely to complete the MOOC survey than men; if this is correct, then the true proportion of female learners on our engineering MOOC would be 23%, which still compares favourably to the 18% female new entrants to UK engineering undergraduate degree courses [41].

4. Research Question B: Social Learning

4.1. Methods

4.1.1. Categories of Learners

To address our second research question around social collaborative learning, we collected overall numbers of learners for each of the four MOOC runs and calculated percentages for different categories of learners. We used the FutureLearn categories of Learner, Active Learner, and Social Learner with the following definitions:

• Joiner: Any person who has enrolled for the course-run;
• Learner: A user who has viewed one or more steps;
• Active Learner: A user who has completed (marked complete) one or more steps; and
• Social Learner: A user who has commented on one or more steps.

It should be noted that Active Learners and Social Learners do not constitute the same group, because it is possible to complete a course step without commenting on it, and similarly it is possible to comment on a step without marking it as complete.

Because learning steps 1–3 are flagged as points where learners should “introduce themselves” by posting a comment, Social Learners who only comment on these first three learning steps we categorise as Introducers.

We also added an extra category of Engaged Social Learner, which we define as those learners who made comments on or after the fourth learning step. Because the fourth learning step is the first time that learners are requested to discuss the course content covered, Engaged Social Learner refines the category of Social Learner by excluding Introducers i.e., those who only make a comment in the introductory steps and then do not comment again. For ease of reference our Learner categories are summarised in

Table 4. Overview of learner categories and quoted percentages.

Category	Definition	Presented Percentages
Joiner	Any person who has enrolled for the course run	n/a
Learners	Users who have viewed at least one step	Percentage of Joiners
Active Learners	Learners who have completed at least one step	Percentage of Learners
Social Learners	Learners who have posted at least one comment on any step	Percentage of Learners
Introducers	Learners who made comments only in the introductory steps (1–3)	Percentage of Learners
Engaged Social Learners	Learners who made comments on or after the fourth learning step	Percentage of Learners

.

4.1.2. Number of Comments

In addition to the above categories, we also collected and analysed descriptive statistical data on commenting behaviour of Learners as follows:

• Number of comments overall;
• Number of comments for each week of the course;
• Number of comments for each learning step;
• Comments per step per Learner (referred to as “comment ratio”);
• Mean comment ratio for each week of the course;
• Mean comment ratio for each course run;
• Mean comments per step within one of three engineering application areas; and
• Number of comments with replies.

It is worth noting that the MOOC consists of a number of learning steps per week, over the three week course, but learners are not expected to make comments on every learning step each week.

Unlike the data on age, gender and learner archetype, the overall data set of numbers of learners and comments was much larger: over 2400 Learners, of whom 720 made comments, with over 4300 comments in total made on the four course runs.

4.1.3. Changes to Comment Prompts

An initial analysis in 2020 of the numbers and percentages of learners participating in the October 2019 course run revealed that only 65 out of 358 Learners (18%) were Social Learners, posting comments and discussing with their fellow learners; much lower than the reported FutureLearn average of 49% [46].

We also noted that in the April 2020 course run (the first MOOC run that occurred during COVID-19 lockdown restrictions), despite a large increase in the number of learners registering for, and participating in the course, and the number of comments increasing on many of the learning steps, and the percentage of Social Learners was still below the FutureLearn average (30%).

Between the April 2020 and June 2020 course runs, the MOOC lead educators considered the learning steps that seem to attract few learner comments and updated them. In considering the textual content of some of the less well-commented steps, it was noted that it was often unclear for learners that comments were explicitly requested. Although the comment facility is available at the end of all learning steps, learners seemed to be more likely to comment where there was some type of commenting prompt at the end of the written text.

The commenting prompts on a number of learning steps were updated prior to the June 2020 run so that every substantive learning step, where the team wanted Learners to comment, had a very clearly labelled commenting prompt section titled “Over To You”, as shown in Figure 5.

Table 5. Examples of discussion prompts that were re-written or added between April 2020 and June 2020 course runs, where # indicates “the number of” and bold formatting indicates the heading of the prompt task.

Step	Step Title	# Comments April 2020	Commenting Prompt Amendment	# Comments June 2020
1.7	Sensors used in medicine now	0	Added comment prompt heading “What about you?”	68
1.4	Welcome to Week 1: engineering for health	18	Rewrote last paragraph: “Have your say Put a comment here if you use some form of technology to track your health (smartwatch, fitbit, clever shoes!). Tell us what it is, what you use it for, and how it has helped you.”	122
2.3	Health and water quality	11	No comment prompt in April 2020—we added: “Exercise: How would you summarise the above article in less than 100 words? Engineers are often asked to read and understand complex situations and technologies, then explain them briefly to different people. Have a go at writing a summary, and post your replies below.”	49
1.11	Healthcare: from sensors to signals	6	We added: “A Thought Experiment for You Imagine you met a friendly alien from outer space, who understood written English (as they do in many science fiction films). They point to your ears and type “What are those for?” You notice they have no ears and probably are not aware of sound. How would you describe what sound is to a being from a place where sound is not heard?”	62

presents examples of how the team amended and added commenting prompts to the steps which received low numbers of comments in the April 2020 run.

Building on previous research around this aspect of online learning design (e.g., [26,47,48,49]) four main principles for restructuring the discussion prompts were followed by the team:

• Making it “obvious”;
• Using open-ended questions;
• Encouraging learners to use their imagination; and
• Using authentic or real-world tasks to put the discussion into a professional context.

These principles are outlined in more detail in Section 6 (Conclusion, recommendation 1).

4.2. Results: Categories of Learners

Potential learners can sign up for FutureLearn MOOCs ahead of the course run, and a proportion of those who enrol for the course (referred to as Joiners)—on average 50%—do not, in fact, participate [46].

The data on numbers of Joiners and Learners for the four Engineering MOOC runs are seen in

Table 6. Numbers of joiners and learners for the 4 course runs.

	October 2019	April 2020	June 2020	June 2021
Joiners	554	1144	1569	726
Learners	358	833	840	432

. The percentages of Learners, Active Learners, Social Learners and Engaged Social Learners for the four Engineering MOOC runs are plotted in Figure 6, alongside comparison data of FutureLearn averages from Jenner [46]. It is worth noting that the comparison data is taken from a survey of 1222 FutureLearn MOOCs in 2016 and 2017.

4.3. Results: Comments and Learning Steps

4.3.1. Comments Per Week and Per Step

We calculated the number of comments posted in each week (

Table 7. Number of comments per week, number of Learners and number of Social Learners for each course run.

	# Comments: Week 1	# Comments: Week 2	# Comments: Week 3	# Learners	# Social Learners
October 2019	150	84	62	358	65
April 2020	519	344	248	833	253
June 2020	752	432	308	840	284
June 2021	344	150	75	432	118

) and the average (mean) number of comments per step (

Table 8. Mean comments per learning step across each course run (excluding steps 1.1 to 1.3; the “introduce yourself” steps).

	Average Comments Per Step	% Change from Previous Run
October 2019	4.63	-
April 2020	17.09	270%
June 2020	22.61	32%
June 2021	9.03	−60%

), for each course run. Both of these metrics varied considerably across the course runs, with increases between October 2019 and April 2020, followed by another smaller increase in June 2020, before a fall in June 2021.

This variation can be explained in part by the different number of Learners and Social Learners participating on the individual course runs, as shown in Figure 6.

4.3.2. Comments Per Step Per Learner (Comment Ratio)

Because the number of learners on each course run varied considerably, in order to investigate differences between the individual course runs a “comment ratio” was calculated for each step. The number of comments on each step divided by the number of Learners on the course run overall (Figure 7, Figure 8 and Figure 9). In this way, a comment ratio of 0.10 indicates that the number of comments on this step equates to 10% of the number of Learners. It should be noted that Learners are able to make more than one comment, so the comment ratio value does not indicate directly the number of Learners who made a comment on that particular step. We also calculated the mean comment ratio for each week, split by course run (Figure 10) and for each course run overall (

Table 9. Mean comment ratio for each MOOC course run, percentage change from previous course run and number of learners per comment.

	Mean Comment Ratio	% Change from Previous Run	Number of Learners “Required” for 1 Comment
October 2019	0.0125	-	79.82
April 2020	0.0202	61%	49.49
June 2020	0.0269	33%	37.16
June 2021	0.0200	−26%	50.11

).

4.3.3. Mean Comment Ratio within Engineering Application Area

Learning steps in the MOOC focussed on three main sub-topics which relate to different engineering applications: medical engineering, audio engineering, and robotics.

In order to establish whether some sub-topics might be more attractive to commenting than others we calculated the mean comment ratio for the steps each sub-topic (excluding introductory steps, weekly round up and summary steps), normalised for each course week (because commenting activity dropped away overall by Week 3 and not all sub-topics appear in all weeks). These are shown in Figure 11 and listed in

Table 10. Mean comment ratio according to three sub-topics: medical engineering, audio engineering, and robotic engineering.

Topic	October 2019	April 2020	June 2020	June 2020	Average of the 4 Course Runs
Medical	0.0133	0.0230	0.0301	0.0261	0.0231
Sound	0.0156	0.0247	0.0302	0.0215	0.0230
Robotics	0.0056	0.0103	0.0164	0.0091	0.0103

.

4.3.4. Comments and Replies

In order to further investigate whether learners were engaging in conversations, which could be an indicator of building an online social community of learners, the number of comments which are replies to other comments were counted, and expressed as a percentage of the total number of comments on the course run (

Table 11. Number of comments (per course run) that are replies to comments, expressed as a percentage of total comments on the course run.

	Replies	% of Comments on Course-Run
October 2019	27	7.28%
April 2020	102	7.26%
June 2020	108	5.84%
June 2021	45	6.47%

).

4.4. Discussion

4.4.1. Overall Numbers of Comments

One of the clearest characteristics of commenting behaviour that can be seen from the data is the drop-off of most commenting measures across the three-week period of each run. The overall lower levels of comment ratio in Week 3 (ranging from 0 to 0.336) in comparison with Week 1 (ranging from 0 to 0.995) reflects other patterns of learner participation found generally in online courses [50,51,52].

It is also worth noting that we found that a not-insubstantial percentage of those who made comments (Social Learners) did not in fact comment on the substantive learning steps, but only commented only at the beginning of the course to introduce themselves. These “Introducers” accounted for 10% of Social Learners on average across the four course runs; we recommend that such learners are identified in future analyses of social learning in MOOCs.

Figure 7, Figure 8 and Figure 9 show that, although comment ratio varies across steps of the course, the majority of steps saw an increase in comment ratio between October 2019 and April 2020 and a further increase in commenting ratio between June and April 2020. However, the majority of steps saw a decreased comment ratio in June 2021 in comparison to June 2020, but still, for the most part, a larger comment ratio than April 2019. This is also reflected in the mean comment ratio values for weeks of the course and for each course.

Although the first increase could be attributed to greater numbers of learners on the course, and lockdown restrictions meaning learners had more time at home and online, the further increase between the two courses in lockdown (April and June 2020) suggests that the restructuring and rewriting of discussion prompts positively affected the rate of commenting. Similar increases are seen across most of the measures we analysed, and as such the impact of COVID-19 lockdown restrictions is considered in more detail here.

4.4.2. Commenting before and during COVID-19 Lockdown Restrictions

All of the commenting and learner data shows an increase between the October 2019 (pre-COVID-19 restrictions) and April 2020 (during COVID-19 restrictions) course runs:

• 375% increase in total comments;
• 232% increase in learners;
• 389% increase in social learners;
• 63% increase in engaged social learners;
• 270% increase in average number of comments per step; and
• 61% increase in average comments per step per learner.

We can easily attribute the introduction of COVID-19 restrictions in the UK (and most other countries where we had online learners) as the main driving force for this increase in Social Learners and commenting behaviour. Most usual social activities were cancelled during this time and schools and colleges shut down; young adults in particular increased their use of social media and online activities [53]. During lockdown in 2020, two-thirds of our MOOC learners were under the age of 25, and as such it is highly probable that the increase in learners’ commenting activity was in part fuelled by the increase in online activities seen during lockdown restrictions.

4.4.3. Commenting after the Restructure of Discussion Prompts

We saw only a small increase in the number of learners between the first MOOC course run during lockdown (April 2020) and the second run (still under lockdown, June 2020): just 0.84%.

Nevertheless, the proportion of social learners and engaged social learners actually increased again by 4% on both measures between these two course runs, and we also saw increases in commenting behaviour in June 2020, including:

• 34% increase in total comments;
• 32% increase in average number of comments per step; and
• 33% increase in average comments per step per learner.

These increases were likely a result of the extensive redesign and re-writing of the discussion prompts as outlined in Section 4.1.3.

It is interesting to consider the content of the learning steps which garnered the highest and lowest comment ratios. Although a full analysis of all steps is beyond the scope of this paper,

Table 12. The learning steps with the lowest average normalised comment ratio across all runs.

Learning Step	Description of Learning Activity or Task
3.1	No task prompt section Welcome video
2.8	No task prompt section Introduction to set of steps
1.19	No task prompt section Information giving (via text or video)
1.17
2.21
3.18
1.22	Task prompt section present Activity is a thought experiment and asks for thoughts
2.13	Task prompt section present Activity requires trying out an activity using a mobile app, posting the results to a Padlet, then commenting
3.8
1.20	Task prompt section present Activity requires doing an experiment and coming back to report on results

lists the steps with the lowest average normalised comment ratio across all runs, and

Table 13. The learning steps with the highest average normalised comment ratio across all runs.

Learning Step	Description of Learning Activity or Task
2.09	No specific task prompt, but requests learner to think and give examples
2.02	Task prompt: post short answer in comment, post image on Padlet
2.10
1.06	Task prompt: think and give examples
3.09
3.24
1.08
1.12
3.05
3.12	Task prompt: respond to audio clips with thoughts

lists the steps with the highest average normalised comment ratio across all runs; a brief description of the learning activity or task is also included.

It can be seen that the majority of the learning steps with the highest comment ratios prompt learners to think about what they have read (or listened to) and give examples of something in the comment section of the page. Two of the most commented learning steps suggested learners post an image on the Padlet board, but also post a short comment on the page.

However, when learners are asked to undertake an activity by using a resource external to the FutureLearn platform (an app or other website) and then report results on the external Padlet board and then navigate back to the MOOC site to comment, it is not surprising that commenting ratios are low for these steps. This suggests that avoiding the requirement for learners to switch their attention from the learning platform will ensure better engagement with the social learning aspects of online learning.

4.4.4. Commenting after Lockdown Restrictions Eased

The proportions of social learners and engaged social learners dropped in June 2021, but continued to be higher than the first course run in October 2019. Similarly, the majority of steps saw a decreased comment ratio in June 2021 in comparison to June 2020, but still on the most part larger than in October 2019.

We suggest that this decreased activity between 2021 and 2020 relates to the easing of lockdown restrictions and the reopening of other (offline) social activities that our learners could engage with. Nevertheless, it is pleasing to see that some of the increase in social learning and commenting behaviour was retained and did not sink back to the pre-lockdown levels This data adds weight to our suggestion that the increase in commenting seen previously was be attributed not only to decreased possibility for social interaction during COVID-19 lockdown, but also the rewriting of the discussion prompts within the course content.

4.4.5. Commenting According to Engineering Application Area

As seen in Figure 11 and Table 10, the steps related to the sub-topics of medical engineering and audio engineering garnered more comments than robotics, suggesting perhaps that the obvious human-centered and social nature of medical and audio topics increased learner engagement. This finding offers an interesting avenue for further investigation in engineering education.

Each sub-topic shows the same pattern of a large increase in mean comments per step for the first course run during lockdown (April 2020), followed by another (smaller) increase for the June 2020 course run, and then a decrease for the final course run, but only to a higher level than that found in the very first course run (pre-lockdown).

4.4.6. Commenting and Interacting

Although we saw an increase in the overall number of comments and average comments per step (both impacted by COVID-19 lockdown restrictions and the rewriting of discussion prompts) there is, in general, only a small amount of interaction between commenters, as evidenced by the low rates of replying to comments. Where there are a small number of replies posted to a comment by another learner, they are predominantly only single reply comments with no associated thread.

In addition, when learners do post a comment, they are for the most part only posting one single comment per learning step, and there is no substantial change to this demonstrated by the different course runs. There are a small number of instances of learners posting more than one comment per learner per step, and the number of times this occurs does increase slightly during lockdown course runs, but since the mean number of comments per learner per step does not increase accordingly, this indicates that although we have more learners interacting, we also see more people posting only once.

It is disappointing that, despite the increase in social learners and average comments per step after lockdown and after the restructuring of discussion prompts, we did not see more evidence that our learners were actually interacting with each other because the Community of Inquiry (CoI) model [47,54] sees “social presence” as one of three key ingredients needed for a worthwhile educational experience (whether offline or online).

However, it is possible that learners were replying to other learner’s comments but without using the reply function in the comment threads, and as such, it is difficult to establish the level of interaction between learners without some type of textual analysis of the comments posted, such as that undertaken by Rovai [30].

5. Limitations and Future Work

5.1. Age, Gender and Learner Type

Because completion of the learner survey is not compulsory, the demographic dataset available is too small to allow an investigation of the impact of age, gender or learner archetype on social learning behaviours or completion rates. Educational researchers are also aware that female students may be more likely to complete surveys than male students, meaning that self-reported demographic data from learners might not reflect the true make-up of the learner group.

Much fuller learner demographic data, combined with machine-learning techniques to analyse learner analytics (e.g., [55,56]) would enable a better understanding of how we can engage online with a more diverse group of learners. In particular, it would be interesting to discover whether engagement and learner outcomes on the engineering MOOC were impacted by participation in commenting, as found by Crues et al. [29].

5.2. Conversation Length and Commenter Interaction

Future work on analysis of our engineering MOOC data should include more detailed analysis of comments e.g., conversation length [57], proportion of replies, length of messages, or textual analysis of the content of comments [30,58], which will allow us to understand whether a higher comment ratio relates to interaction, rather than simply multiple comment posting, and to evaluate social presence and its impact on completion rates and the educational experience more widely. A deeper textual analysis of comments would also enable us to understand more about the relative appeal of the three engineering sub-disciplines covered in the course.

5.3. Padlet Engagement

We did not look at engagement in terms of posting to the Padlet boards that were utilised to collect learner input in some steps, as there is no consistent way to track user activity once they have clicked off the FutureLearn site to an external site such as Padlet. However, a similar attrition of engaged learners was clear to see in the contributions to the Padlet boards, with a much lower rate of engagement in the posting of pictures/links, etc. on the Padlet boards later in the course.

6. Conclusions

We have shown that our engineering MOOC attracted a better gender balance than the average (currently 20% female students) for undergraduate engineering and technology degrees in the UK [41]. In addition, we attracted a younger demographic of learners than those usually participating in technical STEM-related online courses [40].

We focussed on the careful design of online learning content and activities in order to help demystify engineering and dispel some unhelpful stereotypes about engineers and what they do, because such stereotypes present barriers to participation in engineering for those from traditionally underrepresented groups [16,17]. We have shown that the inclusion of human-centred engineering, alongside a focus on the societal, social and creative aspects of engineering can be instrumental in attracting a diversity of learners, consistent with findings of others working to in this area (e.g., [6,9,13,16]).

We included learning activities which highlighted engineering innovation and creativity [7] such as those focussed on robots in space and sound and music in virtual reality. We also chose to emphasise not just the technical underpinning of engineering, but also important social and human-centred aspects, for example in the learning steps concerning healthcare and water hygiene [6,8,10,25].

We have also shown that the careful and structured design of discussion prompts and learner tasks in our engineering MOOC increased both the numbers and percentages of socially engaged learners, which has been shown to improve gender balance in online STEM learning [29,30]. Enabling diverse learners to establish their own “social identity” as engineers, through collaborative and social learning activities, can help to break down the barriers to inclusion in engineering education [16,17,18,59].

On the strength of our experiences of designing and facilitating an engineering MOOC both before, during, and after COVID-19-related lockdown restrictions (when most higher education learning activities were forced fully online) we make the following recommendations for the continued use of asynchronous discussions within engineering education.

6.1. Recommendation 1: Carefully Structure Discussion Prompts

We have seen that the careful re-writing of discussion prompts increased social learning activity on the engineering MOOC. The four principles we outline here as guidelines for writing discussion prompts that will engage learners are consistent with the findings of others who have investigated what students find most effective in online discussion fora.

• Principle 1: Making it obvious/Signpost expected engagement

Allow learners to quickly and easily understand that this section of the learning activity is where discussion with fellow learners is expected. For example, we found that rewriting the discussion prompt by using a consistent phrase (e.g., “over to you…”) clarified the expectation that learners make a comment or to discuss with each other/resulted in increased participation

• Principle 2: Use open ended questions.

Using open-ended (what? who? why? how?) questions rather than closed questions which might garner only a yes or no answer encourages longer answers to questions. If using more than one open-ended question, make sure there are not too many (we found no more than three per learning step worked well), and that they are related, relevant, and presented in a logical order.

• Principle 3: Imaginative framing

Guiding and encouraging learners to use their imagination can set an easier context for thinking about some of the more complex aspects of engineering, and allow users to see the relation to their own personal experience, or to envision imaginative hypothetical situations. Using hypothetical scenarios helps to avoid learners’ fear that there is solely one right answer, and relating to personal experiences allows learners to more strongly identify themselves as potential engineers with relevant experience that is needed and valued in engineering. Note: if questions are related to personal experience, care should be taken with potentially sensitive subjects, and it should be made explicit to the learners that their contributions are public and that they should only share what they feel comfortable with.

• Principle 4: Professionalise the task/Using authentic scenarios.

Using authentic scenarios and relating to activities that engineers might undertake in their professional working context helps learners to understand the value of the learning activity or task. Using real-world examples helps to illustrate the variety of tasks that engineers undertake and widen the understanding of what engineers do, working also to break down potentially negative preconceptions and stereotypes about engineering.

6.2. Recommendation 2: Where Appropriate, Integrate Asynchronous Discussion in Blended Models

Asynchronous discussions have many benefits for learners not solely due to the flexibility they offer, which provides the opportunity for all learners (whether extrovert or introvert) to build social and cognitive presence at their own pace in their own time, but also through their ability to help establish a higher level of learner engagement and inclusion outside of fully online learning environments.

Encouraging discussion and educator/learner interaction through asynchronous discussion can also complement face-to-face teaching, as part of a blended learning provision. For example, online materials for discussion can be provided prior to each in person teaching session, allowing students to prepare their contributions in advance. The facilitator can then summarise the online asynchronous discussions during the in-person session and encourage further contributions as relevant. This type of activity was employed by many in higher education teaching during COVID-19 lockdown times which saw a rapid switch to online learning activities (with face-to-face teaching effectively completely cut out of the teaching toolbox).

We reiterate that careful design of asynchronous discussions should not be restricted to purely online learning experiences. The continued use of interactive and discussion-based elements will have a positive impact on improving diversity and inclusion and should form an integral, key component in engineering education of the future.