What Are We Teaching Engineers about Climate Change? Presenting the MACC Evaluation of Climate Change Education

Round 1

Reviewer 1 Report

I enjoyed reading this manuscript and found the development of the MACC process to be a useful tool for engineering faculty (and indeed faculty across the university) to use in examining the extent to which climate change is addressed within the curriculum.  It was disappointing, although not surprising, to learn that the aspects of mitigation and adaptation included in the tool were almost absent from the curriculum within the course modules examined.  I did find the recommendations for increasing the representation of these climate change related elements helpful both for students and, by extension, as professional development for those currently working in the field.

I did, however, have a number of concerns that I think the authors might consider addressing in a revised draft of the manuscript:

It wasn’t ever clear to me how the 45 modules included in the study were selected and why the process of review wasn’t simply extended to all modules.  I understand that these modules were most likely to reflect certain key topics, but even so it seemed to me that this selection process has the potential to bias the results of the survey.  I can imagine using a tool like this to assess climate-change related content across disciplines, but more detail about how the process was developed and implemented (a kind of step-by-step guide) would be necessary.  This is where a greater understanding of this decision making process would be helpful.

I also wasn’t clear about how the insertion process of language described on page 15 worked.  Were these changes actually made to the syllabi for these modules, or were these simply suggestions offered by the authors for later consideration.  It was unclear to me if course syllabi are created by individual instructors or if there are standard syllabi for these courses, and if instructors have any control over such changes.

On page 18 the authors note that this review of the curriculum was done based on what appears to be a now outdated version of the accredited curriculum. Is the review even relevant now given these changes?

On page 19 the authors note that the two aspects of the GETA framework that were not included both related to values and that changes in expressed values are critical if instructors are going to embrace the recommendations the authors are making.  Given this, it’s unclear to me why these two components of the GETA were not included. Overall, I thought the GETA framework sounded very interesting and important and would have liked to have seen more about this included in the discussion.

My overall response to the article was that this MACC tool has the potential to be very useful, but that additional work in tracking how the results are actually used to support change needs to be conducted. The authors point to the multiple ways in which faculty might be included in future aspects of developing and integrating climate change into the curriculum. It would be most helpful to see how these efforts play out over the next year or two in order to determine whether or not the MACC proves to be a helpful component of these larger change processes. At this point, as just a preliminary description of this first piloting of this one component of what looks to be a much larger and more ambitious curriculum change process, it seemed not to offer enough content to warrant publication in its current form. I’d prefer to wait to see a more fully realized examination of this larger project.

Author Response

Please see attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Author/s

Thank you for your submission of your article. Knowledge, skills and values related to climate change and climate change education are critical for engineers of the future and its integration within the engineering curriculum is pertinent. The paper presents an interesting evaluation of MACC within the engineering curriculum. Here is some feedback for consideration:

1. Has the MACC framework been validated prior to application? If it has, it would be good to include a detailed description of the measures taken to validate the framework.

2. In developing holistic/well rounded engineering graduates, engineering programmes often offer module options from schools outside the engineering faculty (humanities/social science modules for example). Were these not considered as part of the analysis? Why?

3. What ethical considerations did you undertake in conceptualizing and implementing the analysis? 

4. How did you validate the findings of the audit?

5. How do the findings of your study impact continuous quality improvement (CQI) measures for curriculum development, pedagogy, practice and policy for engineering education in the institution?  

Author Response

Please see attached

Author Response File: Author Response.pdf

Reviewer 3 Report

MAAC Mitigation and Adaptation to Climate Change content and toolkit 

A brief summary (one short paragraph) outlining the aim of the paper, its main contributions and strengths.

The aim of this paper is to quantify anecdotal stories and experiences that alarmingly indicate that curriculum across higher education fails to address climate change mitigation and adaptation (referred to as MAAC in this piece) with the required urgency, depth, and coherence. In particular, the authors assert a “dissociation of engineering education from the climate reality as cited in the literature.” After establishing the importance of engineering preparation and how climate reality is related to the work of engineers (reducing GHG, adapting to impacts such as sea level rise and extreme weather events).  On p. 3, the authors state that there is a progressive agenda for engineering curricula, but its delivery is unknown. The methodology of this study seeks to understand the actual preparation engineers are receiving across higher education (specifically, in the UK). After identifying the current state of knowledge, the authors will demonstrate an effective application of mitigation adaptation to climate change (line 84) (MACC) and routes to include climate change themes (line 86). The literature review briefly recaps development of sustainability, environmental, and climate education using current citations. A basic aspect of the problem is that primary and secondary teachers do not feel adequately prepared to teach about climate change, so students do not enter higher education with a strong foundation; thus, they don’t “look” for climate change in their educational experiences, and can even maintain disagreement with anthropogenic causes (194). Meanwhile, society expects engineers to have this knowledge. “The only way that this can be addressed with necessary urgency is to take well-informed and immediate approach to the inclusion of climate change education within engineering higher education.” 

The methodology combines European frameworks for learning objectives (AHEP3 and the updated AHEP 4), and then use a general sustainability framework, the GETA Curriculum Key, to analyze curricular content in Mitigation (M), Adaptation (A), Climate Change (CC). This combined set of key terms is referred to in this article as the MACC Evaluation. The MAAC Evaluation is applied to just one School of Engineering (line 303), but quite comprehensively across modules in this curriculum. The authors propose automated techniques to complete analysis and updates of curricular modules (line 557) across more universities. In addition, the need to assess and retrain current engineering professionals is noted. 

General concept comments 
Article: highlighting areas of weakness, the testability of the hypothesis, methodological inaccuracies, missing controls, etc.

This is very interesting research – and quite terrifying, really, given how much the climate goals and adaptation plans put forth for near future dates like 2030 - 2050 rely on engineering and the preparation of new engineering professionals. Even the possibility of such a statement as “MACC themes were found to be absent from higher education” is of enough concern to warrant publication of this research to inspire other universities to scrutinize their curricula. I would agree that this is a huge problem.

1. My first suggestion is to better define, describe, or rename altogether the MAAC Evaluation Toolkit. If you have previously described this tool, you could cite it, but I believe that you are just describing the coding methodology that you used in this article. By all means, please make such a toolkit, but it was really confusing to refer to a toolkit that was not adequately described. Perhaps you could put the full “toolkit” in an appendix, if it is indeed a thing that I could use tomorrow to go and analyze my own school’s curriculum. You shift suddenly from MAAC as an acronym to MAAC Toolkit as a specific thing. I would suggest taking all those references out and rewrite the way you refer to your key terms and analysis method. 

2. My second suggestion is about describing the case study; you do need to “address the naysayer” in that you are describing the curriculum at one university – true, you say that because it is an accredited university, and because you have used the (since updated) engineering criteria, that your findings should transfer to other universities. I’m not sure that’s the case. 

3. My third suggestion is that, while your idea of using web-scraping tools or automation to analyze mention of these key terms in modules, that is indeed interesting, but doesn’t NECESSARILY tell us what is actually being taught. Do the instructors go “off script” and insert discussion of climate change? Do students bring such interest in to the classes?  It’s hard to imagine a complete bubble where CC is just not discussed. Updating the module descriptions would be a good step and is certainly a worthwhile analysis, but you might describe other ways to know what is actually being taught. For example, an instructor survey, or other research on student attitudes towards CC. You make a fascinating point that MACC is very present in research programs at the same university, but you don’t spend much time on that point – how did you know that? Why is there the imbalance between research and teaching? 

Specific comments referring to line numbers, tables or figures that point out inaccuracies within the text or sentences that are unclear. These comments should also focus on the scientific content and not on spelling, formatting or English language problems, as these can be addressed at a later stage by our internal staff.

Reviewer Comment:  There is a shortcoming here of breadth (only one university) but a strength in depth (very deep analysis of the actual modules that are taught).  This could be better described in the methodology section. Isn’t this a case study, then? Wouldn’t it be relevant to know more about the case – size, number of students/faculty, etc. This came up for me again at line 475 where you describe the sequence of engineering curriculum at this university in the 3 year Master’s courses. That could all go in description of the case. (Note, you use the term “case study” in line 525, but I believe that’s the first place I saw the term, when really it is rather central to the methodology, I think.) 

Reviewer Comment:  Being from the U.S., I am not quite clear what you mean by “modules” – I assumed this is like a course section, but it might be a smaller segment of curriculum, like a unit? Could you clarify this term, module, for an international or U.S. audience? Are you analyzing course syllabi?  Course catalog?  Where are the modules accessed?  

Reviewer Comment: I’d like to see a visual of how elements of AHEP4 and GETA are combined to become the MACC Evaluation tool. The acronym MACC is used very often, mostly as a reference to mitigation & adaptation climate change, but then here on p. 300 it seems to shift to becoming the MAAC Evaluation Toolkit, so now it’s a more specific term. I found that confusing. This came up for me again at line 511 where here you are talking about the MAAC Evaluation Toolkit….but the toolkit is really just the methodology of your study. A toolkit sounds like it is something that already exists. (I actually stopped to google if the MAAC Toolkit was actually a previously existing tool, and found a Toolkit for Becoming a Party to the Convention on Mutual Administrative Assistance in Tax Matters (MAAC). The term “toolkit” is confusing. 

Reviewer Comment: Being from the U.S., I am unfamiliar with AHEP4 or GETA. I understand this analysis focuses on the UK, but does it apply, I wonder, to European universities in general, and also to other countries?  For example, ABET is the engineering accreditation standard in the U.S. but there is also a framework Engineering for One Planet. The Literature Review might include a short section placing the UK standards in context or perhaps briefly exploring engineering standards internationally. 

Reviewer Comment: The MAAC Key Terms Table 1 seems to cover Mitigation and Adaptation, but where is the CC (climate change)? And, I apologize if it is obvious, but to what document do the citations in the table refer? 

Reviewer Comment:  The contrast of experience with research being related to MACC (line 433) is quite interesting and might be brought out earlier – an imbalance of concern for climate change tilted to research, but not in teaching. That is quite interesting. 

Reviewer Comment:  “…MACC themes were found to be absent from higher education.”  While that seems to be the main point, you did share several examples from the modules, so this should probably have a modifier like “largely absent from”. 

Author Response

Please see attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thank you for your thoughtful and thorough response to my earlier comments. Given these responses I am now happy to recommend publication.

Author Response

Thank you for your constructive feedback that has no doubt made the paper much stronger. We have submitted a slightly revised version that has been further proof read and highlighted more clearly that this is a pilot study. 

Reviewer 2 Report

Dear Author/s

The revised manuscript is received with many thanks. The changes made are acknowledged. It would be a good idea to highlight that this paper is positioned as a pilot study to avoid confusion - you could mention this in the abstract and introduction perhaps. All the best with taking this study to its next phase too.

Author Response

Thank you for your constructive feedback that has no doubt made the paper much stronger. We have submitted a slightly revised version that has been further proof read and highlighted more clearly that this is a pilot study. Thank you also for your well wishes for the future of this research. We are very excited to be moving onto the next phases of this research.