Digital Commons @ Michigan Tech Digital Commons @ Michigan Tech Teaching interdisciplinary sustainability science teamwork skills Teaching interdisciplinary sustainability science teamwork skills to graduate students using in-person and web-based interactions to graduate students using in-person and web-based interactions

: Interdisciplinary sustainability science teamwork skills are essential for addressing the world’s most pressing and complex sustainability problems, which inherently have social, natural, and engineering science dimensions. Further, because sustainability science problems exist at global scales, interdisciplinary science teams will need to consist of international members who communicate and work together effectively. Students trained in international interdisciplinary science skills will be able to hit the ground running when they obtain jobs requiring them to tackle sustainability problems. While many universities now have sustainability science programs, few offer courses that are interdisciplinary and international in scope. In the fall semester of 2013, we piloted a course for graduate students entitled “Principles of Interdisciplinary Sustainability Research” at Michigan Technological University. This course was part of our United States National Science Foundation Partnerships in International Research and Education project on bioenergy development impacts across the Americas. In this case study, we describe the course development and implementation, share critical insights from our experience teaching the course and student learning outcomes, and give recommendations for future similar courses.


Introduction
There is now widespread acknowledgement that human and natural systems are inextricably linked and feed back into one another. In fact, many of the world's most pressing and complex problems, such as global climate change, desertification, deforestation, and poverty, result from these feedbacks [1][2][3]. The field of sustainability science has emerged with the aim of addressing these problems, which are urgent and have high damage potential [4][5][6][7][8][9]. Sustainability problems inherently have social, natural, and engineering science dimensions, and these problems cannot be understood or addressed without each dimension. To be successful, sustainability science will require experts working together from a range of scientific disciplines in a truly interdisciplinary fashion [10,11]. Further, because sustainability science problems exist at global scales, interdisciplinary science teams will need to consist of international members who communicate and work together effectively. The challenges of working within an international, interdisciplinary science team are many. National and international science agencies and funders are increasingly stressing the importance of excellent interdisciplinary scientific proposals, especially those centered on sustainability research (e.g., National Science Foundation (NSF) Science, Engineering and Education for Sustainability (SEES) Investment, Inter-American Institute for Global Change Research Collaborative Research Network Program). Yet, year after year, these funding agencies are frustrated by the failure of most proposals to have strong, interdisciplinary cores.
Just as theoreticians and decision makers have called for increased interdisciplinary approaches to answering sustainability questions, experts in learning and educational science have started to assess the degree to which university classes are effectively teaching this material [4,6,[12][13][14]. These authors agree on the importance and interdisciplinary nature of strong sustainability science. One argues that exercises aimed at developing self-efficacy in solving sustainability problems must combine with lessons in "interdisciplinary collaboration" within this context; they also call for assessing the degree to which these skills are taught in higher education [14]. Another reports that the ability to gain skills in interdisciplinary scientific teamwork varies across individuals, and they argue for including substantial dialogue about the challenges of this endeavor in classes aimed at building these skills [13]. Others have found that conveying these skills is particularly challenging and argue that it is important to design classes that give assignments challenging students to work across disciplinary boundaries in formal work within class and informal assignments outside of class [12]. All point to the importance of using new pedagogical methods that create substantial opportunities for in-class deliberation and outside-of-class activities to effectively train students in these skills that continue to challenge many well-established, senior scientists.
Given the urgent need for effective international, interdisciplinary science teams to address complex sustainability problems, academic institutions must move beyond the traditional disciplinary approach to education [15,16]. At the undergraduate level, students gain a strong disciplinary perspective, but also learn about other disciplines through general education courses. Interdisciplinary training should begin in earnest in graduate school, so that students can hit the ground running when they obtain government, non-profit or academic jobs that require them to address complex sustainability problems (see, e.g., [16]). Interdisciplinary training at the graduate level presents several distinctive challenges to educators, especially if the goal is development of interdisciplinary research capacity that enables students to be effective contributors to complex research endeavors straight out of graduate school. If a training program aims to produce students capable of substantive integrative research, then it will be inadequate to adopt a "sampler platter" approach in which students are merely exposed to different disciplines without doing anything with these exposures. Rather, the training program should seek to produce what Stokols [17] calls the "transdisciplinary orientation", which consists of new values, attitudes, beliefs, conceptual skills and knowledge, and behaviors. Priority learning outcomes should include trans-cultural understanding and cooperation, global consciousness, and being able to conduct interdisciplinary work [8]. Trans-cultural understanding and global consciousness are most effectively fostered when students from multiple nationalities learn and work together on complex research projects. Including an international student body in a sustainability course can be readily achieved using an online, Web-based approach that combines live teaching and discussions during class time with discussion boards and group work outside of class [18]. This model has been successfully implemented for several large international interdisciplinary graduate courses [19][20][21].
In the fall semester of 2013, we piloted a course for graduate students entitled "Principles of Interdisciplinary Sustainability Research" at Michigan Technological University (MTU). This course was part of our United States National Science Foundation Partnerships in International Research and Education (PIRE) project on bioenergy development impacts across the Americas, led by our second author. The challenges associated with the administration of an international interdisciplinary science course include managing the logistics associated with connecting students from around the world in different time zones and at different institutions via video conferencing technology, finding appropriate professors who represent a range of disciplines and who have experience working in international interdisciplinary teams, and negotiating the differences between disciplines in methodologies and norms. Our objectives in this paper are to describe the course development and implementation, share critical insights from our experience teaching the course and student learning outcomes, and give recommendations for future similar courses.

Course Context
The NSF PIRE program aims to support international scientific efforts that develop skills in conducting scientific investigations that cross national borders. A primary goal is building teams of scientists from multiple countries that can pursue research projects together beyond the lifetime of the PIRE project. PIRE is part of NSF's SEES portfolio, comprising various types of specific funding opportunities that make up about 20% of NSF's overall research funding to outside organizations. This particular project includes over 40 scientists from Brazil, Mexico, Canada, Uruguay, Argentina, and the United States. The highly interdisciplinary group includes social, natural, and engineering scientists from multiple subdisciplines, as well as a growing number of graduate and undergraduate students. One of the objectives of the project is to mentor young professionals, especially students, as they develop skills in performing international, interdisciplinary science; these skills include securing funding for their projects, implementing quality research, integrating across major disciplinary categories, and publishing disciplinary and interdisciplinary research on sustainability-related topics. The project began in late 2012 and the course described in this manuscript was offered for the first time in late 2013 over one university semester. It included ten students across three partner countries, supervised by eight PIRE faculty members. As a first time, international, multi-university course, students at the host university (MTU) took the course for credit at that university while students at other universities across the US, Brazil, and Mexico took it as thesis or independent study credits. In the future, we plan to work with our NSF PIRE project's participants to get an equivalent course added to their university curricula so that students can take it as an official course listed on their transcript.

Course Development
International, interdisciplinary research in sustainability science cannot be pursued without funding, so a key step in pursuing this type of research is writing a successful proposal. Funding success will typically require a truly interdisciplinary project proposal, with integrated questions that transcend disciplines, that is crafted by a multinational, multidisciplinary team. Given the priority and importance attached to this requirement, we decided to make the production of competitive, international, interdisciplinary scientific proposals by small, multinational, multidisciplinary student teams our primary course learning outcome. Additional learning outcomes for the students in the course included: (1) review and understand scientific literature from multiple disciplines, as well as multidisciplinary and interdisciplinary papers; (2) develop integrated interdisciplinary sustainability science research questions and experimental designs; (3) communicate about that proposal orally and in writing; and (4) work successfully in international interdisciplinary science teams.
We designed the course to include a mix of instructors that reflected the disciplinary diversity of the students and the larger PIRE science team, allowing us to better provide training in interdisciplinary science research. The text we chose for the course was Group Dynamics for Teams (4th Edition, 2013) by Daniel Levi [22]. We selected this book to help the students understand and participate in teams more effectively. Our second author, a policy scientist, was the lead course instructor, and our first author, an ecologist, and third author (an environmental engineer) co-taught it. Our last author provided course support by offering class training in interdisciplinary science teamwork skills. Four M.S. and six Ph.D. students from five institutions throughout the U.S., Mexico, and Brazil enrolled in the course, representing the fields of environmental policy, chemical engineering, forest and soil ecology, avian ecology, environmental and energy policy, economics, and hydrology. Each student obtained three credits for the course from their home institution.
To operate a successful course, our team relied on two main software packages to manage online and real-time interactions. The HUBzero ® platform for scientific collaboration was developed by Purdue University (West Lafayette, Indiana) in 2002 for the purpose of facilitating data-sharing and collaboration among scientific teams. MTU has adopted the HUBzero platform to manage data associated with several research projects, including the PIRE project. A portion of the MTU Sustainability Hub (hub.sfi.mtu.edu) was devoted to the online requirements of this course. This included spaces for downloading weekly readings, uploading assignments, and participating in extended discussions of the course material. The three course instructors were all administrators of the Sustainability Hub space devoted to the course, and were able to modify the system as needed. The students quickly grasped the system, and were able to suggest helpful modifications to the platform over the course of the semester. Real-time course meetings with all of the participants were made possible using the Visimeet videoconferencing software package developed by IOCOM (www.iocom.com). Visimeet is available as a separate software package or a web-based tool, and allows for sharing of audio, video, and other content to all members of a meeting, as well as the ability to record course sessions and distribute them to the group for later viewing.

Course Implementation
Language barriers present a significant challenge to international collaborations. In our course, which was taught in English, we attempted to lessen these barriers in the following ways: (1) all course instructors and attendees were required to have at least a basic understanding of a second language; (2) we asked everyone in the course to speak slowly and clearly at all times; and (3) students were allowed to participate in the in-class discussions by typing to the group if they were not comfortable speaking. To minimize scheduling problems across countries and institutions, we elected to have our three-credit graduate course meet once per week for two hours rather than the usual three. The third hour of class time was allotted for students to participate in online HUBzero discussions of the reading assignments at the time of their choosing, which also gave the international students an additional chance to express themselves in writing if they were not comfortable speaking in English.
The reading assignments began with an article entitled "The challenges of interdisciplinarity" [15], as well as journal articles from each scientific discipline represented in the class ( Table 1). The readings were selected to introduce the students to the variety of discipline specific perspectives on scientific research and sustainability questions, methods, and jargon. We then assigned several chapters of the Group Dynamics for Teams [22] book each week. Students were required to turn in weekly reading journals that briefly summarized the reading and applied the concepts to their own multinational, multidisciplinary student team's work together; these journals included one question designed to generate class and online discussion.
We split the 10 enrolled students into three small teams of three to four, selected to include a diversity of disciplines and nationalities. Our goal was to have each small team turn in a research proposal meeting the standards of a specific, interdisciplinary NSF SEES Request for Proposals (RFP) of their choice at the end of the semester. All teams chose the Dynamics of Coupled Natural and Human Systems (CNH) RFP. The proposals were developed over the course of the semester through a series of smaller, graded assignments so the students received ongoing feedback over time (Table 1). Each team was assigned one course instructor as a mentor. As mentors, we kept in close contact with our team via email and during meetings in each class period, and were primarily responsible for reviewing the teams' assignments. Each mentor also read the assignments from all teams to enable discussions of what the teams were collectively doing well and what needed work. We then tailored our successive lectures to address weaknesses in the group work, such as formulating interdisciplinary hypotheses, integrating separate research tasks, and writing compelling broader impacts sections.
In class each week we led discussions of the reading, received feedback on how each team's work was going, and gave presentations on various aspects of international, interdisciplinary sustainability research proposal development (Table 1). Because the students represented a range of experience levels and backgrounds, we also gave presentations on scientific writing and giving oral presentations. Toward the end of the semester each team was required to give a graded, 30-min presentation on their proposal to the class, which generated feedback from the instructors and from the other students.
We devoted one early class period to a Toolbox workshop, facilitated by our last author. This workshop is part of the Toolbox Project, an NSF-funded research effort that aims to understand and facilitate communication and collaboration in cross-disciplinary scientific research (toolbox-project.org). The Toolbox approach is a workshop-based dialogue method that employs philosophical concepts and methods in the form of dialogue prompts to structure conversation among collaborators about their research assumptions [23]. These prompts express views about core scientific concepts, e.g., "Scientific research must be hypothesis driven" and "Validation of evidence requires replication", and collectively the prompts are associated with a Likert scale (1-5/Disagree-Agree). After indicating their agreement or disagreement with each prompt, workshop participants discuss their views of these concepts with one another in a lightly facilitated discourse. The goals are to achieve a greater degree of reflexivity about one's own assumptions and a greater degree of mutual understanding of the assumptions made by one's collaborators. The Toolbox workshop began with a 20-min preamble by the Toolbox facilitator, followed by a 45-min dialogue and a 10-min debrief discussion. In this workshop, the students used the Scientific Research Toolbox instrument, which comprises 34 prompts gathered into six module sets: Motivation, Methodology, Confirmation, Reality, Values, and Reductionism. Students in this workshop addressed prompts in three of these modules: Motivation, Methodology, and Values. In particular, the dialogue involved candid sharing of views about the importance of application, the role of hypotheses in scientific research, and the nature and influence of values in scientific activity.
During the last few weeks of the semester we invited five scientists affiliated with the PIRE project and representing the range of disciplines in the course to review the proposals generated by each student team. The last class period was devoted to conducting a mock NSF panel review with these reviewers. The three instructors each led a panel but did not participate in the discussions or reviews, and students in the course were encouraged to attend a panel that was not discussing their team. This allowed the students to experience the process of an NSF panel and provided them with detailed, NSF-type feedback on their team's proposal from a range of disciplinary experts who had experience working in international, interdisciplinary science teams. Table 1. Schedule of course activities and assignments for the newly piloted, semester-long graduate course, "Principles of Interdisciplinary Sustainability Research", at Michigan Technological University. In class each week the instructors led discussions of the reading, received feedback on how each team's work was going, and gave presentations on various aspects of multinational, interdisciplinary teamwork and sustainability research proposal development. "Reading Assignment" signifies that the students had to read the paper and/or book chapters and complete a reading journal that briefly summarized the reading and applied the concepts to their own teamwork. The students also had to participate in online HUBzero discussions of each reading assignment before class.

In Class Activities Assignments
Week

Course Outcomes Evaluation
The course, "Principles of Interdisciplinary Sustainability Research", incorporated a number of elements that assisted students in overcoming the obstacles that stand in the way of interdisciplinary research success. First, the use of Levi [22] as the primary course text foregrounded the importance of group dynamics and team process, supporting students in acquiring the skills needed to engage in interdisciplinary team behaviors. Second, attention to disciplinary theories and methods, along with the Toolbox dialogue, provided students with exposure to alternative research worldviews, as well as opportunity to articulate elements of their own research perspective. Experience in communicating across disciplines, and in particular, in identifying and negotiating differences in disciplinary perspectives, is a key determinant in interdisciplinary success [28,29]. Third, exposure to the professional culture of funding and proposal writing is crucial to becoming an effective contributor to and leader of complex sustainability efforts [30]. Fourth, development of model grant proposals provides a meaningful context within which to negotiate disciplinary difference and pursue interdisciplinary integration; further, it functions as an effective measure of interdisciplinary achievement [31].
The mock NSF panel reviews of each team's proposal from the five professors affiliated with the PIRE project gave us an objective assessment of the student's comprehension and application of the course material. In keeping with NSF protocol, each reviewer was asked to rate each proposal as Excellent, Very Good, Good, Fair, or Poor and to write detailed comments on the strengths and weaknesses of the following: Intellectual Merit, Broader Impacts, Contribution to Understanding of Couple Natural and Human Systems, SEES Relevance, Value Added through Interdisciplinarity, Educational and Outreach Activities, and Inclusion of Underrepresented Groups. Overall the reviews were very positive, with comments including: "This is an ambitious, well integrated and nicely detailed proposal. Many parts read as well or better than a lot of NSF proposals I have seen", and "Overall it is very well written and clearly interdisciplinary in character".
The mock NSF panel reviews also allowed us to objectively evaluate the degree to which sustainability was addressed in the student's proposals. We looked for reviews that gave evidence that the proposals: (a) addressed a complex problem with social and natural science components that had high damage potential [8]; and (b) demonstrated systems thinking, where the students integrated social and ecological knowledge in an interdisciplinary fashion [13]. Using these criteria for sustainability, all three student proposals successfully addressed sustainability issues, but to different degrees. The proposal with the highest overall rating received the following reviewer comments: "[This proposal] addresses a number of very important and emerging threats to ecosystems and economic and social well-being", "A true interdisciplinary approach is described", and "The proposal is clearly relevant for the NSF-SEES program, touching on elements of environmental science, social science, education, and practical import". Less positive comments included: "Maybe not sufficient detail in the description of how to sustain integration for the entire project timeline." The proposal with the lowest overall score received the following positive comment: "Identifies relevant and clear questions regarding human and natural systems interactions and sustainability, and provides a somewhat detailed description of the methods to answer those questions." Other reviewers stated: "The social focus is more fully developed than some of the ecological components", "The authors do not successfully integrate the components of their research into an interdisciplinary examination of a coupled natural-social system", and "The attributes of the three research areas of policy, socioeconomics, and ecological impacts are not well connected". Crafting an entire NSF CNH proposal is not an easy task, even for seasoned researchers. From the point of view of the three instructors (three of the four authors of this manuscript), the proposals demonstrated that the students were able to formulate sustainability research questions that encompassed several disciplinary fields. Student teams were also largely successful in employing a systems thinking approach, integrating knowledge from both social and natural sciences to address their research questions in an interdisciplinary fashion.
The Toolbox workshop results highlighted the following: first, nearly all students were committed to the importance of applied science, although there was a strong sentiment that basic science is important-as one student commented, you "can't do applied [science] without extending theory and what is known". Second, while recognizing that not all science need be hypothesis-driven-some is exploratory, for example-the importance of hypotheses for structuring science and separating anecdotal information from evidence was emphasized. Finally, the class group recognized that values are ubiquitous in science, discussing a wide range of values, beliefs, and norms from the ethical to the political to the epistemic. Although the workshop was brief, the conversation was engaged and engaging. The facilitator noted the high quality of the dialogue, especially in light of three factors: (a) the virtual nature of the workshop; (b) the fact that the participants are not research collaborators; and (c) the fact that the participants were still getting to know one another. The dialogue appeared to have the effect of enhancing self-understanding and mutual learning. The class reaction to the workshop was positive, as assessed by a post-workshop questionnaire.
The mock NSF panel reviews and outcomes from the Toolbox Workshop showed that the students made great strides in developing integrated, interdisciplinary sustainability science research questions and experimental designs, writing competitive international interdisciplinary scientific proposals in collaborative teams, and working successfully in international interdisciplinary science teams. Further, the students gained valuable experience reviewing and discussing scientific literature from multiple disciplines, as well as multidisciplinary and interdisciplinary papers. As the instructors, we observed the students struggle during the first few weeks of the course as they were exposed to the new values and methods of thinking of the other disciplines. It was very beneficial for everyone in the class to read multiple examples of research in the natural, social, and engineering sciences and to utilize in-class time for discussions of this research so that questions and doubts could be addressed. When we split the students into smaller interdisciplinary teams they continued to gain knowledge about each other's disciplines, and, eventually, began to integrate this knowledge and methodology with that of their own discipline as they progressed in their proposal development. Some teams struggled more than others with differing levels of motivation and attitudes of members, language barriers and scheduling. However, we found that both reading the Group Dynamics for Teams [22] book and having one instructor assigned to mentor each team helped greatly to prevent minor teamwork problems from becoming more serious.
In our view, the students progressed toward successful acquisition of the "transdisciplinary orientation" recommended by Stokols [17]. The proposal writing exercise in particular created a space in which they acquired the requisite values, attitudes, beliefs, and knowledge that figure into this orientation. In addition, though, the exercise highlighted the complications that can arise from differences that exist across disciplines. In particular, as the instructors we observed differences of vernacular [32], intellectual emphasis [33], professional culture [34], and epistemology and research practice [35]. In general, we believe the ability to function effectively in interdisciplinary groups depends on two key skills. First, interdisciplinary researchers in heterogeneous groups must be good communicators, understood both transactionally in terms of information transfer and relationally in terms of interactive efficiency [36,37]. Second, they must be good integrators, that is, they must be able to integrate information and perspectives from different disciplines into coherent research products [38,39]. We endeavored to impart these skills to the students in our course in the following ways. First, to increase their skill in interdisciplinary communication we included substantial dialogue and discussion about the challenges of interdisciplinary scientific teamwork, as recommended by [13,14]. We also used new methods to create substantial opportunities both for in-class deliberation in small groups and activities outside of class on interdisciplinary communication, as recommended by [4,6,[12][13][14]. Finally, to increase the student's skills in integrating information from different disciplines into coherent research products we combined lessons on sustainability science and grant writing with interdisciplinary group exercises in solving sustainability problems (i.e., the research proposals).

Conclusions and Recommendations
Developing and implementing this course was a great learning experience for the instructors. Based on our experiences piloting this course and feedback from the students and external reviewers, we have developed a set of recommendations for similar courses. First, we recommend placing a strong emphasis in the beginning on helping students from different disciplines develop a common scientific language [32]. Disciplines can vary greatly in their approaches to research problem paradigms, scales, frameworks and goals, and it is important for students to understand these differences before attempting to draft their own interdisciplinary proposals [15]. Even basic information such as the types of questions other scientists ask and the types of data they collect merits time and attention, and feedback from the students suggested that they would have preferred more time to get to know each other's disciplines and interests before beginning the proposal writing. Second, and along these same lines, student feedback emphasized that they would have liked to spend more time working with their assigned groups on smaller assignments before starting the research proposals. This would allow the interdisciplinary, international groups to become more comfortable communicating and working as a group, building social cohesion and making the proposal development less complicated. Third, team work may be facilitated by having each team create a contract outlining the roles and responsibilities of each member and to establish an accountability strategy [39]. Several students indicated that assigning weekly roles for their group members, such as editor, greatly facilitated the proposal writing process. Fourth, spending more time in the beginning of the course giving examples of successful international, interdisciplinary science research and allowing for discussion of what makes these projects truly interdisciplinary may help students more quickly understand how to develop their own interdisciplinary research questions. Finally, we recommend a shorter proposal assignment than the full NSF CNH, which still allows students to formulate interdisciplinary sustainability research questions and methods. We plan to give this course again on a bi-yearly basis and will use course assessments to determine if these modifications are beneficial to students.