Implementing Sustainability Co-Creation between Universities and Society : A Typology-Based Understanding

Universities are under mounting pressure to partner with societal stakeholders and organizations to collaboratively create and implement sustainability-advancing knowledge, tools, and societal transformations. Simultaneously, an increasing number of societal organizations are reaching out to partner with universities to achieve organizational objectives and increase the effectiveness of strategies to further societal sustainability. Using a conceptual framework of “sustainability co-creation”, this study empirically examines the historical and ongoing experiences of five organizations in Japan that actively partner with universities to enhance sustainability activities and strategies to transform society. We examine motivations for partnering with universities, innovative models of practice, factors hampering the co-creative potential of the university, and desired changes to overcome these. Our empirical study leads to the proposal of a typology that might assist in categorizing and understanding key attributes of differing types of sustainability co-creation. We build our typology from two perspectives: First, in terms of the primary objective of the co-creation (ranging from knowledge production to the transformation of society), and second, in terms of the approach taken (ranging from either socially or technologically-centered). We then reflect on the organizations’ experiences to offer several strategies that could increase the effectiveness of the university when partnering with stakeholders in sustainability co-creation. We also highlight several factors effecting the university’s capacity to move beyond knowledge production towards implementation measures to transform society with external stakeholders. Sustainability 2017, 9, 594; doi:10.3390/su9040594 www.mdpi.com/journal/sustainability Sustainability 2017, 9, 594 2 of 28


Introduction
Scientific knowledge on the state and drivers of sustainability issues such as climate change, wildlife and natural habitat loss, pollution, energy systems, and economic inequality is now more abundant and available than at any other time in human history. As major suppliers of scientific knowledge, universities (in using this term, we acknowledge the wide diversity of "universities" (e.g., research/teaching-orientated, public/private, undergraduate/graduate, etc.), the individual disciplines/departments that comprise them, and the unique socioeconomic and political landscapes surrounding each) are faced with increasing pressures to engage in the translation of scientific knowledge into concrete societal actions towards greater sustainability (i.e., changes in policy, institutions, social practices, and technologies). Calls for universities to craft useful knowledge [1] and link knowledge to societal action [2] are widely normalized [3]. Yet an increase in knowledge alone will not suffice for human society to overcome sustainability challenges. The production and translation of knowledge forms but one component of a vast array of transformative actions required to advance the sustainability of human settlements. As Lubchenco [3] (p. 495) famously cautioned, "Science alone does not hold the power to achieve the goal of greater sustainability". Overcoming sustainability challenges also requires continued and radical transformation of our built environment, transport systems, infrastructures, business models, lifestyles, policies, and the way society is organized [4]. Although producing new knowledge can guide more intelligent transformational strategies, much of the transition to sustainability involves "rolling up our sleeves" and implementing what is already known. Therefore, not only must universities and scientists work to produce transformative knowledge [5] moving beyond basic problem description and analysis [6], they are also facing ongoing pressures from governments and funders to develop concrete solutions to specific societal challenges [7][8][9][10]. Since the conception and implementation of transformational knowledge and measures exceeds the expertise and resources of any single academic discipline or societal actor, universities are expected to collaborate with industry, government, and citizenry to this end [11,12].
The university function of sustainability co-creation has emerged in response to these challenges [13,14]. This depicts a role where the university "collaborates with diverse social actors to create societal transformations in the goal of materializing sustainable development in a specific location, region or societal sub-sector" [14] (p. 152). A defining attribute of sustainability co-creation is a shift away from knowledge production as a means in itself. This objective permeates historical conceptions of the university as an "ivory tower" isolated from society [15]. In moving beyond this paradigm to embrace sustainability co-creation, greater emphasis is given to knowledge production as a vehicle for creating solutions to societal problems and triggering societal transformations towards greater sustainability [13,16].
A large body of literature exists on university collaborations with society (in this study, "university collaborations" indicates mutual activities with society undertaken by individual university researchers/students, whole departments or even entire universities) to advance the sustainability of particular cities or regions (see [8,17]). This testifies to a global proliferation of sustainability co-creation in academia [18]. Karatzoglou [19] argues that scholarship is dominated by highly descriptive and mostly single-case accounts of approaches, challenges, and outcomes. Consequently, generalizable insights and theory are lacking, it is argued. Despite such criticisms, the literature offers rich insights into the myriad of forms and paradigms (many of which are still developing) by which sustainability co-creation can unfold between universities and society. Non-exhaustively documented paradigms in case studies include "living laboratories" [20] or "urban living laboratories" [21,22] that extend beyond basic research in the laboratory to trial emerging forms of technological and social innovation in real-life and urban settings; "urban reform" where universities transform surrounding neighborhoods through real estate development and environmental or economic transformation [23,24]; "citizen science" where non-scientist citizens engage in scientific knowledge production by collecting field data [25]; and also "green clusters" [26], where universities contribute to green economic development in a specific geographical area by working with local governments and businesses to foster a concentration of high-tech industries. Such cases of sustainability co-creation are typically examined from the perspective of university research or outreach functions. Yet scholarship features several cases demonstrating that sustainability co-creation with external stakeholders can also be carried out by integrating students and exploiting the education function of the university [16,[27][28][29]. Sustainability co-creation can therefore serve as a powerful means of harnessing often misaligned university missions of research, education, and outreach to place-specific needs and sustainability challenges.
Despite differing labels and conceptual frameworks to understand various manifestations of sustainability co-creation, when seen from the perspective of transdisciplinarity, many common threads are illuminated. Transdisciplinarity describes a process whereby scientists "transcend" disciplinary and organizational boundaries to integrate knowledge from multiple disciplines and societal sectors to create knowledge with practical value for society [30]. Four principles defined by Klein et al. [31] and Binder et al. [32] demonstrate these commonalities, namely, (1) the participation of stakeholders from diverse areas of society; (2) an explicit focus on producing solutions or useful knowledge for a specific geographical location or community of stakeholders; (3) the integration of diverse forms of knowledge from other academic disciplines and societal sectors; and (4) joint production of knowledge and solutions across scientists and stakeholders.
In addition to the literature discussed above, a handful of scholars have examined large numbers of collaborative sustainability initiatives to draw out more generalizable theory. One scholarship branch examines the common roles that university researchers can play in sustainability collaborations with society [33][34][35]. Ranging in intensity of ambition to transform society, these include knowledge production and diffusion, regional governance and planning (including policy shaping), technology transfer and economic development, demonstrations and implementation of experiments with technical or social innovation, and physical reforming of built and natural environments [12,14]. Luederitz et al. [11] draw on an extensive body of literature to propose a framework for understanding and evaluating the inputs, processes, outputs, and outcomes of various sustainability experiments.
As key outputs, they emphasize the commitment of partners to collaborative action, differing types of knowledge, enhanced capacities of partners, societal transformations (institutional, environmental or infrastructural), and the uptake or transferring of results to other societal areas. Other studies have identified factors that can impede the fruitful production of such outputs [10,18]. In addition to resource limitations amongst actors such as time and funding, these include partnership co-ordination issues such as poor integration of the activities and objectives of differing partners, communication difficulties across societal sectors, and poor management and leadership. These point to the need for time and resources devoted to project management just as much as on-the-ground activities.
Despite these insights offered by scholarship, what is noticeably lacking is a broader understanding attained from multiple case studies on how differing models of practice differ or relate to each other. As we pointed out above, there is an increasing push for "transformative science" [5] and university engagements with society that lead to tangible changes in society [6,9]. If available, a typology for categorizing differing forms of sustainability co-creation between universities and societal actors would be useful for scholars, co-creation practitioners, and funders wishing to understand the transformative potential of differing types of practice. We also point to the tendency of the existing literature to examine cases of university and societal collaboration for sustainability chiefly from the perspective of the university (e.g., [36][37][38]. Such perspectives fail to consider that many sustainability co-creation initiatives involving universities are formed and driven by societal organizations. Furthermore, business innovation literature [39,40] has embraced the sustainability co-creation paradigm and suggests that many initiatives do not involve university partners at all. Non-academic societal organizations are therefore critical producers of sustainability knowledge and drivers of societal change [41].
To address these gaps in the literature, we focus on the underrepresented voice and experiences of societal actors in university and multi-stakeholder collaborations for sustainability. By examining multiple cases, our overarching objective was to build a typology-informed understanding of the nature and distinct characteristics of differing types of sustainability co-creation activities in universities, undertaken by five organizations in Japan. Our data were obtained from interviews and documents about past and ongoing collaborations. Three sets of questions guide our study:

1.
What factors motivated organizations to partner with universities? 2.
What is the nature of the partnership with key university partners (i.e., socioeconomic context, objectives, role of the university, and outcomes)? How do these practices suggest differing types of sustainability co-creation? 3.
From the perspective of each organization, what factors hamper the university's effectiveness as a sustainability co-creation partner and what changes are required in universities to overcome these?
Answering such questions would generate important insights for increasing the capacity and effectiveness of the university to work with societal partners in the collaborative pursuit of sustainability. For example, answering the first question would build understanding into qualities and benefits that motivate societal organizations to involve universities in sustainability co-creation. In an age of rapid transformations of university missions, structures, and governance in accordance with evolving stakeholder demands [42,43], this would ensure that desirable qualities are preserved. Conversely, identifying the factors that impede fruitful collaborations with societal partners would give valuable clues as to the future changes required to bring university structures and practices better in line with stakeholder expectations. Lastly, we argue that an integrated view and the identification of differing models of sustainability co-creation would help university actors (e.g., researchers, educators, and administrators) and societal actors (e.g., government funders, business enterprises, and civil society organizations) to understand the differing options on the table when considering ways of collaboratively working to advance societal sustainability. This would also shed light on the potential outcomes of each model, and how each can play a specific, valuable role in wider societal attempts to create transitions towards sustainability.

Background: Conceptual Framework for Understanding Sustainability Co-Creation
The literature illustrates that sustainability co-creation activities between universities and societal partners fall somewhere along a continuum between two acutely distinct objectives. At one end of the continuum lies the primary objective of knowledge production. Conceptions of "co-design" of research agendas [44] and "co-production" of sustainability-relevant knowledge [45] clearly demonstrate this paradigm of collaboration, which focuses on knowledge production between scientists and non-scientist stakeholders from society. Multiple expectations are pinned to collaborative knowledge production. Scholars argue that the inclusion of non-scientist stakeholders such as industry or government practitioners and civilians can supply valuable firsthand knowledge to the scientific process [46], boost the societal relevance of scientific research agendas [47], and enhance the societal assimilation of scientific knowledge [1,45]. By rewarding achievements in knowledge production output such as publications and conference presentations through tenure and funding allocation [7,9], academic incentive systems have historically privileged this type of scientific activity. In contrast, the other end of the continuum concerns collaborations undertaken across societal sectors with explicit ambitions to implement knowledge, create tools and solutions, and transform society [12,14,42,48]. The nature of sustainability co-creation activities differs strongly in accordance with the varying emphasis placed on these objectives. Production, integration, and sharing of codified and tacit knowledge are hallmarks of the first objective. For the second objective, due to explicit transformational ambitions, various kinds of interventional measures to transform society are required [12]. In using the term "transformational interventions", we borrow the concept from literature on climate experiments [49] to indicate the introduction of a novel technology, policy, or social arrangement (typically in a confined geographical area) with the goal of triggering social, economic, or environmental transformations. This conception of the sustainability co-creation continuum is depicted in Table 1 as three levels of intensity: low, medium, and high. Illustrative examples from the literature are also included for each type.
In low-intensity co-creation, objectives and key activities mainly concern joint production, integration, and sharing of knowledge (both codified and tacit) across multiple societal sectors. The principal objective is to produce or synthesize knowledge to later inform the design and implementation of transformational measures such as policies. Luederitz et al. [11] illustrate three types of relevant knowledge production for sustainability. The first is analytical-descriptive type knowledge that focuses on understanding the problem at hand [6]. The second is anticipatory, normative knowledge about the nature of desired future states. The third is transformational knowledge to provide guidance for formulating concrete actions to facilitate progress towards sustainability goals [5]. Due to a focus on knowledge production, direct transformational interventions on society in this paradigm are low. Luederitz et al. [11] describe various outcomes of collaborative knowledge production corresponding with this conception of low-intensity co-creation. These include evidence for decision-making or guiding policy and implementation actions towards sustainability (i.e., reports, datasets, and presentations) and improved capacity of practitioners and scientists to understand the problems examined. A concrete example is the TERIM project from Ludwig-Maximilians-Universität München and partners described by Binder et al. [32]. University researchers collaborated with policy makers, industry practitioners, and citizens to build knowledge of the transition dynamics of Austrian energy regions (analytical-descriptive knowledge), build and test policy scenarios (anticipatory knowledge), and then derive policy recommendations (transformational knowledge) to guide future strategies to accelerate the transition to renewable energy.
Midway on the continuum lies medium-intensity co-creation. This is marked by the execution of interventions to transform society by collaboratively creating or trialing innovative solutions. Conventional knowledge production activities are supplemented with learning from applied settings through bounded experiments (i.e., temporally or geographically) with emerging technologies or novel social configurations in real-world settings [50]. The literature demonstrates that such experimentation typically unfolds in real-world urban settings, which are often labeled "living urban laboratories" [21,51]. Rather than transform society per se, experiments are intended to "demonstrate" sustainability and inspire replication elsewhere. As such, interventions are typically controlled in scale and sometimes non-permanent. Table 1 lists several illustrative examples. These include residential smart grid testing and monitoring in local communities in the Pecan Street Project [52], or trials of compact electric vehicles and experimental social arrangements for biomass generation, tackling urban decline, and encouraging social activities for elderly residents in the Bright Low Carbon Urban Reform Program by the University of Tokyo [53]. Others include living laboratories for involving residents in the trial and co-creation of smart home appliances and smart energy management tools in SusLabNWE by Delft University of Technology and partners [20,54]. As described by Hodson et al. [55], "infrastructure laboratories" (i.e., whole buildings or replicas of actual building sections) for testing smart grid technologies, etc. clearly fall into this category. High-intensity co-creation reflects a transition beyond these controlled interventional measures and goals of knowledge production. It is marked by ambitious objectives to directly transform society through the implementation of knowledge or social or technical innovation [12,14]. Efforts to advance societal sustainability thus move beyond small-scale (temporally and geographically bounded) trials of emerging technologies or novel social configurations towards the creation of full-scale (unbounded) socio-technical configurations and environmental transformations [11]. Evidence suggests that the explicit implementation focus of this type of co-creation can create challenges in establishing scientific rigor and legitimacy [64]. This is not to mention that many projects fix overly ambitious transformational objectives that prove, in practice, difficult to achieve. Being an emerging paradigm of societal interaction, cases of high-intensity sustainability co-creation appear uncommon. However, we list several relevant examples from the literature in Table 1. Many involve built environment-focused measures to reform urban environments, often with interlinked efforts to drive green economic development, social innovation, and socio-technical experimentation. Relevant cases include the Oberlin Project by Oberlin College in the USA [28], the Manchester Corridor by the University of Manchester and partners [61] and the Newcastle Science City Partnership by Newcastle University in the UK [37], and also, District-Urban Lab by the Karlsruhe Institute of Technology in Germany [36]. Others such as Rust to Green by Cornell University in the USA [62] strive to co-create societal transformations by fostering social innovation and capacity building for citizens or municipalities, integrating citizens in town planning, and re-imagining decayed urban spaces. In high-intensity co-creation, although the emphasis is on implementation and societal transformation, codified knowledge generation (i.e., low-intensity co-creation) serves a vital supporting role. For example, newly created or modified urban spaces can serve as living laboratories to generate new opportunities for data collection and scientific monitoring to inform policy [61]. Alternatively, some high-intensity initiatives involve knowledge production by compiling best practices from elsewhere to guide local implementation actions [28].
We emphasize that Table 1 is not normative. All intensities of sustainability co-creation are highly important forms of scientific activities and societal engagement. Depending on the goals of researchers and stakeholders, the nature of societal challenges tackled, and university strengths and expertise, differing co-creation intensities may be more suitable than others. Also, although Table 1 depicts differing co-creation intensities in isolation, in reality there is much overlap. Sustainability co-creation might involve a whole portfolio of activities and projects that each correspond with differing levels of co-creation intensity. It might also involve a progression from lower to higher intensity, such as from knowledge production or solution creation and demonstration towards implementation. Furthermore, this progression can be cyclical and iterative. Studies on cases from living urban laboratories by Evans and Karvonen [61] and Trencher and Geissler [22] demonstrate that implementation-focused activities between universities and societal practitioners can generate important insights for new research agendas. This can spark a new cycle of progression from knowledge production to subsequent trial and experimentation, to implementation, and then back to research.

Sample Selection
Five organizations were selected for empirical analysis in this study. All participated in a roundtable discussion held at the 3rd GPSS-GLI Sustainability Science Symposium. This was hosted by the University of Tokyo at Kashiwanoha, Chiba Prefecture, from 19 to 20 January in 2015. The symposium and roundtable theme concerned university collaborations with society and how sustainability science researchers can build "kizuna" (i.e., bonds or relationships) for sustainability. Five organizations from Japan were invited, each represented by a member with management or supervisory responsibilities. Organizations were selected due to expertise gained from extensive first-hand experience in multiple sustainability collaborations with universities to advance societal sustainability.
Attributes of the surveyed organizations appear in Table 2. Recruitment for the symposium followed a stratified sampling approach; each was chosen to ensure a diversity of profiles in the private and non-profit sector. Organizations range from a large international consulting firm, three NGOs (two international and one local) and a large national, design firm. Reflecting this diversity, Table 2 also shows that differing emphasis is placed by each on the multiple dimensions of sustainability (i.e., environmental, social, and economic) in accordance with the varying sustainability strategies of each organization.

Data Collection
Data were collected in two stages from January 2015 until February 2017. Firstly, from the presentations and discussions in the abovementioned roundtable session, and secondly, from analysis of project documents and nine semi-structured interviews in Japanese during a follow-up phase (see bottom of Table 2). Interview questions were organized into two halves. The first addressed objectives, motivations, and general experiences from various historical or ongoing collaborations with universities. It also explored general perspectives on the effectiveness of universities to collaborate with stakeholders to advance societal sustainability, and factors hampering this. The second half focused on a single case of an innovative collaboration with a university. Interviews took place from July 2015 to February 2017 in person (at the organization's headquarters) or via Skype. Each lasted approximately 60-min and usually involved two university researchers and one or more organizational representatives. Interviews were recorded, transcribed, and then manually analyzed to identify recurring themes in the responses. Following Eisenhardt [65], the accuracy of the observations was ensured by inviting interviewees to contribute to manuscript production by checking the sections relevant to that organization, and also to provide additional information via email where necessary. Hunger and malnutrition alleviation in poverty-stricken areas, particularly in Africa.
Disaster recovery and social capital building in Iwate Prefecture following the 2011 tsunami and earthquake.
Integration of art and science to creatively explore innovative solutions to socio-environmental challenges.

Why Do Organizations Collaborate with Universities?
Addressing our first research question, interviews and workshop discussions with the organizations shed light on important factors motivating the decision to collaborate with universities to enhance sustainability activities. Following Murphy and Simon [66], Table 3 classifies these perceived benefits and motivating factors as either tangible (typically quantifiable) or intangible (typically qualitative) in nature. Tangible benefits and motivating factors typically concern access gained to physical resources such as university equipment, facilities or funding, human resources such as students and researchers, and professional contacts. Such benefits were important motivations for the International Consulting Firm (see Section 4.2.1) when choosing to partner with a local university to foster data scientists and analyze big data collected from various smart city projects. Tangible benefits also include skills procured from university researchers such as expertise in a particular discipline or geographic location, and structured knowledge creation capacities in data collection, analysis, organization, and interpretation of results. For the Community NGO (see Section 4.2.5), tangible benefits in one project also included the capacity-building role of university researchers. They assisted their staff with gaining skills in facilitating and coordinating workshop-style town planning activities. Intangible benefits, relating to the qualities of university knowledge production, are equally significant. Of high importance, we found a wide appreciation for the unbiased and objective nature of university research and societal interactions. For the International Humanitarian NGO (see Section 4.2.2), this allowed a neutral assessment of nutrition intervention efforts in Rwanda that could not have been achieved internally.
"When proposing projects and requesting funding, evidence is crucial. That is, systematic compilation of quantitative knowledge that shows clearly cause and effect (i.e., from a humanitarian intervention) is extremely important. As an NGO, we could, to some extent, probably accomplish this. But if a third-party university did this for us, we could then secure more objective data, and this would hold a higher value to stakeholders in government and corporations." The long-term and non-profit-orientated thinking of university researchers also motivated firms to seek collaboration. For the International Design Firm (see Section 4.2.4), partnering with universities allows a more open exchange of ideas and pursuit of innovation than can be achieved with corporate think tanks, which typically require confidentiality agreements. Lastly, both the International Consulting Firm and International Humanitarian NGO expressed that collaborations with reputable universities significantly improve the societal credibility of activities. As an illustrative example, the International Consulting Firm required trust from local citizens and the municipality to gain access to sensitive personal health and energy consumption data in a big-data-driven smart city project laden with commercial motives (see Section 4.2.1). With a local university housing the data and leading the data analysis, this proved to be a major trust-winning and credibility-boosting factor.

Innovative Cases of Sustainability Co-Creation with Universities
Here we explore our second research question on the various ways the surveyed organizations are engaging in sustainability co-creation with universities. Although our research examined multiple historical collaborations conducted between the organizations and university partners, the following five cases focus on one flagship initiative (either completed or ongoing). Data were sourced from roundtable presentations by organizations, interviews (see Table 2), and document analysis. To maintain a similar analytical focus across cases, we focus on key objectives of the partnership, the role of the university and the logic behind their participation, the nature of the collaborative knowledge production and measures to transform society, socio-environmental conditions of targeted locations, outputs or impacts (actual or potential), and noteworthy challenges encountered.

International Consulting Firm: Data-Driven Economic Transformation and Technological Experimentation
The International Consulting Firm (henceforth "the Firm") is collaborating with Aizu University (henceforth "the University") and the City of Aizuwakamatsu (population 122,600) in Fukushima Prefecture, 300 km from Tokyo. The collaboration aims to overcome local socioeconomic decline and spur a transition to a data-driven economy and smart city. Scores of other IT and electronic manufacturing firms are also involved as secondary partners. This is a case of high-intensity sustainability co-creation with an explicit and long-term strategy to implement knowledge, technical innovation, and economic measures to transform society. Big data and human resource building are key to this. The local city is utilized as a testing arena for emerging technologies and collection of citizen data for analysis. Big data is exploited in both university research and education to provide solutions for transforming energy, health care and transport management, and also to generate new business services and high-income employment opportunities.
Interviews revealed that a host of societal challenges converged to prompt the Firm's initiation of this collaboration. These included a national shortage of data scientists (causing data analysis and sensitive data to be outsourced overseas) and chronic population aging and shrinking in the local city. This, in turn, is driving the deterioration of the economic infrastructure and built environment, as approximately 80% of the University's graduates move to Tokyo for employment. The city has also suffered a major loss of attractiveness as a living destination following the Fukushima disaster and associated radiation concerns.
Targeting the whole city, this collaboration involves a Memorandum of Understanding (MOU) with the entire university for the decade 2011-2021.
Although the Firm initiated and drives the collaboration, the University assumes numerous roles. These include project steering and core-planning; provision of computing and data storage facilities, expert knowledge in IT and graduates for employment; engagement in joint teaching; and participation in research, data analysis, and technology verification experiments. For the Firm, the value of the University's participation lies in its nationally renowned expertise in computer science engineering, advanced computing facilities, and 1100 or so undergraduate and graduate students receiving much education in English. It is expected that top graduates find employment within the Firm and emerging data-science industries in Aizuwakamatsu.
The Firm and University are involved in intimate joint production of applied knowledge. Education underpins the collaboration. This involves jointly taught university courses for regular students and training programs for working professionals to create experts in data analytics and cyber security. The Firm emphasized that the utilization of real data from local residents and the municipality is a defining characteristic of the education collaboration. Secondly, investment from the Firm and national government has enabled the establishment of a data center with advanced cloud storage facilities for data storage, analysis, and R&D. This provides the physical infrastructure for students and newly located firms to launch new big-data-related business ventures with high relevance to smart city priorities such as preventative health care, energy consumption visualization, renewable energy, and transportation. Efforts to halt population decline by building a knowledge-driven economy and generating new employment opportunities for students thus consist of a dual strategy: (1) fostering data scientists with practical skills from industry and (2) providing an advanced university-industry technology base for data storage, analysis, and venture firm creation.
A third set of collaborative activities concerns joint research and implementation of various field tests for smart city technologies. Illustrative examples include trials of energy consumption visualization technologies in 600 local residences and experimentation with wearable ICT devices for measuring and visualizing movement, walking, resting, and sleeping patterns in citizen lifestyles. Citizens and the municipality are actively involved in projects and willingly provide personal data on energy consumption, physical condition, medical history, etc. to university faculty and students for analysis. Data collection and analysis-enabled by field tests with industry, citizens, and the municipality-is integrated into faculty research and student education. For example, data are mined by students, faculty, and industry partners to identify new business opportunities and strategies to increase the efficiency of public service delivery (e.g., snowplow deployment by the municipality). The Firm also emphasized that the collaboration was committed to open innovation and avoiding reliance on patenting and licensing of new data exploitation ideas.
As key outcomes, approximately 240 students and 50 industry professionals have undertaken joint teaching programs each year. Several have found employment in the Firm as data scientists, while students have successfully launched two venture firms. These numbers are expected to increase in the coming years. As other impacts, the municipality has committed to shaping policies and town planning to both support and integrate big data generation and exploitation. The collaboration has also succeeded in gaining support from several thousand residents consenting to provision and utilization of personal energy and lifestyle data for commercial purposes. As previously mentioned, the university's role as the principle data storage and analysis facility was the key to achieving this. As noteworthy challenges, the Firm reported major difficulties during early collaboration stages in bringing the University to understand the missional importance of tackling societal challenges in the local city. Interviews revealed that although the University had an established history in applied research with industry, it regarded its principle mission as preparing graduates for the national IT market place-not intervening in the local economy and urban environment.

International Humanitarian NGO: Evidence-Based Nutrition Program Evaluation
The International Humanitarian NGO (henceforth "the NGO") partnered with several faculty and graduate students from the University of Tokyo and Ochanomizu University for a five-year (2011-2016) collaborative research project. They collected data with the goal of assessing the impacts of the NGO's nutrition program, an Area Development Program (ADP) in Rwanda, spanning 2011-2025. The ADP involves educational initiatives to foster nutrition improvement of children and mothers. Co-creation activities are focused on knowledge production and are characterized by a distinctly people-orientated (i.e., humanitarian) approach to sustainability. Conditions of the rural area targeted include a strong dependence on agriculture, historically chronic levels of poverty and malnutrition, and varying amounts of access to electricity. Targeting multiple communities in Rukara, Eastern Rwanda, the assessment was implemented in conjunction with World Vision Rwanda. The Government of Rwanda is a key stakeholder and assisted with logistics like transport and staff for field visits.
Specifically, the research sought to quantify nutrition-related impacts and stakeholder learning gained through the ADP. Data were sourced through interviews and questionnaires administered to households to estimate the intake amounts and frequencies of key nutritional measures such as calories, protein, vitamins, and minerals, in addition to agricultural output. Measurements were also taken for the height, weight, arm circumference, and body fat of either children (0-5 years) or parents. The survey targeted 240 households and includes a control population in Mwiri. Key roles of university researchers included design of research methods and surveys, collection and analysis of field data, and reporting of results to stakeholders. Both university researchers and NGO actors collaborated closely on the field data collection and subsequent analysis.
Interviews demonstrated that the need for university participation followed realizations that the NGO lacked the capacity to quantitatively and objectively measure the impacts of its nutritional education interventions in Rwanda. The NGO expressed hopes that this decision would improve the credibility of its activities in the eyes of donors. Strong expectations to generate evidence of positive nutritional impacts in Rwanda challenge the need to retain impartiality. Yet since NGO funds alone are insufficient for project implementation, university researchers also contribute research funding and graduate students. With diverse interests and motivations thus vested in the project, maintaining scientific integrity and impartiality is easier.
Data analysis has been finalized and results submitted to the Government of Rwanda. One key finding was that infant growth rates decline significantly after weaning commences. This has re-affirmed the importance of post-weaning nutritional interventions for the NGO. Additionally, the data demonstrated the important role played by the mother in child health (measured by height, etc.). Researchers found that mothers play a key role in distributing food (and thereby nutrition) in households. This is positively affected by involvement in household crop cultivation and the mother's educational level. During interviews, the NGO expressed strong expectations that this co-created knowledge will serve as important, objective evidence to inform future nutrition intervention strategies. Based on this co-created knowledge, the NGO reported future intentions to foster child health by engaging mothers in household agriculture, providing nutritional information, and increasing educational opportunities for women.

International Conservation NGO: Scientific Knowledge for Informing Sustainable Consumption Strategies
The International Conservation NGO (henceforth "the NGO") teamed with a Ph.D. candidate and supervising faculty from the University of Rhode Island during 2009-2010 in a behavioral buying experiment. Conducted at the NGO's headquarters in Tokyo, this assessed the buying behavior of consumers confronted with various choices of seafood products. These included Marine Stewardship Council (MSC)-certified labels. The underdeveloped market for eco-label seafood products in Japan-the world's largest importer of seafood [67]-was the principal social problem motivating this experiment. Similar to the above case, by striving to preserve global fish stocks from a consumption perspective, this collaboration is distinguished by a social and people-centered approach to advancing sustainability. Since its primary objective was scientific knowledge production to inform future societal engagement strategies of the NGO, it corresponds with our conception of low-intensity co-creation.
The experiment involved 160 adults (mainly females aged 30-60) who participated over six days and 16 sessions. To measure eco-label premiums, participants were invited to bid on the pricing of three packaged seafood products (each with or without MSC labels) in an auction format. To assess the effectiveness of differing communication strategies, participants received explanations regarding (i) the meaning of the MSC label; (ii) global overfishing trends; or (iii) both simultaneously. University researchers were involved in the implementation of the experiment, results analysis, and final report drafting (see [68]). In interviews, the NGO reported that the key motivating factors for collaborating with university researchers included their expertise in behavioral economics and seafood consumption. They also highlighted the objectivity and analytical rigor of university research relative to NGO methods like focus groups.
Experimental results revealed that consumers were more likely to purchase eco-labeled products after given explanations for both the unsustainability of current fishing trends (i.e., the problem) and the significance of the eco-label (i.e., the solution). This group paid around 15% more for the eco-labeled product, thereby indicating a potential premium. However, research also concluded that although consumer education is vital, differences in Japanese marketing and distribution systems relative to North America would complicate the propagation of seafood eco-labels. This qualitative evidence informed a decision to re-focus the NGO's public communication strategy away from individual consumers towards large-scale procurers of seafood such as hotels and supermarkets.

National Design Firm: From Knowledge Production to Societal Transformation
The National Design Firm (henceforth "the Firm") concluded a MOU with the Graduate Program in Sustainability Science and Global Leadership Initiative (GPSS-GLI) at the University of Tokyo (henceforth "the University") in June 2015. This aims to catalyze a new model of sustainability science that enrichens scientific enquiry with art, creativity, public enlightenment, and innovative implementation projects. Intended as a loosely defined, self-evolving collaboration mechanism, the partnership involves (i) joint teaching; (ii) provision of internships; (iii) joint publishing in both scientific outlets and print or electronic media for the public; and increasingly, (iv) formation of a platform to recruit other societal partners to co-design and co-implement solutions for tackling real-world sustainability problems. An important long-term goal of this case is to transition from an early focus on knowledge production and a social innovation approach towards implementation and measures to transform societal systems that also integrate technological experimentation. It is envisioned that this would work as a series of loops. As the knowledge was implemented, this would lead to further knowledge creation and implementation opportunities.
As joint teaching activities, Firm designers provide a four-session project-based learning module in creativity and innovation for social challenges. Internship experiences for students are hands-on. They involve research and roles in producing visual communication materials for public education purposes.
In addition, an important collaborative activity for faculty and Firm members concerns joint research and publishing of non-academic publications (see [69]). These collaborative knowledge production activities focus on producing transdisciplinary and highly accessible information for the public on cutting-edge societal issues, solutions and creativity. As one flagship initiative, efforts are currently fixed on raising public awareness and stimulating business opportunities related to the past, present, and potential future role of fermentation technology in the socioeconomic and cultural life of Japan. This project has been branded Jozo 2050 (meaning "ferment" or "brew" in Japanese).
Both a printed and online publication (www.jozo2050.org) are devoted to this initiative. The knowledge production is highly transdisciplinary. It integrates traditional and cutting edge knowledge and practices in fermentation gastronomy from practitioners, scientists, designers and diverse academic fields (e.g., sustainability science, life sciences, data science, humanities, etc.). This is intended to spark a societal conversation about where technology might lead the traditional fermentation industry in the future.
Efforts are also underway to shift from knowledge production to implementation. These aim to combine science, traditional knowledge, and creativity to support the development of new fermentation industries while revitalizing traditional sectors and the surrounding rural communities. Achieving this transition is challenging. Interviewed faculty explained that implementation-oriented projects in tandem with industry and government are unprecedented at GPSS-GLI. Building momentum for a transition from knowledge production to societal transformation strategies requires several years. To cope with differing partner expectations regarding outputs and timelines, the initial stage of the partnership has been framed as a period for mutual learning and free experimentation with idea sourcing and knowledge production. Importantly, this period is shielded from goal-or output-orientated expectations. As explained interviews with the Firm and faculty concerned, this is to build mutual trust between parties and increase the societal visibility of the partnership through a diffusion of jointly produced educational materials.

Community NGO: Social Capacity-Building in Post-Tsunami Reconstruction
The Community NGO (henceforth "the NGO") partnered with faculty and students from Iwate University (henceforth "the University") from 2012 to 2014 in a project called "Youth Committee for Otsuchi's Future". This targeted Otsuchi town (population 12,800) in North-Eastern Japan; an area annihilated by a tsunami exceeding 30-metres in height in March 2011. The collaboration aimed to empower local youth by enabling their participation in decision-making around the post-tsunami reconstruction process. Otsuchi and surrounding towns saw a bourgeoning of public participation events to guide decision-making from March 2011. Yet these were dominated by elderly town members, thereby rendering it difficult for the young to influence post-tsunami reconstruction decision-making. This case is marked by a focus on capacity-building (a people-centered approach) and triggering changes in town planning (as opposed to merely producing knowledge). That said, the collaboration did not involve any interventions to directly transform the social or environmental fabric of the town.
The University and NGO co-implemented approximately 10 workshops over two years. These targeted approximately 15 high school (junior and senior) and 20 undergraduate students from the University, who acted as mentors. During workshops, youth participants were exposed to the town planning process, challenges facing Otsuchi such as population ageing and decline (approximately 800 died and 400 went missing from the tsunami), and alternate future visions from various community and external stakeholders. These accumulated into student-led project conceptions for elements they wished to see incorporated into the reconstruction process: (1) extensive parks and greenery; (2) replacement of street lights; and (3) nurturing of local food culture through events. These were presented to the town mayor in a final workshop. The NGO reported the major outcomes of the project as: enhanced youth interest, understanding and participation in the town planning process; the delivery of youth conceptions of a desirable town future to the municipality; and financial and political commitment from the municipality to continue supporting youth input into post-tsunami reconstruction. As important challenges, interviews revealed that the goal-and outcome-focused nature of the collaboration did not capture the interest of many youth participants. Securing the sustained participation of school students thus proved difficult. This was seen as a consequence of the University's top-down approach of designing the collaboration without explicit strategies to integrate the NGO and locals into project design. This negatively impacted the community support and ownership of the initiative.

Barriers to Fruitful Collaborations with Universities, and Desired Changes
In response to our final research question, interviews and discussions with the five organizations (and some university faculty involved) revealed an array of obstacles to fruitful engagement between universities and societal partners. Five common themes emerged across discussions: (1) missional; (2) university culture and management; (3) education; (4) research project design; and (5) research evaluation. Each is unpacked below.

University Missions
Reflecting on initial attempts to partner with a regional university (Section 4.2.1), the Consulting Firm highlighted difficulties in persuading university decision-makers that initiatives to address the socioeconomic concerns of the surrounding city were an important institutional "mission".
"With up to 80% of graduates leaving town for employment in Tokyo, can you really say that the university is fulfilling its societal mission? To tackle rural depopulation we must provide the same quality jobs as in Tokyo. Yet a university can't do this alone. They must partner with corporations and governments. Getting administration to understand this took some time." It was argued that similar attitudes permeate other regional universities, which tend to see their primary mission as employment preparation for the national market through teaching and basic research. To overcome this, the Firm argued that increased efforts should be made by universities to ascertain societal needs a decade or so in the future, and then orient teaching and research to meet these in conjunction with industry and government.

University Culture and Management
The National Design Firm raised interesting criticisms regarding this point. They highlighted a lack of speed in academic decision-making and projects relative to the corporate world, and critiqued inadequate exchange of personnel between companies and universities. They also asserted that faculty promotion systems rewarding adherence to established protocols impede innovation by deterring risk-taking and trailblazing.
"It is ironic that universities strive for innovation with an extremely historical organizational structure. When attending university faculty meetings, I often feel that departmental management is too conservative and not conducive to fostering a spirit of trying to do something different and change the world. That is, just like banks and governments, conformity and avoidance of risk taking leads to recognition and promotion." It was argued that the university's ability to innovate, partner with society, and adopt external ideas would be improved if corporate culture and stakeholders were integrated into departmental boards. Concrete suggestions included mechanisms to encourage the movement of faculty and personnel to and from corporations and other societal organizations. That said, the capacity of the university to explore or predict future societal conditions and challenges under protection from immediate concerns for commercial utility was highlighted as extremely important. Illustrating this point, the National Design Firm argued that societal innovation required universities and industry to appropriate qualities from each other: "Both the university and the corporation are striving for innovation. ( . . . ) Innovation requires long-term thinking, visioning and exploratory research. Therefore, corporations must appropriate these characteristics from a university, or partner with a university. As for (Japanese) universities, they are now confronted with expectations to savvy the use of their endowments like U.S. institutions, and carry out societal implementation projects. They must therefore become a little like corporations."

Research Project Design
The International Conservation NGO raised many critical points regarding the societal utility of academic research. It was argued that academic researchers tend to overvalue theoretical and methodological considerations during the design of research projects or long-term departmental agendas. It was voiced that researchers and departments lack adequate efforts to balance scientific interests and priorities with actual knowledge requirements of society. Equally missing, it was argued, are concrete measures to enhance the capacity of users to access and exploit new academic knowledge. Reflecting on a stakeholder meeting for establishing long-term research priorities in an academic environmental research institute, the interviewee stated: "Although discussions focused on theory and methodologies, considerations of how the proposed knowledge would actually be used by society were extremely inadequate. In my view, of course the needs of individual researchers are central. Yet it is critical that diverse societal stakeholders are consulted during the design phase to better determine aspects such as the type of knowledge and solutions that are needed to advance societal sustainability." Similarly, multiple organizations expressed a need to prepare dialogue spaces during research design and agenda setting to facilitate the process of matching the needs of universities, users, and societal change agents. As well as enhancing potential societal impacts, organizations asserted this would also improve the societal relevance of university research, and boost ownership and motivation of affected stakeholders towards academic research and implementation projects. Without adequate stakeholder integration in research project design, the Community NGO argued that implementation-orientated university projects risk becoming top-down, goal-orientated, and rigid, which undermines community buy-in. Extensive time in the field building relationships, examining local stakeholders needs, and explicitly defining mutual roles and responsibilities for collaborating partners were emphasized as important avenues for avoiding this.

Research Implementation
While highlighting the importance of basic research, the International Consulting Firm and National Design Firm argued that when seeking to advance societal sustainability, universities need to expand traditional functions to the creation, trial and implementation of tools, technologies, and innovative social arrangements in partnership with industry and government. In particular, the need to better incorporate industry expertise into research was emphasized. Arguments reasoned that business knowhow and integration of cost considerations are crucial to enhancing the potential for societal uptake of research results and social or technical innovation.
The International Conservation NGO expressed that traditional environmental research focuses too much on examining changes in natural environmental systems only. Their interest in collaborating with university partners suffered because traditional disciplinary fields fail to link natural resource studies with consumption trends and patterns, which, in contrast, require an understanding of economics, culture, policy, etc. Interdisciplinary approaches in these fields (i.e., incorporation of knowledge and methods from multiple disciplines) were urged for the future. This is to ensure that scientific knowledge can better guide policy-making and consumption systems towards effecting improvements in natural resources sustainability.

Research Evaluation
Both the International Humanitarian NGO and International Conservation NGO raised concerns that academic research appraisal systems-both internal (i.e., departmental-level) and external (i.e., government implemented)-reduce incentives to consider societal impact and utility when planning and carrying out research. Criticisms were extended against the use of conventional and quantifiable scholarship indicators such as numbers of publications, citations, journal impact factors, and competitive funding amounts, which overlook societal impacts. They argued that widening the research appraisal criteria is crucial if the university is to fulfill its potential to carry out sustainability co-creation with societal stakeholders and move from knowledge production towards measures to transform society. Suggestions were made to account for societal relevance and impacts, and time spent working in communities or with knowledge users.

A Typology for Understanding Sustainability Co-Creation
From our analyses of five cases, two distinct variables have emerged. These provide a useful basis for categorizing the differing models of sustainability co-creation between universities and society so as to understand the essential characteristics of each.
As we show in Figure 1, the first variable concerns the primary objective of the co-creation activities. We previously pointed out, in our background discussion (and also Table 1, that this can be expressed along a continuum. On one hand, objectives can range from knowledge production (low-intensity co-creation) to, on the other hand, ambitions to transform society by implementing sustainability advancing knowledge and practical strategies (high-intensity co-creation). The degree of co-creation intensity is therefore shaped by the partnership objectives, whilst high-intensity co-creation is characterized by various implementation projects and interventions to transform society. Sustainability 2017, 9,594 18 of 28

A Typology for Understanding Sustainability Co-Creation
From our analyses of five cases, two distinct variables have emerged. These provide a useful basis for categorizing the differing models of sustainability co-creation between universities and society so as to understand the essential characteristics of each.
As we show in Figure 1, the first variable concerns the primary objective of the co-creation activities. We previously pointed out, in our background discussion (and also Table 1, that this can be expressed along a continuum. On one hand, objectives can range from knowledge production (low-intensity co-creation) to, on the other hand, ambitions to transform society by implementing sustainability advancing knowledge and practical strategies (high-intensity co-creation). The degree of co-creation intensity is therefore shaped by the partnership objectives, whilst high-intensity co-creation is characterized by various implementation projects and interventions to transform society. Three cases principally concerned scientific knowledge production and, as such, lacked explicit interventional strategies to directly transform society. Two notable examples are the International Humanitarian NGO and International Conservation NGO cases. Both aimed to generate evidence (quantitative and qualitative) to guide the future societal engagement strategies of the partnering NGOs. Interviews demonstrated crucial determinants of organization decisions to partner with universities for this type of sustainability co-creation. NGOs strongly emphasized university qualities emphasized by scholars [70,71] such as the ability to produce objective, trustworthy, and structured knowledge that integrates differing disciplinary or methodological expertise with on-the-ground NGO experiences. Also, rather than being cases of "contracted research" where an organization simply outsources knowledge production to a university [72], both involved extensive involvement by the NGOs in the university researcher-led knowledge production process. Both cases also demonstrated that integration of co-produced knowledge into organizational decision-making regarding future societal engagement strategies is an important co-creation outcome. By integrating both users and producers in the research design and knowledge production process, this led to the creation of "useful" knowledge [1] and a mechanism to "link knowledge and action" [73].
Although the literature tends to emphasize the value of collaboratively produced scientific Three cases principally concerned scientific knowledge production and, as such, lacked explicit interventional strategies to directly transform society. Two notable examples are the International Humanitarian NGO and International Conservation NGO cases. Both aimed to generate evidence (quantitative and qualitative) to guide the future societal engagement strategies of the partnering NGOs. Interviews demonstrated crucial determinants of organization decisions to partner with universities for this type of sustainability co-creation. NGOs strongly emphasized university qualities emphasized by scholars [70,71] such as the ability to produce objective, trustworthy, and structured knowledge that integrates differing disciplinary or methodological expertise with on-the-ground NGO experiences. Also, rather than being cases of "contracted research" where an organization simply outsources knowledge production to a university [72], both involved extensive involvement by the NGOs in the university researcher-led knowledge production process. Both cases also demonstrated that integration of co-produced knowledge into organizational decision-making regarding future societal engagement strategies is an important co-creation outcome. By integrating both users and producers in the research design and knowledge production process, this led to the creation of "useful" knowledge [1] and a mechanism to "link knowledge and action" [73].
Although the literature tends to emphasize the value of collaboratively produced scientific evidence for guiding government or organizational decision-making [73], we also observed another interesting goal of transdisciplinary knowledge production. This was demonstrated in the case involving the National Design Firm and the University of Tokyo. The Jozo 2050 initiative aims to stimulate public awareness and greater appreciation around food fermentation technologies (traditional and cutting edge) in the sociocultural and economic life of Japan. Ultimately, it aims to trigger societal momentum towards a better integration of novel practices, design, and science-based fermentation approaches in gastronomy and food production. Supporting observations from Talwar et al. [74], this case suggests that an important knowledge production goal in the context of advancing societal sustainability can be to integrate knowledge from differing societal sectors and diffuse this to foster public awareness around specific issues, and in this way, nurture support for emerging innovations. This objective contrasts with the widely normalized goal of principally contributing to scientific understanding through traditional means such as academic publications and conference presentations. The National Design Firm case suggests that this alternative goal of collaborative knowledge production implies a shift towards exploitation of non-scientific mediums such as online publications, websites, and videos. Interestingly, this shift is at odds with academic incentive systems, which have traditionally overlooked such outputs.
With regard to high-intensity co-creation, we identified only one model corresponding with this conception. This was the case of the International Design Firm working with a local university, municipality, and other industry partners. This partnership has the explicit goal of transforming the local city (currently facing chronic challenges such as population decline and the deterioration of socioeconomic vitality) into a data-driven smart city. Several intervening measures to transform society were employed to this end. These included educational and economic development, and technological experimentation strategies. Educational measures involved a joint teaching program to build the data scientist workforce needed for a transition to a data-driven economy. Economic development strategies involved the construction of advanced cloud and computing facilities to house new venture developments from graduating students and, additionally, coax data science industries to relocate to the city and create employment opportunities. More direct interventions were observed with the introduction of emerging smart technology experiments in social settings. Several factors appeared to contribute to the ability to carry out high-intensity co-creation in this case. These included considerable investment from industry and government, a long-term (10-year) strategy, and additionally, the explicit formulation of expected roles and commitments from the firm, university, and municipality via a formal MOU.
Knowledge production also played an important role in this case. However we stress that this occurred more as a "means to an end" or "byproduct" of the co-creation process. Specifically, a key outcome of the technological experiments involving local citizens was the generation of data. This was integrated into student education and faculty research. As well as enabling real-world learning in the classroom, data were mined to identify new business opportunities and solutions for increasing the efficiency of public services delivered by the municipality. This is an interesting illustration of how implementation of new technologies can generate new knowledge, then feed back into education and research to further create new ideas for experimentation or business/service creation. Reflecting cyclical conceptions of technological experimentation and knowledge creation in the literature [61], this is further evidence that sustainability collaborations between societal organizations and universities can serve as an umbrella or portfolio of activities (e.g., knowledge production, technological experimentation, human capital building, and economic transformation) that each correspond with differing intensities of co-creation. Interestingly, this case differs visibly from conventional cases of university and industry collaboration centered around technology commercialization and income generation [75,76]. This is due to a commitment to open innovation, the absence of patenting and licensing (due to public funding), and the use of the city as a "living urban laboratory" to trial and diffuse emerging technologies and generate big data.
Although high-intensity co-creation was confined to the above instance, we observed in the National Design Firm case a strong ambition to move from knowledge production towards implementation and measures to transform society (i.e., high-intensity co-creation). As shown in the Jozo 2050 initiative, this involves an incremental strategy of first assembling knowledge from differing areas of society and, secondly, recruiting industrial partners to implement fermentation innovations from R&D firms and academic researchers. This strategy is intended to foster new forms of fermentation industries and stimulate their economic and cultural evolution across Japan. Although such objectives reinforce our initial argument that co-creation can involve a progression from low-to high-intensity activities, this still-developing case suggests that this transition can, in some circumstances, require several years.
A second variable surfacing across the cases was the nature of the approach taken to advance societal sustainability. As we show in Figure 1, approaches may be categorized along a continuum ranging from a technical innovation (technology-oriented) to a social innovation (people-oriented) approach. The International Consulting Firm case is indicative of a mostly technological approach due to its focus on data science and the trialing of new ICT technologies in the local city. As we show in the top-left quadrant, in contrast, three cases demonstrated an entirely people-centered approach to sustainability. These include the humanitarian focus of the International Humanitarian NGO case, the sustainable consumption focus of the International Conservation NGO, and the youth capacity-building focus of the Local Community NGO case. We also illustrate in Figure 1 the desired trajectory of the case of the National Design Firm towards a social innovation orientation balanced with active experimentation with technical knowledge and emerging fermentation technologies. This diversity of approaches in our sample also reflects the orientations of many cases of sustainability co-creation between universities and societal stakeholders that are documented in the literature. For example, going back to the illustrative cases cited in Table 1, one can observe several instances of social capital building approaches to advancing societal sustainability [59,60,62], while, on the other hand, other reported examples demonstrate predominantly techno-centric approaches [22,52]. Still other cases demonstrate hybrid approaches involving both technological and people-oriented, social innovation [36,54].
One question that arose after categorization of the cases with this typology is: What factors contribute to the differing orientations of each case? Based on our observations from the cases, we see several influencing factors. Firstly, the differing dispositions, strengths, and expertise of the societal organization and partnering university, department, or individual researchers are important. For example, the surveyed NGOs are specialized in bottom-up, people-driven approaches, while the International Consulting Firm has accrued vast technical experience in data science and technological innovation in smart cities. Secondly, the nature of the societal conditions and specific societal problem targeted also play an important role. For instance, with reference to the International Humanitarian NGO's case, the utility of technological approaches in promoting health and wellbeing in rural, poverty-stricken areas in developing nations is not as apparent as a people-centered approach.

Increasing the Co-Creative Potential of the University: Potential Strategies and Tensions
In this latter half of our discussion section, we integrate insights from the literature with reflections on the factors reported by organizations in Section 4.3 that hamper the capacity of the university to engage in meaningful sustainability co-creation. Several potential coping strategies are outlined.
The desire to increase corporate knowledge and personnel exchange in university departments, faculty, and boards featured strongly across the interviews. In addition to scholarship [77] and government policy discourse [78,79], calls for increased entrepreneurialism and university-industry personnel exchanges are shared by sustainability science scholars [80]. Clearly, many benefits ensue from university-industry linkages and increased academic receptiveness to corporate culture. These include a better alignment of research and education agendas with industry needs [81]; learning from industry expertise, particularly in commercialization and implementation aspects [72,80]; and improved employment preparation for students [82]. Yet stronger university-industry ties should be promoted with prudence. Some scholars express concern that infiltration of marketplace logic into universities might bring along an emphasis on profit, efficiency, quick fixes, and results. Similar resistance to market shaped science is apparent in calls for "slow scholarship" [83] and "slow science" [84]. Supporting Washburn [71], our findings show that crucial motivating factors for organizations collaborating with universities concern the impartiality, objectivity, and trustworthiness of academic research, which can increase the societal credibility of the organization's activities. These non-tangible qualities are a centuries-old legacy of the "ivory tower" and its historical protection from excessive corporate and government influence [71]. However, as is apparent from fields such as medicine, pharmaceutics, and biotechnology where university-industry exchanges are most prevalent, corporate dollars and intimate linkages with industry can challenge this neutrality and trustworthiness. This occurs from conflicts of interest [85] and aggressive pursuit of profit through licensing at the expense of maximizing the societal impact of research results [75]. As Bok [70] cautions, the benefits of intimate university-industry collaboration are tangible and short-term. Yet the potential trade-offs are invisible, long-term, and ultimately affect public trust in all of science.
Organizations voiced criticisms concerning the lack of societal engagement during the design stage of research and long-term agenda setting. They argued that potential knowledge users and societal stakeholders must work with academics to co-design research projects and agendas based upon societal needs, and additionally, to engineer explicit knowledge-transfer strategies so that societal users can utilize research results. These demands strongly echo scholarly calls for the co-design and co-production of scientific knowledge across organizational boundaries to promote closer science and policy alignment [31,44,47,86]. Yet when emphasizing the benefits of co-designing research agendas and co-producing knowledge, it is easy to overlook potential challenges [82,87]. Firstly, co-designing research agendas can trigger tensions as university researchers strive to balance needs for scientific credibility and novelty with societal demands for practicality and relevance [64]. Secondly, sustainability co-creation requires researchers to navigate vastly different organizing principles, incentives, problems, and desired courses of action [18,86]. Co-designing research and implementation projects with stakeholders therefore requires time to overcome cultural differences, build relationships and trust, and design projects that cater to competing priorities [87].
As a way forward, Dedeurwaerdere [88] proposes that evidence of stakeholder engagement during the design phase be explicitly incorporated into the selection criteria for competitive funding calls. This strategy could be strengthened through government "seed funding". Easily accessible, short-term seed funding over 6-12 months could allow academics to spend time building relationships and integrating societal needs and knowledge transfer strategies into academic projects. Conventional, long-term competitive funding packages could then be allocated in accordance with demonstrated levels of stakeholder engagement, in addition to traditional criteria of scientific rigor and novelty.
Interviews also revealed desires for academics to supplement conventional knowledge production with socio-technical experimentation in societal settings, and knowledge implementation with industry and government. It was voiced that such collaboration should take place after envisioning the long-term needs and knowledge requirements of future society, and aligning research and education programs to these. Calls for implementation-focused and transdisciplinary knowledge production pose interesting challenges to academic incentive structures-both internal and external. These were heavily criticized by organizations as valuing scholarship (e.g., publications) over societal engagement and research with high social impact. These criticisms reflect concerns expressed by scholars [7,88,89].
Regarding internal faculty review structures, in order for the university to attain its full co-creative potential, societal impacts and efforts spent engaging with stakeholders by individual faculty must be taken into account [7,42,90]. This could be accomplished by institutionalizing into departmental review guidelines for faculty promotion and tenure a set of explicit achievement standards (e.g., unmeritorious, meritorious, excellent, outstanding) that define various qualitative conceptions of socially impactful research. These would be unique to each department and university. As for other possibilities, Lambert-Pennington [91] showcased initiatives where faculty are given the opportunity to provide evidence of the societal impact of their research through press articles, letters from community members, and evidence of policy change, for example. While there may be isolated incidents of some university departments reforming faculty review structures in a similar fashion, scholars point out that most research universities are yet to integrate such measures to promote stakeholder engagement [89,92].
Externally, regarding calls from organizations to broaden the evaluation criteria for government funding systems to encompass societal engagement and impacts, it has long been argued that transdisciplinary knowledge production necessitates new evaluation approaches [93]. How this can be achieved in practice is the subject of much debate [90,94]. As one potential model, the United Kingdom Research Excellence Framework (UK-REF) program has developed diverse indicators of social, economic, and environmental impact [95]. University departments demonstrate such impacts by nominating high-impact research projects and collating evidence in a case study format. Government review panels assign departments performance ratings from one to four stars. These results are then integrated into performance-based funding allocation. This approach has not been without problems. Sayer [96], for instance, argues that the disciplinary focus of review panels is unsuited to judging interdisciplinary work. He also contends that the selection of "star" researchers as representatives of each department is problematic since it can lead to inferiority complexes and hierarchies. Also, despite expectations for completely objective appraisals of societal impacts from academic research, a recent UK-REF review of 6679 case studies submitted by 154 universities concluded that "the development of robust impact metrics is unlikely" due to the diversity of reported impacts [97]. This, however, does not defeat the UK-REF initiative's utility. Its main value does not lie in cross-institution comparison. Rather, it stems from the policy signal sent to universities and researchers that challenges traditional, scholarship-based incentive structures.
Ironically, however, excessive expectations for social utility could challenge the university's ability to innovate and contribute to society [71]. Scientific and social utility do not necessarily correspond temporally. Often, the latter becomes apparent years later. An outstanding example is the world average temperature dataset from the University of East Anglia [98]. This has been nominated as a UK-REF best practice case study by the institution [99]. Interestingly, the motivation behind this achievement was primarily a desire to produce scientifically rather than socially useful research [100]. Today, however, the data's tremendous social utility is incontestable, since they now underpin both modern climate science and international political negotiations. When promoting sustainability co-creation and solution-orientated science, prudence is required to not hamper the fundamental ability of researchers to also produce curiosity-driven, scientifically useful research. The world average temperature dataset example from the University of East Anglia illustrates, paradoxically, that this is essential for attaining social utility, which, often, is not apparent until years later.

Conclusions
Drawing on experiences from five organizations in Japan engaging with universities for sustainability co-creation, this study examined differing motivations, models of practice, factors hindering the university's co-creative potential, and, lastly, possible ways forward. Several observations can be made.
First, we analyzed the practices of the organizations through five case studies. These illustrate contrasting orientations towards socially or technologically centered approaches and differing levels of ambitions and measures to transform society. This case analysis led to the creation of a typology for categorizing and understanding key attributes of different models of sustainability co-creation ( Figure 1). We used this to categorize collaborations between universities and societal organizations from two dimensions. This followed our distinction of differing intensities of co-creation that we identified in our literature review. That is, we observed firstly from the literature, and then empirically in our cases, that co-creation objectives can be understood as falling somewhere along a spectrum. This ranges from the primary objective of co-producing knowledge to the goal of transforming society. As was also demonstrated in our cases, the latter goal involves various interventions to transform societal systems by implementing sustainability-advancing knowledge, practices, and technologies.
Our study has addressed the previously raised criticism that scholarship on partnerships between university and societal stakeholders is dominated by individual case analyses. It is argued that this undermines the generation of more widely applicable theoretical insights that might assist in understanding other cases [19]. To this end, our typology of sustainability co-creation might serve as a useful heuristic for categorizing or carrying out analyses across multiple case studies. For example, some scholars have surveyed large numbers of sustainability initiatives involving universities and societal organizations [10,11,18]. Use of this tool could generate important insights into the types of practice that tend to dominate in the chosen sample or across academia. Additionally, the tool could prove useful for stakeholders involved in cases of sustainability co-creation. As we argued, sustainability co-creation often serves as a supporting umbrella for a portfolio of different types of activities. At the level of a single university or societal collaboration, these activities could be mapped out. This would provide firstly a micro-level understanding of the differing orientations of the various activities and, secondly, could assist with setting goals for the transition to higher intensity forms of co-creation.
We found that only one of the five cases corresponded with our conception of high-intensity co-creation. This case, involving the International Consulting Firm, is marked by explicit and ambitious goals to arrest socioeconomic and population decline and transform the local town into a smart city and data-driven economy. It exploited an interesting array of interventional strategies to transform the socioeconomic and technological fabric of the city. This case suggests that considerable resources are required for high-intensity co-creation (e.g., long-term project timelines, formalized commitment from partners, high levels of funding, and technical and business expertise). Although another case (the National Design Firm) harbors ambitions of moving to a higher level of co-creation intensity, we observed that this transition from knowledge production and diffusion to knowledge implementation is requiring several years. Long-term planning and continued partner commitments are therefore essential for high-intensity co-creation.
As we emphasized earlier, our depiction of varying co-creation intensities is not normative. Higher intensity co-creation is not always possible-or appropriate-to the societal situation addressed or the capabilities of various partners. Also, codified knowledge creation and sharing (i.e., low-intensity co-creation) can be inextricably linked and necessary for guiding solutions development, demonstrations, and implementation-focused activities. As evidence, we observed in two cases that collaboratively produced knowledge was used to inform decision-making in the International NGOs regarding strategies taken to transform society. We also observed in the National Design Firm case that transdisciplinary knowledge production was firstly collated and diffused through non-scientific channels to foster public awareness around societal issues. This forms part of a long-term strategy of laying the groundwork for the implementation of newly created knowledge and promising forms of social and technical innovation.
Second, this study paid significant attention to the experiences of societal organizations in sustainability co-creation, which, we argued, are often overlooked in studies focused on university roles. Consequently, we uncovered valuable insights into various non-tangible qualities that are important motivating factors for organizations choosing to partner with universities. One notable factor was the increased credibility of organizational activities when collaborating with reputed universities. This was particularly apparent in the case of the International Design Firm involving training of data scientists and the pursuit of a smart city. It appears that without the involvement of the university as a core partner, the firm would have experienced difficulties winning trust from the municipality and citizens to integrate personal health and energy consumption data into scientific and educational projects with commercial prospects. As other important factors for partnering with universities, organizations also highlighted the objective and reliable nature of university research, as well as academic knowledge creation capacities. These included technical expertise, analytical rigor, and systematic knowledge integration. Conversely, calls from the organizations for greater societal utility and closer ties with industry in research, education, and departmental management pose interesting challenges to these qualities. These attributes must therefore be preserved when promoting more intimate ties with industry (which typically demand greater attention to income generation and outputs) if the university is to maintain its legitimacy in society.
Finally, we also uncovered multiple factors reducing the potential effectiveness of the university as a societal partner in sustainability co-creation. These included a lack of effort to incorporate stakeholders into research and project design, and a lack of innovation or corporate logic in departmental management and organizational structures. We also highlighted university and government incentive and evaluation structures that overvalue traditional scholarship while undervaluing societal impacts and stakeholder collaboration. The university's potential to move beyond traditional knowledge production towards transformational and solution-orientated science for sustainability clearly hinges on the ability to overcome these hampering factors. It also depends largely on future success in legitimizing and fostering emerging forms of sustainability co-creation in academic incentive systems that historically value research outputs over societal engagement.