Co-Creative Problem Solving to Support Rapid Learning of Systems Knowledge Towards High-Tech Innovations: A Longitudinal Case Study

: This article explores co-creative problem solving to support rapid learning of systems knowledge in the concept phase towards innovation. We introduce the term co-creative problem solving to describe the act of collective creation between systems engineers and stakeholders during problem solving. The context of this research is a mature Norwegian industry accustomed to efﬁciency and risk aversion, challenged by late validation of systems design due to poor utilization of systems knowledge. We have explored co-creation between systems engineers and stakeholders such as project managers, business developers, and subject-matter experts through a longitudinal in-depth industry case in the energy domain. The primary outcome is insights into how co-creative problem solving supports rapid learning of systems knowledge in the industry case. We propose a method building on the ﬁndings from the research results to support systems engineers in similar contexts facing similar challenges.


Introduction
Co-creation was coined by Prahalad and Ramaswamy in 2004 [1], focusing on the concept of co-creation between enterprises and consumers with the purpose of value creation and innovation [2][3][4][5]. The use of co-creation in the marketing domain has grown significantly in the last decade [6] and proliferated to other domains such as design, focusing on collective creativity between designers and stakeholders [7][8][9][10][11]. The value of co-creation in systems engineering is less explored. This article seeks to extend cocreation in systems engineering [12], focusing on co-creation between systems engineers and stakeholders such as project managers, business developers, and subject-matter experts. Inspired by Sanders and Stappers [9], we introduce the term co-creative problem solving to describe the act of collective creativity between systems engineers and stakeholders during problem solving in the concept phase of systems development.
People, organizations, and technical functionality contribute to increasing complexity in today's high-tech systems. Checkland and Wilson described such sociotechnical problems as real-world problems and introduced the Soft Systems Methodology (SSM) to address this [13][14][15][16][17]. Jackson [18] introduced Critical Systems Thinking to combine systems approaches and cope with various forms of complexity. We define systems knowledge as knowledge of the system [19] over its life cycle and this underpins the research in systems theory [20]. Due to the ambiguous and uncertain nature of the concept phase, systems engineers should strive for rapid learning of systems knowledge and early validation of systems design. Collective creativity in problem and solution exploration using cocreative workshops [21,22] and creative problem-solving teams [23,24] has been shown to be suitable for this purpose. Co-creation and similar human-centric approaches aim to support the creation of significant innovations in collaboration with customers and users [11,[25][26][27]. We distinguish between incremental and significant innovations, the latter meaning solutions beyond the ordinary (The Boderc research project [28], conducted through the Embedded Systems Institute in Eindhoven, the Netherlands, inspired this terminology). Mature organizations often rely on consolidation and incremental innovation to grow, while significant innovations are much harder to create [29,30]. A challenge for mature companies is that their approaches are often more suitable for incremental innovation than significant innovation. The neglect of the importance of collaboration and creativity in systems development is a typical challenge identified among engineers in the energy domain [31]. West [32] claimed that creativity is relatively easy as most engineers are highly creative. However, transforming creative ideas into innovative concepts is more difficult due to "resistance to change and structural and cultural barriers" [32]. In this article, we seek to bridge these barriers and explore how co-creative problem solving may support the creation of significant innovation in a mature Norwegian high-tech company.
The research presented in this article is part of a research collaboration project with four Norwegian high-tech industry partners and two academic partners in systems engineering and systems oriented design, respectively. In our former research, we identified that the industry needs to support rapid learning of systems knowledge through concept exploration, early validation, and knowledge transfer in the concept phase [33]. Furthermore, we explored the use of co-creative methods and tools [21,22,[34][35][36][37][38] to support the Norwegian high-tech industry in achieving their needs. We concluded on eight success criteria inspired by systems and design theories for a new way of working.
In this article, we apply the success criteria as a guide to explore and analyze the usefulness of co-creative problem solving in an industrial setting. For two years, we have interacted with a development team at one of the industry partners. The industry partner is a large-size global engineering, procurement, construction, and installation (EPCI) supplier with about four decades of EPCI experience in the energy domain. A company accustomed to a business management system based on typical (hard) systems engineering focusing on efficiency and risk aversion. The company is currently developing an innovative renewable energy system for the ocean space named Deep Purple. The innovation is leading the company's transition toward sustainable energy production systems. Deep Purple is an innovative large-scale, complex system of systems, building on the company's excellence in subsea technology. However, the innovation also requires the company to gain new knowledge as they enter a new domain in the renewable market. They need new ways of working to support rapid learning and early validation. Our research has explored the usefulness of co-creative problem solving and gained insights into applying this way of working in the industrial setting.
This article aims at answering the following research questions: RQ1: How may co-creative problem solving support the systems engineers in rapid learning and early validation? RQ2: How may the systems engineers apply co-creative problem solving in an industrial setting? RQ3: What may be the main challenges for the systems engineers to adopt co-creative problem solving as a new way of working?
The primary outcome is insights into how co-creative problem solving supports rapid learning and early validation in the industry case. We propose a method building on the findings from the research results to support systems engineers in similar contexts facing similar challenges.
This article contributes to the body of knowledge in two ways: (1) adding academic rigor to collaborative and creative ways of working in systems engineering, and (2) proposing an industrial-relevant method for systems engineers to apply co-creative problem solving. We apply the notion of rigor and relevance as described by Ivarsson & Gorschek [39].
We structure the article as follows. Firstly, Section 2 provides literature on systems and design practices that have inspired this research. Section 3 introduces the method that evolved in this research, and Section 4 describes the research design. The first stage of that research produces insight into how co-creative problem solving may support systems engineers in the concept phase towards high-tech innovations, aiming to add to the academic rigor (Section 5). The second stage of that research realizes and applies a method for co-creative problem solving in the industry case, aiming to contribute to the industrial relevance (Section 6). Section 7 discusses our findings before concluding in Section 8.

Literature Review
This section reviews the literature on systems and design practices that have inspired this research. We also include literature on creativity practices in engineering and general theory on creativity.
Co-creation stems from the enterprise and marketing domain, describing the cocreation between consumers and enterprises as part of the value creation process [1]. In participatory design, Sanders and Stappers [9] described co-creation as collective creativity in any form. They further narrowed this term into co-design and described this as "the creativity of designers and people not trained in design working together in the design development process", with non-designers typically being users or customers [40]. They emphasized the need to view users and customers as partners rather than subjects in the front end of product development. Contemporary design thinking, as practiced by the Innovation Design Engineering Organization (IDEO, Cambridge, MA, USA), the Stanford Design School (Stanford, CA, USA), and the International Business Machines Cooperation (IBM, Armonk, NY, USA), is by many seen as the recipe for innovations [10]. Design thinking practices co-creation using techniques such as rapid prototyping [41]. Design thinking focuses on empathizing with users to create innovative products or services, providing a more extraordinary user experience. IDEO advocated this mindset to create innovative new products and services and transform creative people and organizations [25]. Jones [7] discussed various types of co-creation and identified possibilities for improving design co-creation methods. He highlighted the importance of continuity and investment in this way of working to provide insight into complex problems. Jones called for a systemic design framework to enable practitioners to select and modify the various co-creation methods.
Various literature within systems engineering described the need for collective and creative approaches to creating new products and systems. Pugh [42,43] highlighted the importance of group work and creativity in concept generation and evaluation (the Pugh matrix) as early as the 1980s. Sage & Armstrong [44] proposed collaborative and creative methods for systems synthesis, such as brainstorming [45] and the morphological box approach [46]. Lippert and Cloutier [47] described an extended use of TRIZ [48] to support systems engineers in creating innovations within digital systems engineering. White [49] proposed a practical methodology for complex adaptive systems engineering (CASE) to improve traditional systems engineering in sociotechnical systems engineering. His methodology covered organizational and team aspects, such as the need for brainstorming approaches and user experimentation. The methodology aimed to be an iterative and adaptive way of working for the team to operate on the "edge of chaos" effectively [49].
Looking towards systems architecting, Maier and Rechtin [50] described a great systems architect as being as skilled as an engineer and as creative as an artist. Sillito [51] emphasized the need for an analytic, inventive, and creative process in systems architecting to thoroughly understand the problem and solution domain and create suitable architectures. He distinguished between architecting and architectural modeling, both activities interacting and creating a system fit for purpose. The main objective of the systems architect is to understand how the system behaves and communicate this to others effectively [51]. Muller [52] described facilitating workshops as one of the tools of the Systems 2021, 9,42 4 of 20 systems architect. Workshops enable crossfertilizing and sharing systems insight such as in product specification, design, or business strategy. To conduct effective workshops, Muller proposed timeboxing and multi-views in iteration [53]. Multi-views use a CAFCR [54] framework to view the system from the customer, application, functional, conceptual, and realization perspective iteratively as the concept matures. Timeboxing is to set an appropriate timebox to achieve this within the duration of the workshop.
McFadzean [55,56] supports the importance of multiple perspectives in the engineering of innovative products and systems. She called for a more vigorous use of creativity techniques in engineering and proposed a framework for creative problem-solving teams. The framework enables teams to select appropriate techniques based on their level of experience and need. McFadzean [23,24] further elaborated on the role of a neutral facilitator in such teams. The facilitator needs to address soft issues within the team to establish trust and deal with conflicts. She also described the effectiveness of such teams to depend on their experiences in using creative problem-solving techniques. According to McFadzean, some people may find creative problem-solving techniques uncomfortable. This discomfort can reduce the effectiveness of such techniques. In the worst case, people may not participate at all.
Using groups to stimulate creativity is also supported by Paulus and Nijstad [57]. They found ideas produced by a group more innovative in a productive group setting, considering factors such as trust, attitude, and the number of participants. They proposed the usage of a facilitator and a leader to cope with unproductive team dynamics. The group should be diverse enough to provide knowledge on the subject but not too diverse as this may cause misunderstandings and conflicts. A diverse group enables a change of perspective and stimulates activation of long-term memory, resulting in more innovative ideas. Paulus et al. also found positive effects of combining individual ideation with group ideation to avoid participants biased by each other's opinions and ideas.
Dorst and Cross [58] and Dorst [59] described the importance of coevolving on the problem and solution space by going back and forth on problem and solution exploration to support creative designs. De Bono [60] emphasized the importance of exploring the problem landscape thoroughly to gain insight instead of jumping to a solution. Furthermore, De Bono [60] described lateral thinking as a way of thinking to explore the problem space and stimulate creativity. Without a thorough understanding of the problem and solution spaces, engineers and designers are likely to jump to solutions and develop systems not fit for purpose. Bonnema, Veenvliet, and Broenink [61] (p. 9) identified solution focus as a challenge among engineers in development teams and claimed that "many engineers think in solutions". Daly, Yilmaz, Christian, et al. [62] found that the generation of multiple concepts was crucial in creating innovations to avoid such fixation. Furthermore, Murray, Studer, Daly, et al. [63] emphasized the importance of problem exploration perspectives to create innovative designs.

A Method for Co-Creative Problem Solving
This section introduces the method that we realized and applied in the industry case. Figure 1 illustrates the method, composed of a three-stage process and a timeline. The threestage process evolved as we explored sessions in the industry case and bears similarities to best practices for conducting effective meetings such as [64]. Systems and design practices focusing on collective creativity in problem and solution exploration (Section 2) inspired the timeline.  The method distinguishes from an expert-led co-creation workshop in primarily two ways:  it supports the use of inexperienced facilitators to conduct the sessions, as the systems engineers in the industry case lacked such experience;  it emphasizes the use of brief sessions to fit within a busy workday for the participants in the industry case.
In Stage 1, the problem owner and the facilitator gain an understanding of the problem. They identify the need of the session and decide on the scope. The primary outcome of this stage is a mutual understanding of the problem owner's need for the session's outcome. In Stage 2, the problem owner and the facilitator identify the purpose of the session and choose relevant participants. They plan the timeline and select specific techniques fitting the purpose and the participants. The primary outcome of this stage is the timeline, including specific tasks and the choice of participants. In Stage 3, the problem owner and the facilitator conduct the session in collaboration. The facilitator facilitates the participants through the tasks in the timeline while the problem owner leads the session. The primary outcome of the session is rapid learning and early validation.
The timeline in Figure 1 consists of five main tasks for a structured problem and solution exploration. It is the combination and order of the tasks that are important in its cocreative problem-solving capabilities. The timeline aims to create a collective understanding and shared ownership for the participants regarding both the problem and solution space. Furthermore, the timeline combines individual and group exercises to prevent blocking and to enable the participants to build on each other's perspectives. Task #1 Purpose informs the purpose of the session to get the participant to aim towards the problem identified in Stage 1. In Task #2 Individual exercise, each participant reflects upon the problem and provides their views using simple artifacts, such as sticky notes.
Furthermore, in Task #3 Consensus, the facilitator facilitates discussion of these views in plenum and adapts the session's problem description accordingly to share ownership. In Task #4, Creative problem solving, the facilitator facilitates the participants in a creative problem-solving technique to support the participants in challenging systems boundaries and exploring multiple perspectives. Finally, in Task #5 Sum-up, the facilitator sums up the session and connects to the problem description. This task verifies that the solution exploration fits the session's problem and stimulates reflection among the participants. The method distinguishes from an expert-led co-creation workshop in primarily two ways: • it supports the use of inexperienced facilitators to conduct the sessions, as the systems engineers in the industry case lacked such experience; • it emphasizes the use of brief sessions to fit within a busy workday for the participants in the industry case.
In Stage 1, the problem owner and the facilitator gain an understanding of the problem. They identify the need of the session and decide on the scope. The primary outcome of this stage is a mutual understanding of the problem owner's need for the session's outcome. In Stage 2, the problem owner and the facilitator identify the purpose of the session and choose relevant participants. They plan the timeline and select specific techniques fitting the purpose and the participants. The primary outcome of this stage is the timeline, including specific tasks and the choice of participants. In Stage 3, the problem owner and the facilitator conduct the session in collaboration. The facilitator facilitates the participants through the tasks in the timeline while the problem owner leads the session. The primary outcome of the session is rapid learning and early validation.
The timeline in Figure 1 consists of five main tasks for a structured problem and solution exploration. It is the combination and order of the tasks that are important in its co-creative problem-solving capabilities. The timeline aims to create a collective understanding and shared ownership for the participants regarding both the problem and solution space. Furthermore, the timeline combines individual and group exercises to prevent blocking and to enable the participants to build on each other's perspectives. Task #1 Purpose informs the purpose of the session to get the participant to aim towards the problem identified in Stage 1. In Task #2 Individual exercise, each participant reflects upon the problem and provides their views using simple artifacts, such as sticky notes.
Furthermore, in Task #3 Consensus, the facilitator facilitates discussion of these views in plenum and adapts the session's problem description accordingly to share ownership. In Task #4, Creative problem solving, the facilitator facilitates the participants in a creative problem-solving technique to support the participants in challenging systems boundaries and exploring multiple perspectives. Finally, in Task #5 Sum-up, the facilitator sums up the session and connects to the problem description. This task verifies that the solution exploration fits the session's problem and stimulates reflection among the participants.

Research Approach
Action research forms the basis of this research and allows the researcher to acquire knowledge of real-world problems and improve [65]. In this research, we conducted a longitudinal study of about two years within the development team in the concept phase of developing the Deep Purple system at the industry partner. Deep Purple is a complex system of systems leading the company's transition toward sustainable energy production systems. Thus, we selected the participants in the development team as we consider Deep Purple as a representative industry case to explore co-creative problem solving. The main author actively engaged with the team in the concept phase of developing the Deep Purple system. The participants represent both genders and hold formal education in engineering and systems engineering disciplines. Table 1 shows the profile of the team members, including years of relevant work experience, which reflect the participants' age.  Figure 2 illustrates the research design. The first rectangle shows our previous work [33], where we identified the industry needs and the success criteria ( Table 2). The two following rectangles of Figure 2 show the two stages of research presented in this article. In Stage 1, we explored and analyzed nine sessions to gain insight into how co-creative approaches supported the systems engineers in the industry case to achieve the success criteria. In Stage 2, we used the insights gained from Stage 1 to synthesize and apply a method to support the systems engineers in co-creative problem solving. Systems 2021, 9, x FOR PEER REVIEW 6 of 21

Research Approach
Action research forms the basis of this research and allows the researcher to acquire knowledge of real-world problems and improve [65]. In this research, we conducted a longitudinal study of about two years within the development team in the concept phase of developing the Deep Purple system at the industry partner. Deep Purple is a complex system of systems leading the company's transition toward sustainable energy production systems. Thus, we selected the participants in the development team as we consider Deep Purple as a representative industry case to explore co-creative problem solving. The main author actively engaged with the team in the concept phase of developing the Deep Purple system. The participants represent both genders and hold formal education in engineering and systems engineering disciplines. Table 1 shows the profile of the team members, including years of relevant work experience, which reflect the participants' age.  Figure 2 illustrates the research design. The first rectangle shows our previous work [33], where we identified the industry needs and the success criteria ( Table 2). The two following rectangles of Figure 2 show the two stages of research presented in this article. In Stage 1, we explored and analyzed nine sessions to gain insight into how co-creative approaches supported the systems engineers in the industry case to achieve the success criteria. In Stage 2, we used the insights gained from Stage 1 to synthesize and apply a method to support the systems engineers in co-creative problem solving.

Methodology in Stage 1-Exploring Co-Creative Approaches in the Industry Case
We planned sessions guided by the formerly identified success criteria ( Table 2) combined with inspiration from design and systems practices, including creativity practices in engineering and general theory on creativity. We adapted to the team's daily challenges during concept development and planned for sessions when a team member had a specific problem and needed to gather stakeholders and discuss. The researchers and team members facilitated the sessions, neither holding any previous facilitation skills. We did not aim for one session to fulfill all criteria; instead, we used the criteria to gain insight into how co-creative approaches could support the team to achieve the criteria. Focusing on customer Systems oriented design 7 Enabling creativity Systems thinking, participatory design 8 Focusing on user Participatory design, systems oriented design, design thinking 1 The positioning reflects the authors' opinions of the main academic field(s) to support the given criterion based on the findings from the former research results.
During the sessions in Stage 1, we conducted participant observations and passive observations. After the sessions, we conducted informal interviews, focus groups, and questionnaires. The questionnaire included a set of statements derived from the success criteria using a five-point Likert scale [66]. The scale ranged from strongly disagree, disagree, neutral, agree, to strongly agree. The questionnaire also included open-ended questions for participants to report on benefits and concerns. The Likert scale responses were analyzed using a Net Promoter Score (NPS) [67,68]. Promoters reply strongly agree, while the detractors reply neutral, disagree, or strongly disagree. Agree is neither promoter nor detractor and hence left out of the NPS score. Appendix A provides the questionnaire results, including NPS. Table 3 provides an overview of the sessions conducted in Stage 1, the context of the sessions, the number of participants and respondents to the questionnaire, and corresponding methods for data collection. We collected data during the planning of the sessions, during the sessions, and from the participants after the sessions using questionnaires and focus groups. To analyze the sessions, we found inspiration in the framework proposed by Midgley, Cavana, Brocklesby, et al. [69] for evaluating systemic problem structuring methods. The framework focuses on "the use of a particular method (or set of methods) in a context for particular purposes, giving rise to outcomes". We view the method as the proposed method (Section 3), the context as the industry case, the purposes as the sessions, and the outcome as the goal to support rapid learning of systems knowledge guided by the success criteria (Table 2). While analyzing the sessions in the industry case, we found it beneficial to differentiate between the session's applicability and usability to distinguish between factors impacted by the industry case (the context) and the success criteria (the outcome). Applicability describes how well the session was conducted in the industry case, including planning and structure, while usability describes the session's ability to achieve the formerly identified success criteria. We categorized the notes into applicability and usability and used the success criteria to code [70] the notes on usability. An example of an interview-note made in S3 and coded with SC3 is "Good suggestions which put the technical solutions into a wider perspective. This will help us to introduce our ideas and solutions to the stakeholders". Assigning the code SC3 "showing business potential" to this note describes the session's ability to view the concept from a business perspective.
Three main capabilities indicating support for rapid learning of systems knowledge in the sessions emerged from analyzing the coded notes on usability. From analyzing the notes on applicability, we identified three impacting factors for applying useful sessions. We made a qualitative assessment of the coded notes for each session and scored each capability in each session using a five-point scale from 1 (very low) to 5 (very high). Sessions providing very low support for the capability scored 1, while sessions providing very high support for the capability scored 5.

Methodology in Stage 2-Realizing and Applying a Method for Co-Creative Problem Solving
We integrated the findings from Stage 1 into a method to support co-creative problem solving in the industry case. We passively observed the development team for an initial evaluation while applying the method in two sessions, Session A and B. After the sessions, we conducted informal interviews with the facilitator and the problem owner. Table 4 provides details on the application of the method in the two sessions.

Validity of Data
This research primarily builds on qualitative and participative research methods. While this approach is valuable to gain an in-depth understanding of industry challenges in their relevant context [65,68,[71][72][73], there is a risk of researcher bias and challenges in the generalization of the research results.
To ensure the trustworthiness of our findings and reduce the risk of researcher bias, we used Maxwell's eight-point checklist for qualitative research [74]. We actively engaged with the systems engineers in the industry case through a longitudinal study to gain an in-depth understanding of their challenges and potential solutions. We conducted nine co-creative sessions with systems engineers and stakeholders to ensure rich data collection from multiple contexts. Furthermore, we triangulated data collection using various methods to analyze the results from multiple perspectives.

Results from Stage 1-Exploring Co-Creative Sessions in the Industry Case
This section describes the main results from exploring and analyzing nine sessions in the industry case using the research methodology presented in Section 4.1. Table 5 provides an overview of the techniques and artifacts explored in the sessions. In Sessions 2 to 8, we applied brainstorming [45], brainwriting [75], and Gigamapping [76] for ideation. In addition, we used other techniques to stimulate creativity, such as coevolving on problem and solutions spaces [58,77], combining individual and group exercises [57], and challenging operational scenarios of concepts through adding or removing constraints [78]. We also explored CAFCR [54], focusing on connecting desired systems qualities to the systems realization and identifying gaps and trade-offs. We explored the use Systems 2021, 9, 42 9 of 20 of a playful warm-up exercise inspired by design thinking in Session 5. Furthermore, we used large paper plots, visualizations, markers, sticky notes, flip overs, and a whiteboard.  Table 6 shows an overview of the number of qualitative field notes from the sessions, including data from observations, interviews, questionnaires, and reflections. The observation notes include field notes made during the session. The interview notes include the notes made during informal interviews, focus groups, and open-ended questions in the questionnaire after the sessions. Furthermore, the questionnaire notes include the most significant promoters and detractors of the statements in the questionnaire (Appendix A). Finally, the reflective notes include reflections made by the researchers after the sessions. The two last rows show the number of notes on the applicability and usability of the sessions, respectively. Applicability describes how well the session was conducted, including planning and structure, while usability describes the session's ability to achieve the formerly identified success criteria (Table 2).

Analysing the Sessions' Applicability
We identified three main impacting factors for applying useful sessions from analyzing the notes on applicability. These are the understanding of the upfront problem and need, session length, and session structure. The subsequent sections elaborate on these factors.

Understanding of the Upfront Problem and Need
Throughout the sessions, we identified the importance of two roles: the facilitator and the problem owner. The problem owner is the person responsible for the progress of the problem and has an incentive to take on this role. In Session 1, the facilitator took time to discuss with problem owners upfront and carried out a session that satisfied the problem owners' expectations. In Sessions 2 and 4, the problem owner was also acting as facilitator. While this worked well in Session 4, we observed Session 2 suffering from a schedule overrun. A neutral facilitator applying appropriate timeboxes could have prevented this overrun, both during planning and execution. In Session 4, we posed the following questions to the problem owner and facilitator during planning: Q1. What does the problem owner want to get out of the session (purpose)? Q2. Who should participate (background and knowledge)? Q3. What is the appropriate duration of the session?
The questions were based on findings from Sessions 1 to 3 and inspiration from best practices for effective meetings, such as [64]. Q1 immediately arose when we planned for Session 3. When the problem owner and facilitator aligned on the problem and need, Q2 followed when selecting techniques. From Sessions 5 to 7, we divided the roles of the problem owner and the facilitator more clearly and used the questions (Q1-3) during planning. These sessions provided a useful outcome to the problem owners.
In Session 8, the facilitator and the problem owner used little time to discuss the problem upfront. During the session, it became clear that the planned exercises did not match the need of the problem owner. This session did not play according to the planned agenda, and the outcome was poor.

Session Length
We found that the session length depended on session purpose and commitment from the participants. We found the participants perceived longer sessions as a waste of time in a busy workday. In the sessions with subject-matter experts from other parts of the organization, we observed reluctance to attend more extended sessions due to practical reasons such as lack of cost accounts.
In Session 4, we planned for only one hour, as all participants were subject-matter experts from other parts of the organization. The problem owner was satisfied with the outcome. However, some of the participants reported concerns about too little time. In Session 9, we planned for a full day as the scope of the session was quite large. Likewise, in this session, two participants were subject-matter experts from other parts of the organization. At the last minute, the subject-matter experts canceled the session and replaced it with a two-hour skype meeting. The subject-matter experts were not dedicated to support the innovation and lacked a cost account to attend a full-day session. Sessions 3 to 7 were around 2 h, and we found these sessions to provide a useful outcome within an acceptable timeframe.

Session Structure
In Session 1, the participants showed little responsibility to keep to the scheduled time. The facilitator often cut off discussions between the participants to keep to the Systems 2021, 9, 42 11 of 20 scheduled time. There was minimum use of techniques to engage the participants during the discussions. After lunch, the participants were tired and started to wear out. The session could not go through the planned agenda within the scheduled time slot, even though it had a 1-day duration. Session 2 suffered from a schedule overrun. In this session, the participants were engaged using artifacts such as sticky notes, guide questions, and visualizations. The problem owner planned for a session length of about three hours. However, it ended up as three subsessions with a total length of 11 h.
For Sessions 3 to 9, we aimed to activate the participants and conduct structured sessions using engaging techniques and artifacts and shorter lengths. We conducted sessions according to the following structure:

•
The problem owner introduces the problem or possibility, good to put visuals or statements up on the wall (in Session 3, this was a specific sentence in a research application form, in Session 5, these were system qualities of the subsea storage concept).

•
The participants use sticky notes in an individual exercise to reflect upon the problem.

•
Each of the participants present their sticky notes and puts them up on the wall for discussion.

•
Creative problem-solving technique, choose the appropriate technique for purpose and participants. • Sum up by discussing main findings, if relevant make an action list.
We chose timeboxes (Timeboxes set an appropriate length of a task that enables the participants to produce about 80% complete. The reasoning behind this is that 80% is often achievable quickly, while the last 20% is considerably more challenging to achieve and not worth the effort in sessions [53]) fitting to the minimum time achievable for each activity, considering the appropriate session length. The core of the session, being techniques for creative problem solving, required a significant amount of the time. In Session 5, we planned for 60 min for this; the facilitator chose to stop after 45 min due to saturation. In Session 7, we used around 30 min. From Session 3 onward, we found this way of planning agendas to be effective. The feedback and the observations show that the participants appreciated using timeboxes to push them through the activities.

Analysing the Sessions' Usability
To analyze the notes on usability, we coded the notes according to the success criteria in Table 2. Table 7 shows the number of coded notes in Sessions 1 to 9 for each success criterion. Three main capabilities indicating support for rapid learning of systems knowledge emerged from analyzing the coded notes. The capabilities are problem and solution exploration, collective creation, and creative thinking. To gain further insight into how the sessions supported these capabilities, we scored the usefulness of each session using a five-point scale. Sessions providing very low support for the capability scored 1, while sessions providing very high support for the capability scored 5. Figure 3 provides the scoring results. The subsequent sections elaborate on these scorings. Grasping complexity - Showing business potential 1 Sharing knowledge 2  7  2  7  3  1  10  1  -5  Visualizing  1  1  1  -2  ---1  6 Focusing on customer - Enabling creativity  --3  -3  5  1  -2  8  Focusing on user  1  1  1  1  ----2  3  tential  1  1  2  3  -1  ---4  Sharing knowledge  2  7  2  7  3  1  10  1  -5  Visualizing  1  1  1  -2  ---1  6 Focusing on customer - Enabling creativity  --3  -3  5  1  -2  8 Focusing on user 1 1 1 1 ----2 Figure 3. Scoring how the sessions achieve the main capabilities.

Problem and Solution Exploration
We define problem and solution exploration as exploring both the problem and the solution space in the session, considering enabling technology, business case, customer needs, and user needs. In Sessions 1 and 2, we observed the participants primarily focusing on solution exploration and less on problem exploration. In Sessions 3 to 9, we planned for a structured agenda and found this supported the participants in reflecting on and discussing both the problem and solution exploration. Sessions 3, 4, and 6 scored high on problem and solution exploration, while Session 7 scored medium. In these sessions, we observed the problem owner strive to select participants holding the relevant knowledge.
In Session 4, the context of the session was to explore enabling technology for a specific concept that could impact the company's strategy for future applications. We observed the problem owner selecting subject-matter experts holding the knowledge of the enabling technology to validate the current concept early and explore options. In Session 6, we observed the problem owner include business developers to explore possibilities for an early phase pilot project. Further, in Session 5, we observed the problem owner gain awareness of the lack of operational knowledge. As a result, the problem owner included the subject-matter experts holding this knowledge in Session 9.

Collective Creation
We define collective creation as exploring or maturing an idea or concept in the session with a high degree of uncertainty and complexity. Typically, this requires knowledge

Problem and Solution Exploration
We define problem and solution exploration as exploring both the problem and the solution space in the session, considering enabling technology, business case, customer needs, and user needs. In Sessions 1 and 2, we observed the participants primarily focusing on solution exploration and less on problem exploration. In Sessions 3 to 9, we planned for a structured agenda and found this supported the participants in reflecting on and discussing both the problem and solution exploration. Sessions 3, 4, and 6 scored high on problem and solution exploration, while Session 7 scored medium. In these sessions, we observed the problem owner strive to select participants holding the relevant knowledge.
In Session 4, the context of the session was to explore enabling technology for a specific concept that could impact the company's strategy for future applications. We observed the problem owner selecting subject-matter experts holding the knowledge of the enabling technology to validate the current concept early and explore options. In Session 6, we observed the problem owner include business developers to explore possibilities for an early phase pilot project. Further, in Session 5, we observed the problem owner gain awareness of the lack of operational knowledge. As a result, the problem owner included the subject-matter experts holding this knowledge in Session 9.

Collective Creation
We define collective creation as exploring or maturing an idea or concept in the session with a high degree of uncertainty and complexity. Typically, this requires knowledge sharing and striving for early failures. In Sessions 1 and 2, we observed techniques supporting analytical thinking and less exploration. In Session 2, several participants found it challenging to discuss the concept, reporting reasons such as high uncertainty and lack of knowledge. From Sessions 3 to 9, we planned sessions to support the participants in collective creation using techniques to cope with complexity, such as Gigamapping, CAFCR, and ideation. Sessions 3 to 7 scored high and very high on achieving collective creation. In these sessions, we found the participants appreciated the possibility of sharing their knowledge and perspectives on early phase concepts using techniques such as free-format Gigamapping. In Session 3, the participants reported "rapid identification of possibilities", indicating that the session supported the team in early failures. However, we also reflected after Session 4 that "failing early or perhaps being comfortable admitting that we try to fail is difficult to achieve", indicating that failure is considered harmful in this industry, which focuses on high quality and risk reduction.
We found most participants appreciated the use of creative exercises in the sessions and that participants more experienced using such techniques also appeared to be more confident using them. This confidence became clear in Session 3, where the participants engaged in Gigamapping without further instructions from the facilitator. However, we also observed participants reluctant to engage in creative exercises.

Creative Thinking
We define creative thinking as exploring multiple perspectives and challenging perceived boundaries during problem and solution exploration. In Session 1, we observed a lack of techniques to stimulate creative thinking, such as visualization, engaging techniques, and timeboxes. From Sessions 2 to 9, we made use of techniques and visualizations to support creative thinking.
Sessions 3, 5, and 6 scored high or very high on creative thinking. In these sessions, we found the participants to appreciate the creative artifacts reporting "visual", "creative", "brainstorming", and "sticky notes" as benefits. In Session 6, we found the use of brainwriting combined with group discussion especially beneficial. This combination enabled the participants to think for themselves before discussing their views. Session 6 resulted in 52 ideas in one hour. The participants claimed that the ideas held a higher level of innovation than assumed from a "normal" meeting in the debrief.
Session 9 planned to include creative exercises; however, the subject-matter experts replaced this session with a two-hour "normal" meeting. The problem owner stated in the debrief of the meeting: "This was no creative session; the creativity had happened before the meeting". Furthermore, the problem owner stated that he/she missed the possibility to interact with the subject-matter experts in creative exercises to gain a deeper systems insight.

Results from Stage 2-Realizing and Applying a Method in the Industry Case
This section describes the main results from realizing and applying a method in the industry case using the research methodology presented in Section 4.2. We integrated the capabilities and impacting factors identified in Stage 1 into a method. Furthermore, we applied the method in two sessions (Sessions A and B) in the industry case for an initial evaluation. In Sessions A and B, we passively observed the systems engineers applying the proposed method. After the sessions, we conducted informal interviews with the facilitator and the problem owner.
In both sessions, we found the two first stages of the method, understanding of the problem and need and session planning, supported the problem owner to reflect and discuss the problem before conducting the session. Further, splitting the roles of the problem owner and the facilitator forced the problem owner to discuss and reflect on the problem in collaboration with the facilitator before conducting the session. The problem owner enjoyed the possibility to reflect and discuss the problem before the session. The facilitator enjoyed being involved and gaining ownership of the problem.
In Session A, both the problem owner and facilitator found the method provided valuable structure to the session. The structure pressured the participants to share perspectives and explore problems and solutions in a brief time. The timeboxing avoided the participants getting lost in detailed discussions. However, we observed during planning that they failed to plan for an individual exercise and did not include this in the session. In Session B, we observed that the problem owner and the facilitator had difficulties planning for a creative problemsolving technique fitting to the context of the session. They ended up planning to conduct a creative technique on the go. The session was highly talkative, the problem was unclear, and the facilitator could not conduct a creative technique on the fly. We found that both the facilitator and the problem owner appreciated the method during the debrief. However, both missed using a creative problem-solving technique and would plan to use one next time. The facilitator stated that he/she found it challenging to apply a creative exercise as the purpose of the session was unclear. As the facilitator said during the debriefing of Session B, "without creativity techniques, it will only be a lot of talks".

Discussion
This section answers the research questions and compares and contrasts the results with existing work in the body of knowledge. The end of this section provides contributions to theory and implications for practice. RQ1: How may co-creative problem solving support the systems engineers in rapid learning and early validation? Guided by eight success criteria, we planned and conducted nine sessions in the industry case to gain insight into how to realize a method for co-creative problem solving. We identified three main capabilities for a new method: problem and solution exploration, collective creation, and creative thinking. We scored the sessions to gain further insights into how the sessions supported the capabilities and argue that the identified capabilities may support the systems engineers in rapid learning and early validation towards high-tech innovations. Figure 4 illustrates the primary connections between the success criteria on the left and the capabilities on the right. The solid lines illustrate a strong connection between the success criteria and the capabilities, while the dotted lines illustrate a weaker connection. method during the debrief. However, both missed using a creative problem-solving technique and would plan to use one next time. The facilitator stated that he/she found it challenging to apply a creative exercise as the purpose of the session was unclear. As the facilitator said during the debriefing of Session B, "without creativity techniques, it will only be a lot of talks".

Discussion
This section answers the research questions and compares and contrasts the results with existing work in the body of knowledge. The end of this section provides contributions to theory and implications for practice. RQ1: How may co-creative problem solving support the systems engineers in rapid learning and early validation? Guided by eight success criteria, we planned and conducted nine sessions in the industry case to gain insight into how to realize a method for co-creative problem solving. We identified three main capabilities for a new method: problem and solution exploration, collective creation, and creative thinking. We scored the sessions to gain further insights into how the sessions supported the capabilities and argue that the identified capabilities may support the systems engineers in rapid learning and early validation towards high-tech innovations. Figure 4 illustrates the primary connections between the success criteria on the left and the capabilities on the right. The solid lines illustrate a strong connection between the success criteria and the capabilities, while the dotted lines illustrate a weaker connection.  The importance of a proper problem and solution exploration towards innovation is well supported in literature [55,56,58,60,62,63,77]. Bonnema, Veenvliet, and Broenink [61] (p. 9) identified solution focus as a challenge among engineers in development teams and claimed that "many engineers think in solutions". We identified a timeline to support a structured problem and solution exploration and thus prevent such fixation. The timeline supports a proper problem exploration in the first half and solution exploration in the latter. Furthermore, we found it important that the participants held the required knowledge to conduct problem and solution exploration. Such knowledge typically includes insight into customer needs by focusing on the customer, systems operational context by focusing on the user, and enabling technology and business case by showing business potential.
Co-creation and co-design focus on collective creativity in value creation [1] and design development [9]. Systems-centric approaches such as systems architecting and systems engineering emphasize collaboration, creativity, multiple perspectives in concept generation [42,43], and systems development [44,50,51,54]. To support collective creation, we aimed for knowledge sharing of early phase concepts and strived for early failure by sharing multiple perspectives using techniques such as CAFCR [54]. We also applied techniques to support the participants in grasping complexity, such as Gigamapping [76].
Furthermore, we applied a variety of means to enable creativity and support creative thinking. Human-centric approaches, such as design thinking, emphasize creativity to explore innovative concepts [25]. Firstly, the sessions applied engaging and creative techniques, such as brainwriting [75]. Secondly, the sessions applied time constraints using timeboxes [53] to push the participants to engage and create. Thirdly, the timeline combines individual and group exercises to ensure all participants share their perspectives and build on each other's ideas and thoughts. Paulus et al. [57] emphasized the importance of combining individual and group exercises to enhance creativity. Finally, the sessions applied visualizations to stimulate creativity, either in the form of collective creation of sketches or preparing visualizations before the session. Kerzner, Goodwin, Dykes, et al. [79] highlighted the importance of visualization in creative workshops.
RQ2: How may the systems engineers apply co-creative problem solving in an industrial setting? Building on the findings from exploring the nine sessions, we realized a method consisting of a three-stage process and the timeline to support the systems engineers to achieve the capabilities in the industry case. The three stages are an understanding of the upfront problem and need, session planning, and the actual session, while the timeline combines five main tasks for a structured problem and solution exploration. Furthermore, the timeline aims to create a collective understanding and shared ownership of both the problem and solution space.
The method splits the roles of the problem owner and the facilitator. Paulus et al. [57] and Muller [52] described a similar approach for creative groups and workshops. The problem owner holds the system's insight and need for progress on the problem and has the incentive to take on this role. The facilitator leans on the problem owner for support during the session to share knowledge of the problem. Furthermore, the method makes use of brief sessions using inexperienced facilitators. The systems engineers in the industry case lacked facilitation experience, and shorter sessions are often easier to facilitate than full-day workshops. Using internal facilitators may also enable free discussions without confidentiality issues and avoids expensive facilitators who do not hold the necessary systems knowledge. Facilitation skills are not necessarily that common among engineers. However, this may vary in different organizations and cultures. We foresee the need for more experienced facilitators when including external and unfamiliar stakeholders such as customers. Conducting brief sessions with a few familiar participants may provide un-trained facilitators with facilitation experiences in a safe environment.
While exploring sessions, we experienced stakeholders reluctant to attend more extended sessions due to practical reasons such as lack of cost account and busy workdays. Aiming to cope with such practicalities, we strived to balance an acceptable session length in the industry case with the time needed to gain valuable insight through the timeline. We found sessions with a duration of about two hours to provide valuable outcomes when planned well, including about six participants familiar with the session context. Sessions including several participants unfamiliar with the session context may require a longer time.
RQ3: What may be the main challenges for the systems engineers to adopt co-creative problem solving as a new way of working? After realizing the method, we applied it in the industry case in two sessions, Sessions A and B, for an initial evaluation. In Session A, we found the method supported the systems engineers in planning and conducting a structured problem-solving session with stakeholders. In Session B, we observed the facilitator and the problem owner failing in the upfront planning, resulting in a session with participants mainly talking. In both sessions, the facilitator and the problem owner strived to include participants holding the needed knowledge for early validation of essential aspects such as business case, value proposition, and enabling technology. Due to the context of the industry case, we could not include external stakeholders, such as customers. We would have liked to include external stakeholders to evaluate how the method supports systems knowledge through co-creation with beneficiaries such as customers or users. The problem owner and the facilitator need to consider the necessity and possibility of customer inclusion during planning. When observing the systems engineers applying the method without interference from the researchers, we found that they neglected creative thinking during planning. In Session A, they did not include the individual exercise. In Session B, they failed to plan for a creative problem-solving exercise. The problem owner and the facilitator appreciated the method's ability to plan and structure the session. They stated that they missed using a creative exercise in debrief of Session B.
McFadzean found the usability of a method focusing on collective creativity in engineering teams dependent on the participants' willingness and ability to engage in such activities [23,24]. This way of working is unfamiliar to some people and challenges their comfort zones and their perception of acceptable ways of working. Setting session rules or informing participants upfront of what creative problem-solving techniques are may support people to feel less uneasy and prepare for engaging in this new way of working. While exploring the sessions, we observed that once the participants familiarized themselves with creative problem-solving techniques, most seemed to approve and appreciate this way of working. McFadzean [23,24] proposed a framework to ease the selection of techniques based on needs and level of experience. We propose to make use of McFadzean's framework [56] as a guide during session planning. By gradually building experience in using co-creative problem solving, we expect the effectiveness of the method to increase. Jones [7] emphasized the importance of continuity and investment in this way of working to provide insight into complex problems. Following the recommendations by Jones, we propose a systemic use of the method to enhance co-creation between systems engineers and stakeholders and support a long-term outcome.

Contributions to Theory
This article contributes to the body of knowledge by extending co-creation in systems engineering and proposes an industrial-relevant method for systems engineering to apply co-creative problem solving. We identify and define three main capabilities for co-creation in systems engineering to support rapid learning of systems knowledge in a context of socio-technical complexity. By exploring and scoring co-creative sessions in the industry case, we gain insights into achieving the capabilities in an industrial setting. We map the formerly identified success criteria to the capabilities and propose that these criteria can be means for future validation.

Implications for Practice
The proposed method indicates partial support for the systems engineers in an indepth understanding of the problem and a structured problem and solution exploration. In the context of a mature Norwegian high-tech industry accustomed to efficiency and risk aversion, we foresee the method supporting this industry in exchanging knowledge and perspectives leading to rapid learning of systems knowledge and early validation of systems design. Thus, the method offers a step towards innovations that create value for beneficiaries such as customers, users, and businesses.

Conclusions
Guided by eight success criteria, this article explores co-creative problem solving through nine sessions in a real industry case. Furthermore, this article identifies and defines three main capabilities to provide rapid learning of systems knowledge in the concept phase towards innovations. These capabilities are problem and solution exploration, collective creation, and creative thinking. Building on the findings from exploring the nine sessions, we propose a method to support systems engineers using a three-stage process and a timeline for conducting co-creative sessions to achieve the capabilities. Our findings from an initial evaluation imply that the method partly supported the systems engineers in a structured problem and solution exploration focusing on multiple perspectives and shared ownership. Furthermore, our findings imply that the main challenges in applying the method are the systems engineers' experience in using creative problem-solving techniques and the willingness to engage in such activities.
Previous research on co-creation includes value co-creation in the marketing domain and co-creation in other domains such as design, focusing on collective creativity between designers and stakeholders (co-design). This article aims to extend co-creation in systems engineering to support systems engineers in rapid learning of systems knowledge to cope with complexity in systems development. Collaboration and creativity have been found to be essential in systems development, such as during concept evaluation and systems synthesis. However, co-creation in systems engineering is less explored. This article looks towards systems and design practices and proposes a co-creative method between systems engineers and stakeholders. The method enables the exchange of knowledge and perspectives and offers a step towards innovations that create value for beneficiaries such as customers, users, and businesses.

Limitations and Future Research Directions
Action research forms the basis of this research in a longitudinal study of about two years within a development team in the concept phase of an innovative complex system of systems. Action research and similar participatory approaches are applied in research on systems engineering to gain an in-depth understanding of industry challenges and improve. However, these research approaches increase the risk of researcher bias and challenge the generalization of research results. We strived for valid research results by focusing on triangulation and rich data collection.
We focused our research primarily on one industry case. Hence, we cannot claim that our findings nor the proposed method fit other contexts and needs than described in this article. Due to the context of the industry case, we could not include external stakeholders in the initial evaluation of the proposed method. We propose further research to evaluate how the method support rapid learning of systems knowledge by including external stakeholders such as customers or users.
Based on our research, the company established a new work process for their innovation projects, including the proposed method. The willingness to adopt the method in the industry case indicates that the systems engineers found the method valuable and aim to continue to use it. However, we need further research over a longer time to fully evaluate how such a method supports the systems engineers to achieve the identified capabilities. We propose to continue the use of the success criteria as means for future validation. Acknowledgments: The authors are grateful to the people in the Deep Purple team for being our guinea pigs in this research.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results. Table A1 provides an overview of the statements and the NPS scores in the survey used in S1 to S5. The leftmost column shows the success criteria (SC) that the statements derive from. The rightmost columns indicate the most significant promoters and detractors in bold (NPS ≥ 3, NPS ≤ −3).