Lessons Learned from the Introduction of Systems Engineering

Abstract

The introduction of Systems Engineering is an approach for dealing with the increasing complexity of products and their associated product development. Several introduction strategies are available in the literature; nevertheless, the introduction of Systems Engineering into practice still poses a great challenge to companies. Many companies have already gained experience in the introduction of Systems Engineering. Therefore, as part of the SE4OWL research project, the need to conduct a study including expert interviews and to collect the experiences of experts was identified. A total of 78 hypotheses were identified from 13 expert interviews concerning the lessons learned. Using exclusion criteria, 52 hypotheses were validated in a subsequent quantitative survey with 112 participants. Of these 52 hypotheses, 40 could be confirmed based on the survey results. Only four hypotheses were rejected, and eight could neither be confirmed nor rejected. Through this research, guidance is provided to companies to leverage best practices for the introduction of their own Systems Engineering and to avoid the poor practices of other companies.

1. Introduction

The complexity of products and product development is increasing owing to various trends in engineering such as digitalization, globalization, and sustainability. Companies are facing new challenges due to the change from traditional mechatronic products to autonomous, interactive, and dynamically networked products. In particular, the increasing interdisciplinarity of their systems and the increase in networking with other systems are drivers for growing complexity in product development. One approach to mastering this complex and interdisciplinary product-development process is Systems Engineering (SE) [1]. “Systems Engineering is a transdisciplinary and integrative approach to enable the successful realization, use, and retirement of engineered systems, using systems principles and concepts, and scientific, technological, and management methods” [2]. The companies that introduce Systems Engineering expect to gain added value as a result. This added value is to be created through an improved understanding of the overall system, e.g., in the enhanced ability to detect errors in product development earlier, parallelize development activities, and ensure traceability and transparency in the development process. Despite all the potential benefits that Systems Engineering offers, it is currently only used in a few pioneering companies across the entire product life cycle and across the various company divisions [1]. The distribution of Systems Engineering is very much industry dependent [3]. Systems Engineering is much more established in the aerospace and automotive industries than in the mechanical engineering and automation sectors. In addition, Systems Engineering is used more frequently in large companies than in small and medium-sized enterprises [1]. Medium-sized companies (SMEs) have limited resources, making it particularly difficult to access and build up SE expertise. In addition, the mechanical and plant engineering sector has a very heterogeneous structure, and therefore the requirements regarding Systems Engineering also vary from company to company [4,5].

In the following work, we discuss the challenges to, and lessons learned from, companies using Systems Engineering that can already be found in the literature. Working from these conclusions, the following research study was set up. The focus of this research is explicitly on Systems Engineering. Related topics such as the need for automotive companies to comply with standards such as ASPICE [6] are not discussed. However, in practice, these topics should be analyzed in a combined manner, as they are highly interdependent. An example of such a combined view is the suggestion to use SPICE compatible process evaluations for processes as a basis for a modern SE maturity model [7].

1.1. Current Lessons Learned from the Introduction of Systems Engineering in the Literature

As regards related studies on Systems Engineering, there are several studies that discuss the benefits and challenges of Systems Engineering in Germany, e.g., [1,3,8]. The study “Engineering in Germany—The status quo in business and science, a contribution to Advanced Systems Engineering” [1] is the most recent one, from the year 2021. Taking all the studies into consideration, the challenges companies face during the introduction of Systems Engineering can be divided into the topics of organizational structure, qualification, motivation, general conditions, and procedure. The introduction of Systems Engineering represents a deep change within a company and requires a suitable organizational structure. The changes affect many people, and companies struggle to manage multitudes of stakeholders. The successful implementation of Systems Engineering requires a comprehensive understanding of SE as well as of internal processes within the company [1]. The qualifications of employees and the lack of in-house expertise are also a challenge. Most employees do not have the expertise required for Systems Engineering [1]. The needs of stakeholders in terms of depth and breadth of expertise must be considered [9]. The importance of employees’ qualifications has increased significantly in recent years [10]. Qualifying and building internal expertise is therefore an important step for companies to take when introducing SE. A significant obstacle is the lack of intrinsic motivation [1] and the lack of acceptance both among employees and in the entire organization [3]. However, the willingness of all stakeholders is required for a successful implementation. Currently, companies lack suitable motivators and best practices for dealing with employees during the introduction of SE [1]. Moreover, the introduction is limited by general conditions. For example, companies expect that the introduction of Systems Engineering will have a negative impact on their day-to-day business. In addition, a change in the component- and discipline-oriented mindset in the company is required. Furthermore, companies need recommended procedures for the introduction of SE [1]. While there are many different approaches to introducing SE, the standards and guidelines for Systems Engineering do not provide companies with the support they need [11].

In respect of Systems Engineering introduction strategies, a barrier for many companies in the introduction of SE is the lack of adequate and practical approaches to introduction [3]. In previous research, different SE introduction strategies were developed. In BRETZ’s dissertation [7], available implementation strategies are discussed. It is noticeable that these approaches do not contain any information about their testing or evaluation. An exception to this is the dissertation by SCHIERBAUM [11]. SCHIERBAUM’s systematics include an overview of the potential benefits of Systems Engineering for mechanical engineering, a process model, a requirements analysis, and a toolbox that provides process steps, methods, and roles, and their interrelationships. These guidelines were developed and validated in pilot projects. Furthermore, the step-by-step guide of the “Special Interest Group Systems Engineering Implementation” (SIG SEI) of INCOSE Netherlands was the result of experiences and best practices from previous implementations. A rough guideline for the introduction of Systems Engineering was developed, which is divided into eight steps [12]. Nevertheless, we are unable to find a statement about the evaluation of the approach to introduction.

As regards lessons learned in respect of the implementation of Systems Engineering, a review of the literature detailing lessons learned from introducing SE into organizations reveals that such content is rare. The 2016 study by Fraunhofer IESE “Systems Engineering study: challenges and best practices” [8] provides some approaches. The research can be divided into three main parts. In the first part, previous studies are summarized and presented (

Table 1. Overview of the investigated studies on Systems Engineering in [6].

Name of the study	Publisher	Year	Content
Model-Based Systems Engineering (MBSE) Methodologies	Jet Propulsion Laboratory and the California Institute of Technology	2016	to provide a cursory description of some of the leading model-based Systems Engineering (MBSE) methodologies used in industry
Improving the Integration of Program Management and Systems Engineering	MIT Consortium for Engineering Program Excellence	2012	to give an overview of the understanding of 3000 INCOSE members (systems engineers) and 5000 PMI members (program managers) about how program management and Systems Engineering are integrated within their organization
Systems Engineering in Industrial Practice	Fraunhofer IPT, Heinz-Nixdorf-Institute, Unity Consulting & Innovation	2013	to gain a clear representation of the capability of Systems Engineering and to obtain the current level of use of SE in practice and in activities in training and further education
Systems Engineering Effectiveness	National Defense Industrial Association Systems Engineering Division (NDIA-SED) with the Institute of Electrical and Electronic Engineers Aerospace and Electronic Systems Society (IEEE-AESS) and the Software Engineering Institute (SEI) of Carnegie Mellon®	2012	to identify SE best practices used in projects, collect performance data on these projects, and identify relationships between the application of these SE best practices and project performance
Model-Driven Development	Fraunhofer IESE	2013	to investigate how to use model-driven development approaches in the practical development of software and systems

).

The studies and surveys presented in [8] did not meet the goal of Fraunhofer IESE. Therefore, in a second step, a separate study was set up, the objective of which was to “collect the state of the practice regarding Systems Engineering in the German-speaking region, focusing on challenges and solution approaches in terms of best practices (work processes, methods, and tools)” [8]. Twenty interviews with eighteen different companies were carried out. A survey of the status quo was carried out; it included such questions as which future challenges still exist in the companies, which language is used in them, and where the greatest potential for improvement is still perceived. In the last part of the study, selected industry cases are presented, and the individual lessons learned from the chosen cases are summarized. However, the study does not focus on concrete lessons learned in respect of the introduction phase of Systems Engineering. Furthermore, a shortcoming of the study is that the statements were only made by a small group of participants and were not validated in a quantitative analysis.

We found a further source during our research on lessons learned. The publication from the year 2000 by COOK, “What the Lessons Learned from Large, Complex, Technical Projects Tell Us about the Art of Systems Engineering” [13], examined projects over a 15-year period. The goal was primarily to find a value-for-money argument for the application of SE. Again, it can be concluded that this study focuses more on potential cases where it makes sense to introduce Systems Engineering. The study provides reasons why Systems Engineering delivers added value. Concrete lessons learned in respect of the introduction phase of Systems Engineering—suggested in the title of the publication—are not in fact given here either.

1.2. The Need for Action and the Objective of the Study

It can be concluded that previous studies offer only rudimentary lessons learned on the introduction of Systems Engineering [8,13]. The main focus of the known studies [1,3,8,13] is generally the provision of an overview of the current state of Systems Engineering in specific companies. Nevertheless, the introduction of Systems Engineering represents a demanding transformation process for companies, which is associated with numerous challenges [1]. Current introduction strategies often lack methods, tools, and the consideration of change management. The companies also identify the introduction methods as inadequate, thus highlighting a barrier to the introduction of Systems Engineering [14].

In order to solve the challenges faced by companies, the cooperative research project SE4OWL (Systems Engineering for OstWestfalenLippe) aims to create a set of tools for the company-wide introduction and maintenance of Systems Engineering in medium-sized companies [14]. The project SE4OWL involves three companies from the field of mechanical and plant engineering in Germany. These three companies already have SE expertise and are contributing their experience to the project. However, the depth and breadth of their SE experience as well as their company-wide requirements are very different. This reflects the complexity of SMEs and the challenges they face. We identified the fact that, in order to achieve the project goal and to provide SMEs with tailored assistance in the introduction of Systems Engineering, the experience of experts in the introduction of SE needs to be captured. This journal publication is intended to fill the gap identified in the literature on lessons learned in respect of the implementation of Systems Engineering.

Accordingly, we designed an expert study to examine experiences of the introduction of Systems Engineering. The aim was to derive hypotheses from expert interviews and to validate them with the help of a questionnaire. Qualitative research is an important aspect of this endeavor. Suitable lessons learned in respect of SE introduction are to be collected in order either to be reused or to be avoided in future introduction projects. This requires the experience of experts who have previously accompanied or performed a Systems Engineering introduction. On one hand, these experts can report on factors that can influence the introduction of Systems Engineering, and on the other hand, they can reflect on lessons that they have learned in their previous introduction projects. Based on these experiences, hypotheses can be formulated that will form the basis for the subsequent quantitative research and the toolkit that builds on it. The subsequent quantitative research should ensure that others can also confirm and thus validate the lessons highlighted by the experts. The findings can firstly be applied and reflected upon by companies themselves and secondly be transferred back to the research project to assist in the development of further strategies for assisting SMEs.

2. Methodology

Our research design is divided into two segments. In order to support companies with the introduction of Systems Engineering, a qualitative interview study was conducted. In the second part, the results were verified in a quantitative study. The following section discusses the methodologies that were used in the qualitative and quantitative research.

2.1. Qualitative Interview Study

Explorative expert interviews were conducted between 13 July and 11 August 2021, as part of the SE4OWL project. For this purpose, 13 experts were interviewed online via the Microsoft Teams platform using a semi-structured questionnaire about their experiences with the introduction of Systems Engineering. This interview was structured as follows:

• Personal data. We asked, for example, what function the person has in the current company and how many introduction projects the person has been involved in.
• Information about the project(s). Subsequently, the projects were discussed in more detail by asking about the general conditions under which they were carried out (e.g., what the scope of the project was, whether it a small pilot project, who initiated the project, etc.).
• Lessons learned. There followed explicit questions about lessons learned (in respect both of best practices and worst practices). For this, we had a collection of guiding questions:
  • What best practices were you able to derive from your introduction projects?
  • What was most difficult during the introduction? What was difficult?
  • What went well? What went smoothly?
  • What reactions were noticeable during the change? Were there any particular negative/positive reactions from specific roles? How were these dealt with?
  • Which general conditions should be ensured so that an introduction can succeed?
  • Does the introduction depend on certain factors?
  • How should an SE introduction project be set up?
  • Should the topic of qualifications be considered during the introduction? If so, what are the lessons learned?
  • Are there best practices for sequencing processes, (roles), methods, and tools?
  • Should a company directly start with model-based development (MBSE)?
  • Who is now responsible for SE in the company? Have new roles been created?
  • Are all roles to be held accountable in the rollout? Which stakeholders should be involved?
  • What are you doing to ensure the topic of Systems Engineering is evolving?
  • What is your position on the topic of Digital Twin Product Lifecycle Management? Is this topic related to SE for you?
• Conclusion and outlook. Participants were still given the opportunity to highlight or add information. Finally, they were informed about the progress of the study.

As the study was exploratory, no detailed questions were formulated, and only some basic questions were defined. In the interviews, the experts’ answers were then addressed, and questions were asked that could be derived from these answers. Participation in the interviews was voluntary.

The experts interviewed all have several years of Systems Engineering experience, with their average length of experience being nine years (

Table 2. Overview of the experts for the qualitative interview study.

Interview	Function at the Current Company	Industry Sector	SE Experience in Years	Gender
1	Senior expert	Scientific research	9	female
2	Senior expert	Scientific research	4	male
3	Managing Director	Software development	7	male
4	Division Manager SE	Mechanical engineering	10	male
5	Team Leader MBSE	Technical consulting	15	male
6	Division Manager	Management consulting	11	male
7	Requirements Engineer	Mechanical engineering	7.5	male
8	Agile Coach and SE Employee	Automotive industry	20	male
9	Area Manager Virtual Product Development	Electrical appliances	6	male
10	Research Associate	Scientific research	3	male
11	Senior Vice President	Medical technology	12	male
12	SE Employee	Automobile industry	5	male
13	SE Coordinator	Electrical appliances	7	male

). They belong to different industries and have different functions in their companies. Most of the experts have introduced SE projects in different companies. Some experts have only implemented SE introduction projects in their current companies.

As regards data analysis, MAYRING’s qualitative content analysis was selected for the evaluation of the expert interviews. This analysis method is suitable for the development of hypotheses [15] and offers the advantage of an evaluation routine for its rules-based approach, the process model, and the step-by-step material processing. Thus, this analysis is applicable to a larger amount of material [16]. Since the interviews represent a large amount of material, qualitative content analysis according to MAYRING was preferred to other methods of analysis, such as the grounded theory according to GLASER and STRAUSS.

In the qualitative content analysis according to MAYRING, the source material is first determined and the research questions of the analysis are defined. The research questions are derived from the identified need for action: What lessons are learned from the introduction of Systems Engineering? What should be done or avoided when introducing Systems Engineering? Subsequently, the individual interpretation steps of the analysis were determined. The general process model, which is given by MAYRING, must be adapted to the respective question and the material [15]. In the analysis, a mixed form of deductive category application and inductive category formation was used. First, the source material was roughly structured in terms of content using a category system in order to achieve an overview of the large amount of information. The categories are derived from the challenges of the companies; they are: structural organization, qualification, motivation, general conditions, and procedure. However, some of the experts’ answers were very broad in terms of content owing to the openly formulated questions. Subcategories were therefore subsequently formed inductively. The analysis was carried out step by step using the process model shown in Figure 1.

2.2. Quantitative Interview Study

Based on the qualitative interview study, a quantitative verification of the hypotheses was carried out. However, the number of hypotheses formulated was too large to allow them to be validated in the form of a questionnaire. Therefore, criteria were established with the project team to select the most relevant hypotheses. Accordingly, the hypotheses were checked in terms of redundancy due to existing hypotheses, comprehensibility, and SE specificity. If they did not meet the requirements, they were not included in the questionnaire. This process reduced the number of hypotheses to be queried to 52.

The questionnaire first collected personal data, such as age, SE experience, and work location. There was no query in respect of the industrial sector. Subsequently, the hypotheses on the categories of organizational structure, qualification, motivation, general conditions, and procedure were queried (cf. explanation in Section 1.1). At the end of the questionnaire, participants were able to provide further comments, questions, or remarks in a free text field. Owing to the large number of different hypotheses, the questionnaire was designed with closed questions and predefined answers in which the participants could indicate their disagreement or agreement with the respective hypotheses. An ordinal scale with six response levels was used for this purpose (see Figure 2). Every question starts with this: “In my experience, for the introduction of Systems Engineering, it is useful to …” and continues with the respective hypothesis, which in this case is: “… use external help at the beginning (e.g., consulting in process analysis, coaching, etc.)”.

The questionnaire was distributed during webinars on the topic of “Lessons Learned from the Introduction of Systems Engineering” and via LinkedIn. One of the webinars was an international webinar organized with INCOSE 1 and GfSE 2 on 30 March 2022, and one was a webinar via the VDI 3 association in Germany on 7 April 2022. Both organizations address the Systems Engineering community. Participants were able to engage online via links. The period for participation in the survey was limited to the period from 30 March 2022 to 13 May 2022. In this time, the questionnaire was completed in full by 112 participants. Using the collected data, a univariate analysis was performed to accept or reject the hypotheses. Furthermore, the correlation between the different age, experience, and work-location groups and their level of agreement or disagreement was examined using several bivariate analyses. For this purpose, the absolute and relative frequencies of the answers were calculated first. Excel was used for this analysis. Since the response options are ordinal scales, the median was calculated for the evaluation of the hypotheses. Arithmetic operations are not suitable for ordinal data [17]. However, the median can be taken as an adjusted mean for ordinal qualitative characteristics and is a valid statistical measure [17,18]. Therefore, the response options were initially coded with numbers from 0–5, with 1 representing “disagree” and 5 representing “agree”. The 0 represented “no statement”. If the value was between 0.5 and 2, the hypothesis was rejected, for 0, 2.5 and 3, the hypothesis was neither accepted nor rejected, and for all values above 3.5, the hypothesis was accepted.

3. Results

3.1. Qualitative Interview Study

A total of 309 core statements from the 13 expert interviews were analyzed. These could be summarized into 164 different statements, of which 94 were unique statements. Some of these unique statements could be summarized, resulting in 78 hypotheses, which were assigned to the categories in Figure 3.

Approximately half of the hypotheses could be assigned to the “procedure” category. Regarding motivation, 22 hypotheses were identified. Eight hypotheses were formulated in the qualification category and five each in the organizational structure and general conditions categories.

3.2. Quantitative Study

In the following section, the results of the quantitative study are described in more detail. These results are again divided into the familiar categories. The motivation category is subdivided into acceptance building and communication. The procedure category is divided into preparation, initiation, processes, methods, and instruments, as well as roles, continuity, parallel topics, and collaboration. The following diagrams always show the absolute frequencies of the answers of the 112 respondents.

Table 3. Overview of the personal data of the 112 participants in the quantitative study.

Age in Years	Number	SE Experience in Years	Number	Work Location	Number
20–30 30–40 40–50 50–60 60+	12 28 32 31 9	1–2 3–5 6–9 >10	14 27 25 46	Germany Europe international	64 19 29

gives an overview of the personal data of the 112 participants. More than half of the participants came from Germany. Some 25% of the participants came from the international community; the remainder were from Europe. It should be emphasized that 41% of the participants had more than 10 years of experience in Systems Engineering. This fact in particular ensures that the results of the study are of a high quality. In addition, deviations between the groups were examined, but these were predominantly small and did not reveal a clear pattern. Nevertheless, stronger deviations are pointed out in the corresponding thematic sections. Here, the deviations within disapproval (0.5–2), neutrality (2.5–3), or agreement (3.5–5) are not examined in detail, as they are not relevant for this research.

The following section discusses the individual lessons learned in more detail. The results are shown in Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12. Figure 4 shows the questionnaire results for the hypotheses on the “organizational structure”. The graph shows the absolute frequencies of the response options “agree”, “somewhat agree”, “neither agree or disagree”, “somewhat disagree”, “disagree”, and “no statement” for the individual hypotheses.

From the graph (see Figure 4), it can be seen that 59 respondents agreed with the statement “… use external help at the beginning“. Of the remaining respondents, 31 somewhat agreed with it, 15 neither disagreed nor agreed, 6 selected “somewhat disagree”, none chose to disagree, and one person opted for “no statement”. The median is 5, which means that this hypothesis is accepted.

The hypothesis “…build up internal systems engineering competencies in the course of the introduction” achieved a median of 5, as did “…define roles and responsibilities” and “…have a top-level manager who advocates and drives systems engineering”. The hypothesis “…locate systems engineering organizationally centrally in the company” achieved a median of 4. Thus, all of the organizational structure hypotheses are accepted. As mentioned above, we also examine at this point whether age, SE experience, or location has an influence on the experts’ assessment. For this topic area there were no major deviations between the answers.

Based on the median, which is between 4 and 5, all the hypotheses of the category “qualification” (see Figure 5) can also be confirmed. Here, there were also no particular irregularities with regard to the group examinations of age, SE experience, or place of work.

In the category “Motivation—Building up acceptance” (see Figure 6) only two hypotheses were accepted (“…build up acceptance for systems engineering by demonstrating its benefits” (median 5) and “…address the problems of employees with systems engineering” (median 4)). The hypotheses “…build up acceptance by showing that not much will change” (median 3) and “…neglect/particularly promote certain stakeholders in the process of building up acceptance” (median 3) could neither be confirmed nor refuted. However, it could be refuted that employees should be forced to deal with Systems Engineering (median 2) and that employees who resist the introduction of Systems Engineering too strongly should be terminated (median 2).

In this category (see Figure 6), there was an irregularity in the analysis of the different groups. For the second hypothesis “…build up acceptance by showing that not much will change”, the groups with less experience (0–2; 3–5 years, median 2.5; 2), the youngest groups (age 20–30; 30–40) and the oldest group (age 60+), as well as the international community, tended to reject the hypothesis with a median of 2. The last statement “…address the problems of employees with systems engineering” (Figure 6) was rated by the 20–30 age group with a median of 5. In contrast, it was rated 3 by the 50–60 age group. Thus, this hypothesis would be rated “agree” by the 20–30 age group, whereas the 50–60 age group would tend to have a neutral opinion.

In the category “Motivation—Communication” (see Figure 7) almost all hypotheses except two could be confirmed. The second statement, “…use videos to convey the introduction of systems engineering in small companies”, could neither be confirmed nor refuted. The hypothesis that the term Systems Engineering should be avoided was rejected. Its rejection reflected the highest amount of disagreement of all the hypotheses within this quantitative research, and it achieved a median of 1.

For the fourth statement (see Figure 7), “…not overestimate the possibilities of systems engineering”, the 20–30 age group and the people with less experience (0–2; 3–5 years) tended to be neutral towards this statement, with a median of 3. All the other groups recorded a median of 4 and tended to agree with this hypothesis.

Based on their median of 5, the two hypotheses “…provide enough resources” and “…to promote mentoring and a coaching mentality” from the “General Conditions” category (see Figure 8) can also be confirmed. There were no irregularities in the analysis of the different groups.

In the category “Procedure—Preparations” (see Figure 9), each hypothesis was also confirmed. Every hypothesis reached a median of 5 except for the fourth one, “…create a systems engineering management plan (“SEMP”, according to INCOSE)”, which has a median of 3.5. This is because some groups were rather neutral towards this statement. Age groups 20–30, 30–40, 60+, and work location groups from Germany, as well as persons with 3–5 years of SE experience achieved a median of 3 while for the other groups it was 3.5 or 4.

In the category “Procedure–Initiation” (see Figure 10), almost all hypotheses could be confirmed. Only the two hypotheses “…start the initiation with reverse engineering” and “…introduce systems engineering holistically” could not be confirmed, but nor could they be rejected. The other hypotheses reached a median of 4 or 5 and were therefore accepted.

In this category (see Figure 10, there were noticeable differences of opinion between the participant groups. The statement “…introduce systems engineering holistically” was approved for the age groups 20–30, 30–40 and 50–60, while the others (40–50, 60+) were neutral to this statement. Respondents with 6–9 years of experience were also the only group neutral to the hypothesis. The other groups scored a median of 3.5 or 4 and thus would have agreed with the statement. Regarding the work location, it was found that the international community would also have agreed with a median of 4. For the statement “…start the initiation with reverse engineering”, the participants with SE experience of 6–9 years, age 50–60, or with a European work location tended to reject this statement (median 2), whereas the other groups were neutral (median 3).

In the category “Procedure—processes, methods, and tools” (see Figure 11), only one of the five hypotheses could be confirmed. The results of the questionnaire showed that the order in which the topics of processes, methods, and tools should be approached could not be clarified because both statements reached a median of 3, which means that the two hypotheses regarding the order can neither be accepted nor rejected. Only the hypothesis that the topics should be considered in synergy could be confirmed, having a median of 4. The statement “…keep the tool in the introductory phase” could also neither be confirmed nor rejected with a median of 3. The hypothesis “…take a complete tool package as a small or medium-sized company” has the highest number of “no statement” responses of the whole survey with 30 statements. This hypothesis was also rejected based on its median of 2.

In the category “Procedure—Processes, methods, and tools” (see Figure 11) there were noticeable variances of opinion. The answers differ greatly between the different groups and also change from acceptance of the hypothesis to rejection of it according to the median of the different groups. Processes, methods, and tools is the preferred order for those with considerable experience, as the statement was adopted by the 6–9 years and 10+ years groups with a median of 4 and 3.5. The less experienced (0–2 years) preferred the order of methods, processes, and tools and reached a median of 4 there. Age also shows differences in terms of order. The 60+ age group agrees with the statement “…proceed in the order processes, methods, and tools” with a median of 4, while the others are neutral to it. For the statement “…proceed in the order methods, processes, and tools”, the level of agreement varies greatly. The 20–30 and 50–60 age groups disagree with the statement with a median of 1.5 and 2, the 30–40 and 60+ groups are neutral to the statement with a median of 3, and the 40–50 group agrees with the statement with a median of 4. Regarding work location, respondents from Germany are neutral to the order. The European community prefers the order of processes, methods, and tools (median 4) and rejects the other order (median 2). The international community prefers the order of methods, processes, and tools (median 4) and is neutral to the other order (median 3).

The statement “…to keep the tool in the introductory phase” was approved by the group with 3–5 years of experience (median 4), while those with 6–9 years of experience rejected it (median 2). The other experience groups were neutral to the statement (median 3). The 20–30 age group agreed with this statement with a median of 4. The other experience groups were more neutral to the statement with a median of 2.5 and 3. When comparing the work location, it is noticeable that the European community rejected this hypothesis with a median of 1. The other groups are neutral to the statement with a median of 3.

In the last section of the questionnaire, “Procedure—Roles, continuity, parallel themes and collaboration” (see Figure 12), all hypotheses could be confirmed. The median ranged between 4 and 5. There were no irregularities in the analysis of the different groups.

In evaluating all the results, we examined whether the distribution differed according to experience, age, or location of the participant. In most cases, there were only minor variations in the level of agreement with the statements of the different groups. The greatest differences occurred in the “Procedure—Processes, methods, and tools” section (see Figure 11), where the medians between the group showed a difference of up to 2.5. Otherwise, the respondents’ experiences in introducing Systems Engineering are similar. Therefore, it can be concluded that the experiences of the respondents in introducing Systems Engineering are similar. Neither the age of the respondents nor the number of years of experience matter significantly. Furthermore, the location of the Systems Engineering introduction is not significant, as the results of the experience are very similar for Germany and Europe, and internationally.

Other findings that could be concluded from the respondents’ comments are briefly mentioned in this section. These comments could not be verified in the survey. Therefore, they are only intended to provide additional ideas. First, it was noted that SE or MBSE is indispensable in mastering complexity, regardless of the size of the company. Furthermore, the team within the implementation process was mentioned. This team is essential and its members should be compatible in terms of personality, attitude, and motivation. The team members should have broad and complementary knowledge. Regarding the question of whether SE should be centrally located or whether there should be an SE department, it was suggested that there should be a mix. The respondent stated that he used a central group with long-term assignment to projects. This allowed for a central exchange for systems engineers to share lessons learned on the different projects and enabled the program managers to have a Systems Engineering resource. Regarding motivation and preparation, it was also noted that systems thinking must be strengthened and should also be reflected in the objectives. In addition, middle management should be convinced of the value of SE and should support the introduction project. The results and their achievements should also be celebrated in order to improve employee motivation.

4. Discussion

In this article, the process of identifying lessons learned for the introduction of Systems Engineering and the quantitative verification of the hypotheses were presented. The findings of this study are based on 13 in-depth interviews with experts who have an average of 9 years of SE experience. When interpreting the results of the expert interviews, it should be noted that, although the interviews were conducted with individuals from various industries, not all sectors are specifically covered. Consequently, certain insights from these industries could be missing; this may have a profound influence on the introduction of Systems Engineering in the industry sector in question.

Further limitations concern the collection of quantitative data. One respondent indicated that his answer of “neither disagree or agree” meant that it was situation dependent. In the quantitative survey, this situational dependency could not be queried, but all the hypotheses were kept universal. Therefore, the results only provide general guidance for adoption and may not produce the desired outcome in certain situations. Similarly, there was no question about the industry in which the participants worked. Therefore, no analysis can be made of the differences between the hypotheses from participants who carried out SE introductions in small or medium-sized companies and the hypotheses from participants who carried out SE introductions in large companies such as the automotive industry. Nevertheless, it should be noted that the answers among the participants were very similar. It is therefore doubtful whether a deviation in the hypotheses presented could be due to differences in the participants’ industrial sectors.

Future research could refer to particular industries or situations in which the introduction is located. This could provide specific guidance for companies in a particular industry and in a particular situation. Lessons learned must therefore be carefully considered before they are applied in a company.

5. Conclusions

The complexity of products and product development is increasing owing to various trends, and this confronts many companies with challenges. One approach to mastering the complex and interdisciplinary product development process is Systems Engineering. From 13 expert interviews conducted in the SE4OWL research project, a total of 78 hypotheses on lessons learned for the introduction of Systems Engineering could be formulated. These hypotheses were grouped into the following categories: “organizational structure”, “qualification”, “motivation”, “general conditions”, and “procedure of the introduction”. For the quantitative analysis, these hypotheses were reduced to 52, based on exclusion criteria. Of these 52 hypotheses, 40 could be confirmed, based on the survey results. Only four hypotheses were rejected, and eight hypotheses could neither be confirmed nor rejected. It was also determined that there are no significant overall differences between the distribution of the answer options and thus the agreement or disagreement with the hypotheses. This applies equally among different age groups and among participants with limited or extensive experience, and also at national, European, and international level.

This research provides support through knowledge, gained through experience, about how the introduction of Systems Engineering can be carried out more successfully by using best practices. This will assist companies to control their increasingly complex product development processes with the help of a successful introduction of Systems Engineering. Furthermore, the study now provides input in the research project focused on providing assistance to SMEs that is tailored to their needs.