Managed Evolution of Automotive Software Product Line Architectures: A Systematic Literature Study

Abstract

The rapidly growing number of software-based features in the automotive domain as well as the special requirements in this domain ask for dedicated engineering approaches, models, and processes. Nowadays, software development in the automotive sector is generally developed as product line development, in which major parts of the software are kept adaptable in order to enable reusability of the software in different vehicle variants. In addition, reuse also plays an important role in the development of new vehicle generations in order to reduce development costs. Today, a high number of methods and techniques exist to support the product line driven development of software in the automotive sector. However, these approaches generally consider only partial aspects of development. In this paper, we present an in-depth literature study based on a conceptual model of artifacts and activities for the managed evolution of automotive software product line architectures. We are interested in the coverage of the particular aspects of the conceptual model and, thus, the fields covered in current research and research gaps, respectively. Furthermore, we aim to identify the methods and techniques used to implement automotive software product lines in general, and their usage scope in particular. As a result, this in-depth review reveals that none of the studies represent a holistic approach for the managed evolution of automotive software product lines. In addition, approaches from agile software development are of growing interest in this field.

1. Introduction

The automobile has become the most technically complex consumer product [1]. The fulfillment of increasing customer requirements and strict legal requirements with regard to the reduction of fuel consumption and pollutant emissions, as well as the higher demands on safety and new driver assistance systems resulted in a steady increase in the deployment of onboard electronics systems and software. Software-intensive systems and functions are the major drivers for innovations in cars today [2]. In premium vehicles, software is responsible for up to 80% of the innovation [3]. Electronics and software account for up to 40% of the production costs of a car [4].

The requirements for automotive electronics differ significantly from other areas of consumer electronics (cf., e.g., [5,6]). For automotive functions, hard real-time requirements exist while the storage and computational power on ECUs have to be kept as small as possible. Systems in an automobile must be reliable, safe, and secure in all situations as system failures can endanger human life. In addition, long product life cycles exist: the OEM has the duty to offer service and spare parts for at least 15 years after the purchase of a vehicle due to the long product life cycles. In contrast, software is changed at comparatively short intervals. During the production period and even during the development phase, many new versions of a piece of software are developed. As a consequence of short innovation and long life cycles, a huge number of versions and configurations exist, which inter alia makes maintenance very difficult.

In order to reduce software development costs, the automotive industry aims for a high degree of software reuse. Reuse is achieved both through cross-product development for different vehicle variants and through reuse in subsequent products. However, the increasing complexity and variability of automotive software systems is making it increasingly difficult to reuse and extend these systems. From one vehicle generation to the next, functionality differs mostly not more than 10%, while much more than 10% of the software is rewritten [6]. This has led to a growing interest in software product line approaches to automotive systems [7].

Since the 1990s, software product lines have been introduced as a major addition to existing reuse approaches [8,9,10,11]. Clements et al. [11] define a software product line (SPL) as a family of systems that share a common set of core technical assets, with preplanned extensions and variations to address the needs of specific customers or market segments.

In general, software product line engineering consists of two key processes, domain engineering and application engineering [12]. The aim of the domain engineering process is to define and realize the commonality and the variability of the software product line. The process of application engineering is responsible for deriving product line applications from the platform established in domain engineering by exploiting the variability of the software product line [12].

Nowadays, many challenges in the area of automotive software product line development are mentioned in literature [7,13] (see Section 2.1.1). Motivated by the huge set of challenges in this domain, many studies exist proposing particular methods and techniques to support the product line-driven development of software in the automotive sector (see, e.g., [13]). However, these approaches generally consider only partial aspects of development. An overview of the set of available methods and techniques would help researchers and practitioners in the field of automotive software product line engineering. As Raatikainen et al. [14] state in their study, there is a need for a combination of existing solutions in the area of software product lines rather than for novel approaches. This study examines existing approaches with respect to a holistic model, so that one gets an order along a process view (which is domain specific). For the introduction of a software product line approach it is crucial to know which activities (process steps) are supported by it and which are not.

Moreover, existing studies either focus on the domain of automotive software engineering in general, or they address the whole field of software product line engineering without regarding the specific requirements of the automotive domain. Thus, a study providing a holistic view of software product line engineering in the automotive domain is still missing.

1.1. Research Approach and Contribution

In this paper we present an in-depth literature review based on a conceptual model of artifacts and activities for the managed evolution of automotive software product line architectures.

Our study is grounded in [13]. For this, following Kitchenham et al. [15] and Petersen et al. [16,17], we use [13] as a so-called scoping study that we utilize to investigate a specific topic in more detail. Since the study in [13] only considers papers until 2015, we have additionally included relevant papers from 2016 to 2021 in our study.

We aim at collecting information about the current state-of-the-art holistic approaches for managed evolution of automotive software product lines and use our conceptual model as a reference model to evaluate the state of the art. We are interested into the coverage of the particular aspects of the conceptual model and, thus, the fields covered in current research and research gaps, respectively. Furthermore, we aim to identify the methods and techniques used to implement automotive software product lines in general, and their usage scope in particular. The paper supports engineers to get an overview of existing methods and techniques in automotive software product line engineering and helps researchers to find research gaps.

1.2. Outline

The paper is organized as follows: Section 2 provides a conceptual model of artifacts and activities on automotive software product line engineering and summarizes the related work. In Section 3, we describe our research design, and present the results of our study in Section 4. We conclude the paper in Section 5.

2. Related Work and Background

2.1. Related Work

In this section, we give an overview of the related work. We regard existing surveys on automotive software engineering and software product lines in general. Subsequently, we define a conceptual model for the managed evolution of software product line architectures as a basis for our systematic literature study.

2.1.1. Automotive Software Engineering

Haghighatkhaha et al. [13] published a systematic mapping study that analyzes and classifies the literature related to the field of automotive software engineering. This review includes 679 articles from multiple research sub-areas, published between 1990 and 2015. They analyze research activities, topics, types, and methods and reveal research gaps in automotive software engineering. A classification of the 679 articles is listed in an excel sheet including general information for each article such as abstract, title, year, authors, and research topic.

Pretschner et al. [6] provided a comprehensive overview of the state of the art in automotive software engineering. They identified research challenges in automotive software engineering, in particular the integration, evolution, maintenance, and reuse, and explored the potential benefits of a seamless model-based development process as a possible solution. Furthermore, the study provides a roadmap for research in this area.

Clarke et al. [18] identified important areas in software engineering that will have a significant impact on future automotive systems. The authors introduced global software development, software product lines, service-oriented architectures, and mathematical approaches applied to software engineering as possible future research directions for the automotive domain.

Grimm [19] discussed major challenges of automotive software engineering and the most important technological core competencies required to meet these challenges. Among other things, future work will focus on the following fields: elaboration of a software product line approach for future in-vehicle software architectures, model-based development of distributed systems, and integration of processes, methods, and tools from the different areas of mechanical, electrical, and software engineering.

Gruszczynski [20] gave an overview of software engineering technologies in the automotive industry and identified future research directions. Fabbrini et al. [21] provided a picture of the achievements and the open issues in the European automotive industry and suggested future research directions.

Thiel et al. [7] presented some challenges that automotive engineering faces today and discuss contributions software product line approaches could make to provide solutions for these challenges.

Antinyan, in his paper “Revealing the Complexity of Automotive Software” [22], identified four aspects that an approach to automotive software evolution must mitigate. These are the complexity of the requirements, the complexity of source code, the architecture complexity, and the complexity in creating variants. All these areas are represented by the conceptual model (see Section 2.2), which serves as the baseline of this study.

2.1.2. Software Product Line Engineering

Since the 1990s, software product lines have been introduced as a major addition to existing reuse approaches [8,9,10,11]. Clements et al. [11] define a software product line (SPL) as a family of systems that share a common set of core technical assets, with preplanned extensions and variations to address the needs of specific customers or market segments.

Pohl et al. [12] propose a holistic approach on software product line engineering consisting of two key processes, domain engineering and application engineering. The domain engineering process aims at defining and realizing the variability and commonality of the software product line. The process of application engineering aims at deriving product line applications from the platform established in domain engineering by exploiting the variability of the software product line [12].

An important activity in software product line engineering constitutes variability management [11,12]. Most existing approaches in variability management can be classified as feature modeling and decision modeling [23]. The main difference between the approaches is that feature modeling supports both commonality and variability modeling, whereas decision modeling focuses exclusively on variability modeling [24].

In the following, we present surveys on various topics of SPL:

Harman et al. [25] presented a survey on search-based software engineering (SBSE) for SPLs and highlighted some directions for future work. They identified the most active areas in SBSE for SPLs: SPL testing, SPL feature selection, product line architecture (PLA) improvement, and SPL feature extraction.

Furthermore, several surveys on product-line testing have been conducted: Engström and Runeson [26] presented a systematic mapping study on SPL testing. The main challenges identified in the paper are the large number of tests, the balance between effort for reusable components and concrete products, and handling variability. Lee et al. [27] surveyed the current SPL testing approaches and highlight the challenges and key research perspectives in SPL testing. They defined a reference SPL testing processes as the survey framework. Oster et al. [28] also addressed SPL testing and presented a survey on the state of the art of model-based testing (MBT) approaches for SPL. They defined a conceptual process model for SPL testing which is used for a comparison of the different testing approaches. Furthermore, they highlighted the challenges and open research topics in SPL testing.

Thüm et al. [29] proposed a classification of SPL analyses and survey and classify 123 existing approaches for the analysis of SPL. They also provided a research agenda to guide future research on SPL analyses.

Chen et al. [30] discussed the findings from a systematic literature review in variability management in SPL. They presented the chronological backgrounds of various approaches in variability management and identified certain gaps that need to be filled by future research: among other things, they concluded, that only a few approaches address systematic process support for variability management and that there is only limited support for evolution of variability. In addition, they stated the inability of most approaches to scale to large and complex product lines.

Schobbens et al. [31] presented a survey of existing feature diagram variants. Based on the regarded feature diagram variants, they proposed a generic formalization of the syntax and semantics of feature diagrams.

Additionallly, recent systematic literature studies [14,32,33] are still identifying research gaps. Chacón-Luna et al. [32] criticize the accuracy of empirical studies in particular and the fact that most of the approaches are evaluated on artificial scenarios. Marques et al. [33] also note that while case studies are prevalent, only a few industry studies of an appropriate size are publicly available. Furthermore, SPL approaches cannot be compared due to a lack of consensus regarding their formalization. Raatikainen et al. [14] suggest that the goal must be not to develop new and novel SPL methods in the future but to make better use of the existing ones in terms of actionability, context-sensitivity, and evidence and quality in the resulting syntheses.

2.2. Conceptual Model for Managed Evolution of Automotive Software Product Line Architectures

As discussed in [34] classical holistic approaches to software product line engineering, such as [12], have to be adapted to the special requirements of the automotive domain. In the automotive sector, it is not possible to carry out all further developments within the product line. Rather, there may be further developments that do not take place in the product line but at the level of the individual products. The reasons for this may be the high time and cost pressure, but also the fact that sophisticated further developments are initially to be tested within the scope of a prototype implementation. These further developments, which are separate from the product line, have to be transferred into product line development at a later stage. This goes beyond the classical domain/application engineering approaches, such as [12].

In addition, we are often facing an eroded software architecture. Thus, we first have to repair the architecture as discussed by Cool et al. [35]. Architecture repair typically involves the two approaches of recovery and discovery. Recovery is based on reverse engineering techniques by which the implemented architecture is extracted from source artifacts whereas discovery hypothesizes its intended architecture [36].

A challenge is to minimize architecture erosion in the long term: the product line architecture is designed initially and develops over time. Further development must ensure that the product architecture remains consistent with the product line architecture. Thus, in order to prevent architecture erosion in the future, architecture conformance checking is required for all further developments.

Based on these prerequisites, we use a conceptual model with activities for managed evolution of automotive software product line architectures (cf. [34,35]) as shown in Figure 1. Figure 1 depicts in the left part the activity of discovery and recovery. This activity aims at repairing an initially eroded software architecture. It is performed once before the long-term development cycle can begin. The long term evolution cycle consists of two layers: The cycle Product line (PL) (see Figure 1) contains the activities for the development of the product line. The cycle Product (P) includes the activities for the development on the product specific level. These two cycles enable a parallel development, and even the activities within one cycle can be performed independently. The dependencies of an activity on artifacts of the previous activity are indicated by the circular arrow in the middle of the two cycles. Despite this, a parallel execution of single activities is also possible: a new PLA can be developed in activity PL-Design while an implementation is realized in activity PL-Implement An external decision-making process is necessary to start a new prototyping or to transfer a prototype implementation back to the product line. Therefore, we have introduced the large, blue arrows between the two layers.

Next, we will briefly introduce the activities of the conceptual model in Figure 1.

Table 1. Explanation of the activities in Figure 1.

Activity	Objective	Input/Output
Recovery and Discovery	Recovery of the implemented PLA from the source artifacts (developed products) and discovery of the intended PLA.	Input: Source artifacts (developed products). Output: Implemented and intended PLA.
PL-Requirements	Specification and validation of software system and software component requirements by requirements engineering.	Input: Requirements. Output: Software system and software component requirements.
PL-Design	Further development of PLA with consideration of design principles. Application of measuring techniques to assess quality of PLA.	Input: Software system/component requirements and documentation from product development. Output: New PLA (called “PLA vision”).
PL-Plan	Planning of a set of iterations of further development toward the PLA vision taking all affected projects into account.	Input: PLA vision. Output: Development plan including the planned order of module implementations and the planned related projects.
PL-Implement	Implementation including testing as specified by the development plan for product line development.	Input: Development plan for product line. Output: Implemented module versions.
PL-Check	Minimization of product architecture erosion by architecture conformance checking based on architecture rules.	Input: Architecture rules and set of implemented modules to be checked. Output: Check results.
PL to P	Defining a project plan by selecting a project from the the product line.	Input: Development plan for product line. Output: Project plan.
P-Requirements	Specification of special requirements for a certain vehicle product including vehicle related parameter settings.	Input: Requirements in particular from calibration engineers. Output: Vehicle related requirements.
P-Design	Designing product architecture and performing architecture adaptations taking product specific requirements into account. Compliance checking with PLA to minimize erosion.	Input: Project plan and product specific requirements. Output: Planned product architecture.
P-Plan	Definition of iterations to be performed on product level toward the planned product architecture.	Input: Product architecture. Output: Development plan for product development.
P-Implement	Product specific implementations including testing as specified by the development plan for product development.	Input: Development plan for product development. Output: Implemented module versions.
P-Check	Architecture conformance checking between PLA and PA.	Input: Architecture rules and set of implemented modules to be checked. Output: Check results.
P to PL	Providing product related information of developed product for integration into product line development.	Input: Developed product. Output: Product documentation and implementation artifacts of developed products.

(cf. [34,35]) summarizes the objectives of the 13 activities, including input and output artifacts.

The cycle for the product line architecture needs as input the requirements from requirements engineering (PL-Requirements) and all the artifacts developed on product level. The activities PL-Plan and PL-Design serve to design, plan, and evolve the PLA.

In PL-Implement, the planned implementation artifacts are implemented on level of the product line, whereas in P-Implement the implementation of the product specific artifacts takes place. To create a fully functional software status for a particular vehicle project, the project plan is transferred (PL to P), which contains module descriptions and descriptions of the integration plan of the logical product architecture with the associated module versions.

Furthermore, special requirements for a particular project are taken into account (P-Requirements). To build a new product, we start with an initially planned PA derived from the product line (P-Design). Iterations to be performed are planned in P-Plan. An iteration contains the planned implementations and the elements of the PA.

Each planned project refers to a set of implementation artifacts, called modules constituting the product architecture (PA). P-Check and PL-Check minimize PA erosion by architecture conformance checking. In P-Design, we apply architecture conformance checking to check conformance between the planned PA and the PLA.

3. Research Design

In this section, we describe our research design. Section 3.1 describes our overall approach, followed by the research questions in Section 3.2. Section 3.3 describes the data collection procedures. Finally, in Section 3.4, we describe the analysis procedures and discuss the validity procedures implemented.

3.1. Research Method

The study at hand presents an in-depth literature study, which is grounded in [13]. For this, following Kitchenham et al. [15] and Petersen et al. [16,17], we use [13] as a so-called scoping study that we utilize to investigate a specific topic in more detail. Since the study in [13] only considers papers until 2015, we have additionally included relevant papers from 2016 to 2021 in our study. To investigate our research questions (Section 3.2), we implemented the following procedure:

Step 1

In the first step, we analyzed the scoping study [13] and carried out a data cleaning based on the inclusion/exclusion criteria resulting from the research questions.

Step 2

In the second step, we removed multiple occurrences of papers in the data set following the steps described in [37].

Step 3

On order to include further relevant papers from 2016 to 2021, we collected papers by defining and applying a search string. Similar to steps 1 and 2, we carried out a data cleaning on the collected papers.

Step 4

In the fourth step, we read the papers and applied the rigor-relevance model as proposed by Ivarsson and Gorschek [38].

Step 5

The fifth step comprised a quality assessment following the approach described in [39] (see also Appendix B).

Step 6

In the sixth step, we prepared the in-depth review. In this regard, we used a refined conceptual model for software product lines [34], which we used to further classify the papers in the data set.

Step 7

Finally, we used the aforementioned conceptual model and the categorized papers to conduct the in-depth review to answer the research questions.

The research approach above was implemented in a team of researchers with well-defined task distribution to improve the validity of the findings.

3.2. Research Questions

With the paper at hand, we aim to understand the current state-of-the-art in holistic approaches for a managed evolution of software product line architecture in automotive software engineering (

Table 2. Research questions addressed with the study at hand.

	Research Question and Rationale
RQ${}_1$	What Is the current state of the art in holistic approaches for managed evolution of automotive software product lines? We aim at collecting information about such holistic approaches and use our conceptual model (cf. Section 2.2) as a reference model to evaluate the current state of the art. Specifically, we are interested into the coverage of the particular aspects of the conceptual model and, thus, the fields covered in current research and research gaps, respectively.
RQ${}_2$	What particular methods and techniques are used to implement a managed evolution of automotive software product lines? We aim to identify the methods and techniques used to implement automotive software product lines in general, and their usage scope in particular. For this, we analyze the available literature and categorize and evaluate the contributions found according to a given schema (Section 2.2).

). Specifically, we define our working hypothesis as follows: There is no holistic approach for a managed evolution of automotive software product lines.

3.3. Data Collection Procedures

In this section, we describe the data collection procedures. The data collection is based on a previously conducted study [13], which we use as a scoping study. In the following, we provide a brief summary of the scoping study’s contribution in Section 3.3.1, before we describe the selection process of papers relevant to the study at hand in Section 3.3.2. In addition, we describe how we included papers from 2016 to 2021 Section 3.3.3, as the conducted study in [13] only considers papers until 2015. Finally, in Section 3.3.4, we describe how we selected the final set of papers for analysis.

3.3.1. Overview of the Scoping Study

As basic data source, we use the scoping study [13] and the complementing published dataset. The scoping study has collected papers on automotive software engineering, which were categorized into the seven research areas listed in

Table 3. Automotive software engineering research areas according to ISO/IEC 12207 PRM as found in [13].

Research Area and Topics	Studies Per Area
Agreement Processes (AGR) Agreement Support Group	5
Organizational Project-Enabling Processes (ORG) Organizational Project-Enabling Support Group	48
Project Processes (PRO) Project Support Group	14
Technical and Software Implementation Processes a (ENG/DEV) System/Software Architecture and Design (131 studies). System/Software Qualification Testing (127 studies). Software Implementation (62 studies). System/Software Integration (44 studies). System/Software Requirement Engineering (35 studies). Software Construction (22 studies). Software Maintenance (18 studies).	439
Software Support Processes (SUP) Software Verification and Validation (71 studies). Software Quality Assurance and Review (48 studies). Software Documentation and Configuration Management (3 studies).	122
Software Reuse Processes (REU) Software Reuse	72

^a SUMMARIZED.

.

In total, the scoping study [13] includes 679 papers, which were categorized into seven research areas and, finally, were broken down into 14 specific research topics. This dataset served as input for the data collection, which is explained in detail in the following sections.

3.3.2. Study Selection from the Scoping Study

As introduced in Section 3.2, the scope of the study at hand is the field of software product lines. For this, we used the scoping study and applied a multi-staged selection procedure for studies of interest:

(1)

Analyze the scoping study’s data and identify all papers on software product lines (incl. synonyms):

(a)

Selection of papers based on title and abstract;

(b)

Selection based on keywords (while testing the selection criteria, we decided to exclude the keywords reusability, reuse, derive, and feature, since they generated too many false positives) (incl. variants, upper/lower case, etc.): SPL, family, reference architecture, variability, variant model, variability management, feature model, feature tree, feature-oriented, derivate;

(c)

Result: This stage resulted into 87 paper (candidates); most of the papers selected are in the REU group (Table 3).

(2)

Rating of the study candidates (Section 3.3.2)

(a)

Application of the rigor-relevance model [38];

(b)

Definition of a threshold and selection of papers;

(c)

Final data cleaning and preparation of the in-depth analysis.

(3)

In-depth analysis (Section 3.4)

3.3.3. Collection and Selection of Papers from 2016 to 2021

To include papers in this work that were published after 2015, we performed another search for relevant papers. For this purpose, we defined a suitable search string and applied it to Google Scholar. In particular, we performed the following steps:

(1)

Definition of a suitable search string.

(2)

Result:“automotive software” AND “software product line”.

(3)

Application of the search string to Google Scholar for the years 2016–2021.

(4)

Result: This stage resulted in 263 papers (candidates).

(5)

Selection of papers based on title and abstract, done by two reviewers.

(6)

Result: This stage resulted in 20 papers (candidates).

To distinguish the papers from the scoping study [13] and the selected papers from 2016 to 2021, we gave the prefix P to the ID of the scoping study’s papers and the prefix E to the papers of the extended search.

3.3.4. Final Paper Selection

In the final paper selection, we analyzed the 107 selected candidate studies and, in a first step, applied the rigor-relevance model as proposed by Invarsson and Gorschek [38]. This model grades papers on the two parameters rigor (FROM-TO) and relevance (FROM-TO). To select the studies of interest, we defined the threshold $rigor+relevance\ge 2$ to ensure that:

(1)

All high-quality papers (i.e., high-scored papers) are in the result set;

(2)

Papers that have a high relevance score but a poor rigor score are included;

(3)

Medium-scored but balanced papers are included.

Applying the rigor relevance model to the candidate studies, we selected 56 studies. These studies were (again) checked with a particular focus on multiple occurrences. In order to finally clean the dataset, we decided to prefer journal papers to conference papers, as we assume follow-up special issue papers to have a higher maturity/quality. Eventually, we selected 51 primary studies for inclusion into the data analysis. The final evaluation and classification of the different studies is summarized in

Table 4. Evaluation and classification of the primary studies selected for analysis.

Id	Ref	RTF a	CTF b	Rigor	Relevance
P8	[40]	Evaluation	Model	1.0	2.0
P22	[41]	Solution Proposal	Framework/Method/Technique	1.5	2.0
P43	[42]	Evaluation	Lessons Learned	3.0	3.0
P71	[43]	Evaluation	Framework/Method/Technique	1.5	2.0
P72	[44]	Evaluation	Lessons Learned	3.0	3.0
P79	[45]	Evaluation	Framework/Method/Technique	1.0	2.0
P86	[46]	Evaluation	Lessons Learned	2.0	4.0
P90	[47]	Evaluation	Model	2.0	3.0
P94	[48]	Evaluation	Framework/Method/Technique	1.0	2.0
P100	[49]	Validation	Framework/Method/Technique	2.5	1.0
P129	[50]	Evaluation	Model	2.0	4.0
P173	[51]	Evaluation	Framework/Method/Technique	1.0	2.0
P215	[52]	Evaluation	Lessons Learned	2.5	4.0
P218	[53]	Experience Paper	Lessons Learned	0.5	2.0
P220	[54]	Experience Paper	Lessons Learned	0.5	2.0
P221	[55]	Experience Paper	Lessons Learned	0.5	2.0
P223	[56]	Experience Paper	Framework/Method/Technique	1.0	2.0
P279	[57]	Evaluation	Framework/Method/Technique	1.0	3.0
P281	[58]	Validation	Guideline	1.0	2.0
P285	[59]	Evaluation	Framework/Method/Technique	1.5	4.0
P289	[60]	Evaluation	Framework/Method/Technique	1.0	2.0
P300	[61]	Evaluation	Lessons Learned	1.0	2.0
P310	[62]	Evaluation	Framework/Method/Technique	1.0	2.0
P332	[63]	Evaluation	Framework/Method/Technique	1.0	1.0
P343	[64]	Evaluation	Framework/Method/Technique	1.0	2.0
P365	[65]	Evaluation	Framework/Method/Technique	1.0	1.0
P377	[66]	Solution Proposal	Framework/Method/Technique	1.0	1.0
P404	[67]	Evaluation	Lessons Learned	0.5	2.0
P468	[68]	Experience Paper	Lessons Learned	1.0	3.0
P493	[69]	Experience Paper	Framework/Method/Technique	2.0	3.0
P503	[70]	Experience Paper	Lessons Learned	0.0	2.0
P580	[71]	Solution Proposal	Framework/Method/Technique	1.0	3.0
P588	[72]	Evaluation	Framework/Method/Technique	1.0	3.0
P589	[73]	Evaluation	Framework/Method/Technique	1.0	2.0
P660	[74]	Evaluation	Framework/Method/Technique	1.0	2.0
P580	[71]	Solution Proposal	Framework/Method/Technique	1.0	3.0
P588	[72]	Evaluation	Framework/Method/Technique	1.0	3.0
P589	[73]	Evaluation	Framework/Method/Technique	1.0	2.0
P660	[74]	Evaluation	Framework/Method/Technique	1.0	2.0
E8	[75]	Evaluation	Framework/Method/Technique	1.0	2.0
E53	[76]	Evaluation	Framework/Method/Technique	1.0	3.0
E59	[77]	Evaluation	Lessons Learned	2.0	4.0
E66	[78]	Evaluation	Framework/Method/Technique	2.0	2.0
E127	[79]	Evaluation	Framework/Method/Technique	1.0	4.0
E130	[80]	Evaluation	Framework/Method/Technique	1.0	4.0
E132	[81]	Evaluation	Framework/Method/Technique	2.0	2.0
E173	[82]	Experience Paper	Guideline	3.0	2.0
E175	[83]	Experience Paper	Lessons Learned	2.5	2.0
E177	[84]	Experience Paper	Lessons Learned	2.0	2.0
E178	[85]	Experience Paper	Lessons Learned	2.0	2.0
E179	[86]	Evaluation	Framework/Method/Technique	1.0	1.0
E183	[87]	Solution Proposal	Framework/Method/Technique	1.0	3.0
E187	[88]	Evaluation	Lessons Learned	1.0	2.0
E224	[89]	Solution Proposal	Framework/Method/Technique	2.0	1.0
E230	[90]	Experience Paper	Lessons Learned	2.5	2.0

^a Research type facet, according to [91]. ^b Contribution type facet, according to [16,17].

.

3.4. Analysis and Validity Procedures

The analysis procedures applied to the final dataset of 51 papers also followed a multi-staged approach. In our analysis procedure, we applied a number standard analyses on the data already provided by the original scoping study [13]. In particular, we used the provided data for analyzing the research type facets [91] and the contribution type facets [16,17]. Furthermore, we applied the rigor-relevance model [38] for implementing study selection (Section 3.3.4) and further data analyses.

Grounded in these basic measures, we used the conceptual model for a managed evolution from Section 2.2 as a classification schema. Five researchers classified the papers according to the conceptual model from Section 2.2, which we used as a classification schema. For each element in the classification schema, the individual researcher had to state whether or not a study makes a major contribution to a specific element, e.g., product line design or product implementation. The results were integrated and checked for agreement using Fleiss’ $\kappa$ [92]; the overall agreement in the studies’ classification was 0.72 (substantial agreement). This classification resulted into three categories:

Category 1

This category includes those papers for which all five researchers agreed that a study has a major contribution in a specific category. Finally, 8 papers were assigned to this category.

Category 2

This category includes those papers for which three or four researchers agreed that a study has a major contribution in a specific category. Finally, 30 papers were assigned to this category.

Category 3

This category includes those papers for which one or two researchers only agreed that a study has a major contribution in a specific category. Finally, 13 papers were assigned to this category.

The categories above are used in further in-depth analyses to, notably, evaluate the specific contributions and the maturity of the contributions of the individual studies. Finally, we used the classification schema from Section 2.2 and the researchers’ ratings to generate a mapping table, which guides the in-depth content analysis.

As we reused an already published dataset resulting from an independently conducted scoping study [13], we implemented several measures to improve the validity of our findings. First and foremost, we relied on researcher triangulation, i.e., we always ensured that one researcher from the team was not involved in a task and performed the quality assurance of that very task, e.g., in the rating of the contributions of the study (Section 3.4), five researchers evaluated the studies and a sixth researcher checked the evaluations and computed the agreement levels. Furthermore, we shuffled the teams, e.g., data collection, quality assessment, and data analysis were performed by different teams of two to three researchers.

4. Study Results and Discussion

This section summarizes the results of our analysis. First, we provide general demographic information about the dataset, before we answer the individual research questions. Finally, we discuss our findings in Section 4.4 and discuss the threats to validity in Section 4.5.

4.1. Result Overview

After the study selection, 51 papers remained in the result set. Figure 2 illustrates the publication frequency including the 3-year trend (red line) and the 5-year trend (black line). The general publication frequency shows a growing interest in the research on automotive software product lines starting in 2011.

Based on the initial classification from the scoping study [13], we evaluated the remaining 51 papers for their research and contribution type classification, which is illustrated in Figure 3. This classification shows the majority of the papers contributing frameworks, methods and techniques, and that most studies are classified as evaluation research. It has to be noted that the papers from our result set have been filtered using the rigor-relevance model (see also Table 4). Hence, due to the selection procedure as described in Section 3.3.4, we expected mostly papers of type evaluation research, and result set contains 32 papers of type evaluation research, which we consider an indication towards practically relevant contributions. Furthermore, 12 more papers from the result set have been classified as experience papers, i.e., practical experience regarding automotive software product lines have been reported.

In a nutshell, we argue that our paper selection strategy resulted in a dataset that comprises those studies reporting practical relevant knowledge about automotive software product lines, which we investigate in more detail in the following sections.

4.2. State of the Art in Holistic Approaches for Managed Evolution of Automotive Software Product Lines

In this section, we aim to answer research question RQ${}_1$ (Table 2). As described in Section 3.4, five researchers classified the papers according to the conceptual model from Section 2.2. The classification was used to generate a mapping table for the in-depth content analysis. Based on this mapping table, Figure 4 shows the assignment of the papers to the conceptual model from Section 2.2. In Figure 4, a paper is annotated to an element if at least two researchers state that the paper makes a major contribution to that specific element. A paper can be assigned to more than one element if it contributes to several activities of the conceptual model.

Table 5. Assignments of the finally selected primary studies to the activities of the classification schema (Section 2.2) and the percentages of counts.

Activity	Papers	%
Recovery and Discovery	[40,48,50,51,53,59,60,70,75,78,81,88]	13
PL-Requirements	[41,45,53,54,65,66,77,81]	9
PL-Design	[40,41,42,43,45,46,47,50,51,52,53,55,56,57,58,59,61,63,65,66,67,69,70,71,72,74,75,77,82]	32
PL-Plan	[47,50,52,66,68,71,75,87,89]	10
PL-Implement	[56,62,66,69,72,76,77,87,88,89]	11
PL-Check	[49,65,69,76]	4
PL to P	[41,54,57,59,61,66,87]	8
P-Requirements	[66]	1
P-Design	[50,66,67]	3
P-Plan	[66]	1
P-Implement	[62,66,76,87]	4
P-Check	[44,76]	2
P to PL	[41,48]	2
Rest (unassigned)	[64,73,83,84,85]

shows the assignments of the selected primary studies to the activities of our classification schema. According to our schema, none of the analyzed studies presents a holistic approach for managed evolution of automotive software product lines. Yet, some studies provide a certain coverage. For instance, study P377 [66] has been assigned to nine categories thus covering three quadrants of our schema. It has also to be mentioned that this study only received a score of 1.0 in the rigor-relevance evaluation (Table 4), i.e., it has barely received the minimum score required for inclusion in the result set (cf. Section 3.3.4). The two primary studies P22 [41] and P129 [50] have been assigned four times each, and studies P218 [53], P365 [65], and P493 [69] have been assigned thrice. The remaining studies have been assigned once or twice only. In the subsequent Section 4.3 we will regard all studies proposing an “overall approach/process” in detail.

To facilitate the evaluation we count the assignments of studies to activities of the classification schema and display the percentages of counts in Table 5. In total, there are 92 assignments to the 13 activities of the classification schema, i.e., a study has been assigned on average to two activities. Two studies (P343 [64], P589 [73]) have been assigned to category “Rest (unassigned)”. Here, none of the 13 activities in our classification schema has received at least two assignments for the corresponding study.

Table 5 reveals that the contribution of the studies concentrate on PL-Design (32%), followed by Recovery and Discovery (13%), PL-Plan (10%), PL-Requirements (9%), and PL to P (8%). Based on the accumulation of studies on the initial activities of the schema, we hypothesize that the studies in question are primarily concerned with the development of new SPL rather than the long-term evolution of the product line. For example, architecture conformance checking related activities required to prevent architecture erosion in the long term are underrepresented in the studies: only four studies (P72 [44], P100 [49], P365 [65], P493 [69]) can be found which our researchers have mapped to this appropriate fields.

Remarkably few studies address the product specific development cycle (application engineering) and only study P72 [44] exclusively addresses the product specific development cycle. One reason for this could be that no specific methods and techniques are required for the product specific development cycle and, as in the classic SPL approach (cf. [12]), product development is carried out exclusively by derivation from the product line.

4.3. Methods and Techniques to Implement Automotive Software Product Lines

In this section, we aim to answer research question RQ${}_2$ (Table 2). Specifically, we identify the methods and techniques used to implement automotive software product lines in general, and their application scope in particular.

Table 6. Methods and/or techniques of the papers included in this study.

No.	Methods and/or Techniques	Papers	PL a	P b
1	Architecture evolution process	[46,50,52]	x	x
2	Cost/effort estimation	[43,48,51,74]	x
3	Safety analysis	[44,71,72,73,89]	x
4	Description languages	[67,76]	x	x
5	Architecture reengineering	[60]	x
6	Model transformation	[49,56,58,59,61,78,86]	x
7	Model-based requirements engineering	[41,45]	x	x
8	Overall approach/process	[50,53,59,66,70,75,77]	x	x
9	Reference architectures	[42,47,50,53,82]	x	x
10	SPL merging	[48,51,55,81,86]	x
11	Testing/verification	[62,64,87,88]	x	x
12	Variability management	[40,54,57,63,65,68,69,71,72,73,79,90]	x	x
13	Agile software development	[79,80,83,84,85,87]	x

^a Activities related to product line development (domain engineering) including Recovery and Discovery. ^b Activities related to product development (application engineering).

provides a categorization, which is built based on the in-depth content analysis (see

Table A2. Data extraction and study quality assessment sheet used in this study.

Category	Questions
Data Extraction	The data extraction was performed using the following key questions: Which methods and or techniques are used? Are the same methods used for domain and application engineering? Are the methods/techniques used evaluated (see quality assessment)? Is there a holistic approach? Are the different activities (Section 2.2) fully covered? Are there gaps? What are the consequences for a holistic approach?
Quality Assessment	The quality assessment was carried out following the list of questions from Kitchenham and Charters ([39], p. 28). Each question was rated on a 5-point scale with: 1 = not fulfilled at all, 2 = partially fulfilled, 3 = basically fulfilled, 4 = fulfilled to a large extent, and 5 = completely fulfilled and very well implemented/documented. Please note that the numbers for the individual questions are taken from ([39], p. 28). In Table A3, we refer to these numbers to present the quality assessment for the particular studies.

). Furthermore, the table provides an overview of the different studies and how their major contributions are applied, i.e., in the scope of the product line as such and in the context of particular products derived from that product line. In the following, we first provide an overview before we provide the detailed structured following the categories listed in Table 6.

4.3.1. Overview of Methods and Techniques Used for Automotive Software Product Lines

From the content analysis of the 51 selected primary studies (see Appendix B, Table A2), we derived 13 categories—each representing a cluster of methods and/or techniques. As already discussed in Section 4.2, on the one hand, most of the papers propose one method or technique and provide—if at all—a scoped evaluation only. On the other hand, most of the proposed methods and techniques are subject of only one paper. That is, Table 6 lists the topics of interest and, at the same time, provides some information about topics of interest in automotive software product line research, however, based on the result set, we can only derive a “fragmented” picture as several aspects are studies, yet in a fairly isolated manner. Nonetheless, the result set shows that some topics are studied from different perspectives, which indicates the variety of the subject field.

In the following sections, we discuss the different method/technique clusters individually and in more detail.

4.3.2. Architecture Evolution Process

This section discusses studies P86 [46], P129 [50], and P215 [52], which were found addressing the architecture evolution process category—notably the maintenance and the long-term evolution of software product line architectures.

Gustavsson and Eklund [46] discuss the architects’ work approach concerning the maintenance in the context of an evolving/changing product line architecture (PLA). The authors conducted a series of expert interviews with architects from Scania and Volvo Car. They found that the process for managing changes of the PLA is very similar and that the found processes from both companies can be mapped to a generic process, which consists of the five steps need, impact analysis, solution, decision, and validation. All of these findings are based on conclusions concerning the interviews and are not verified by a further case study.

The work of Lind and Heldal [50] is primarily focused on reference architectures, but also addresses the evolution of architectures. They distinguish between revolutionary and evolutionary architecture processes, which are both discussed in [50]. In particular, the authors argue that reference architectures have to be continuously maintained, notably, it is necessary to evaluate reference architectures continuously to identify bottlenecks, which is key to initiate refactoring and to support the architecture’s evolution. Hence, Lind and Heldal present an empirically grounded reference architecture development process, which contains activities that are primarily performed for evolutionary architecting, e.g., activities “Synthesize, Evaluate, and Verify & Validate Architecture”. Such activities describe the architecture design, the measurement of architecture quality, and the validation against the requirements in the context of an evolving architecture.

Axelsson [52] discusses the revolutionary architecture process (RAP) and evolutionary architecture processes (EAP). The author specifically focuses on the interplay of both processes and how the EAP is performed in practice. In an empirical study, reasons for changes in an architecture are discussed, complemented by a discussion of affected attributes, technical aspects involved, and decisions made. The study shows that RAP and EAP differ significantly. Additionally, Axelsson states that most literature mainly describes RAP, whereas EAP lack studies and evidence.

In summary, the primary studies assigned to this category use similar activities for analyzing/describing the evolution of (automotive) software product line architectures. Key is the process-support of the architecture processes, notably for architecture development and maintenance/evolution. A cross-company analysis conducted by Gustavsson and Eklund [46] did reveal similarities, which underpins the usefulness of a generalized/holistic concept.

4.3.3. Cost/Effort Estimation

This section discusses studies P71 [43], P94 [48], P173 [51], and P660 [74], which were found addressing the cost/effort estimation category. A number of studies consider the cost/effort estimation a key activity, especially in the course of planning a product line development endeavor.

Kiebusch et al. [43] propose metrics to measure the size of an automotive software product line. They argue that neither of the existing methods adequately measures the (unadjusted) size nor estimates the cost of process-oriented automotive software product lines. For this, the authors propose the process-family-points (PFP) analysis method to allow for size measurement and effort estimation.

Yoshimura et al. [48,51] address effort estimation methods with two studies: the approach proposed in [48] describes a software clone analysis approach, and the approach discussed in [51] proposes an estimation process based on the return on investment (ROI). Both approaches are integrated into a process for merge potential assessment of existing variants, i.e., how to (economically) reintegrate variants back into the product line. In [51], the authors discuss an application of this approach and present lessons learned and open issues. As lessons learned they discuss, e.g., that software cloning may not be a good way to realize product line engineering, and that ROI predictions can strongly motivate the management to invest in product line engineering, and that architecture-centric clone analysis is a useful and practical approach to assess the merge potential of the existing systems. Open issues are, e.g., clone visualization, clone refactoring, and clone error reduction.

Gustavsson and Axelsson [74] provide a method of evaluating system designs with the purpose of enabling practitioners to systematically think about the future development of a system. For this, they use the Real Options Theory [93,94], which adds the possibility to put an economic value on the system adaptability attribute and, thus, motivates architects to also anticipate the actual value of future developments of an architecture.

In summary, studies [43,51,74] use different methods to perform cost and effort estimation actives in the course of planning software product line development. Furthermore, the studies provide process models for practically applying these methods, and the studies present evaluations of the respective approaches.

4.3.4. Safety Analysis

This section discusses studies P72 [44], P580 [71], P588 [72], P589 [73], and [89], which were found to address the safety analysis category.

Three studies address safety analysis for the development of safety-critical automotive software product lines. Rana et al. [44] address the problem of selecting the appropriate software reliability growth model (SRGM) of more than 100 currently existing SRGMs. Growth models are used for evaluating the maturity or release readiness of a software before its release and, respectively, for an optimal allocation of the test resources required. Rana et al. use a statistical model to identify the distribution of defects, which helps selecting an appropriate growth model. A case study conducted at Volvo Car Group is utilized to evaluate the proposed approach. They evaluate six standard distributions on defect inflow data from four large software projects and show that beta distribution provides the best fit to the defect inflow data. Second, Käßmeyer et al. [71] present an improved safety engineering approach for software product line development. Their approach provides an integrated change impact analysis by combining their approach with variability management. They apply their approach to an industrial example, a small part of an advanced driver assistant system (ADAS), to illustrate the benefits. As demonstrated by the example, changes are propagated in one model for both variant management and safety engineering. Pett et al. [89] apply a risk-based change-impact analysis to an automotive architecture, combining risk-based testing, product sampling, and configuration prioritizing. They use information of the changes applied in a software update to prioritize relevant system variants for checking compatibility of an update to existing variants in the field. The approach is evaluated on five versions of a body comfort system.

De Oliveira et al. [72] propose an approach to support the generation of fault trees and FMEA analyses for products derived from a software product line. Their approach aims at reducing the effort required for performing safety analyses for the products. The study proposes a process model for model-based safety analysis, which starts with a product line hazard analysis, followed by a process step “Augmentation of PLA with failure logic” that describes how product line architecture design elements, i.e., product line components, can fail and how they contribute to the occurrence of hazards. Further steps include the definition of software product line configuration knowledge, product derivation, and safety assessment. In [73], de Oliveira et al. propose an approach to support the automated construction of modular product line safety cases. The approach uses different techniques such as architecture failure modeling, functional failure modeling, and component failure modeling. Note that three studies [71,72,73] are also assigned to the category variability management (Section 4.3.13) in which we provide a further discussion.

4.3.5. Description Languages

This section discusses studies P404 [67] and E53 [76], which were categorized in the description languages category. For the development of a software product line, a suitable description technique is key for supporting the specification of variability. Kim et al. [67] propose a concept to support the functional view (software behavior) in the component-based software development (CBSD). The approach introduces “signal flows” and “mode-dependent signal flows” for the specification of a component. In this regard, important information is provided at the component level to support the understanding of the software’s behavior and to also understand dependencies among software components when reusing and adapting software components. The approach is proposed in the context of product line development in the automotive sector. However, the study does not provide extensive details concerning the degree of appropriateness of the approach for product line-driven development. The approach rather addresses a more general problem of software development in the automotive sector. The study [76] introduces the so called EMAB architecture description language. The EMAB meta model consists of two views on the architecture: the logical architecture (design layer) and the technical software architecture (implementation layer). For both layers, mappings between the elements can be defined. The description language serves as a basis for an architecture conformance checking approach, also described in the study. The goal is to minimize software architecture erosion in the long-term. The benefits of the approach are demonstrated on a real world case study, brake servo unit (BSU) software system from automotive software engineering.

4.3.6. Architecture Reengineering

This section discusses study P289 [60], which addresses the architecture reengineering. Architecture erosion has become a major challenge in automotive software engineering, which results in a considerable effort for maintenance and software system evolution. Furthermore, architecture erosion is a major problem affecting software reuse [35]. In this context, Strasser et al. [60] propose an approach for reengineering an eroded software product line architecture. First, all relevant variation points and the associated functional requirements of a component are identified. A variability analysis is performed by the product line UML-based software engineering (PLUS; [95]) approach, which is also used to describe appropriate variability models. Based on the analysis results of the product line extraction process, architecture components are identified and designed in the next step. This approach can also be assigned to the recovery and discovery activity.

4.3.7. Model Transformation

This section discusses studies P100 [49], P223 [56], P281 [58], P285 [59], P300 [61], E66 [78], and E179 [86], which were assigned to the model transformation category. Several studies utilize model transformation techniques to transform artifacts of software product line development into different models, such that techniques, e.g., for model analysis, can be applied.

For instance, White et al. [49] propose an approach to debugging feature model configurations and automating configuration evolution, called CURE (Configuration Understanding and REmedy). Configurations and feature models can be transformed into constraint satisfaction problems (CSP) to automatically diagnose errors and repair invalid feature selections.

Merschen et al. [56] present a prototypical framework for the analysis of embedded software product lines. They analyze artifacts by transforming them into models, which are used in an analysis process based on model transformation languages. The automated preprocessing is implemented as model transformations in ATLAS transformation language (ATL) and epsilon transformation language (ETL).

Leitner et al. [58] introduce EAST-ADL2 in an automotive software product line including a transformation from AUTOSAR to EAST-ADL2. Basic variability information can be automatically extracted during the transformation step. Different mapping strategies are analyzed to generate a correct model and to reduce losses in the transformation process. Furthermore, they describe the implementation of the transformation process.

Polzer et al. [59] present a framework for model-based product lines of embedded systems. The framework supports the (semi-)automated extraction of models from existing requirement, test, and implementation artifacts.

Wille et al. [78] propose a variability mining procedure that semi-automatically identifies variability information in a set of related models that were realized by clone-and-own approaches. They generate a delta language automatically based on the results of the variability mining. The procedure is evaluated using IBM Rational Rhapsody state charts from an SPL and MATLAB/Simulink models from an industrial case study.

Kehrbusch et al. [86] propose an automated syntactical similarity analysis for software component interfaces to support the software product line extraction and maintenance. They want to identify identical or similar components under development and to be able to extract a generic interface for the establishment of a generic component. This generic component can then be reused in the different analyzed project contexts.

Finally, Wang [61] presents a study on the application of model transformation techniques on the development of automotive software product lines. The paper aims at understanding the state-of-the-art techniques and to identify model transformation challenges in product-line-based automotive software—notably to help developers choosing the model transformation technique appropriate for the respective situation. Wang distinguishes between model transformation at the same abstraction level and model transformation across different abstraction levels. The transformations are implemented by using the tool GReAT (Graph Rewrite And Transformation). Wang presents the results of a case study with a simplified enhanced cruise control system (eCCS). As lessons learned, he states that model transformations with well-design transformation rules yield consistent implementations across vehicle variations and can thus reduce the efforts to create and maintain the design variations for different vehicles. In addition, Wang reveals that current transformation features partially meet the needs of derivative design, and still require improvement.

In summary, the studies [49,56,58,59,61,78,86] show that model transformation techniques play a key role in the development of automotive software product lines. By means of model transformations, tools and techniques can be used in the different process steps. The effort to create and maintain design variations, e.g., for different vehicles, can be reduced. Well-design transformation rules are crucial for the effectiveness of a model transformation.

4.3.8. Model-Based Requirements Engineering

This section discusses studies P22 [41] and P79 [45], which were assigned to the model-based requirements engineering category. Requirements engineering requires appropriate support by methods and techniques to capture and manage the requirements for a software product line to maintain the high degree of variability.

For this, Gleirscher et al. [41] present an approach for integrating innovation management with requirements and technology management. Requirements-based innovations are usually motivated by newly elicited requirements or needs originating from market research, whereas technology-based innovations are motivated by new or emerging technologies (e.g., specific platform components and platform services). To identify innovations, they use models of feature hierarchies, platform service hierarchies, and a platform component models. Furthermore, to accomplish innovations they define activities for requirements-based innovations and technology-based innovations.

Aoyama and Yoshino [45] present an aspect-oriented approach for the requirements engineering in software product line development. Automotive systems often deal with a wide spectrum of interwoven functional and nonfunctional requirements. They apply a multi-dimensional aspect-oriented modeling and analysis to generate multiple software product lines for automotive systems. The approach separates intersecting non-functional requirements (NFR) into primitive concerns (aspects) and introduces quantitative metrics of each NFR/aspect.

4.3.9. Overall Approach/Process

This section discusses studies P285 [59], P377 [66], P503 [70], P129 [50], P218 [53], E8 [75], E53 [76], and E59 [77], which were assigned to the overall approaches/processes category. Several studies propose an overall approach and/or outline a process for automotive software product line development. In some cases, specific methods and techniques are proposed, such that the respective studies are also assigned to other categories in Table 6.

Polzer et al. [59] present a framework for model-based automotive software product lines in which model transformations play an important role. The framework supports the creation of context-specific views, which provide a detailed description of the domain engineering and application engineering process tasks and artifacts. The approach also addresses the recovery and discovery of an evolved product line using a model-extraction process. Finally, the authors define a product-derivation process for application engineering.

Hardung et al. [66] propose a framework to improve the reuse of automotive software. Their framework is based on the product line practice (PLP; [11]) process model. The authors explain how to perform the modularization of the product line core assets which are contained in a function repository. The process also defines how to develop products from that function repository using a standard software core. Finally, the authors list tools to support the processes.

Tischer et al. [70] present experiences regarding the introduction of product lines at Bosch Gasoline Systems. Their study outlines the different relevant areas for SPL-based development, e.g., architecture development, product-line scoping and core asset development, market-oriented development, measurement of product line success, and product quality management. Furthermore, they discuss why the product line’s deployment at Bosch has been delayed.

Lind and Heldal [50] and Eklund et al. [53] both propose an architecture-centered development approach and present detailed process descriptions. One focus of both studies is on reference architectures so that the studies are discussed in detail in category reference architectures (Section 4.3.10).

Grewe et al. [75] propose an approach for extracting, designing, and managing architecture concepts. First, they propose methods to extract initial architectures by recovery/discovery techniques. Second, they show architecture design principles/pattern that they worked out in their automotive domain projects. In addition, they suggest techniques for measuring of architecture quality. The approach is evaluated on a real world example, a longitudinal dynamics torque coordination.

Bilic et al. [77] identify and define a product line engineering (PLE) process at the engines control department of Volvo CE. For this purpose, they analyze the existing model-based systems engineering activities and discuss the implications of the migration from the current development process to a model-based PLE-oriented process. They also identify hindering factors of the current development process, e.g., ambiguous and incomplete information.

In summary, the studies of this category describe an overall development approach for automotive software product line development, but at different levels of detail and with different focal points: the studies [50,53] are architecture-centered and contain more detailed process descriptions. In contrast, [70] remains fairly vague and does not contain a process description; instead, it provides an experience report. The work in [77] introduces a model-driven process using languages such as OVM and SysML. The other studies [59,66] propose frameworks for software product line development: [59] focuses on model-based development using techniques such as model transformation. The framework proposed in [66] allows for classifying software according to possible ways of reusing it, and it links the development process to the environment, i.e., repositories and tools. However, specific guidelines for developing automotive software product lines are not provided.

Compared to our classification schema (Section 2.2), neither approach [50,53,59,66,70,77] covers all activities. For instance, recovery and discovery is only supported by [59]. Yet, the framework in [59] does not address a product-specific development of implementation artifacts. Instead, implementation is only performed during domain engineering and serves as the basis for product derivation.

4.3.10. Reference Architectures

This section discusses studies P43 [42], P90 [47], P129 [50], P218 [53], and E173 [82], which were assigned to the reference architecture category. Several studies propose reference architectures as a key element for the development of software product lines.

Martínez-Fernández et al. [42] present a survey on the benefits and drawbacks of using the software reference architecture AUTOSAR, an open industry standard for the automotive software architecture between suppliers and manufacturers. They conducted an online survey that addressed experienced AUTOSAR practitioners. They found standardization and reuse as most popular benefits, and complexity and initial investment as most remarked drawbacks and risks of using AUTOSAR.

Eklund and Bosch [47] proposed a reference architecture for embedded open software ecosystems consisting of 17 key decisions resulting in four architectural patterns. The architectural patterns are device abstraction, data and service provision, device and information composition, and safety-critical, certified and open application access. These patterns have to be instantiated as a product line architecture for platform design. Furthermore, they define quality attributes for the reference architecture: composability, deployability of new functions, stability over time, configurability, consistent user interface, and dependability. Their approach is demonstrated with a prototypically implemented architecture that satisfies selected key decisions and quality attributes.

Lind and Heldal [50] study the research question “How can a reference architecture for automotive Systems be developed in a component-based setting, and how is it utilized in product development to increase reuse?”. They focus on the definition of a development process with a reference architecture as a core artifact. The process is divided into E/E architecture development and E/E systems development. Their reference architecture is developed and maintained during E/E architecture development and serves as the basis for developing a product-specific architecture for E/E systems development. The proposed development process has been validated in several steps by projects at Saab Automobile AB. The result of the validation shows that the process works well at Saab.

Eklund et al. [53] propose an architecture-centered development process and describe how reference architectures are used to improve the development process. The authors describe the design, verification, dissemination, and maintenance of the reference architecture. Based on their experiences, they conclude that dissemination and maintenance require more resources than the development of the reference architecture. The proposed process results on the experiences with architecture-centered development at Volvo Cars.

Oliinyk et al. [82] give guidance to practitioners how to structure automotive product lines and their feature models. They analyze the strengths and weaknesses of alternatives in structuring automotive product lines and their features and investigate to what degree the selected structures are realizable with existing tool support.

Summarized, the studies [50,53] focus on the development process and how reference architectures are embedded into the development process. In contrast, the work of Eklund and Bosch [47] focus on the particular design of a reference architecture grounded in design decisions and patterns. The study [82] gives guidance on how to structure automotive product lines and their feature models. The benefits of drawbacks of the AUTOSAR reference architecture are presented in [42]. As mentioned in [47], AUTOSAR assumes an integration-centric approach in which all integration work concerning software components is done by an OEM. Yet, the authors of [47] argue that it is desirable to move away from integration-centric development of software towards a development in a open software ecosystem.

4.3.11. Software Product Line Merging

This section discusses studies P94 [48], P173 [51], P221 [55], and E132 [81], which were assigned to the SPL merging category. If independently developed systems, e.g., diesel and gasoline software systems, share a high degree of common functionalities, merging those products into one product line is reasonable.

The two studies by Yoshimura et al. [48,51] primarily focus on effort estimation techniques for software product line merging (as already discussed in the category cost/effort estimation, Section 4.3.3). Both studies also outline merging strategies. However, details about the actual process of performing the merge are missing in both studies.

Tischer et al. [55] present their experiences from merging two large-scale software product line development projects (a diesel and gasoline software system), which was motivated by the high synergy potential. The merge was performed at the different levels: organization, software architecture, development environment, and processes and methods. The study provides a comprehensive overview of the challenges and solutions chosen to merge these large-scale systems.

Ignaim and Fernandes [81] show a practical evolution-based approach to migrate and evolve a set of variants of a given product into an SPL. The approach starts with a reverse engineering phase to synthesize the feature model (FM) The FM is then upgraded and refined in the forward engineering phase. Here, the SPL is evolved by new customer requirements in an incremental way.

Kehrbusch et al. [86] extract SPL by automated syntactical similarity analysis (see category model transformation). The analysis supports the identification of similarities between two interfaces and can also be applied to monitor the evolution of software component interfaces.

4.3.12. Testing/Verification

This section discusses studies P310 [62], P343 [64], E183 [87], and E187 [88], which were assigned to the testing and verification category. Due to the high degree of variability that results in a potentially huge number of products, testing and verification of a software product line are challenging tasks.

Lochau et al. [62] present an approach of pairwise testing in the SPL context by providing a mapping between feature models and a reusable test model in the form of statecharts. Therefore, they investigate the relationship between feature-based coverage criteria and model-based coverage criteria. A further contribution is a reasoning about the applicability of this approach. The approach is validated by a case study from automotive software engineering.

Scheidemann [64] proposes an incremental process using approximation algorithms for minimizing the number of configurations needed to verify the completeness of the configuration space. For this purpose, the author introduces the concept of “locality sets” containing architectural elements that realize a functionality, which is concerned with a particular requirement. The proposed algorithms either choose the minimum set of configurations necessary to verify all requirements for all configurations, or to maximize the verification coverage of the software product line as a whole by choosing the “best” configurations. A further contribution is a technique for automatically determining commonalities in architecture and requirements.

Ebert et al. [87] present methodology called TIGRE to reduce redundant testing effort by intelligent planning and reporting of the test execution. The approach consists of two phases: the initial identification and documentation of relevant data for product line testing and a second phase focusing on the usage of this data for testing activities during each agile sprint. The study also discusses the lessons learned from applying the methodology in an industrial environment.

Shahin et al. [88] propose an approach to lifting behavior alteration analysis, including configurable visualization of analysis results. They apply an existing declarative analysis to a set of automotive software product lines from General Motors. Finally, they discuss the lessons learned throughout the project.

4.3.13. Variability Management

This section discusses studies P8 [40], P220 [54], P279 [57], P332 [63], P365 [65], P468 [68], P493 [69], P580 [71], P588 [72], P589 [73], E127 [79], and E230 [90], which were assigned to the (general) variability management category. Managing the variability within a software product line constitutes a challenging task and is addressed by nearly one third of the studies from the result set.

Jin Sun et al. [40] propose a component-based development process based on variability types. They identify several variability types that may occur in ECU-related development: variability of software components, variability of logic components, variability of sensor components, variability of actuator components, variability of setpoint generator components, variability of output device components, and variability of component interfaces. Furthermore, they propose a process for developing ECUs that focuses on managing the variability by using the introduced variability types.

Boutkova [54] present their experiences with variability management in the requirements specification process at Daimler passenger car development. Based on these experiences, Boutkova proposes a new feature-based variability management (FBVM) approach. This approach is extended by a decentralized variability modeling approach that supports variability modeling of individual components and systems as well as modeling a product as a whole.

Graf et al. [57] propose a graph-based approach for variant modeling and management. The approach focuses on managing, modeling, and combining the different kinds of knowledge in software product line development, i.e., combining local expert knowledge with domain knowledge provided by application groups, and the integration of overall management knowledge.

Brink et al. [63] propose an approach to link hardware and software variability models. Their approach distinguishes between software and hardware variants using separate variability models. They further distinguish two different kinds of dependencies between hardware and software product lines.

Millo and Ramesh [65] propose an approach to link the design-level variability with the requirements-level variability. The requirements and designs are expressed as extended finite state machines, so-called finite state machines with variability (FSMv). The variability between designs and requirements is based on a conformance relation between design and requirements models of the software product line features. An algorithm is proposed for checking conformance between the models and is implemented on the verification tool SPIN.

Manz et al. [68] propose an approach for enabling traceability, integrated configuration of software variants, and links using integrated feature modeling. An integrated feature model considers the different development phases, abstraction layers, and development artifacts.

Kato and Yamaguchi [69] present an approach for variability management focused on pair-wise feature interactions. They accumulate the occurrence of the feature interactions for all past products using a feature interaction matrix to visualize feature interactions.

Käßmeyer et al. [71] propose a process for a model-based change impact analysis. They use methods and techniques for safety analysis (see category safety analysis, Section 4.3.4) and configuration/variant management, and focus on the combination of both.

De Oliveira et al. [72] propose an approach to support safety analysis for software product lines. Their approach provides guidelines regarding the use of model-based development, safety analysis, and variability management tools. In [73], de Oliveira et al. also address safety engineering in software product lines. They propose a method to support automated construction of modular product line safety cases. The method uses outputs provided by model-based development, safety analysis, and variability management tools.

Hayashi and Aoyama [79] propose a structural analysis method of variability for multiple product lines using an extended model of OVM (orthogonal variability model). The approach is applied to agile product line engineering (APLE) of their automotive software systems (see category agile software development).

Wägemann et al. [90] conduct interviews with the automotive industry experts. They want to investigate the usefulness (or necessity) of automated PLA design space exploration for current industrial practice. The study derives a set of challenges for automated design space exploration, and distinguishes between technical issues, domain issues, and end-user issues.

Summarized, variability management is addressed by the selected studies in different ways using different methods and techniques. Several studies deal with the modeling of variability (e.g., [40,54,57,63,65,68,69,79] and with the tracing of variability information across different levels of abstraction, such as requirements and design [65], hardware and software [63], or safety analysis assets and design assets [71,72,73]. Three studies focus on the combination of variability management and safety analysis and are thus also classified in category safety analysis [71,72,73] (Section 4.3.4).

4.3.14. Agile Software Development

This section discusses studies E127 [79], E130 [80], E175 [83], E177 [84], E178 [85], and E183 [87], which were assigned to the agile software development category. Agile development is also gaining importance in the area of product lines and promises many benefits.

Hayashi and Aoyama [79] (see category variability management) show how software product line engineering and agile software development can be integrated towards agile product line engineering (APLE). After studying existing work, they see a lack of variability management in the APLE and thus address this in their study. To demonstrate their approach APLE is applied to the development of automotive system using ultrasonic sensors at DENSO. In a further study [80], Hayashi et al. propose an agile development method for multiple product lines by iteratively reusing process assets in application engineering.

The three studies by Hohl et al. [83,84,85] investigate the mapping of agile development to automotive software product lines. The challenges for the adoption of agile development are mainly of organizational, technical, and social nature [83]. For example, dependencies between departments and suppliers are challenging and must be considered [84]. However, agile software product line engineering is promising and can add value to existing development approaches [85].

Ebert et al. [87] apply their TIGRE method (see category testing/verification) in an agile development process. Here, they test the product line in agile sprints. As lessons learned, the study shows that agile software development required agile tools for data management.

Summarized, all studies show attempts how to integrate agile development into the field of automotive software product line engineering. It is also noticeable here that all studies are from the last five years.

4.4. Discussion

We conducted this review with the objective to understand the current state of the art in holistic approaches for a managed evolution of software product line architecture in automotive software engineering. Furthermore, we aimed to identify the methods and techniques used to implement automotive software product lines in general, and their usage scope in particular. To achieve these objectives, we performed an in-depth review by using a refined conceptual model for automotive specific software product line development [34] as well as the results of a scoping study [13] in the field of automotive software engineering. In the following, we summarize the most important findings of this review according to the research questions.

4.4.1. RQ1: What Is the Current State of the Art in Holistic Approaches for Managed Evolution of Automotive Software Product Lines?

The in-depth review reveals that none of the studies represents a holistic approach for managed evolution of automotive software product lines according to the classification schema (Section 2.2). On average two activities of our classification schema are addressed by the selected primary studies (see Table 5). In addition, according to the results of Section 4.2, most studies refer to activities related to product line development (domain engineering) including Recovery and Discovery (86%). The analysis of the methods and techniques in Section 4.3 produces comparable results.

4.4.2. RQ2: What Particular Methods and Techniques Are Used to Implement a Managed Evolution of Automotive Software Product Lines?

From the content analysis of the primary studies, we clustered the proposed methods and techniques into 12 categories (see Table 6). Most of the papers propose one method or technique and provide—if at all—a scoped evaluation only. Based on the result set, we can only derive a “fragmented” picture as several aspects are studies, yet in a fairly isolated manner. Nonetheless, the result set shows that some topics are studied from different perspectives, which indicates the variety of the subject field.

Another finding is the accumulation of methods and techniques addressing the first activities of the schema such as the product line design (see Section 4.2). This indicates that the focus is on how to build up a product line, which is called revolutionary architecture process (RAP) in [52]. Axelsson [52] states that most literature mainly describes RAP, whereas evolutionary architecture processes (EAP) lack studies and evidence (see Section 4.3). Methods and techniques required for long term evolution (i.e., EAP according to [52]) such as architecture conformance checking are only addressed by few studies.

Remarkably, there is a low number of contributions concerning the product specific development cycle, which may indicate a research gap or it may indicate, e.g., that only few specific methods and techniques are required for product development within a product line based development approach. However, further developments that are separate from the product line, have to be transferred back into product line development at a later stage. The corresponding activity P to PL of the conceptual model was solely subject of two studies [41,48] according to the result of the categorization depicted in Table 5. However, the proposed methods and techniques of both studies [41,48] have another focus (see Section 4.3) and do not support the actual objective of activity P to PL. Here, we hypothesis a further research gap which has to be addressed in future research towards a holistic approach for managed evolution of automotive software product line architectures.

4.5. Threats to Validity

Since the study at hand is an in-depth study grounded in a previously conducted scoping study, the dataset we relied on suffers from potential selection bias introduced by the main study. To improve the internal validity, we implemented a researcher triangulation for the refinement of the dataset. For the team of researchers, we built sub-teams that carried out specific tasks and the remaining researchers not involved in that very tasks performed the quality assurance activities.

5. Conclusions

We prepared the in-depth review by using a refined conceptual model for automotive specific software product line development as well as the results of a scoping study in the field of automotive software engineering. First, we aimed at collecting information about the current state of the art in holistic approaches for managed evolution of automotive software product lines and used our conceptual model as a reference model to evaluate the state of the art. Specifically, we were interested into the coverage of the particular aspects of the conceptual model and, thus, the fields covered in current research and research gaps, respectively. The in-depth review reveals that none of the studies represents a holistic approach for managed evolution of automotive software product lines according to the used classification schema.

Next, we aimed to identify the methods and techniques used to implement automotive software product lines in general, and their usage scope in particular. We clustered the identified methods and techniques into 13 categories. On the one hand, most papers propose one method or technique and provide limited, if any, evaluation. On the other hand, most of the proposed methods and techniques are the subject of only one paper. Based on the results, we can only draw a “fragmented” picture, since several aspects are studied, but in a rather isolated way. Nevertheless, the set of results shows that some topics are studied from different angles, indicating the diversity of the subject area. A further finding is the accumulation of studies addressing the first activities of the schema such as the product line design and thus aiming at building up new software product lines. In contrast to this, approaches for long term evolution are underrepresented and should be addressed in future research.

It is evident from the study that the topic of applying agile software development to automotive software product line development has become very important in recent years. In the papers after 2016, more than every third paper addresses this topic. The results in this field are promising and it can be assumed that the results from this field will soon find their way into practice.

The small number of contributions dealing with the product-specific development cycle is noticeable, which may indicate a research gap or, for example, that only a few specific methods and techniques are required for product development within a product line-based development approach. However, there may be further developments that do not take place on the product line but at the level of the individual products. These further developments that are separate from the product line must be transferred back into the product line development at a later time. Here, we see a research direction for future work.

Furthermore, real industry examples should be increasingly used for future research as the studies show that many approaches lack practicability and applicability. It has been shown that the individual approaches always represent only a partial aspect, and integration is usually not possible as different formalizations and no overall language or model underlies them. Therefore, we see the formalization and modeling as a future research area to achieve a seamless integration of the individual approaches.