Digital Transformation in Aircraft Design and Certification: Developing Requirements for Modeling Regulatory Documentation

Abstract

Aircraft design and development is complex and regulated by increasingly stringent regulatory documentation. While many disciplines manage design complexity with well-established digital tools, digital transformation of the certification process remains in the early stages of implementation. Models are often used to provide explicit structures to facilitate digital transformation. While several modeling approaches have been applied to regulatory documentation, a gap remains for an established list of requirements for developing effective models in the context of digital transformation. This paper proposes a list of requirements using a requirements elicitation framework adapted from the International Council on Systems Engineering (INCOSE) Needs and Requirements Manual. The adapted research methodology includes problem identification, needs assessment, and requirements development processes. The resulting requirements are validated against needs statements and verified against selected INCOSE requirement statement criteria. Four requirements are selected for a detailed feasibility assessment, which compares the efficacy of process mapping, Unified Modeling Language (UML), and ontological modeling methods to realize the requirements.

1. Introduction

Aircraft are safety-critical systems regulated by law [1,2]. Aircraft design and development programs are complex, involving thousands of people across many disciplines, spanning upwards of 7 years, costing several billion dollars, and delivering fewer than 1000 units per year [3,4,5,6,7,8].

Aircraft design companies must demonstrate compliance with regulatory requirements which are developed and managed by authorities [9] such as the European Aviation Safety Agency (EASA) [10]; the Federal Aviation Administration (FAA) in the United States [11]; and Transport Canada Civil Aviation (TCCA) [12,13]. Regulatory information is communicated from the authority to aircraft development companies in the form of documentation, which can be divided into two main categories: regulations and guidance material. For example, the FAA oversees over 1800 aircraft design regulations for commercial transport-category aircraft within the Code of Federal Regulations (eCFRs) [14,15]. The FAA is also responsible for thousands of pages of associated guidance material and supporting documentation such as advisory circulars (ACs), airworthiness directives (ADs), and notices to air missions (NOTAMs) [16,17]. Advisory circulars are issued by federal regulators to “help the civil aviation community understand how to comply with current regulations and standards in aviation” [18]. Of the 764 active advisory circulars authored and maintained by the FAA, 308 are related to aircraft design compliance and certification [19]. Regulations have become progressively more stringent since the 1920s [11,20], and the certification process can account for as much as 50% of development costs [21,22,23].

The imperative to manage information throughout the aircraft design process has created an interest in digital transformation and a market for digital solutions [24,25,26,27,28]. While the use of digital tools such as Computer Aided Design (CAD) [29] and finite element analysis (FEA) [30], multidisciplinary design analysis and optimization (MDAO) [31], and digital twins [25,32] has been demonstrated to contribute to program success; digital transformation of the certification process remains in the early stages of implementation [33,34]. Many definitions of digital transformation have been proposed, most of which focus on improving processes by leveraging digital technologies [35,36]. Applications of digital transformation include changes to processes, advancement of technologies, and knowledge management [35,37,38].

Models are routinely used to provide structure for digital transformation [38]. Software tools often rely on formalized models to provide a communication link between subject matter experts (SMEs) in the application and digital domains [39,40,41]. Modeling prior to digital implementation is useful for developing specifications [42,43] and can transform information into a formal, explicit format for machine applications [44].

Models are frequently used to support aspects of the aircraft certification process. For example, Jeyaraj and Liscouët-Hanke [45] investigate a model-based approach for deriving safety requirements in conceptual aircraft design. Bamrah et al. [46] use a parametric CAD model to evaluate particular risk and zonal safety of different conceptual aircraft configurations. Franzén et al. [47] use ontological modeling to support the conceptual design space and decision making in developing search and rescue aircraft. The authors use a model-based systems engineering approach to transform stakeholder needs into aircraft functions and then assess a design space using ontological modeling and logical reasoning. While these modeling initiatives contribute to design space organization and alignment with certification regulations, the models do not contain explicit relationships to regulatory documentation. Additionally, using multiple independently developed models on the same aircraft development program would require extensive integration work to allow for communication across models. Models of regulatory documentation would contribute to certification process model maturity, and a set of requirements for those models would contribute towards the integration and overall digital transformation of the certification process.

Requirements engineering is an area of research that has been integrated into design through the use of international standards such as The International Council on Systems Engineering (INCOSE) Needs and Requirements Manual [48], ISO/IEEE Systems and Software Requirements Engineering [49], the Radio Technical Commission for Aeronautics (RTCA) DO-178C Software Considerations in Airborne Systems and Equipment Certification [50], and the Society of Automotive Engineers (SAE) International Aerospace Recommended Practice (ARP) 4754B Guidelines for Development of Civil Aircraft and Systems [51]. In these standards, models are presented as tools for requirements development. For example, the INCOSE Needs and Requirements Manual ([48], pp. 138–140) encourages the use of models to manage and trace requirements and suggests several Unified Modeling Language (UML)-based diagrams as being suitable for this application. The SAE ARP4754B ([51], p. 42) recommends identifying “modeling standards and libraries in order to establish a common understanding”. While the standards refer to the use of models for capturing and managing requirements in the design and development of a product or system, an opportunity remains for the establishment of an explicit set of requirements to assess the effectiveness of those models.

This paper proposes a set of requirements for developing an effective model of regulatory documentation in the context of digital transformation and is organized as follows: aspects of the INCOSE Needs and Requirements Manual [48] are selected to form the research methodology, and include problem identification, needs assessment, and requirements development processes. Problems are identified in the literature and serve as the structure for the needs assessment, which is conducted using a regulatory documentation use case. The resulting needs statements are transformed into requirements, which are then verified, validated, and assessed for feasibility.

2. Literature Review

The literature review focuses on two primary areas: requirements and modeling. The first Section 2.1 reviews literature that discusses requirements for effective models, followed by Section 2.2 which reviews requirement elicitation techniques. Section 2.3 and Section 2.4 examine problems identified in the literature in engineering model development and regulatory documentation model development, respectively. Section 2.5 presents a summary of problems identified in the literature and the research question addressed in this paper.

2.1. Requirements for Effective Models in the Literature

While model selection criteria is a well-researched topic in statistical analysis applications [52,53], research into requirements for model selection and development remain sparse in the design engineering literature. Shaked and Reich [54] compare three process modeling techniques and generate a list of requirements in the context of development process design. The requirements developed by the authors focus on supporting the representation and flow of activities, including traceability across multiple levels of abstraction, and accommodating change. Radhakrishnan and McAdams [55] compare analytical and computational models for the design of a vehicle component, identifying model selection criteria as model error and model development cost. Previous work by Cartile et al. [56] compare process mapping, UML, and ontological modeling approaches in their ability to address natural language ambiguity and complex relationship challenges in regulatory documentation. However, the authors do not establish a formal set of requirements for determining model effectiveness.

2.2. Requirements Elicitation

Requirements describe what a product will do without specifying how [57]. While many different requirement categorizations have been proposed for complex system-of-systems products such as aircraft [48,51,58,59,60], Frank [61] differentiates requirements for the technical modeling environment; the requirements for the domain; and requirements for the interface between the domain and other applications. Deckers and Lago [62] distinguish between technical specifications for creating a modeling environment and using the modeling environment in an application.

Popularized in the field of software development, requirements engineering involves the elicitation, analysis, specification, verification, validation, documentation, and management of requirements [49,58,63]. INCOSE [48] proposes a Needs, Requirements, Verification and Validation (NRVV) framework to requirements development. While the complete INCOSE framework addresses the entire product lifecycle, the aspects of the framework most relevant to this research include identifying the problems and opportunities in the domain of interest; the needs assessment; and developing, verifying, and validating requirements statements. INCOSE ([48], pp. 18–19) defines a need statement as the “result of a formal transformation of one or more sources or lifecycle concepts into an agreed-to-expectation”, and a requirement statement as “result of a formal transformation of one or more sources, needs, or higher-level requirements into an agreed-to-obligation”.

The predominant requirements elicitation technique involves human-to-human communication, where a design SME assesses customer needs using tools such as interviews, surveys, and group workshops [48,64]. This is especially true in the aerospace domain, where the interpretation and understanding of regulations and guidance material is knowledge held tacitly by aircraft design and certification SMEs [2]. Downer ([2], pp. 87–88) describes the relationship between regulators and aircraft design SMEs as “designee-dependency”, where selected SMEs are delegated as representatives of the regulator in making findings of design compliance against certification requirements [65]. SMEs gain in-depth knowledge of aircraft design and how it relates to certification requirements through years of experience, and that knowledge is not captured explicitly in writing [2,66,67]. The challenges that remain in the human-to-human requirements elicitation process often center around communication barriers and misunderstanding between parties [68,69].

More recently, a growing interest in natural language processing (NLP) tools and large language models (LLMs) has driven document-centric requirements elicitation approaches [70,71]. Wein and Briggs [70] use the natural language found in project design documents to automatically generate requirements, which are then compared to the requirements manually generated at the beginning of the project. The authors note that the ability to automatically generate correct requirements depends on the quality of the source documents used.

Problems identified in the literature in terms of model development and modeling aircraft regulations are discussed below.

2.3. Problem Identification in Model Development

Wynn and Clarkson [72] conduct a systematic literature review of design process models and observe several challenges in model development, including model complexity; model iteration; interaction; and use of generic models.

2.3.1. Complexity and Iteration

In terms of model complexity, Cardoso et al. [73] explain that selecting practical constraints to determine scope and granularity of a model is important for computational feasibility, cost of model development, and keeping the model up to date. Chi et al. [74] and Avila et al. [75] discuss the importance of keeping models up to date so that the model can be relied upon as a true source of information and prevent model obsolescence [76]. In a survey of software practitioners conducted by Forward and Lethbridge [77,78], the participants reported keeping models up to date as the biggest challenge and expressed a need for improved traceability between models and code to facilitate maintenance. Noy [79] refers to the effort required to keep a model up to date as model maintenance.

2.3.2. Interaction Between Models

Challenges in interaction between models is supported by Browning [80]. Browning discusses integrating the many viewpoints of the design process, commenting that disparate models lacking an overarching architectural framework can contain overlapping, incomplete, or contradictory information. The ability to exchange usable information between models is referred to as interoperability, and is identified at the system, syntax, data structure, and semantic levels [81,82]. Cartile [83] discusses the differences in semantics between modeling methodologies as one of the barriers to process model integration. The observed lack of model formalization is supported in the literature [84,85].

2.3.3. Models in Application

Duffy et al. [86] present a modeling framework for design reuse, and discuss application of past design requirements, domain knowledge, libraries and models on future programs. Trujillo and Madni [87] highlight the importance of developing generic models for reuse on different product development programs in the space sector. The authors discuss models as a platform for the retention and reuse of design knowledge such as decisions and supporting rationale, where “Design knowledge reuse involves redeployment of knowledge generated and proven on previously flown missions.” ([87], p. 5). In the commercial aircraft sector, design knowledge reuse is especially pertinent to certification where the reuse of existing designs provides historical safety and reliability performance data while reducing certification costs [88]. Wynn and Clarkson [72] conclude that practitioners would benefit from a systematic approach to generic model development for reuse across programs.

2.4. Problem Identification in Modeling Regulatory Documentation

Early regulatory modeling efforts were aimed at making regulatory information retrievable during compliance audits [89]. More recent research is focused on developing semantic frameworks for regulatory language, using Extensible Markup Language (XML) for web-based regulations [90]; natural language processing with ontological modeling [91]; and a subset of artificial intelligence (AI) called Bidirectional Encoder Representations from Transformers (BERT) for regulatory language classification [92,93].

Aerospace researchers have recently taken an interest in modeling regulations and guidance material and have noted a common theme of ambiguity, in that “a word with multiple senses is considered to be semantically ambiguous.” ([94], p. 1). Systems modeling language (SysML) has been used to model regulations for small [95,96] and large [97] fixed-wing aircraft. Fazal et al. ([97], p. 7) discuss the challenges in reflecting the structure of the regulations in a model, as well as challenges caused by the “degree of inconsistency in the language and style within the regulations themselves.” Ray et al. use a pre-existing generic BERT language model and extend it to aerospace by using eCFR Part 23/Part 25 regulatory text and CubeSat documentation to develop an aerospace-specific LLM called aeroBERT-NER [98]. Ray et al. [98] discuss having to manually label aerospace-specific language and convert the organization of the regulations into a format that is more machine-readable as a solution to regulatory ambiguity. Despite recent advances in digital tools such as large language models, the translation of natural language into machine-readable formats remains a manual process for technical domains where specialized terminology does not appear in general domain text [99].

Rushby [100] proposes the use of logic to structure guidance material and notes the interchangeable use of technical terms throughout an aircraft systems development guidance material document. Eito-Brun et al. [101] use a natural language processor to develop ontological models of a standard for software development in the space sector. The authors argue that using standards as the basis for ontological models ensures a common vocabulary and represents a shared understanding of processes and activities. Szeto and Das [102] use natural language processing neural networks to help clarify ambiguous language in notices to air missions (NOTAMs) documents.

2.5. Summary, Gaps, and Research Question

While several studies have explored methods for modeling regulatory documents, a gap remains for establishing a formal set of requirements for developing effective models. The research that is the subject of this paper aims to answer the question: What are the requirements for developing an effective model of regulatory documentation in the context of digital transformation?

The unique contribution of this paper is to identify the problems, assess the needs, and develop a formal set of requirements against which regulatory documentation modeling methods can be assessed for effectiveness in the digital transformation context using an adapted requirements elicitation methodology.

Several problems reoccur across the literature as opportunities to elicit requirements for modeling aircraft regulatory documentation. These problems include ambiguity, complexity, change tracking, and knowledge reusability, which form the main points of investigation.

3. Research Methodology

The research methodology is adapted from the framework for requirements elicitation proposed by INCOSE [48], and includes problem identification, needs assessment, and requirements development, shown in Figure 1. A regulatory documentation use case is used to conduct the needs assessment and requirements development processes.

The problems identified in the literature include ambiguity, complexity, change tracking, and knowledge reusability, as shown in Figure 1. The use case domain and regulatory documents selection is described in Section 3.1, and the needs assessment process shown in Figure 1 is structured as follows: ambiguity needs are assessed using a natural language processing tool and contextual analysis in Section 3.2.1; complexity needs are assessed using scope and cross-referencing in Section 3.2.2; change tracking needs are assessed in the context of version control and change management in Section 3.2.3; and knowledge reusability needs are assessed across product development programs in Section 3.2.4. Each needs assessment concludes with needs statements which are transformed into requirements using INCOSE criteria presented in Section 3.3.

3.1. Selecting Regulatory Document Use Case

A use case research approach is applied as a means of requirements elicitation, which includes the selection of both a domain and documentation. Examples of aircraft design domain use cases include clean-sheet design, derivative design, significant modifications, integration of new technology, and component design. All aircraft design domains are required to comply with regulatory requirements; however, each domain will differ in terms of applicable regulatory sections and corresponding guidance material depending on the type of aeronautical product and design.

The process of selecting a use case and relevant documentation is where subject matter experts (SMEs) and research partnerships plays an important role. Experienced SMEs have the tacit knowledge necessary to understand the relationships between the regulations and guidance material and can be consulted to identify the most relevant documents and document sections in the chosen domain. This research was conducted in collaboration with two industrial partners: a Canadian aerospace research and development company specializing in aircraft certification; and a Canadian aircraft modification company. The aircraft modification sector provides the industrial context, with subject matter experts from both partners providing research validation.

The aircraft modification sector is a complex subset of aircraft design and development in which changes are made to an existing aircraft. Significant modifications to an aircraft are those that affect the general configuration, principles of construction, or the assumptions used for certification [103]. The modified aircraft undergoes a certification process where evidence must be provided that the modified aircraft is as safe or safer than the original unmodified aircraft [83]. Examples of modifications include converting an aircraft’s mission capabilities from passenger-carrying to firefighting [104] and upgrading engines from piston to turboprop [105]. More recently, existing aircraft have been used as test beds for new technologies, such as retrofitting aircraft with all-electric [106] and hybrid-electric [107] propulsion systems.

The Code of Federal Regulations (eCFRs) Section §21.101 [108] and the Advisory Circular AC21.101-1B—Establishing the Certification Basis of Changed Aeronautical Products [103] are selected as the use case regulatory documents. The AC 21.101-1B provides guidance for the certification of changed products and includes suggested methods of compliance to eCFRs §21.101. These documents focus on the process of identifying which regulations are applicable to the changes made to the aircraft so that its design remains as safe or safer than the original configuration. The equivalent documents in the European regulatory system, EASA, are the guidance material document GM 21.A.101 Establishing the certification basis of changed aeronautical products [109] and the regulation CS 21.A.101 [110]. The Canadian advisory circular equivalent, AC 500-016 Issue No. 1 [111], currently points to the FAA’s AC21.101-1B to provide interim guidance for complying to the equivalent Canadian Aviation Regulation, CAR 521.158 [112], in establishing the certification basis for a changed aeronautical product. While the methodology described in this section would be applicable to any regulatory documentation use case, the AC21.101-B and eCFR §21.101 FAA documents are selected for their relevance to the aircraft modification sector and subject matter expertise contributing to this research.

3.2. Needs Assessment

The needs assessment is structured using the problem themes identified in the literature and listed in Section 2.5, including ambiguity, complexity, change tracking, and knowledge reusability.

3.2.1. Ambiguity

The ambiguity in the natural language of regulatory documents is investigated in this assessment. The natural language processing (NLP) software, Atlas.ti version 25 [113], is used to identify frequently occurring keywords and their associated noun phrases with a function called concept search. The concept search function is distinct from a word frequency function. While a word frequency function identifies parts-of-speech frequencies and provides lists of word categories, such as nouns, verbs, and adjectives; the concept search provides a list of concepts in any part-of-speech and ranks their frequencies depending on the number of noun phrases associated with that concept [114]. The concept search tool also provides the contextual excerpts in which the noun phrases occur.

The results of the natural language processing concept search tool for the AC 21.101-1B are shown in Figure 2. This analysis only includes the main body of the text; the front matter and appendices are excluded.

Figure 2a shows the concept map for the AC 21.101-1B. The top five concepts and corresponding frequencies of occurrence include change (317), design (140), certification (113), basis (110), and product (106), with certification selected as an example. Figure 2b shows the noun phrases corresponding to certification, and in Figure 2c the context in which the noun phrases appear. For example, of the 109 occurrences of the word basis, over 90% are paired with the word certification. Certification basis is therefore selected as a noun phrase for further analysis.

The concepts identified by the natural language processor are then analyzed in the document for further context. The objective of this process is to identify ambiguous use of concepts, as well as identification of more nuanced concepts that fall outside of the scope of natural language processing capabilities.

For example, the context of certification and certification basis are further analyzed directly in the text, a sample of which is shown in Figure 3a using an excerpt from chapter 3.1 of the AC21.101-1B. Concepts and noun phrases identified by the natural language processing tool are shown in Figure 3b. Figure 3c shows concepts identified through manual contextual analysis.

Figure 3a shows three instances of the concept certification, two of which include the noun phrase certification basis, as classified in Figure 3b. The final certification basis as a deliverable for an aircraft design program takes the form of a collection of information captured in an explicit format, such as a list in a document artifact (as described in Appendix H of the AC21.101-1B). However, the context in which certification and certification basis occur change their meaning throughout the text. For example, Figure 3c shows that an instance of certification is associated with of a type design change. Certification is used in this context to indicate a process. Another example shown in Figure 3c is the use of certification basis in association with proposal, indicating that a process exists in which a certification basis is proposed. Neither of these examples explicitly include the word process and cannot be automatically detected by a natural language processing tool.

• Needs statement

The following statement is identified from the needs assessment corresponding to the problem of ambiguity:

-

A model of regulatory documentation needs to support the nuanced semantics of the domain in the interest of managing the ambiguity inherent to the natural language used in the documentation. Explicitly defined formal semantic structure is necessary in the context of digital transformation.

3.2.2. Complexity

This section investigates contributors to regulatory documentation complexity, including scope and cross-referencing.

Scope

Model scope is an important consideration for managing model complexity. To illustrate an example of model scope, previous research conducted by Cartile et al. ([56], p. 9) modeling aspects of the AC21.101-1B is critiqued below.

Five keywords were identified from the first step of a process model found in Chapter 3 of the AC.21.101-B. The keywords included Identify, Proposed, TypeDesign, Changes, and AeronauticalProduct. Contextual analysis was conducted to identify other surrounding keywords and their relationships with one another. The resulting keywords were modeled in an ontology, shown in Figure 4.

The ontological model shown in Figure 4 depicts the (a) class hierarchy; (b) object property or relationship entities; and (c) graphical viewpoints of the AC 21.101-1B model. The class hierarchy includes the list and classification of concepts identified through the NLP and analyzed for context and other related concepts. Concepts are related to one another using object properties and represented visually using the graphical viewpoint.

The drawback from this approach was the rapid escalation of the model’s scope. The initial 5 keywords led to the identification of over 40 related keywords. The resulting ontological model in Figure 4 is more of a taxonomy than an ontology, in that it lists classes in a hierarchical fashion but does not attempt any logical organization of the semantics. Further inclusion of keywords would result in a disorganized, busy model, that would quickly become computationally unfeasible and unusable.

Cross-Referencing

An element of regulatory documentation complexity is the use of explicit cross-referencing. An example of explicit cross-referencing is shown in Figure 5, which illustrates the relationships between eCFR Section §21.101 [108] and other sections of the eCFRs.

The eCFR §21.101 paragraphs (a) and (b) shown in Figure 5 cross-reference over ten other sections of the regulations, each of which contain many sub-sections. For example, eCFR §21.101 paragraph (a) references eCFR Parts 34 and 36, where eCFR Part 34 contains 20 regulations, and eCFR Part 36 contains 23 regulations and 7 appendices [115]. These requirements must be consulted in addition to the eCFR §21.101 requirement.

Cross-referencing is also present in guidance material. For example, while the AC21.101-1B [103] primarily supports the eCFR Section §21.101 [108], the AC also references eCFR Sections §21.16, §21.17(b), §21.19, §21.21(b)(2), §21.24, §21.25, §21.27, §23.2, §25.2, §27.2, §29.2, §31, §33, and §35 [115]. While guidance material references both the regulations and other guidance material documents, regulations only reference other sections of the regulations and make no reference to guidance material.

• Needs statement:

The following statements are identified from the needs assessment corresponding to the problem of complexity:

-

A model of regulatory documentation needs to have a well-defined objective and constrained scope to ensure computational feasibility and model usefulness.

-

Models need to be able to represent the complexity of the regulations while maintaining a constrained scope by taking a modular approach to model development.

-

Cross-referencing across regulatory documentation would need to be reflected in and across models. Each model would therefore need consistency in terms of model architecture and semantics to ensure model interoperability.

3.2.3. Change Tracking

While relatively infrequent, regulatory documentation does undergo change. Changes to regulations are tracked by amendment level, as shown in Figure 6.

The title of the regulation eCFR §21.101 Designation of applicable regulations [108] is shown as an example in Figure 6, with the corresponding amendment levels shown below it. The first issuance of the regulation is shown first as Doc. No. 28903, 65 FR 36266, 7 June 2000, which corresponds to the docket number of the document that proposed the change to the regulation; the federal register volume and page number; and the date of issuance, respectively. The “as amended by” statement indicates this regulation has undergone changes, which are then assigned amendment numbers such as Amdt. 21-90 as well as corresponding references to where the change was documented. For example, Amdt. 21-90 is documented in federal register volume 72 on page 63,404 dated 8 November 2007, but does not reference a docket number.

Changes to regulatory guidance material, however, are tracked by a version control letter and issuance dates. For example, Figure 7 shows an excerpt from the AC 21.101-1B. The version 1B dated 2016 cancels version 1A dated 2010.

One of the challenges in changes to regulatory documentation is the relationship between guidance material and regulations. Different amendments of regulations likely correspond to the versions of guidance material that were active around the same dates; however, this information is not explicitly stated. Guidance material also does not state which amendment level of the regulations it refers to, only referencing the section where the regulation can be found.

Guidance material does, however, specify version letters and numbers when referencing other guidance material documents. For example, the AC 20-174 Development of Civil Aircraft and Systems [116] references the ARP4754A. The most recent version of the AC 20-174 was published in 2011, while the ARP4754 has since been updated to version B.

As aircraft operate for many years, regulatory bodies must maintain all legacy versions of the regulations and guidance material, and track how those documents have changed over time.

• Needs statements:

The following statements are identified from the needs assessment corresponding to the problem of change tracking:

-

Models need to be structured in a way to be able to reflect the changes made in the source documentation to stay up to date and prevent obsolescence.

-

In order to implement the changes between regulatory documents and models, the models need to be able to support relationships between the modeled entities and their location within source documentation.

3.2.4. Knowledge Reusability

The complexity of aircraft design and development drives the need for knowledge capture and reuse across programs. The AC21.101-1B [103] lists 27 examples of significant changes to small airplanes (eCFR Part 23), and 29 examples for large airplanes (eCFR Part 25). Examples of significant Part 25 aircraft modification programs include a passenger-to-cargo conversion, a fuselage stretch, a propulsion system conversion, and an avionics upgrade program. If seeking certification from the regulatory body in the United States, each of these modification programs would likely need to comply to eCFR 21.101 and could use the AC21.101-1B to develop each respective certification basis. Each of the changes in these examples affects different areas, systems, and components of the aircraft, and must demonstrate compliance to different regulatory requirements depending on the modification and year of the original design.

• Needs statements:

The following statements are identified from the needs assessment corresponding to the problem of knowledge reusability:

-

A model needs to include generic provisions to accommodate for and be applied to any aircraft design program.

-

Aircraft modification has extensive product variability, in that any aircraft from any year may undergo any type of change. Each aircraft modification program has a different project scope and complexity. A generic model would need to be able to scale to any size, accommodating both smaller and larger programs, and include any number of stakeholders.

3.3. Transformation of Needs into Requirements

Requirements statements are developed using INCOSE requirements development guidelines [48,117]. The following criteria for developing requirements are selected as most applicable to the context of this research:

• Singular (INCOSE C5): The statement should state a single capability, characteristic, constraint, or quality factor.
• Feasible (INCOSE C6): The statement can be realized within application constraints with acceptable risk.
• Verifiable/Validatable (INCOSE C7):

  o

  The statement is structured and worded such that its realization can be verified to the approving authority’s satisfaction.

  o

  The statement is structured and worded such that its realization can be validated to the approving authority’s satisfaction.

INCOSE [48] uses the terms verification and validation in the context of requirements statements as follows: Verification focuses on the wording and form of requirements and is the process of confirming that requirements meet a set of characteristics for writing well-formed statements. Validation focuses on the message of the requirements and is the process of confirming that the requirements clearly communicate the intent of the needs and are formulated in language that is understandable to designers.

4. Results and Discussion

The needs statements developed through the use case needs assessment methodology described in Section 3 are transformed into requirements, which are presented below. The requirements are validated against needs statements and verified against INCOSE requirements criteria.

4.1. Requirements Summary

The resulting requirements are presented in

Table 1. Requirements with corresponding needs statements, problems, and assessment sections.

Requirement	Need	Problem	Assessment Section
Requirement 1 Models must accurately reflect the natural language and intended meaning found in the regulatory documentation.	Support the nuanced semantics of the domain	Ambiguity	Section 3.2.1
Requirement 2 Models must include explicit definitions.	Explicitly defined formal semantic structure	Ambiguity	Section 3.2.1
Requirement 3 Models must be modular.	Modular approach to model development	Complexity	Section 3.2.2
Requirement 4 Each model must be purpose specific.	Well-defined objective	Complexity	Section 3.2.2
Requirement 5 Each model must have a constrained scope.	Ensure computational feasibility and model usefulness	Complexity	Section 3.2.2
Requirement 6 Models must have consistent architectures.	Ensure model interoperability	Complexity	Section 3.2.2
Requirement 7 All modeled entities must be traceable to their source document and location within that source document.	Reflect the changes made in the source documentation	Change tracking	Section 3.2.3
Requirement 8 Models must be generic and reusable on different programs.	Applied to any aircraft design program	Knowledge reusability	Section 3.2.4
Requirement 9 Models must be scalable.	Scale to any size	Knowledge reusability	Section 3.2.4

and answer the research question “What are the requirements for developing an effective model of regulatory documentation in the context of digital transformation?” The requirements listed in the first column of Table 1 correspond to need, problem, and needs assessment reference sections.

4.1.1. Requirements Validation

Each requirement was derived from the italicized keywords taken from the needs statements, validated according to the intent of that need, and translated into language suitable for model designers. For example, the need corresponding to Requirement 1 states that a model should support the nuanced semantics of the domain. This statement necessitates clarification to translate what that would mean in terms of model design within the application context, and results in a requirement statement of models must accurately reflect the natural language and intended meaning found in the regulatory documentation.

4.1.2. Requirements Verification

The requirements are verified against the INCOSE criterion of singularity, where each requirement statement is limited to one constraint. While INCOSE recommends avoiding the use of conjunctions such as “and”, exceptions are made when joining actions that will be verified together. For example, Requirement 7 states that all modeled entities must be traceable to their source document and location within that source document, where the location further specifies the traceability requirement.

The feasibility criterion, which focuses on realizability within application constraints, is further expanded upon by INCOSE, where “feasible implies the existence of a possible solution to satisfying a concept, need, or requirement” [117]. Several possible solutions are proposed and verified against selected requirements in the next section to demonstrate feasibility.

4.2. Proposed Modeling Solutions for Requirements Feasibility Verification

Several preliminary model design solutions are presented and assessed for their ability to meet selected Requirements 1, 2, 6, and 7. The proposed solutions are intended to demonstrate the feasibility of these requirements.

Previous work by Cartile et al. [56] compared process mapping, Unified Modeling Language (UML), and ontological modeling approaches in the context of aircraft regulatory documentation. In the context of this research, the ability of process mapping, UML, and ontological modeling methods are verified against the selected requirements to assess the feasibility criterion.

4.2.1. Modeling Methods Verified Against Requirements 1 and 2

Requirement 1. 

Models must accurately reflect the natural language and intended meaning found in the regulatory documentation.

-

Process mapping: process mapping does not include any formal semantic capabilities. The ability to reflect regulatory natural language is at the discretion of the process map developer. Process mapping does not provide any methodology for semantic assessment.

-

UML: UML includes semi-structured semantics, in that the method provides base type entities which can be extended to provide domain-specific semantics. UML does not provide any formal method for semantic assessment, resulting in the natural language included in a UML model being dependent on the UML model developer and selected semantic input.

-

Ontological modeling: ontologies are based in description logic and are focused on developing formal semantic models. Ontologies include reasoning capabilities for assessing logical contradictions in semantic structures.

Requirement 2. 

Models must include explicit definitions.

-

Process mapping: process mapping does not include any definition capabilities.

-

UML: UML does include definition capabilities, but only as text-based metadata. Definitions and constraints are not part of any rule-based automated checking methods in base UML capabilities.

-

Ontological modeling: the logic-based nature of ontological modeling includes both text-based definition and rule-based reasoning capabilities. Text-based metadata can be included using the annotation function, and defined class membership is automatically assessed using logical reasoners.

The modeling method that meets both Requirements 1 and 2 is ontological modeling. Requirements 1 and 2 are both feasible, in that one or more modeling methods can realize these requirements.

4.2.2. Architectural Framework Verified Against Requirement 6

The contextual analysis of ambiguous regulatory natural language shown in Figure 3 is expanded upon to illustrate a potential solution for meeting Requirement 6 Models must have consistent architectures.

The natural language processing tool and contextual analysis were used to identify several other entities in the aircraft regulatory document AC 21.101-1B [103], the results of which are shown in Figure 8.

The entities shown on the right-hand side of Figure 8 include keywords FAA and applicant; concepts certification process and certification basis; and manually identified processes Identify baseline product, Identify proposed change, Type design change certification, and Certification basis proposal process. Architectural framework literature suggests using a repeatable class structure to facilitate interoperability between models [118]. The entities listed in Figure 8 are categorized into five classes shown on the left-hand side, labeled Activity, Process, Artifact, Organization, and Role.

-

Process mapping: process maps do not inherently provide means for implementing an architectural framework.

-

UML: UML entities reflect a class-based structure which can be used to realize an architectural framework.

-

Ontological modeling: ontologies reflect a class-based structure which can be used to realize an architectural framework.

A consistent architecture can be feasibly achieved by developing a repeatable class structure across models to meet Requirement 6. UML and ontological modeling can implement a repeatable architectural framework.

4.2.3. Traceability Framework Verified Against Requirement 7

The manually identified concepts shown in Figure 8 are used to illustrate a potential solution in Figure 9 for meeting Requirement 7 All modeled entities must be traceable to their source document and location within that source document.

Modeling approaches should include bi-directional, searchable relationships between the modeled entities and their source documentation. Figure 9 shows several AC21.101-1B entities that are to be included in a model and their corresponding locations within the document. For example, one of the instances of Identify baseline product can be found in AC21.101-1B Chapter 3.2.1. The inclusion of this relationship will provide forward and backward traceability between the entity and the source to meet Requirement 7 so that the model can be kept up to date with changes made to the regulatory documentation.

-

Process mapping: while process map arrows can be specified as bidirectional by including an arrowhead at each end of the connector, process mapping tools do not include search functions. Arrows only provide a visual traceability representation between entities.

-

UML: UML includes searchable bi-directional relationship capabilities through relationship entities, the semantics of which can be specified in both directions and used to connect entities.

-

Ontological modeling: ontological models include searchable bi-directional relationship capabilities through the object property function and inverse property characteristic, which can be used to connect entities.

A choice must be made for how source documentation is represented in a model. Hard linking modeling entities to source documentation such as URL hyperlinks incurs risk of error, as source documentation may be restructured and break the modeled links. Representing source document titles and sections as entities in models would be a better solution, as the document structure can be reflected independently from an online host.

Bidirectional relationships in UML and ontological modeling can be feasibly used to connect entities to other entities representing source documentation and meet Requirement 7.

4.2.4. Requirement Feasibility Verification Summary

The following

Table 2. Summary of selected requirements feasibility verification.

Requirement	Process Mapping	UML	Ontological Modeling
Requirement 1 Models must accurately reflect the natural language and intended meaning found in the regulatory documentation.	Cannot feasibly meet requirement	Cannot feasibly meet requirement	Can feasibly meet requirement
Requirement 2 Models must include explicit definitions.	Cannot feasibly meet requirement	Cannot feasibly meet requirement	Can feasibly meet requirement
Requirement 6 Models must have consistent architectures.	Cannot feasibly meet requirement	Can feasibly meet requirement	Can feasibly meet requirement
Requirement 7 All modeled entities must be traceable to their source document and location within that source document.	Cannot feasibly meet requirement	Can feasibly meet requirement	Can feasibly meet requirement

summarizes the feasibility of process mapping, UML, and ontological modeling approaches to meet selected Requirements 1, 2, 6, and 7.

The visual nature of process mapping, while useful for human-to-human communication, does not meet any of the requirements listed in Table 2, meaning that process mapping would not provide adequate modeling functionality to facilitate the digital transformation of regulatory documentation. While UML does include a class-based structure that can support a repeatable architecture as well as bi-directional relationship capabilities to support traceability and meet requirements 6 and 7, UML does not provide formal semantic structure capabilities to meet requirements 1 and 2. The approach that can most feasibly meet the regulatory documentation modeling requirements for the context of digital transformation is ontological modeling.

4.3. The SME

Absent from the problems identified in the literature and a limitation of the requirements elicitation methodology is the mention of subject matter experts. Identifying the reliance of the aircraft design and certification process on subject matter expertise is itself implicit knowledge that is not well documented or acknowledged in the literature. The intent of digitally transforming regulatory documentation is not to replace the SME but rather leverage software tools and automation to help support the SME throughout the design and certification process. Taking the role of the SME throughout the certification process into account and providing mechanisms to support decision making expertise are important considerations in model development.

5. Conclusions and Future Work

Models of regulatory documentation are necessary for the digital transformation of the aircraft design certification process. While several regulatory modeling initiatives exist, a gap remains in the literature for an established set of requirements for developing effective models in this context. In this paper, a requirements elicitation methodology is presented and a list of requirements for modeling regulatory documentation is proposed.

The methodology used in this paper is adapted from an INCOSE requirements elicitation framework and includes problem identification, needs assessment, and requirements development using a regulatory documentation use case. Nine requirements result from this approach, which are then validated against the identified needs and verified against selected INCOSE requirements statement criteria. The feasibility criterion is assessed in detail for four selected requirements by comparing three modeling methodologies of process mapping, UML, and ontological modeling. The feasibility assessment concludes that ontological modeling is the most effective modeling method for meeting the requirements and developing models of aircraft regulatory documentation in the context of digital transformation.

Future work for this research will include investigating ontological modeling as a means to meet all nine requirements. A resulting modeling methodology that met the established requirements could be used to model any aircraft regulatory document and inform the development of new regulatory infrastructure in the interest of digital transformation.