Quality Models for Artificial Intelligence Systems: Characteristic-Based Approach, Development and Application

The factors complicating the specification of requirements for artificial intelligence systems (AIS) and their verification for the AIS creation and modernization are analyzed. The harmonization of definitions and building of a hierarchy of AIS characteristics for regulation of the development of techniques and tools for standardization, as well as evaluation and provision of requirements during the creation and implementation of AIS, is extremely important. The study aims to develop and demonstrate the use of quality models for artificial intelligence (AI), AI platform (AIP), and AIS based on the definition and ordering of characteristics. The principles of AI quality model development and its sequence are substantiated. Approaches to formulating definitions of AIS characteristics, methods of representation of dependencies, and hierarchies of characteristics are given. The definitions and harmonization options of hierarchical relations between 46 characteristics of AI and AIP are suggested. The quality models of AI, AIP, and AIS presented in analytical, tabular, and graph forms, are described. The so-called basic models with reduced sets of the most important characteristics are presented. Examples of AIS quality models for UAV video navigation systems and decision support systems for diagnosing diseases are described.

In industrial systems, healthcare, transportation, weapons systems, etc., the impact of AI is becoming increasingly tangible and sustainable, and on the other hand, it is very controversial in terms of implementation. This is due to:

•
The versatility and complexity of decisions taken in the development and application of systems in which AI tools are built; • The variability of the physical and information environments which are not always defined by the parameters in which they operate. The number and extent of external influences, such as cyberattacks aimed at artificial intelligence and based on AI methods are growing and expanding; • The accumulation of expert information and expansion of knowledge bases that can be used to improve the efficiency of these systems. The principle of human-centeredness in its creation and application must be balanced to reduce the risks of wrong decisions due to subjective reasons; • They increase the importance of ethical and safety aspects during use. This factor is especially important and specific to AISs. According to [22] and other documents focusing on the humanitarian aspects, human dignity, personal and collective security, and well-being are valued in the development and implementation of AISs.
These conditions, in contrast to "traditional" systems, complicate the formulation of specifications and verification of compliance with the requirements for creation and modernization. In addition, the number and variety of AI and AISs traits that need to be considered are growing, especially, ethics, clarity, credibility, etc. [23][24][25]. In turn, the methods of evaluation are diversified, which should be based on a clear idea of the nature and interdependence of the characteristics of artificial intelligence.
It should be emphasized that the increase in the number of publications and standards is accompanied by a significant disturbance in the characteristics of AI, which, on the one hand, determines, and on the other hand is due to a certain contradiction of definitions. Therefore, it is extremely important to research to harmonize and hierarchy characteristics, which objectifies and simplifies the development of tools for standardization, evaluation, and provision of requirements for the creation and implementation of AI systems.

Aim, Objectives, and Structure
The study aims to develop a model of the quality of artificial intelligence and AI systems based on the definition and ordering of traits. Objectives: • Formulate the principles and justify the sequence of analysis and development of quality models of AI and SSI as an ordered sets of characteristics; • Analyze and classify the characteristics, determine their relationship and principles of use of AI and AI systems, provided in known sources, including standards and guides developed by leading institutions; • Propose quality models of AI, AI platforms (AIPs) as parts of AISs, and systems using different forms of representation for further use, primarily as an evaluation of individual characteristics and quality in general; • Demonstrate the profiling of AI and AISs quality models for systems using artificial intelligence (drone video navigation system (UAV) and urological disease diagnosis system).
The article is structured as follows. Section 2 grounds the principles of developing quality models and their sequence. Section 3 provides an overview of the references that provide definitions and describe the hierarchical relationships between characteristics. The codification of definitions is proposed, and the characteristics are separated from the principles of development and application of artificial intelligence. Section 4 proposes approaches to formulate definitions of AI characteristics based on the analysis of existing ones and their harmonization considering different groups of references, the final table with definitions and classification of AI quality traits is formed. The different forms of representation of dependencies and hierarchies of characteristics are presented in Section 5, and then it is used to analyze the relevant hierarchies based on the analysis of key sources. Section 6 describes the quality models of artificial intelligence and AI platforms, presented in parentheses, tables, and graphs. The so-called basic models with reduced sets of characteristics are given because of their importance. Section 7 describes examples of quality models for AI systems. Lastly, Section 8 provides conclusions and describes areas for further research.
The main contribution of the research includes a set of streamlined AI, AIP, and AIS characteristics, and quality models making it possible to specify requirements to systems and assess them during development and application.

. Quality as a Generated AI Characteristic
The set of characteristics of artificial intelligence systems analyzed in the article is united by the concept of "quality" similar to how it is usually performed for software, where there are stable quality models that have developed and improved over 50 years of evolution [26][27][28]. The concept of "quality" of AI, in our opinion, is an acceptable generalizing feature, even though in some works it is used as a partial feature of AI, or study considers the quality of artificial intelligence purely in the context of software quality [29].
The software quality context is indeed very important, but it should be used as an approach to form a more general AI quality model. In [30], a position similar to the position of the authors of this study on the importance of AI quality is formed, although this work narrows the content of quality somewhat, as the set of AI traits analyzed is limited. Therefore, we further use the concept of AI and AIS quality as a system-forming, top-level entity in the hierarchy of all characteristics according to the general interpretation of quality according to ISO 9001: 2015 as the extent to which a set of objects-specific characteristics (in this case AI) to which a set of inherent characteristics of an object meets requirements). It is a guarantee for the development of an orderly set of characteristics of artificial intelligence.
Importantly, the quality of AI and AI system should be distinguished from the quality of the product that is the result of the AIS application. For example, the quality of images should be distinguished from the quality of AI with which the image has been processed. Although, there is an effect of AI (AIS) quality on product quality.

AI, AI Platforms, and AI Systems
The quality of the AI system consists of the quality of AI as a generalized but specific object and the quality of the software and hardware platform (called as AIP), through which AI is implemented. This study considers only those components of the quality (characteristics) of AIPs that should somehow consider the specificity of AI as opposed to standard quality characteristics of software and hardware. It should be added, as the analysis showed, that there are quality characteristics that are common to AI and AIP. In this case, we will refer them to the characteristics of the quality of AI, but also consider them at the system level. The characteristics (models) of AI and AIP quality are combined in a general quality model. It is needed to note that requirements to common AI and AIP characteristics are implemented at the two stages: firstly, at the development of the models/algorithms considering specific features of AI functionality and design, and, secondly, at the development of hardware and software by the use of methods considering these features.
In addition, the ISO25010 standard [31], which describes the software quality model, divides the characteristics into two sets, namely subsets of the "product quality" and "quality in use" characteristics. In [30] there is a description of a fragment of such a distribution. Therefore, it may be the second feature to classify AI quality traits for more detailed analysis.

Characteristics as a Key Conception
The key concept used in the study is "characteristic"-a component of quality, which describes the different characteristics of the AI, AI platform, and AI system. The characteristic is the basis for formulating requirements for the AI system and its components by:

•
Considering the relevant characteristics in the development of the system specification, i.e., inclusion in the list of requirements; • The definition and choice of metrics according to which the value of the property is evaluated, namely scales and methods of measurement; • Substantiation of the necessary "limits" of this property, i.e., requirements to the qualitative or quantitative level thereof defined by the corresponding metrics.
If the characteristic Ch1 depends on the characteristic Ch2, then we will call the sub-characteristic of the characteristic Ch1. Ch1 and Ch2 should be located at the upper and next lower levels of the hierarchy of the quality model, respectively. For each of the characteristics (sub-characteristics), metrics must be defined for their evaluation, as well as formulated requirements for the values of these characteristics, and develop a profile of requirements and their quality [32].
Quality characteristics form a matrix classification, the general view of which is presented in Figure 1. We will call it the quality characteristics classification card (QCCC). QCCC columns correspond to the set of characteristics of artificial intelligence (AI characteristics), platform (AIP characteristics), including their common part (AI&PC), and the rows correspond to the sets of product quality characteristics (QPC), quality in use (QUC) and many characteristics that include both types of quality (QPUC). • Substantiation of the necessary "limits" of this property, i.e., requirements to the qualitative or quantitative level thereof defined by the corresponding metrics.
If the characteristic Сh1 depends on the characteristic Сh2, then we will call the subcharacteristic of the characteristic Сh1. Ch1 and Ch2 should be located at the upper and next lower levels of the hierarchy of the quality model, respectively. For each of the characteristics (sub-characteristics), metrics must be defined for their evaluation, as well as formulated requirements for the values of these characteristics, and develop a profile of requirements and their quality [32].
Quality characteristics form a matrix classification, the general view of which is presented in Figure 1. We will call it the quality characteristics classification card (QCCC). QCCC columns correspond to the set of characteristics of artificial intelligence (AI characteristics), platform (AIP characteristics), including their common part (AI&PC), and the rows correspond to the sets of product quality characteristics (QPС), quality in use (QUС) and many characteristics that include both types of quality (QPUC). Thus, the concept of artificial intelligence assessment is thus to develop a quality model as a general and orderly set of characteristics of the actual artificial intelligence, the corresponding software and hardware platform, and the AI system as a whole. Consider the stages of its implementation in the construction of quality models.  Thus, the concept of artificial intelligence assessment is thus to develop a quality model as a general and orderly set of characteristics of the actual artificial intelligence, the corresponding software and hardware platform, and the AI system as a whole. Consider the stages of its implementation in the construction of quality models.    The main phases are as follows:

The Order of Quality Models Construction
• In the first stage, the set of characteristics and principles of use of AI is formed based on the analysis of references, their coding is performed and the corresponding table is developed (Table 1, Section 3.1); • Then, the separation of principles and guidelines for the use of AI from its characteristics, the results of which are given in Table 2 (Section 3.2); • The analysis and harmonization of characteristics definitions according to the presented approach is performed. The result is  The main phases are as follows: • In the first stage, the set of characteristics and principles of use of AI is formed based on the analysis of references, their coding is performed and the corresponding table is developed ( Table 1, Section 3.1); • Then, the separation of principles and guidelines for the use of AI from its characteristics, the results of which are given in Table 2 (Section 3.2); • The analysis and harmonization of characteristics definitions according to the presented approach is performed. The result is • A partially formalized procedure for building a hierarchy of AI quality traits is being developed. Accordingly, hierarchies between brackets, table, and graph forms are represented (Section 6.1); • Then the models of AI, AI systems, and AI platforms quality are presented in visual graph form. Table forms of AI and AIP quality are given in Tables 5 and 6 (Section 6.2), respectively. In addition, basic models are built, which are part of the overall quality models (Section 6.3); • Finally, examples of profiling of quality models for two UAV video navigation systems and decision support for the diagnosis of urological diseases are provided (Sections 7.1 and 7.2).

Forming and Codification of AI Characteristics
The order for solving this problem is as follows. 1.
The forming of a set of characteristics related to or may be related to AI, its platforms, and systems is as follows: the entire reference base, formed based on selected publications, was analyzed, and selected 75 characteristics named or defined in these references (Table 1).

2.
Selected AI characteristics were coded and provided alphabetically. The codification is performed using three Latin letters, which ensures their uniqueness. 3.
All references to the definitions in Table 1 are provided in the two right-hand columns: • Column with an indication of the reference where the characteristic is mentioned "Referenced in" provides links to publications where the relevant AI characteristic is only mentioned but not defined; • Column with an indication of the reference where the definition of the characteristic is given "Defined in" provides references to publications where the characteristics of AI or AI systems are defined in any verbal version.
During the analysis and harmonization of the definitions of these characteristics, all references are divided into three groups:

•
(GS) normative documents-standards, guidelines, technical reports of leading institutes and organizations, namely ISO, UNESCO, NIST, etc., as well as professional dictionaries and guides. These documents define the relevant characteristics to which we refer and take; • (GR) scientific publications-articles, monographs, posts, etc., which provide their definitions of characteristics, or are based on the definitions provided in the links of the first group; • (GV) academic dictionaries are used to define characteristics in the general sense when their definitions are missing in the documents of the first and second groups, or when they need to be clarified due to inconsistencies in the definitions given in the references of these groups.
Consequently, for each of the attributes in Table 1, the link in the column «Reference where the definition of the characteristic is given» is provided in a separate line (from the first to the third line according to the groups: GS, GR, and GV). Table 1, thus, collected 75 different terms related to characteristics and systematized references to relevant, most representative sources. Then their substantive analysis is performed.

Analysis of AI Principles
In the first phase of the analysis of all terms collected in Table 1, those that are not real characteristics but can be attributed to the principles related to the development of requirements, creation, and use of AI systems were identified.
Principles are generalized attitudes (values) and views that underlie the development, evaluation, and application of artificial intelligence, AI platforms, and AI systems.
Based on this, a set of characteristics is formed, which directly determines the various components of quality in a broad sense, and which can be measurable. Therefore, the terms "quality" and "quality model" are used as generalizations by analogy with the quality models of software and information systems.
In the process of analyzing key references related to the principles and characteristics of AI [13][14][15][16][17][18]22], as well as other sources provided in Table 1, the following are identified:

•
The boundary between principles and characteristics is difficult to find because principles can dissolve into properties, and characteristics can be generalized into principles. Some characteristics are even identified with the principles; • Even between different principles (groups of principles) there is a significant intersection in their components, which are in fact characteristics; • The key difference between the characteristics and the principles is the possibility of their measurement (evaluation). Therefore, considering, firstly, the priorities of engineering practice, we have referred to the characteristics of the relevant concepts when such a possibility exists, i.e., there are or can be proposed criteria (scales and assessment methods) for it.
These conclusions are illustrated in Table 2, which provides the results of analyzing two important documents [2,22] and suggestions for the delineation of principles and characteristics.
In addition, in many sources and some sectoral regulations, such as [113,114], the principles, characteristics, and relevant requirements are formulated for both the AI systems themselves and for the personnel who develop and are responsible for their use of AI systems. It should also be noted that there are significant characteristics and differences in the list and detail of principles and characteristics for different industries, namely defense systems, healthcare, law, and education [13,19,20,113]. There are strong developments on standards for AI and AI systems in such areas, especially made by the IEEE [11,12] and other institutions, which is the subject of a separate analysis.
Thus, according to the analysis of Table 1, which contains 75 terms, with the excluded terms in Table 2 (seven terms related to the principles), it is necessary to analyze and harmonize the definitions for the remaining (68) characteristics.

Harmonization of AI Characteristics Definitions
The analysis and harmonization of the definitions of the characteristics of AI and AI platforms were performed as follows.

1.
The analysis of definitions considered that individual characteristics can be identical, i.e., those that have different names but the same essence. Of the subsets of such properties, only one remained for further use. For example, of the characteristics "governance" and "controllability", which means "ability to be controlled", the characteristic "controllability" was left as more general. Plus, under the characteristics "explicability" and "explainability", which means "ability to be explained", for further consideration the characteristic "explainability" is chosen.

2.
Some characteristics with insignificant differences were combined, and these differences were considered in the relevant definitions. For example, the characteristic "human oversight and determination" was absorbed by the characteristic "human oversight", considering the characteristics of AI supervision suggested in the absorbed characteristic in the final definition.

3.
Several characteristics have been excluded because, in our opinion, they do not have specific features for AI and AI platforms, but are common to technical systems or their software and hardware. Such characteristics include, in particular, "confidence" and "compliance". 4.
For characteristics relevant to AI and AI platforms, the definition is provided by: • Repetition (citation) or insignificant adjustment of the definition of one of the documents, which is the most adequate and accurate, according to the authors (marked with the letter R-referred). The definition of the attribute "integrity" was given, for example, by [31]; • Harmonization of definitions based on definitions provided in various publications (marked with the letter H-harmonized). The essence of harmonization was to identify key terms and combine the essential components of different definitions of the characteristic being analyzed. The definition of the trait "integrity" was obtained, for example, by combining the essential components of the definitions of this trait proposed in [6,31,33]; • Definition provided by the authors in the absence or in their opinion unsatisfactory wording for the description in the available sources (marked with the letter A-author). Thus, for example, the definition of the characteristic "resiliency" was obtained.
The relationship with the characteristics of different groups (AI, AI platforms, or AI and AI platforms) is determined in the wording itself, and the relationship with the type of quality characteristic (product quality QP and quality in use QU) is determined by a separate column.
The results of the analysis and harmonization of AI characteristics definitions are given in Table 3. Thus, 46 characteristics were selected, of which: The results of the analysis are presented by the classification map of quality characteristics ( Figure 3) according to the template developed in Section 2.1 (Figure 1). In this map, the corresponding characteristics in each cell are given, considering two parameters according to the object of quality assessment and the type of quality characteristics.  Quality characteristics in the classification map of quality characteristics cells ( Figure  2) are written alphabetically without considering their importance or interdependence. Therefore, the next step should be to determine the dependencies of the characteristics and build a model of AI quality.

Description of AI Characteristics Hierarchies
Given a large number of characteristics of AI, AI platforms, and AI systems (Table 3), we present a description of the hierarchies of characteristics, determine the presence, and hierarchy of relationships to organize, and provide ease of assessment. Let us analyze how in the well-known sources, firstly, such leading organizations as ISO, UNESCO, NIST, and the EU special commissions, form the corresponding dependencies between the characteristics of AI.
The relationship of dependence and hierarchy between characteristics can be described in the form of simple parenthesis. This form is a multiple representation based on Quality characteristics in the classification map of quality characteristics cells (Figure 2) are written alphabetically without considering their importance or interdependence. Therefore, the next step should be to determine the dependencies of the characteristics and build a model of AI quality.

Description of AI Characteristics Hierarchies
Given a large number of characteristics of AI, AI platforms, and AI systems (Table 3), we present a description of the hierarchies of characteristics, determine the presence, and hierarchy of relationships to organize, and provide ease of assessment. Let us analyze how in the well-known sources, firstly, such leading organizations as ISO, UNESCO, NIST, and the EU special commissions, form the corresponding dependencies between the characteristics of AI.
The relationship of dependence and hierarchy between characteristics can be described in the form of simple parenthesis. This form is a multiple representation based on a hierarchy of characteristics, which is as follows: where A, B, C, F, H-sub-characteristics of characteristic X (or quality characteristics of AI in general); D, E-sub-characteristics of C; G, I, J-sub-characteristics of H.
According to this notation, Table 4  This expression is obtained through verbal analysis of the content of the document and its structuring.

Analysis of AI Characteristics Hierarchies
The conclusions of the analysis of Table 4 are as follows:

•
The number of references, where in some ways the ratios of dependence on properties are given, is rather small. Of the more than one hundred references analyzed in Table 1 and considered in Table 3, the number of distinctive hierarchies that can be identified is 15; • The most commonly used characteristics with described their hierarchies are trustworthiness TST (6 sources), explainability EXP (5 sources), responsibility RSP (3 sources), and ethics ETH (2 sources); • The maximum number of hierarchy levels is 3, for example in [2] the TST property depends on the sub-characteristics of NMA, NMO, ACN, and the sub-characteristics of TRP and ACN depend on TRC, EXP and ADT, and RDR, respectively; There are many inconsistencies in the proposed hierarchies: • Firstly, the composition of the sub-characteristics, which are attributed to the relevant characteristics. For example, the (functional) safety of SFTs is attributed in some sources to the trustworthiness TST [2], which in our opinion is appropriate, and in others [38]-to the ethics of ETH, which is incorrect enough given their purpose; • Secondly, by assigning them to a certain level of the hierarchy. Some characteristics (e.g., explainability EXP) in some sources [36] are characteristics of the first level of the hierarchy, and in others [17]-the second level or even the third one [2]; • Thirdly, the interpretation of these characteristics in general, due to inconsistencies in their definitions.
The discrepancies identified are most likely due to the fact that: • Different documents have different directions due to different target audiences and domains (technical, ethical, legal, etc.), and are therefore insufficiently consistent; • The definition of many characteristics is not provided in sufficient detail and clearly; • Documents were prepared in parallel for a relatively short period (2-3 years) for such a complex and important problem; Authors of non-institutional publications have focused on solving partial problems, so hierarchical dependencies were either not considered at all or were fairly specific [115]. Therefore, it is necessary to build a correct and consistent hierarchy of characteristics based on definitions and analysis of the relationship of dependency.

The Order of Building and Forming of AI Systems Quality Models
Construction of the AI systems quality model is performed in the following order: Stage 1. Dividing the set of properties of the AI systems S ChAIS (Table 3) on the set of characteristics of pure AI, i.e., those who have specific characteristics of artificial intelligence, and those who are characteristics of software and hardware platforms which implement AI − S ChAIP : it also true that S ChAI ∩ S ChAIP = ∅, since there are common characteristics for AI and AI platform.
According to the results of the analysis given in Section 4, we have two subsets of S ChAI , S ChAIP and subset S ChAI−ChAIP , which consists of joint characteristics of AI and AI platform.
Stage 2. Building of hierarchy for the quality model QS ChAI based on the analysis of properties set S ChAI . When building a hierarchy in these and subsequent stages, the following procedure is used: Step 1. Each of the characteristics of S ChAI are compared with all others and choose those that are dependent on others, and are those on which all others are not dependent (the ratio of dependence is determined by experts). Such characteristics must be attributed to the first level of the hierarchy S ChAI−1 (with power m 1 ); Step 2. Characteristics not included in S ChAI−1 , that is, formed a set S ChAI−2 = S ChAI \ S ChAI−1 are divided into m 1 subsets of S ChAI−2i , which do not intersect and affect the corresponding characteristics of the set S ChAI−1 : Step 3. Steps 1 and 2 are repeated for each of the subsets S ChAI−2i with power m 2i , which makes it possible to form the second and third levels of the hierarchy.
This procedure continues, in the case of more levels in the hierarchy. Stage 3. Building of hierarchy for the quality model QS ChAIP based on the analysis of properties set S ChAIP according to the procedure described for the Stage 2.
Stage 4. Combining hierarchies of sets of attributes S ChAI ∪ S ChAIP into a general quality model QS ChAIS .
Thus, quality models are presented in three forms: • Parenthesis (analytical) form, already used in the analysis of hierarchies of characteristics described in well-known publications (Table 3); • Tabular form, in which the columns specify the levels of dependence on the characteristics, and their number is equal to the number of levels of the hierarchy for depth, and the rows specify the characteristics and sub-characteristics according to their dependence on individual groups; • Graph form, the most obvious and convenient for further use to assess the quality of AI. In the graph, the vertices correspond to the characteristics and sub-characteristics, and the edges correspond to the relationship between them.

AI Quality Model QS ChAI
According to the step-by-step procedure, we form a set of characteristics of the first level. It includes the following characteristics S ChAI−1 = {ETH, EXP, LFL, RSP, TST} because they are the most commonly used and directly affect the quality of AI.
The The tabular form of the model is presented in Table 5. The asterisks (*) in the table indicate the characteristics common to AI and AI platform.
The graph form of the model is shown in Figure 4. The tabular form of the model is presented in Table 5. The asterisks (*) in the table indicate the characteristics common to AI and AI platform.
The graph form of the model is shown in Figure 4.   Figure 4. The graph representation of the AI quality model ℎ .

AI Platform Quality Model ℎ
The peculiarity of this model is that it combines many of the actual platform characteristics of ℎ 1 , as well as characteristics of ℎ 2 , which is also part of many AI quality characteristics. At the platform level, these characteristics should also be considered when assessing the quality of AI platform.
Generally, the AI platform quality model is formed from a set of properties of ℎ 1 by the same procedure as the previous AI model. The set of the first level consists of the following characteristics The tabular form of the model is presented in Table 6. The asterisks (*) in the table indicate the characteristics common to AI and AI platform.
The graph form of the model is shown in Figure 5, where the additional vertex AIG combines the characteristics of the set ℎ − ℎ .

AI Platform Quality Model QS ChAIP
The peculiarity of this model is that it combines many of the actual platform characteristics of S ChAIP1 , as well as characteristics of S ChAIP2 , which is also part of many AI quality characteristics. At the platform level, these characteristics should also be considered when assessing the quality of AI platform.
Generally, the AI platform quality model is formed from a set of properties of S ChAIP1 by the same procedure as the previous AI model. The set of the first level consists of the following characteristics The tabular form of the model is presented in Table 6. The asterisks (*) in the table indicate the characteristics common to AI and AI platform.
The graph form of the model is shown in Figure 5, where the additional vertex AIG combines the characteristics of the set S ChAI−ChAIP .

AI System Quality Model
ℎ AI system quality model combines the previous two ( Figure 6). HMA  HMO  RDR  ACN  CSL  CMT  CMH  TRP  INP  INR  VFB  DVS  RSL  RBS  SFT  SCR  ACP   BIS  NDS  TRC  SWB  PRV  ING  OBC   ETH  EXP  LFL  RSP   TST   AI   ACS  ACR  ADT  AVL  CNT  EFS  INF  RLB  MNT  SST  USB   AIG   DGV  MTR  TRF  GRN AIP AIS ACR Figure 6. The graph representation of the AI system quality model

FRN GRS
The proposed quality models of AI, AI platform, and AI system can be extended and detailed due to additional characteristics that consider the specific characteristics of different disciplines. On the other hand, these models are quite extensive and can be simplified under certain circumstances and according to a certain procedure.

AI System Quality Model
ℎ AI system quality model combines the previous two ( Figure 6). HMA  HMO  RDR  ACN  CSL  CMT  CMH  TRP  INP  INR  VFB  DVS  RSL  RBS  SFT  SCR  ACP   BIS  NDS  TRC  SWB  PRV  ING  OBC   ETH  EXP  LFL  RSP   TST   AI   ACS  ACR  ADT  AVL  CNT  EFS  INF  RLB  MNT  SST  USB   AIG   DGV  MTR  TRF  GRN AIP AIS ACR Figure 6. The graph representation of the AI system quality model

FRN GRS
The proposed quality models of AI, AI platform, and AI system can be extended and detailed due to additional characteristics that consider the specific characteristics of different disciplines. On the other hand, these models are quite extensive and can be simplified under certain circumstances and according to a certain procedure. The proposed quality models of AI, AI platform, and AI system can be extended and detailed due to additional characteristics that consider the specific characteristics of different disciplines. On the other hand, these models are quite extensive and can be simplified under certain circumstances and according to a certain procedure.
6.3. Developing of Basic AI and AI System Quality Models 6.3.1. Building of a Basic AI Quality Model QS ChAIb The basic AI quality model is developed to make it more compact and engineerfriendly to evaluate real AI systems. The basic model can be obtained by optimizing it "vertically" and "horizontally".
The basic AI quality model differs from the original in the following: • Vertical optimization is performed by presenting the model in two levels. Subcharacteristics of the third level are considered at the level of metrics of the corresponding characteristics of the second level; • The relevant components of the characteristics that are removed or combined can be considered at the level of criteria used for evaluation and weighed accordingly when evaluating the top-level characteristics; • The RSP characteristic is removed because it intersects with other characteristics of this level: (a) ETN and LFL-in terms of responsibility for compliance with ethical and legal norms; TST-in terms of responsibility for complying with the requirements of the user as a whole. Moreover, the requirement to inform him in case of possible breach, which is part of the responsibility, can be considered compulsory and therefore considered in the assessment of reliability; (c) EXP-in terms of suitability for verification and provision of information in the event of a breach of the relevant rules and requirements that are part of the characteristics of TRP, VFB; • LFL characteristics are combined with ETF, as they are similarly worded and differ only in references to ethical and legal norms, the boundary between which is not always clear. After unification, the characteristics of ETN are formulated as follows: ethics-the ability of AI to comply with current moral standards, laws and regulations regarding the results of surgery; • The characteristics of HMA and HMO are combined because they are usually considered together and can complement each other at the metric level. The new definition of HMA is the ability of AI to enable the user to make autonomous informed decisions about the use of AI based on control and to interfere in some way in its functioning; • Accountability of ACN and causability CSL causality are combined with transparency TRP as they can be considered as additional transparency metrics. Transparency can then be defined as the ability of AIs to describe, test and repeat models, individual components and algorithms according to which decisions are made, to determine cause-and-effect relationships and to report on performance in a defined form do; • Acceptability of ACP is excluded as separate, since it is in fact a "soft" component of the actual credibility, the definition of which does not require adaptation. Thus, the parenthesis view of the basic AI quality model can be represented as follows: This model is also described by a graph (Figure 7), which is a sub-graph of the general QS ChAI model and includes 19 characteristics. Optimization was carried out as follows: • One lower level of detail (sub-characteristics: DGV, MTR, TRF, and GRN) is removed; • Accessibility of ACS characteristic is absorbed by availability AVL because availability is often seen as synonymous or as a component of readiness. The definition of AVL after this merge does not need to be adapted, as it includes the term "availability"; • The IFS feature is absorbed by the maintainability MNT and usability USB characteristics. First, it intersects with maintainability, as the provision and use of information is necessary for the recovery, prevention, and modernization of AI system. Second, the information provided to the user for the convenience of using AI system is an important component of the quality of human-machine interfaces.  Optimization was carried out as follows: • One lower level of detail (sub-characteristics: DGV, MTR, TR moved; • Accessibility of ACS characteristic is absorbed by availability AV ity is often seen as synonymous or as a component of readines AVL after this merge does not need to be adapted, as it includes ity"; • The IFS feature is absorbed by the maintainability MNT and usab istics. First, it intersects with maintainability, as the provision an is necessary for the recovery, prevention, and modernization of the information provided to the user for the convenience of us important component of the quality of human-machine interface Optimization was carried out as follows: • One lower level of detail (sub-characteristics: DGV, MTR, TRF, and GRN) is removed; • Accessibility of ACS characteristic is absorbed by availability AVL because availability is often seen as synonymous or as a component of readiness. The definition of AVL after this merge does not need to be adapted, as it includes the term "availability"; • The IFS feature is absorbed by the maintainability MNT and usability USB characteristics. First, it intersects with maintainability, as the provision and use of information is necessary for the recovery, prevention, and modernization of AI system. Second, the information provided to the user for the convenience of using AI system is an important component of the quality of human-machine interfaces.

Building of a Basic AI System Quality Model QS ChAISb
This basic model is a simple combination of basic models QS ChAIb and QS ChAIPb (Figure 9).  Figure 9. The graph representation of the basic AI system quality model ℎ .

Case Study
Consider examples of building a quality model for real AI systems based on the proposed models. Such models can be used to justify the requirements for the developed systems or to check their implementation and adapt design solutions. The process of building models for real AI systems can be called profile development or requirements profiling. Profiling is implemented by defining quality characteristics at each level of the model hierarchy that is important to the system being analyzed.
This task is solved for the two systems. The basic model of AI system quality was chosen for further consideration and use ( Figure 9).
The purpose of the case study is to illustrate how the proposed model can be used to obtain a profile, i.e., models for a particular system with artificial intelligence, considering the characteristics of its (system) use, as well as requirements for its non-functional characteristics. Thus, the solution to this problem is to determine the set of characteristics (formally separating the subgraph from the general graph of the quality model), which must be considered for a particular system (class of systems).
The procedure for obtaining a profile of quality characteristics (requirements) that are important for the system was as follows: • Each of the characteristics and sub-characteristics of the model (Figure 9) was analyzed separately; • The experts (authors of the article and developers of analyzed systems) determined the need to include the characteristics in the AIS quality model; • the decision to include these characteristics in the quality model was made by consensus after considering the opinions of the experts. From our point of view, such a consensus-based approach is acceptable considering the complexity of the tasks and the lack of principal differences in opinions of the experts.

Quality Model of the UAV's Video Navigation System as an AI System
The first example relates to the UAV's video navigation system (VNS), which can be used as a separate device or as part of the UAV fleet [116] in monitoring, reconnaissance, etc. The system is based on a convolutional neural network (CNN) with a multilevel structure [117]. The architecture consists of a convolutional neural network for visual features, an extreme learning machine for estimating position bias, and an advanced classifier of extreme information values to predict UAV barriers.

Case Study
Consider examples of building a quality model for real AI systems based on the proposed models. Such models can be used to justify the requirements for the developed systems or to check their implementation and adapt design solutions. The process of building models for real AI systems can be called profile development or requirements profiling. Profiling is implemented by defining quality characteristics at each level of the model hierarchy that is important to the system being analyzed.
This task is solved for the two systems. The basic model of AI system quality was chosen for further consideration and use ( Figure 9).
The purpose of the case study is to illustrate how the proposed model can be used to obtain a profile, i.e., models for a particular system with artificial intelligence, considering the characteristics of its (system) use, as well as requirements for its non-functional characteristics. Thus, the solution to this problem is to determine the set of characteristics (formally separating the subgraph from the general graph of the quality model), which must be considered for a particular system (class of systems).
The procedure for obtaining a profile of quality characteristics (requirements) that are important for the system was as follows: • Each of the characteristics and sub-characteristics of the model (Figure 9) was analyzed separately; • The experts (authors of the article and developers of analyzed systems) determined the need to include the characteristics in the AIS quality model; • the decision to include these characteristics in the quality model was made by consensus after considering the opinions of the experts. From our point of view, such a consensusbased approach is acceptable considering the complexity of the tasks and the lack of principal differences in opinions of the experts.

Quality Model of the UAV's Video Navigation System as an AI System
The first example relates to the UAV's video navigation system (VNS), which can be used as a separate device or as part of the UAV fleet [116] in monitoring, reconnaissance, etc. The system is based on a convolutional neural network (CNN) with a multilevel structure [117]. The architecture consists of a convolutional neural network for visual features, an extreme learning machine for estimating position bias, and an advanced classifier of extreme information values to predict UAV barriers.
The quality model of VNS as an AI system with built-in CNN is shown in Figure 10. The characteristics which should be considered are marked in gray color. The model features are as follows: • At the first level, all three main characteristics of AI quality include: ethics ETH, explainability EXP, and trustworthiness TST; • Under the sub-characteristics of ethics the fairness FRN is left, because for VNS the component of FRN definition is important-to reduce the risks of anomalies due to erroneous assumptions and errors in the process of minimizing model setting; for explainability EXP all sub-characteristics are included except for interactivity INR, taking into account the autonomous operation of the UAV; for trustworthiness TST all sub-characteristics are also included, except for diversity DVS, as the application of the multi-versatility principle in on-board systems is limited by the need to minimize overall mass and energy performance; • For the AI platform level, all characteristics are included except for the audibility ADT, controllability CNT, and sustainability SST with respect to the purpose and functions of the VNS.
The quality model of VNS as an AI system with built-in CNN is shown in Figure 10. The characteristics which should be considered are marked in gray color. The model features are as follows: •  Figure 10. The graph representation of the quality model of VNS as an AI system with built-in CNN.

Quality Model of the Decision Support System for the Diagnosis of Urological Diseases as an AI System
The second case illustrates the construction of a quality profile for a decision support system for the diagnosis of urological diseases, based on a distributed three-tier system of neural network modules with additional training [118]. At the first level-the diagnostic urological office-the patient is examined directly and the relevant parameters of urological fluorograms are measured and processed to support the doctor's decision about the patient's diagnosis using neural network modules; at the second group level-the regional network of hospitals-the accumulation and exchange of cases of urological diseases is carried out to ensure intensive training of neural network modules of all hospitals; at the third level, the interregional level, information is exchanged between groups for further study, taking into account the details of each region.
The model of the quality of the decision support system for the diagnosis of urological diseases (SDUD) as an AI system with a built-in convolutional neural network is shown in Figure 11. Its features are as follows:

Quality Model of the Decision Support System for the Diagnosis of Urological Diseases as an AI System
The second case illustrates the construction of a quality profile for a decision support system for the diagnosis of urological diseases, based on a distributed three-tier system of neural network modules with additional training [118]. At the first level-the diagnostic urological office-the patient is examined directly and the relevant parameters of urological fluorograms are measured and processed to support the doctor's decision about the patient's diagnosis using neural network modules; at the second group level-the regional network of hospitals-the accumulation and exchange of cases of urological diseases is carried out to ensure intensive training of neural network modules of all hospitals; at the third level, the interregional level, information is exchanged between groups for further study, taking into account the details of each region.
The model of the quality of the decision support system for the diagnosis of urological diseases (SDUD) as an AI system with a built-in convolutional neural network is shown in Figure 11. Its features are as follows: • At the first level, the model includes all three characteristics (ethics ETH, explainability EXP, and trustworthiness TST); • For ethics, all sub-characteristics of fairness FRN are included for obvious reasons due to its definition; for explainability EXP all sub-characteristics are included: fairness FRN, graspability GRS, and human agency HMA except redress RDR, considering the fact that it is a decision support system for diagnosis; • Explainability of EXP is represented by all six sub-characteristics as it is important for the medical system; • Trustworthiness is represented by four of the six characteristics, except for diversity DVS and safety SFT, given the lack of functions for forming control effects on patients; • For the level of the AI platform, only controllability CNT is not included due to the lack of functions for forming control effects and the peculiarities of the purpose of VNS. for the medical system; • Trustworthiness is represented by four of the six characteristics, except for diversity DVS and safety SFT, given the lack of functions for forming control effects on patients; • For the level of the AI platform, only controllability CNT is not included due to the lack of functions for forming control effects and the peculiarities of the purpose of VNS. Therefore, if we compare the quality models for these systems, we can deduce a more saturated model for the decision support system for the diagnosis of urological diseases as an AI system relative to the basic-its profile contains 23 characteristics and sub-characteristics out of 27, and for VNS-19 out of 27.
Graph of quality model for VNS in fact is a subgraph of graph for SDUD. The graph of SDUD has the following additional characteristics and sub-characteristics which should be considered for assessment of AIS quality: characteristics ADT and SST, GRS, HMA, INR, and SFT.

Discussion
Despite a large number of publications and the availability of a number of high-level documents issued by reliable national and international institutions, it must be concluded that there is no structured and complete set of AI characteristics, which can be called a quality model similar to an existing and generally accepted software quality models.
The main result of the study is the set of quality models for artificial intelligence, AI platforms, and AI systems. These models are based on the analysis and harmonization of definitions and dependencies of quality traits specific to the AI itself and AI systems. Therefore, if we compare the quality models for these systems, we can deduce a more saturated model for the decision support system for the diagnosis of urological diseases as an AI system relative to the basic-its profile contains 23 characteristics and sub-characteristics out of 27, and for VNS-19 out of 27.
Graph of quality model for VNS in fact is a subgraph of graph for SDUD. The graph of SDUD has the following additional characteristics and sub-characteristics which should be considered for assessment of AIS quality: characteristics ADT and SST, GRS, HMA, INR, and SFT.

Discussion
Despite a large number of publications and the availability of a number of high-level documents issued by reliable national and international institutions, it must be concluded that there is no structured and complete set of AI characteristics, which can be called a quality model similar to an existing and generally accepted software quality models.
The main result of the study is the set of quality models for artificial intelligence, AI platforms, and AI systems. These models are based on the analysis and harmonization of definitions and dependencies of quality traits specific to the AI itself and AI systems. According to our conclusions, some of the characteristics were common to these two entities in the sense that they could be provided at the level of development of the AI itself and its platform.
We tried to select characteristics and build quality models in such a way as to eliminate duplication, ensure completeness of presentation, determine the specific peculiarities of each of the characteristics, and distinguish between the characteristics of the AI and the platform on which AI has been deployed. Clearly, it is extremely difficult to create a model that fully meets such requirements, so the proposed options need to be supplemented and improved considering the rapid development of technologies and applications of AI.
Quality models are presented in this study in various forms-analytical (brackets), tabular, and graph forms, which are convenient for substantive and formal analysis. These models develop the results of [115] and offer the possibility to obtain partial quality profiles for specific developments, considering the details of the respective systems, as demonstrated in the two cases. They can be used as a basis for metric-based AI quality assessment.
The proposed quality models are open and can be supplemented and detailed according to the specific purpose and scope of the AIS. In our opinion, based on the proposed models, it is possible to develop an intersectoral quality standard and requirements for AI (AI systems and AI platforms).

Future Research
Further research needs to be conducted in the following areas: • Profiling (addition and detailing) of models for specific industries (healthcare, law, industrial systems, mobility, etc.) considering evolution issues [119]. Such profiling should be accompanied by an overview of the characteristics and sub-characteristics added based on experience in the development and use of AI, for further generalization in the quality models of AI, AI systems, and AI platforms; • Development of metrics and algorithms for evaluating AI and AI platforms for each of the proposed characteristics and quality in general. It is advisable to collect and analyze information on various criteria for their inclusion in the general database; • Development of tools and case-oriented methods for assessing the quality of AI, AI systems, and AI platforms [120]. They can be based on general assurance case approaches [121,122] as well as functional and cybersecurity assessment approaches [123]; • Application of internal validation as an additional procedure which can be embedded into AIS assessment [124]; • Development of content quality models including different aspects of image quality assessment and so on.